Specifications
Japanese TTS
Natural Japanese speech synthesis
Japanese ASR
Japanese speech recognition
Voice Chat
Interleaved Japanese audio/text
Quick Start
- liquid-audio
- llama.cpp
Install:Multi-Turn Chat:Japanese ASR:Japanese TTS:
Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
๐ New: LFM2.5-VL-450M โ our smallest vision model is now available! Learn more โ
1.5B Japanese-focused audio/text model for TTS, ASR, and voice chat
| Property | Value |
|---|---|
| Parameters | 1.5B (1.2B LM + 115M audio encoder) |
| Context Length | 32K tokens |
| Audio Output | 24kHz |
| Supported Language | Japanese |
pip install liquid-audio
pip install "liquid-audio[demo]" # optional, for demo dependencies
pip install flash-attn --no-build-isolation # optional, for flash attention 2
import torch
import soundfile as sf
from liquid_audio import LFM2AudioModel, LFM2AudioProcessor, ChatState, LFMModality
HF_REPO = "LiquidAI/LFM2.5-Audio-1.5B-JP"
processor = LFM2AudioProcessor.from_pretrained(HF_REPO).eval()
model = LFM2AudioModel.from_pretrained(HF_REPO).eval()
chat = ChatState(processor)
chat.new_turn("system")
chat.add_text("Respond with interleaved text and audio.")
chat.end_turn()
chat.new_turn("user")
wav, sampling_rate = sf.read("question_jp.wav", dtype="float32")
wav = torch.from_numpy(wav).unsqueeze(0)
chat.add_audio(wav, sampling_rate)
chat.end_turn()
chat.new_turn("assistant")
text_out, audio_out, modality_out = [], [], []
for t in model.generate_interleaved(**chat, max_new_tokens=512, audio_temperature=1.0, audio_top_k=4):
if t.numel() == 1:
print(processor.text.decode(t), end="", flush=True)
text_out.append(t)
modality_out.append(LFMModality.TEXT)
else:
audio_out.append(t)
modality_out.append(LFMModality.AUDIO_OUT)
audio_codes = torch.stack(audio_out[:-1], 1).unsqueeze(0)
waveform = processor.decode(audio_codes)
sf.write("answer_jp.wav", waveform.cpu()[0], 24_000)
chat = ChatState(processor)
chat.new_turn("system")
chat.add_text("Perform ASR in japanese.")
chat.end_turn()
chat = ChatState(processor)
chat.new_turn("system")
chat.add_text("Perform TTS in japanese.")
chat.end_turn()
export CKPT=/path/to/LFM2.5-Audio-1.5B-JP-GGUF
export INPUT_WAV=/path/to/input.wav
export OUTPUT_WAV=/path/to/output.wav
./llama-liquid-audio-cli -m $CKPT/LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
-mm $CKPT/mmproj-LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
-mv $CKPT/vocoder-LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
--tts-speaker-file $CKPT/tokenizer-LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
-sys "Perform ASR in japanese." --audio $INPUT_WAV
./llama-liquid-audio-cli -m $CKPT/LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
-mm $CKPT/mmproj-LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
-mv $CKPT/vocoder-LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
--tts-speaker-file $CKPT/tokenizer-LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
-sys "Perform TTS in japanese." -p "ใใใซใกใฏใไปๆฅใฏใฉใฎใใใชใ็จไปถใงใใใใใ" --output $OUTPUT_WAV
./llama-liquid-audio-cli -m $CKPT/LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
-mm $CKPT/mmproj-LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
-mv $CKPT/vocoder-LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
--tts-speaker-file $CKPT/tokenizer-LFM2.5-Audio-1.5B-JP-Q4_0.gguf \
-sys "Respond with interleaved text and audio." \
--audio $INPUT_WAV --output $OUTPUT_WAV
Was this page helpful?