Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Welcome to Aria

Aria’s Rust libraries turn speech into text and text into spoken audio. Use them to add microphone input, transcription, or speech playback to your application. They also power the Aria voice assistant.

This guide assumes you know Rust and Cargo.

Start here

First, add Aria to your project. Then choose a task:

For a first transcription, start with a short WAV file. That lets you check the model and its output before adding microphone capture.

Whisper and Kokoro run on your machine. Downloading models and build dependencies needs network access. Read Privacy before processing personal recordings.

For everyday use of the voice assistant, read the User Guide.

Licensing

The library crates used here are Apache-2.0. The Aria application, CLI, and macOS app are GPL-3.0-only. Depending on aria-stt, aria-tts, or aria-kit does not require a dependency on the application crate, aria.

Engine dependencies and model weights have their own licenses. Review those before distributing your application. The repository records bundled third-party material in THIRD-PARTY-NOTICES.md.

Setup

Start with a small Cargo project: add the dependencies for your task, check that they build, then run an example. You can use the same dependencies in an existing project. No Aria service or configuration file is required.

Build tools

Install stable Rust and Cargo with rustup. These examples target macOS and Linux.

Whisper and Kokoro need a C/C++ compiler, CMake, and Clang/libclang. On macOS, install the Xcode command-line tools.

On Linux, microphone capture and library playback also need ALSA development headers and pkg-config. For the system voice, install espeak-ng or espeak under /usr/bin. macOS provides the system voice through say.

Create a project

cargo new speech-demo
cd speech-demo

Run the remaining commands from speech-demo, unless a page says otherwise. cargo add adds the latest release of each crate to Cargo.toml.

Transcription

cargo add aria-stt --features whisper

aria-stt has no engine enabled by default. The whisper feature selects the native Whisper implementation.

For microphone input, also enable the microphone feature:

cargo add aria-stt --features microphone

Speech output

For the system voice, run:

cargo add aria-tts --features system

For Kokoro audio generation, run this instead:

cargo add aria-tts --features kokoro

If your application uses both Whisper and Kokoro, add both aria-stt and aria-tts. To use both the system voice and Kokoro, run cargo add aria-tts --features system,kokoro.

Check the build

cargo check

The first build downloads the crates. Microphone capture and Kokoro also need ONNX Runtime, which the build downloads and verifies when needed. Download speech models separately, as shown in each example.

If you already have compatible ONNX Runtime libraries, you can select them with ORT_LIB_LOCATION:

ORT_LIB_LOCATION=/opt/onnxruntime/lib \
  cargo check

After the check passes, you’re ready to transcribe a file, record from a microphone, or generate speech.

File transcription

Start by turning a short WAV recording into text with Whisper. Once that works, you can choose a language or correct project terms in the result. Add the aria-stt dependency and check your build tools before continuing.

Model and audio

Whisper needs a GGML model from the whisper.cpp model collection. This example uses ggml-base.en.bin, a small English model:

curl -fL -o ggml-base.en.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.en.bin

Use the model’s absolute path below.

The input must be a nonempty WAV containing 16 kHz, mono, signed 16-bit PCM (uncompressed audio), with at most 15 minutes of audio. MP3, stereo WAV, and other sample rates are not converted automatically. To convert a recording with ffmpeg:

ffmpeg -i input.m4a \
  -ar 16000 \
  -ac 1 \
  -c:a pcm_s16le \
  speech.wav

Use a short recording with a known sentence for your first test.

Replace src/main.rs with:

use aria_stt::{prelude::*, whisper::Whisper};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let engine = Whisper::load("/absolute/path/ggml-base.en.bin")?;
    let transcription = Transcription::streaming(engine);
    let transcript = transcription.transcribe_file("speech.wav")?;
    println!("{}", transcript.as_str());
    Ok(())
}

Replace the model path, put speech.wav in your project directory, and run:

cargo run \
  --release

The program prints the recognized sentence. Whisper::load loads the model; Transcription uses it to process the recording. Reuse transcription for later files to avoid loading the model for each one.

transcribe_file uses automatic language selection; an English-only model still recognizes English only. Transcription::streaming also works with a completed file and returns the complete result here.

If model loading fails, check that you downloaded a GGML file and supplied an absolute path. Model files must be owned by your user or root and must not be writable by group or others (for example, chmod 644). If input loading fails, check the WAV format above. Silence or empty recognized output is reported as TranscriptionError::NoSpeech.

Choose a language

If you know the recording’s language, you can select it explicitly. Use a RecordingReader to read the WAV and pass the language to transcribe. Replace src/main.rs with:

use aria_stt::{prelude::*, whisper::Whisper};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let engine = Whisper::load("/absolute/path/ggml-base.en.bin")?;
    let transcription = Transcription::streaming(engine);
    let mut audio = RecordingReader::from_wav_file("speech.wav")?;
    let language: Language = "en".parse()?;
    let transcript = transcription.transcribe(&mut audio, &language)?;
    println!("{}", transcript.as_str());
    Ok(())
}

The selected language must be supported by the model.

Project spellings

Recognition may spell a project name differently from how you write it. Text layers let you correct those spellings after recognition. Keep the model-loading line and replace let transcription = ... with:

use aria_stt::{prelude::*, whisper::Whisper};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let engine = Whisper::load("/absolute/path/ggml-base.en.bin")?;
let transcription = Transcription::streaming(engine)
    .with(RemoveHesitations)
    .with(Vocabulary::new(["aria-stt", "Cargo.toml"])?);
let _ = transcription;
Ok(())
}

Layers run in .with(...) order. RemoveHesitations removes a fixed set of English fillers. Vocabulary can turn “aria dash stt” into aria-stt and “cargo dot toml” into Cargo.toml. It corrects text after recognition; it does not train or prompt the model, and it is not general fuzzy spelling correction.

With live transcription, layers process each finalized segment separately. A phrase split across segments cannot be matched as one vocabulary term.

Test your spellings

Vocabulary works without a model, so you can test it directly. Put this in tests/vocabulary.rs:

use aria_stt::Vocabulary;

#[test]
fn vocabulary_formats_project_terms() {
    let vocabulary = Vocabulary::new(["aria-stt", "Cargo.toml"]).unwrap();

    assert_eq!(
        vocabulary
            .correct("aria dash stt and cargo dot toml")
            .as_deref(),
        Some("aria-stt and Cargo.toml")
    );
    assert_eq!(vocabulary.correct("unchanged"), None);
}

Run cargo test. correct returns Some when it changes text and None when it leaves text alone. Add cases for your own terms and for ordinary text that must stay unchanged.

To use the same transcription with microphone input, continue to Microphone transcription.

Microphone transcription

With a Whisper model ready, you can transcribe microphone input using the same engine. Enable the whisper and microphone features of aria-stt. If you still need a model, follow Model and audio.

On macOS, grant microphone access to the application running the example. On Linux, install ALSA development headers and pkg-config before building. Start with a short utterance and a working default input device.

Stop with Enter

Start with a recording you stop yourself. This program records until you press Enter, then transcribes the recording and prints the completed text.

Replace src/main.rs with this program, using the absolute path to your Whisper model:

use aria_stt::{prelude::*, whisper::Whisper};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let engine = Whisper::load("/absolute/path/ggml-base.en.bin")?;
    let transcription = Transcription::streaming(engine);
    let session = Session::microphone(&transcription, &Language::Automatic)?;

    println!("Speak, then press Enter to finish.");
    std::io::stdin().read_line(&mut String::new())?;

    println!("{}", session.stop()?.as_str());
    Ok(())
}

Run with cargo run --release, speak a sentence, and press Enter within a minute. The default microphone limit is 60 seconds; exceeding it produces an error rather than a successful transcript.

stop() ends capture, processes the remaining audio, and returns the transcript. Dropping the session without calling stop() requests a stop without returning a transcript.

Live text

The first example prints the transcript after you stop recording. To show text as it becomes available, read finalized segments: pieces of recognized text that will not change. This version also stops recording automatically after silence. Replace src/main.rs with the following, keeping your model path:

use aria_stt::{prelude::*, whisper::Whisper};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let engine = Whisper::load("/absolute/path/ggml-base.en.bin")?;
    let transcription = Transcription::streaming(engine);
    let mut microphone = Microphone::open()?;
    let (recording, _stopper) = microphone.record(RecordingMode::StopOnSilence {
        trailing_silence: Default::default(),
    })?;
    let mut session = Session::over(&transcription, &Language::Automatic, recording)?;

    while let Some(segment) = session.next_finalized()? {
        println!("{}", segment.as_str());
    }

    println!("Complete: {}", session.stop()?.as_str());
    Ok(())
}

record also returns a Stopper. Call its stop() from another thread or a UI control to end the recording early; here, silence ends it.

Speak soon after starting the program, then pause. The default stop condition is 1.5 seconds of silence after speech. If no speech starts within ten seconds, capture returns an error. The 60-second recording limit still applies.

Append finalized segments to your displayed text. The complete transcript contains those segments too; replace the display with it, or use it elsewhere, without appending it a second time. A short utterance may finish before any intermediate text appears.

Here, silence ends the recording. Calling stop() on Session::over waits for that end before completing transcription; it does not stop capture itself.

Recording lifetime

Microphone::open keeps capture open while the microphone object lives. A recording uses a private temporary WAV that can contain background speech. It starts with up to about one second of audio from before record(). The file is removed when the recording objects holding it are dropped.

For how to reuse the model and manage sessions, read Engines and sessions.

Generate speech

Use the system voice when you want text read aloud without downloading a model. Use Kokoro when your application needs audio samples for its own player or file writer. Start with the example that matches the output you need. Both examples use the synthesis dependencies.

System voice

Enable the system feature of aria-tts, as shown in setup. On macOS, this uses say. On Linux, install espeak-ng or espeak at its standard /usr/bin location.

Replace src/main.rs with:

use aria_tts::{prelude::*, system::SystemVoice};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let voice = SystemVoice::new()?;
    voice.speak(
        &SpeechText::new("Hello from Aria.")?,
        None,
        SpeakingRate::standard(),
        &CancelToken::never(),
    )?;
    Ok(())
}

Run cargo run --release. You should hear the sentence through the system voice. The call returns after playback finishes or fails. None selects the platform’s default voice; SpeakingRate::standard() speaks at 175 words per minute. CancelToken::never() lets this example run without a cancellation signal.

The system voice runs in a child process. It needs no playback feature and provides no raw audio samples (PCM), so it cannot generate an audio file.

Kokoro audio

Enable kokoro instead of system in the aria-tts dependency. Download the model and voices from Kokoro’s model-files release:

curl -fLO https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/kokoro-v1.0.onnx
curl -fLO https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/voices-v1.0.bin

The voices file is a NumPy archive despite its .bin extension.

This example collects generated samples and prints their count. Replace src/main.rs with the following, using the absolute paths to your two files:

use aria_tts::{kokoro::Kokoro, prelude::*};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let engine = Kokoro::load(
        "/absolute/path/kokoro-v1.0.onnx",
        "/absolute/path/voices-v1.0.bin",
    )?;
    let speech = Synthesis::new(engine);
    let mut session = speech.start(&SpeechText::new("Hello from Aria.")?)?;
    let sample_rate = session.sample_rate();
    let mut samples = Vec::new();
    let mut buffer = [0.0; 1024];

    loop {
        let count = session.read(&mut buffer)?;
        if count == 0 {
            break;
        }
        samples.extend_from_slice(&buffer[..count]);
    }

    println!("Generated {} samples at {sample_rate} Hz", samples.len());
    Ok(())
}

Run cargo run --release. This prints the sample count and sample rate; it does not play the audio. Kokoro supplies mono f32 PCM at 24 kHz and uses US English text processing. Its standard voices archive includes the default af_heart voice.

read() waits for samples, completion, or an error. A zero count means successful completion; an error means generation failed. Consume only buffer[..count]. To avoid storing the whole utterance, send each block to your player or file writer.

Dropping the session requests cancellation and waits for its thread to stop. The engine may take time to stop, so keep that cleanup off the UI thread.

Kokoro playback

Run cargo add aria-kit --features playback. It adds aria-kit, or enables the feature on an existing entry. Keep the Kokoro-loading and Synthesis::new lines from the previous example. Replace the sample-reading code, from let mut session through println!, with:

use aria_tts::{kokoro::Kokoro, prelude::*};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let engine = Kokoro::load("/absolute/path/kokoro-v1.0.onnx", "/absolute/path/voices-v1.0.bin")?;
let speech = Synthesis::new(engine);
speech.speak(&SpeechText::new("Hello from Aria.")?)?;
Ok(())
}

This plays through the default output device and returns when the utterance finishes. Linux playback also needs ALSA headers and pkg-config at build time.

Text and voices

SpeechText::new trims input and rejects empty text or text over 8 KiB. Voice::new validates a name’s form; whether it exists depends on the engine. SpeakingRate::new accepts a finite multiplier from 0.25 to 4.0.

For Kokoro, start_with lets you choose a voice and speaking rate and supply a cancellation token. Use names from your voices archive. Keep model loading separate from each utterance, as explained in Engines and sessions.

Engines and sessions

The examples use two kinds of object: a loaded engine and a session for one recording or utterance. Keeping them separate lets you load a model once and reuse it for later requests.

Libraries

aria-stt provides transcription, aria-tts provides synthesis, and aria-kit provides microphone capture, audio playback, and shared speech types.

The engine crates re-export their common types in prelude. That is why transcription imports Transcription, RecordingReader, and Microphone from aria_stt, and synthesis imports Synthesis and CancelToken from aria_tts. Add aria-kit directly only for playback.

Transcription

Load Whisper once
  → construct Transcription
  → provide a WAV or microphone recording
  → receive finalized text

Whisper loads and runs the recognition model. Transcription chooses how to process the recording and applies any text layers. Session manages transcription of one recording.

For a completed file, transcribe_file returns one Transcript. For live input, Session::next_finalized returns segments as they become final, and stop returns the complete transcript. Finalized segments stay unchanged.

The libraries do not insert text into another application. Your code decides whether to display, save, or deliver it.

Synthesis

Load Kokoro once
  → construct Synthesis
  → start an utterance
  → read audio samples until completion

Kokoro supplies the model and voices. Synthesis starts one worker per utterance. Its session gives you the sample rate and generated audio samples. Dropping the session requests cancellation and waits for the thread to stop.

The system voice has a different output: it speaks through the platform’s voice program and provides no audio samples.

Blocking calls

Model loading, transcription, audio reads, and speaking are synchronous calls. Run them on a worker thread when integrating with a UI or an asynchronous application. Streaming can produce output before an operation finishes, but reading the next result can still block.

Privacy

Whisper, Kokoro, and the system voice run on your machine and make no network requests. Only builds and model downloads use the network. Your code decides what to save or send.

Temporary recordings

Microphone capture writes each recording to a private WAV in aria-<uid>/ under the system temporary folder ($TMPDIR, or /tmp). The folder is 0700 and the file 0600. It holds everything the microphone heard, including background voices and about one second from before record().

Dropping the recording deletes the file. If your process crashes or is killed, the file stays until you or the system remove it. Deletion is not secure erasure and cannot remove snapshots or backups.

Your files and output

transcribe_file reads your WAV and leaves it in place. Transcripts and generated audio stay in memory until your code saves them.

Model files

Use model files from sources you trust, and check their licenses before shipping. Native loaders require an absolute path, current-user or root ownership, and a file that group and others cannot write. These checks do not isolate engine code.