This bounty is for bringing up the Higgs Audio v2 model using TTNN APIs on Tenstorrent hardware (Wormhole or Blackhole).
Higgs Audio v2 is a cutting-edge text-audio foundation model from Boson AI that redefines expressiveness in audio generation. Released in 2025, it offers state-of-the-art capabilities:
Exceptional expressiveness: Industry-leading performance on emotion and prosody benchmarks
Multi-speaker dialog: Natural multi-speaker conversations with distinct voices
Voice cloning: Zero-shot voice cloning from reference audio
Diverse audio generation: Speech, sound effects, music, and environmental sounds
Massive training scale: Trained on 10 million hours of audio data (AudioVerse dataset)
Custom tokenizer: Unified semantic and acoustic tokenization
DualFFN architecture: Enhanced LLM for acoustic modeling with minimal overhead
Apache 2.0 license: Fully open source and commercially usable
The model achieves 75.71% win-rate on EmergentTTS-Eval emotions and best-in-class multi-speaker generation quality.
The goal is to enable this model to run on TT hardware for high-throughput, low-latency audio generation across diverse use cases including virtual assistants, audiobooks, content creation, and interactive media.
What Success Looks Like
A successful submission will fulfill all requirements in the following stages. Payout is made after all three stages are completed.
Stage 1 — Bring-Up
Implement Higgs Audio v2 using TTNN APIs (Python)
Implements the full generation pipeline:
LLM backbone with DualFFN architecture
Audio tokenizer (semantic + acoustic features)
Audio decoder (token-to-waveform conversion)
Model runs on either N150 or N300 tenstorrent hardware with no errors
Supports multiple generation modes:
Text-to-speech: Generate speech from text only
Voice cloning: Generate speech with reference audio
Multi-speaker dialog: Generate conversations with distinct speakers
Produces valid audio output on sample texts (English and multilingual)
Output is verifiable (audio quality assessment, compare with PyTorch reference)
Achieves baseline throughput target:
At least 30 tokens/second for autoregressive generation
Real-time factor (RTF) < 0.5 for typical sentences
Accuracy evaluation: Token-level accuracy > 95% against PyTorch reference
Audio quality: Passes intelligibility and expressiveness tests
Clear instructions for setup and running the model
Stage 2 — Basic Optimizations
Use optimal sharded/interleaved memory configs for LLM layers
Implement efficient sharding strategy for:
Token embeddings (text + audio tokens)
DualFFN transformer layers
Multi-head attention mechanisms
Audio tokenizer encoder/decoder
Fuse simple ops where possible (e.g., layer normalization, attention patterns, activation functions)
Store intermediate activations in L1 where beneficial
Use recommended TTNN/tt-metal LLM flows
Leverage TT library of fused ops for attention and MLP blocks
Validate each submodule’s output against PyTorch reference before integration
For example the LLM backbone:
Start with standard transformer layers
Add DualFFN architecture (key innovation)
Optimize attention mechanisms
Implement efficient KV-cache
Experiment with different sharding strategies
Use TTNN profiling tools to identify bottlenecks
Test diverse use cases
Open a draft PR early to get feedback on your approach
Result Submission Guidelines
Beyond the model implementation itself, contributors must submit the following material as proof of work.
However, feel free to open a PR at any time if you want us checking that you are on the right track.
Just understand that payout is only made after all 3 stages are completed.
Background
What Success Looks Like
Guidance & Starting Points
Possible Approaches
Result Submission Guidelines
Resources