[Bounty $2500] Bring up microsoft/phi-1 on Wormhole

Bounties
1

Tenstorrent Bounties

Issue ID: I_kwDOI9Wqc86rlxM5

Bounty $500-2500 Bring up microsoft/phi-1 on Wormhole (N150/N300)

:memo: Background

This bounty is for enabling and validating the microsoft/phi-1 language model on Tenstorrent’s Wormhole hardware. Phi-1 is a transformer-based LLM designed with efficiency and compactness for code and text tasks, making it attractive for research, developers, and edge deployments. By running Phi-1 on Tenstorrent hardware, we aim to demonstrate practical inference, benchmark performance, and establish a production-grade implementation for community use.

:bullseye: What Success Looks Like

To claim this bounty, contributors must complete all three stages below:

Stage 1 - Bring Up

  • The microsoft/phi-1 model must compile and run end-to-end inference on Wormhole (N150/N300) with no blocking errors.
  • The model produces coherent output for text generation tasks
  • Integrate phi-1 support into our model directory, maintain the same coding style as used in TT repository and measure performance (see current benchmarks)
  • Include a simple demo script to run inference and show output.

Stage 2 - Basic Optimizations

  • Store temporary results in L1 memory when possible.
  • Fuse activation functions directly after matmul operations.
  • Apply sharding to tensors where applicable.
  • Leverage existing tt-transformers base modules (avoid duplicating code).
  • Document any parameter tuning or known issues.

Stage 3 - Deeper Optimization

  • Adapt model to exploit TT hardware features, e.g.:
    • Employ SDPA operator for efficient attention computation.
    • Minimize unnecessary tensor moves by reordering computation steps.
  • Refactor for modularity and internal code reuse, following production standards.

:compass: Guidance & Starting Points

  • Use the tt-transformers base in tt-metal/models/tt_transformers for model implementation; reuse existing modules where possible.
  • Reference the model bring up guide.
  • Avoid full codebase duplication—implement in a clean, maintainable way.

:magnifying_glass_tilted_right: Possible Approaches

  • Start from an existing transformer implementation in tt-transformers (Llama/Qwen) and adapt for Phi-1.
  • Use modular inheritance to minimize code changes.
  • Profile bottlenecks, optimize memory layout, and use TTNN/tt-metal profiling tools.
  • Reach out in issues or discussions for hardware-specific advice.

:bar_chart: Result Submission Guidelines Notes

Beyond the model implementation itself, contributors must submit the following proof of work. Feel free to open a PR at any stage for feedback, but payout is only made after all 3 stages are completed.

  • Functional model implementation
  • Validation logs showing output correctness and accuracy
  • Performance report + perf header for final review

The performance report generated will be used as a key piece of information for us to decide if the program is reasonably performant.

Links:

For reference our estimated max throughput for phi-1 is 141 tokens/sec/user on Wormhole. A

:books: Resources