Introducing Infall-TT: fine-tuning for Tenstorrent

Nov 20, 2026 · By Synthetic Minds

Note: Infall-TT is not available to the public yet. The code, commands and results in this post are examples of what we are building and may change before release.

Infall-TT

Waitlist now open

We’re excited to introduce Infall-TT, our first open-source project. It is being built to bring quantization, fine-tuning and, later, inference to Tenstorrent Blackhole and Galaxy Neural Accelerators. Infall-TT is in developer preview and is not available to download yet. Some highlights:

There is nothing to install yet. The package and the source code are not published, so the commands below are previews of how it will work. To be reminded the day the beta opens, join the waitlist.

Why we built it

Tools like Unsloth made fine-tuning open models fast and approachable on NVIDIA GPUs. Tenstorrent’s Blackhole chips are a different machine: they compute on tiles of 32×32 values and keep weights in their own block-float formats. A port of a GPU library leaves that on the table, so Infall-TT is written for Blackhole from the first line.

Native BFP4

In BFP4, every 16 consecutive weights share one 8-bit exponent and each keeps a sign and 3 magnitude bits. That is 4.5 bits per weight, read directly by the chip’s matrix engine. Infall-TT reproduces Tenstorrent’s packer bit for bit, so every quantized weight we produce is exactly what the chip will hold.

In the preview, quantizing a model is a single command:

# one command
infall-tt quantize Qwen/Qwen3-8B-Base --out qwen3-8b-bfp4 --layout mixed

To keep the model the same model, we add GPTQ on the block-float grid, exponent search, low-rank corrections and dynamic per-layer layouts, and we measure every change against the original BF16 model.

Accuracy

The hard part of 4-bit is not making the model smaller. It is keeping it the same model. Each technique takes another bite out of the loss:

Each technique takes another bite out of the 4-bit loss

Qwen3-0.6B-Base, WikiText-2 perplexity increase vs BF16. Lower is better.

Plain rounding4.50 bits per weight
+30.9%
GPTQ on the BFP grid4.50 bits per weight
+15.5%
+ exponent search4.50 bits per weight
+12.6%
+ low-rank corrections4.87 bits per weight
+11.3%
+ dynamic layout5.31 bits per weight
+7.6%
+ distillation recovery5.31 bits per weight
+5.2%
+ longer recovery5.31 bits per weight
+4.6%
0%10%20%30%

Calibration on C4, evaluation on WikiText-2 test (held out). Bits per weight count the stored 4-bit and 8-bit weights, their shared exponents and the BF16 corrections. 0.6B is the hardest case for 4-bit; larger models lose less.

And every recipe, at every bit budget, measured against the original BF16 model:

Accuracy for every bit budget

Each point is one quantization recipe. Down and to the left is better.

Plain roundingGPTQ on the BFP gridinfall-tt recovery recipes
0%10%20%30%456789Bits per weightPerplexity increasePlain rounding: +30.9%BFP8: +0.06%+4.6% at 5.3 bits

Qwen3-0.6B-Base, WikiText-2. Same setup as the chart above.

Fine-tuning

Fine-tuning on a 4-bit base is our QLoRA for Tenstorrent. The adapter layer already trains correctly on the Blackhole simulator, and PEFT export already loads in the official peft library. This API is a preview and still in development:

import infall_tt
model, tokenizer = infall_tt.load_model("Qwen/Qwen3-8B-Base", load_in_4bit=True) # frozen BFP4 base
model = infall_tt.lora.apply(model, rank=16, alpha=32)
dataset = [{"prompt": q, "response": a} for q, a in my_examples] # your own data code
trainer = infall_tt.Trainer(model, tokenizer, dataset, max_steps=500, learning_rate=2e-4)
trainer.train()
model.push_to_hub("your-org/qwen3-8b-finetuned", kind="adapter")

Free for one card, enterprise for the cluster

We plan to release infall-tt as open source under Apache 2.0, free and covering everything for a single Neural Accelerator. infall-tt-enterprise will be under a commercial license and will add multi-accelerator training on P300, QuietBox and Galaxy, with support from the team that builds it.

What’s next

State-of-the-art 4-bit accuracy on larger models, then GGUF export for llama.cpp and Ollama, reinforcement learning (GRPO and DPO), inference on a single card, and Galaxy-scale training.

Work with us

We are always seeking sponsorships to accelerate our research and grow Infall-TT, for the open-source community and for enterprises. If you would like to sponsor us, join us, contribute or build with Infall-TT, email us at hello@syntheticmindslab.com.

hello@syntheticmindslab.com

To be reminded when the beta opens, join the waitlist.

A note on results: device results so far come from ttsim, Tenstorrent’s bit-exact Blackhole simulator. It reproduces the chip’s arithmetic exactly but not its timing, so we publish no speed or throughput numbers until we measure on silicon. Accuracy is measured against the original BF16 model on held-out text the method never saw.

Fine-tuning for Neural Accelerators is coming.

Waitlist now open