Qwen logo

Qwen3.8-27B-BFP4-Infall-TT

BFP4 quantization of Qwen/Qwen3.8-27B by Qwen · Updated November 21, 2026

BFP4TenstorrentApache 2.0

What is Qwen3.8-27B-BFP4-Infall-TT?

Qwen3.8-27B-BFP4-Infall-TT is a Tenstorrent-native BFP4 (bfloat4_b) quantization of Qwen/Qwen3.8-27B, released by the Qwen team. We quantized it with the beta preview of Infall-TT, solving the weights with GPTQ directly on Tenstorrent’s BFP4 grid, so every value already lies exactly on the format tt-metal uses on Blackhole.

It is a non-GGUF, native checkpoint in the standard Hugging Face format: the same files, tensor names and shapes as the original, about 56 GB to download. In BFP4 the weights take about 4.5 bits each, roughly 15 GB for the model.

Specification

Original modelQwen/Qwen3.8-27B
Model providerQwen
Release dateNovember 21, 2026
LanguagesMultilingual (as the original model)
Model typeCausal language model with vision encoder
Parameters27B
QuantizationTenstorrent BFP4 (bfloat4_b), quantized with Infall-TT by Synthetic Minds
FormatHugging Face safetensors (18 shards), values stored in BF16 on the BFP4 grid
Download sizeAbout 56 GB (about 15 GB once compressed to BFP4 by tt-metal)
LicenseApache 2.0
Model developerSynthetic Minds
Target hardwareTenstorrent Blackhole: P150A/B, P300C, Galaxy Blackhole

Quantization quality

Measured against the original BF16 model on held-out text the quantizer never saw: 40 windows of 2,048 tokens per set (81,880 predicted tokens each). “Tenstorrent default” is what tt-metal produces from the original weights: every weight rounded to the nearest value on the same BFP4 grid.

WikiText-2 perplexityC4 perplexityKL divergence (C4)Top-1 agreement (C4)
BF16 (original)6.48912.1930100%
Tenstorrent default BFP47.205 (+11.0%)12.839 (+5.3%)0.05985.7%
Infall-TT BFP46.750 (+4.0%)12.600 (+3.3%)0.04287.7%

Lower perplexity and KL divergence are better. Higher top-1 agreement (how often the quantized model’s most likely next token matches the original’s) is better.

Perplexity increase over BF16 (lower is better)
Tenstorrent default BFP4WikiText-2+11.0%
Infall-TT BFP4WikiText-2+4.0%
Tenstorrent default BFP4C4+5.3%
Infall-TT BFP4C4+3.3%

Why Infall-TT is better. Round-to-nearest treats every weight on its own, so the small errors add up through the network. Infall-TT solves for the weights layer by layer with GPTQ directly on Tenstorrent’s BFP4 grid: each rounding error is compensated by adjusting the weights not yet rounded, using real text to see which errors matter, and the shared exponent of each block is chosen to minimize the error. On the same format and the same memory, this cuts the perplexity increase from 11.0% to 4.0% on WikiText-2 and from 5.3% to 3.3% on C4, and the KL divergence by 42% (WikiText-2: 0.093 → 0.054) and 29% (C4).

How to use it

This repository contains the full weights in the standard Hugging Face format. The quantized values are stored in BF16, so any tool that reads the original checkpoint reads this one. When tt-metal loads the weights, it automatically compresses every projection and the LM head to BFP4. Every value in this checkpoint already lies exactly on that BFP4 grid, so the compression keeps the Infall-TT weights unchanged instead of rounding the originals.

Download it and point tt-metal’s Qwen3.8-27B demo (or its vLLM integration) at it instead of the original weights. No code changes are needed:

huggingface-cli download synthetic-minds/Qwen3.8-27B-BFP4-Infall-TT --local-dir Qwen3.8-27B-BFP4-Infall-TT
export MODEL_WEIGHTS_DIR=$PWD/Qwen3.8-27B-BFP4-Infall-TT
python models/demos/qwen38_27b_qb2/demo/text_demo.py --full --length 128 --generate 128

The weight layout was checked against tt-metal’s models/demos/qwen38_27b_qb2 implementation.

What was quantized

Method

Limitations

Frequently asked questions

What is Qwen3.8-27B-BFP4-Infall-TT?

Qwen3.8-27B-BFP4-Infall-TT is a Tenstorrent-native BFP4 (bfloat4_b) quantization of Qwen/Qwen3.8-27B, made by Synthetic Minds with the beta preview of Infall-TT. It uses GPTQ solved directly on Tenstorrent's BFP4 grid, so the weights already lie exactly on the format that tt-metal uses on Blackhole.

What is BFP4?

BFP4 (bfloat4_b) is Tenstorrent's block floating point format: 4-bit mantissas with a shared exponent per block, about 4.5 bits per weight including the shared exponents. In BFP4 the 27B model takes roughly 15 GB.

How is it better than Tenstorrent's default BFP4?

Tenstorrent's default rounds every weight to the nearest value on the BFP4 grid. Infall-TT instead solves the weights layer by layer with GPTQ on that same grid, compensating each rounding error and choosing each block's shared exponent to minimize error. On the same format and memory, WikiText-2 perplexity rises 4.0% over BF16 instead of 11.0%, and C4 perplexity rises 3.3% instead of 5.3%.

How do I use it?

Download the repository, set MODEL_WEIGHTS_DIR to it, and point tt-metal's Qwen3.8-27B demo or its vLLM integration at it instead of the original weights. No code changes are needed. The checkpoint is in standard Hugging Face format, with values stored in BF16 that already lie on the BFP4 grid, so tt-metal's load-time BFP4 compression keeps them unchanged.

Has it been run on Tenstorrent hardware?

Not yet. It was quantized and evaluated on our development server using Infall-TT's bit-exact emulation of Tenstorrent's block-float formats. We publish no speed claims, because speed depends on tt-metal and the hardware, not on this checkpoint.

Can I use it with llama.cpp or Ollama?

No. This is a native Tenstorrent checkpoint in Hugging Face safetensors format, not a GGUF. For GGUF builds that run on any machine with llama.cpp, see our GGUF models.

What license is it under?

Apache 2.0, the same license as the original Qwen3.8-27B model.

Support our work

Infall-TT is built by Synthetic Minds without access to Tenstorrent hardware: the framework is developed and tested on Tenstorrent’s ttsim simulator. Your support, on Buy Me a Coffee, will help us acquire a TT-QuietBox 2, so we can run and verify models like this one on real silicon, which the simulator cannot do at this size. You can also join the Infall-TT waitlist or get in touch at hello@syntheticmindslab.com.

License

Apache 2.0, the same license as the original model. Original model: Qwen3.8-27B © the Qwen team. Quantization © Synthetic Minds | Infall-TT framework.