Ai2 and LMU retrofit Llama and Qwen to read raw bytes

Researchers from LMU Munich, the Allen Institute for AI (Ai2) and three universities published a way to make trained language models read raw bytes. Their Nature paper, out October 7, 2026, calls it byteification. The conversion takes 49.1 billion tokens of training, less than 1% of a typical pretraining run.

Ai2 first released two such models, Bolmo 7B and Bolmo 1B, in December 2025. At the time, VentureBeat reported that companies handling many languages or messy user text were turning to byte-level models. The paper adds two more: Bwen 8B, made from Qwen3 8B Base, and Blama 8B, made from Llama 3 8B.

Byte-level models see every letter that tokenizers hide

Most models divide text into tokens, which are words or word fragments. The model behind ChatGPT splits "LMU München" into three chunks, so it never sees the letters in München, LMU's release explains.

Byte-level models read the bytes computers store text in, roughly one per letter. Until now they've trailed token-based models, partly because catching up meant costly training from scratch, Ai2 says.

The retrofit keeps the trained model at the core. Small new layers group the bytes into patches, pass them through the original transformer and turn the output back into bytes. The Bolmo models started from Olmo 3 7B and OLMo 2 1B.

The converted models spell better but lose some ground on code

In the paper's tests, the byteified models beat every earlier byte-level model of similar size on average. Bolmo 7B scored 16.5 points higher on STEM tasks than Meta's BLT 7B. All three 7B and 8B models came close to the models they were built from.

They also clearly beat their source models on character-level tasks, helped by training data built to teach those skills. The paper's own control shows the cost. Against an Olmo 3 model trained on the same data, Bolmo 7B slipped slightly on maths and lost more on first-try code generation.

The paper is peer reviewed, but the scores come from the authors' own test suite, and we found no independent re-run yet.

When Bolmo first came out, an r/LocalLLaMA thread drew 24 comments. Read on October 11, about a fifth were excited and about a fifth questioned what bytes add beyond spelling. Several asked when it would run in llama.cpp.

It's not a new idea, and the main benefit of this paper is a cheaper training process which makes it easier to experiment with these.

A Reddit user in r/LocalLLaMA, December 2025

That's a snapshot of reactions to the first release, not a survey.

The models, code and training data are public, and Ai2's Nature blog post says the new Bwen and Blama checkpoints are on Hugging Face. Early-stage checkpoints are there too, for researchers who want to extend the method.