Artificial Analysis launches AA-Robotics v0.1: frontier models drive a real robot arm, GPT-6 Astra leads at 69, but tasks take ~30 minutes vs 20 seconds for a human

Why it matters

【Key points】

Independent evaluator Artificial Analysis released AA-Robotics v0.1 (research preview) on Oct 14. It tests whether frontier language models can directly drive a real 6-axis robot arm (5 joints plus gripper) from coding-agent harnesses (GPT in Codex, Claude in Claude Code, Gemini in Antigravity, all at high effort), via a simple HTTP API and four cameras, zero-shot, on eight tabletop tasks: press a button, place a toy, pour a drink, tidy blocks into a lidded box, shape matching, write "hello world", build the tallest tower, measure a box. A human judge scores the end state.

Scores: GPT-6 Astra 69.4, GPT-6.1 Sol 64.6, Gemini 4 Argon 55.3, Claude Opus 5.5 48.8, Claude Fable 5.1 46.3. Models take about 30 minutes per task (Argon 52) versus about 20 seconds for a human, nearly 100x slower. Sol is cheapest at $1.93 per task, Astra $16.36; about 70% of spend is input. No model fully solved box measurement (within 2 mm). Typical failures: missed grasps, wrong action order that topples their own towers, and misjudging completion (Sol built a passing tower at ~17 min, then kept refining and failed).

【Verifiable】

Launch article: https://artificialanalysis.ai/articles/aa-robotics

Official X post: https://x.com/ArtificialAnlys/status/2110500262628806787

【Our read】

1. This is the first public, like-for-like real-hardware comparison of general frontier models' physical skills: simple pick-and-place works, but speed is two orders of magnitude off.

2. The ranking differs from coding and knowledge leaderboards (GPT leads, Claude trails), suggesting embodied tasks reward visual grounding and self-checking more than raw reasoning.

3. Caveats: eight tasks, one arm, few attempts, research preview. Small score gaps should not be over-read; watch for task expansion and benchmark-specific tuning.