Apple and EPFL paper: with the same model and a 24-hour budget, a single-session coding agent with little more than a shell matches or beats four open-source ML-engineering harnesses, but on GLM 5.2 each run costs about 6x as much

Why it matters

【Key points】

Apple's ML research site lists a paper with EPFL (arXiv 2609.40303, submitted Sep 30) asking whether multi-agent orchestration, search trees and retrieval subagents still pay off now that models can write, run and debug code themselves. A minimal baseline, Malena, is a single long-running OpenCode session with file, shell, background-job and submission tools. With the same model, hardware and 24-hour budget it matched or beat AiScientist, Arbor, MLEvolve and ScienceFlow on all 17 harness-backbone pairs on 30 MLE-bench tasks and all 6 pairs on 40 NatureBench tasks, across GLM 5.2, Kimi K3, DeepSeek V4 and Gemma 4 31B. The biggest gain was moving from chat to a shell-and-filesystem environment; no search strategy added a significant gain beyond that. Pedro Domingos' Oct 8 post "Harnesses are useless" drew about 1,160 likes.

【Verifiable】

Apple: https://machinelearning.apple.com/research/harness-autonomous-ml-engineering

Paper: https://arxiv.org/html/2609.40303

X: https://x.com/pmddomingos/status/2108153590929891733

【Our read】

1. "Useless" overstates it. Malena is a thin harness; what proved redundant is outer orchestration from the single-completion era. Harness work is shifting from scheduling the model to giving it a good runtime.

2. Simple is not cheap. On GLM 5.2 a Malena run cost about $12.12, roughly 6.2x AiScientist's $1.95, because one session resends its growing context every turn. The complexity moved into the token bill.

3. Mind the scope: Kaggle-style tasks, single worker, contamination risk and wide intervals. On the smaller Gemma 4 31B, MLEvolve led on the mean, so weaker models still benefit from scaffolding.