OpenAI open-sources MultiLangBench and says GPT-6 Astra now passes 87.4% of SQLite-from-scratch tests across 20 languages, with GPT-6.1 Sol reaching 97.9% on COBOL migration

Why it matters

【Key points】

On Oct 19 (posted by @OpenAI at 22:30 UTC), OpenAI said its latest models are much better at writing and migrating code in "low-resource" languages such as COBOL, Fortran, Visual Basic and OCaml, and open-sourced a benchmark called MultiLangBench. These languages have little public code to learn from but still run banks and governments; OpenAI says COBOL supports 95% of ATM transactions.

MultiLangBench has a model build a SQLite-compatible database engine from scratch in one of 20 languages, checked against real SQLite. At maximum reasoning effort GPT-6 Astra averages an 87.4% test pass rate, versus 26.8% for GPT-5.6 Sol. COBOL rises from 10.9% to 72.1% (still the hardest), Fortran 21.4% to 83.5%, Visual Basic 27.5% to 89.4%. The cheaper GPT-6 Luna averages 40.3% at an estimated $0.77 per sample, against $3.48 for GPT-5.6 Sol.

A second eval migrates COBOL programs to C++, Java, Kotlin, Python or Rust; hidden tests require byte-identical output. GPT-6.1 Sol scores 86.6% at low reasoning effort (GPT-5.6 Sol: 5.4%) at about 61% of the cost, and 97.9% at higher effort, close to saturation.

【Verifiable】

OpenAI post: https://openai.com/index/low-resource-programming-languages/

MultiLangBench repository (Apache 2.0): https://github.com/openai/MultiLangBench

OpenAI on X: https://x.com/OpenAI/status/2112310345863930168

【Our read】

1. Legacy-code modernization is a coding job enterprises pay well for; the race is moving from popular languages to old systems in banks and governments.

2. Both evaluations were built and run by OpenAI and compare only against its own older models, with no other vendors' results. Only MultiLangBench is public; the COBOL migration eval is not, so outsiders cannot check it yet.

3. With MultiLangBench open, others can run the same tests. The repo converts 24k of SQLite's 51k tests and recommends running just 272 routinely, so check coverage before comparing scores.