Modernizing COBOL or Fortran with Codex? OpenAI's new evals show GPT-6.1 Sol jumping from 5% to 87% on COBOL migration

#What changed

On October 19, OpenAI posted on X and published a research note on two new evaluations for "low-resource" programming languages, meaning languages with far less public code and documentation for models to learn from than Python or JavaScript.

• MultiLangBench asks a model to build a SQLite-compatible database engine from scratch in 20 languages, graded with tests adapted from SQLite's own suite. At maximum reasoning effort, GPT-6 Astra averaged an 87.4% test pass rate, up from 26.8% for GPT-5.6 Sol, and improved in every language. COBOL is still the hardest at about 72%. GPT-6 Luna reached 40.3% at an estimated $0.77 per sample versus $3.48 for GPT-5.6 Sol, about 78% cheaper.
• A COBOL migration eval asks the model to port COBOL programs to C++, Java, Kotlin, Python, or Rust, and only counts a migration as passing when hidden tests match the original outputs byte for byte. GPT-6.1 Sol scored 86.6% at low reasoning effort, compared with 5.4% for GPT-5.6 Sol at the same setting, while costing 39% less. At higher effort it reached 97.9%.

OpenAI says it is releasing the MultiLangBench setup and test suite on GitHub with per-language results, and it is asking teams which languages and workflows it should work on next.

#Who is affected

Teams that maintain or modernize COBOL, Fortran, Visual Basic, or OCaml code, such as banks, insurers, government agencies, and scientific and engineering groups. It also matters if you tried Codex on these languages before and gave up because older models wrote code that looked right but behaved differently. OpenAI points developers to the API to try the latest models; in Codex, it applies wherever GPT-6.1 Sol, GPT-6 Astra, or GPT-6 Luna are available on your plan.

#What to do now

• If an older model failed you on legacy code, rerun one real task with GPT-6.1 Sol. The big migration jump was measured at low reasoning effort, so start there and raise the effort only on the programs that still fail.
• Before asking Codex to port anything, pin down how the current system behaves. Capture real inputs and outputs from production runs and turn them into tests the new version has to match exactly. That is the same bar OpenAI used to grade its own migration eval.
• Spell out COBOL-specific behavior in the repo's AGENTS.md. OpenAI's example is fixed-point arithmetic: COBOL drops extra decimal digits unless rounding is explicitly requested, so a rewrite that rounds instead can run cleanly and still change financial results. Rules for rounding and missing-record handling belong there too.
• Migrate one program at a time and have someone who knows the business rules review each diff before it ships.
• Check the per-language MultiLangBench results for the language you actually use before planning a large project.

#What is not confirmed

These are OpenAI's own benchmarks, and DotsFeed has not run them. Passing an eval is not the same as a safe production migration; OpenAI itself says production use still needs expert review and testing against the system's business rules and dependencies. The cost figures are estimated API costs, not how much of a ChatGPT or Codex subscription allowance a migration will use. The post does not give GPT-6.1 Sol's MultiLangBench score.

#Sources

• OpenAI on X: https://x.com/OpenAI/status/2112310345863930168
• OpenAI, Advancing coding in low-resource programming languages: https://openai.com/index/low-resource-programming-languages/