Gemini 4 Neon Claims Top Spot on Models Engineering Index

Each model’s score on the Models Engineering Index is a composite index across ten coding benchmarks, adjusted for benchmark difficulty and missing results, resulting in an average score of 50. The index also considers cost and token use, both adjusted to reflect relative performance against the typical model (1×).

The latest rankings show that Gemini 4 Neon has secured the top position. Key competitors include AstraOpus 5.54, CarbonSonnet 5.54, and Hydrogen 6.2, among others. The performance scores of these models range from 4 to 6.2, with some variations in cost and token use.

Notably, the index is far from perfect. Models are tested on different subsets of benchmarks, leading to varying degrees of evidence for each score. There are no error bars, making small gaps in performance less meaningful. Scores are relative to the models listed, so changes in the model roster can shift rankings. Benchmarks are weighted equally, regardless of their size or reliability.

Cost in the index refers to the reported cost of each benchmark run, reflecting prices at the time and setup of each benchmark, not necessarily the model’s cost in a real-world workload scenario. Results are reproduced as published, without independent verification, and may be affected by contamination or tuning to the benchmarks. These benchmarks serve as a proxy for real engineering work.

Data from Cognition, Cursor, the Laude Institute, Proximal, Scale Labs, and Vals AI are the source for the index. The full dataset is available on GitHub.

While the Models Engineering Index provides a valuable reference for AI model evaluation, it is important to consider its limitations and the ongoing efforts to improve the index.

Source: https://git-aarvin.github.io/mei/