Technology
OpenAI's chief scientist proposes shared AI development limits measured by perplexity
Coverage
2 reports · 1 in Arabic, 1 in English · over 2 hours
OpenAI chief scientist Jakub Pachocki has proposed that AI developers agree on shared limits to model development, measured by a benchmark called perplexity.
- The proposal: Pachocki proposed developers commit not to exceed agreed perplexity thresholds until their ability to monitor AI behavior reaches a set milestone.
- Model data: OpenAI published internal perplexity figures, citing about 0.99 for GPT-3.5, roughly 0.91 for GPT-4, and below 0.8 for current models.
- 2027 outlook: Pachocki said a wave of fundamental advances in pre-training methods could begin in 2027, with AI already contributing to its newest models' design.
- Caveats: Pachocki said differing training and test data across companies make direct comparison of published perplexity figures insufficient to judge model capability.

