Expert-validated STEM QA

arXiv:2608.28591v1 introduces a high-quality, expert-validated STEM dataset (N=398) spanning Physics, Chemistry, Biology, and Mathematics. Developed by 241 domain experts, the dataset features a balanced taxonomy, rigorous multi-round reviews, and verifiable QA formats. Benchmark results show low frontier-model performance (<25%), with post-training on a private expanded version yielding a notable 15% relative gain on the STEM subset of a separate dataset. The authors have open-sourced a portion for the AI research community.

Source: cs.AI updates on arXiv.org. Summary: The paper presents a curated STEM QA dataset designed to address gaps in existing datasets, emphasizing expert validation, balanced taxonomy, and consensus-driven revisions.

Impact: The dataset serves as a challenging benchmark for AI models in STEM and demonstrates potential utility for model training, as evidenced by performance improvements after additional training on a private, larger dataset. A portion of the dataset has been open-sourced for researchers.

Source: cs.AI updates on arXiv.org — https://arxiv.org/abs/2608.28591