Terminal-Bench-Science: Evaluating AI agents on scientific research workflows

 (terminal-bench-science.ai)

30 points | by matt_d 2 hours ago

5 comments

  • jerpint 6 minutes ago
    The fact that opus 5 is outperforming fable is odd to me

    From personal experience, opus 5 feels net inferior to fable on almost every aspect (for coding tasks)

  • akshay_akula 1 hour ago
    Evals on actual research workflows is the right direction, most agent benches are toy tasks.
    [-]
  • rubslopes 2 hours ago
    I'm glad to see that GPT Sol beats Opus at least in Mathematical Sciences, because that's my need right now, and I much prefer GPT's prose style.
  • mlmonkey 35 minutes ago
    Sad to see no mention of Gemini ...
  • vatsachak 1 hour ago
    Damn. These things aren't AGI... but I don't care.

    Luna is good enough for me to give a parser spec and have it write one.