DataMining Get a demo →

Datasets · Delivered

DeliveredDatasets

This page lists the delivered datasets we can make public: 10 in total, covering academic reasoning, software engineering, and agent evaluation. Each dataset entry states its delivery scale, QA mechanism, and data fields. Samples and full documentation are available on request.

Academic Reasoning

Expert-level academic problems. Questions are originally written by a competition coach team, not rewrites of public question banks.

L3 frontierMathMultimodalEnglish

HLE-Math

HLE (Humanity's Last Exam) was built by the Center for AI Safety and Scale AI with domain experts from around the world. It is a high-difficulty multimodal academic question-answering benchmark; answers must be clearly verifiable and hard to obtain through simple retrieval.

Our deliveryNewly authored by the competition coach team in the style of HLE-Math, strictly multimodal, produced with an automated pipeline.

Delivery scale
2,000 problems, scalable on request
QA mechanism
Formal solver verification (Z3 / SymPy), cross-validation across multiple models, double-blind expert review
Stock status
Available now
Data fields
Problem, answer, worked solution, options, problem image, option images, problem ID, Pass@4 evaluation result, and four-round model dialogue with verdict
Original benchmark: the original HLE benchmark ↗
L3 frontierPhysicsMultimodalEnglish

HLE-Physics

HLE (Humanity's Last Exam) was built by the Center for AI Safety and Scale AI with domain experts from around the world. It is a high-difficulty multimodal academic question-answering benchmark; answers must be clearly verifiable and hard to obtain through simple retrieval.

Our deliveryNewly authored by the competition coach team in the style of HLE-Physics, strictly multimodal, produced with an automated pipeline.

Delivery scale
2,000 problems, scalable on request
QA mechanism
Cross-validation across multiple models, double-blind expert review
Stock status
Available now
Data fields
Problem, answer, worked solution, options, problem image, option images, problem ID, Pass@4 evaluation result, and four-round model dialogue with verdict
Original benchmark: the original HLE benchmark ↗
L3 frontierChemistryMultimodalEnglish

HLE-Chemistry

HLE (Humanity's Last Exam) was built by the Center for AI Safety and Scale AI with domain experts from around the world. It is a high-difficulty multimodal academic question-answering benchmark; answers must be clearly verifiable and hard to obtain through simple retrieval.

Our deliveryChemistry problems newly authored by the competition coach team in the HLE style, strictly multimodal, produced with an automated pipeline.

Delivery scale
1,000 problems, scalable on request
QA mechanism
Cross-validation across multiple models, double-blind expert review
Stock status
Available now
Data fields
Problem, answer, worked solution, options, problem image, option images, problem ID, Pass@4 evaluation result, and four-round model dialogue with verdict
Original benchmark: the original HLE benchmark ↗

Software & Engineering

Long-horizon software engineering and engineering optimization tasks, each with an executable verification environment.

L1 basicEngineering OptimizationCodeEnglish

Frontier-Eng

A generative optimization benchmark for real-world engineering tasks, with 47 tasks across five areas: computation and quantum information, operations research and decision science, robotics and control, optics and communication systems, and physical science and engineering design. Within a fixed interaction budget, the model repeatedly generates or modifies executable code, and domain validators and simulators return performance scores and constraint feedback.

Our deliveryConsistent with the original benchmark; delivers executable task packages and manifest files.

Delivery scale
Same as the original benchmark
QA mechanism
Unit tests run successfully
Stock status
Available now
Data fields
Task packages (zip) and dataset manifest (csv)
Original benchmark: Frontier-Engineering ↗
L1 basicSoftware EngineeringCodeEnglish

DeepSWE

A long-horizon software engineering benchmark built by Datacurve, with 113 original tasks across 91 active open-source repositories and five programming languages. Tasks require the model to understand the codebase on its own, design a solution, complete the changes, and run tests; each task has an isolated execution environment, a functional verifier, and a reference solution.

Our deliveryConsistent with the original benchmark; delivers F2P / P2P test sets.

Delivery scale
Delivery capacity: 200 tasks / week
QA mechanism
Cross-validation across multiple models, double-blind expert review
Stock status
Available now
Data fields
F2P and P2P test sets
Original benchmark: DeepSWE (Datacurve) ↗
L1 basicSoftware EngineeringCodeEnglish

FrontierSWE

An ultra-long-horizon coding agent benchmark built by Proximal; v2 has 34 tasks across five categories: system implementation, scientific computing, performance optimization, visual reasoning, and AI research. Each task provides a full Linux environment and a 20-hour execution budget, and a deterministic automated validator evaluates the final artifact.

Our deliveryConsistent with the original benchmark; delivers F2P / P2P test sets.

Delivery scale
Delivery capacity: 200 tasks / week
QA mechanism
Cross-validation across multiple models, double-blind expert review
Stock status
Available now
Data fields
F2P and P2P test sets
Original benchmark: frontier-swe-v2 (Proximal Labs) ↗
L2 advancedProfessional TasksCodeMultimodalEnglish

Agents' Last Exam（ALE）

An agent evaluation benchmark led by Berkeley RDI together with more than 300 industry experts, targeting long-cycle, high-value, verifiable-outcome tasks. It covers 55 target industries, has collected more than 1,500 tasks, and aims for 5,000.

Our deliveryProduced by experts using real, licensed professional cases, with executable reference solutions and independent scorers. Delivered in Harbor Task format; each task has its own bootable Docker environment.

Delivery scale
Delivery capacity: 200 tasks / week
QA mechanism
Unit tests run successfully, cross-validation across multiple models
Stock status
Quickly reproducible (configurable)
Data fields
Harbor Task format task packages, with review oracles and standalone Docker environments
Original benchmark: agents-last-exam (Berkeley RDI) ↗

Agents & Office Automation

Multi-app, long-horizon agent evaluations, scored by final state.

L1 basicAgentLong-horizon reasoningEnglish

AutomationBench

The public official set has 600 tasks, 100 per domain, plus 200 simple basic tasks not counted toward scores, covering 47 simulated SaaS applications. Each task initializes a simulated enterprise environment, requires the agent to complete business workflows on its own, and checks the final state with programmatic assertions.

Our deliveryConsistent with the original benchmark.

Delivery scale
Delivery capacity: 200 tasks / week
QA mechanism
Double-blind expert review
Stock status
Quickly reproducible (configurable)
Data fields
Task packages and demo samples
Original benchmark: AutomationBench (Zapier) ↗
L1 basicAgentLong-horizon reasoningEnglish

Toolathlon（The Tool Decathlon）

108 tasks, 32 software applications, and 604 tools, with an average of about 20 interaction rounds. The official release emphasizes diverse applications, realistic initial environment states, and long-horizon workflows; each task comes with a script that evaluates the execution result.

Our deliveryConsistent with the original benchmark.

Delivery scale
Delivery capacity: 200 tasks / week
QA mechanism
Double-blind expert review
Stock status
Quickly reproducible (configurable)
Data fields
Task packages and demo samples
Original benchmark: arXiv:2510.25726 ↗
L2 advancedLegalAgentLong-horizon reasoningEnglish

Harvey LAB（Legal Agent Benchmark）

The launch announcement describes 1,250 tasks across 24 legal practice areas, with more than 75,000 expert-written grading criteria. Each task includes work instructions, client matter materials, and required deliverables, simulating a law firm partner assigning work to a lawyer who processes the materials and submits the result.

Our deliveryConsistent with the original benchmark.

Delivery scale
Delivery capacity: 200 tasks / week
QA mechanism
Double-blind expert review
Stock status
Quickly reproducible (configurable)
Data fields
Task packages and demo samples (hard subset, 5 tasks)
Original benchmark: Harvey launch announcement ↗

Want to see samples from a specific dataset?

Tell us your model focus and evaluation goals, and we will provide demos and field documentation on request.

Contact us →