# AssistantEval published data

AssistantEval publishes results and recorded evidence at [assistanteval.com](https://assistanteval.com/). These downloads are snapshots of completed evaluations. Read the [methodology](https://assistanteval.com/method.md) for the grading definitions.

## Find the results

- [Leaderboard CSV](https://assistanteval.com/data/results.csv): one row per assistant, with task success, behavior measures and recorded timing.
- [Results index JSON](https://assistanteval.com/data/index.json): `assistants` contains aggregated results; `tasks` identifies the scenarios; `runs` lists each public run ID, assistant ID, task ID and outcome. `research` contains the repeated-trial and paired-scenario aggregates.
- Use a run's `id` to open `https://assistanteval.com/runs/<id>` or retrieve `https://assistanteval.com/data/runs/<id>.json`.
- Each run JSON includes its task, `turns`, service `requests`, `state_changes` and `verdict`. The verdict contains the individual grading checks. Compare these records when inspecting why a run passed or failed.

The index's `generated_at_ms` is the publication update time. Individual run timestamps describe the original recordings. A grade update does not represent a new assistant conversation.

## Inspect runs with Harbor

Download [available trajectories](https://assistanteval.com/data/harbor.zip), or retrieve one bundle at `https://assistanteval.com/data/harbor/<id>.zip` using an ID from the results index.

With [Harbor](https://harborframework.com/) installed, extract a bundle into an empty directory and open its recorded jobs:

```sh
unzip harbor.zip -d assistanteval-runs
harbor view assistanteval-runs --jobs
```

The bundles contain conversations, observed service requests and responses, grading evidence and recorded state. They support offline inspection; they do not include executable tasks or resettable service environments. Harbor's job timestamps describe export creation.

## Inspect a task and its starting environment

The worked tile-order example publishes its [task definition](https://assistanteval.com/data/samples/tile-order-task.json) and [synthetic starting state](https://assistanteval.com/data/samples/tile-order-environment.json). Run records include their own task context and setup.

## Data usage

The leaderboard and CSV are licensed [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), with credit to AssistantEval (assistanteval.com).

Run records and tasks may be read, verified, quoted and cited. Training, fine-tuning, evaluating AI models or redistributing them as a dataset requires written permission. See the [data terms](https://assistanteval.com/terms) and contact [hello@assistanteval.com](mailto:hello@assistanteval.com) for licensing.

Scored runs with `trajectory_available: false` remain included in aggregate results. Their JSON contains summary metrics only; no conversation, grading evidence or Harbor bundle is available. The index's `trajectory_counts` distinguishes scored runs from downloadable trajectories. These unavailable records cannot be replayed from this release.