Research and leaderboards
EnvLoop has two different comparison surfaces. The Console's Project leaderboard compares selected Jobs inside your project. The public Leaderboard presents published benchmark results with their protocols and evidence. Adding a Job to a project board does not publish it publicly.
Compare your own Jobs
Read saved results, ranks, scores and coverage in Jobs → Metrics → Project leaderboard. The Console provides no action to add results.
Use the Job link to inspect the original Trials and artifacts before interpreting a difference. Matching Dataset and Runner names alone are insufficient: fixed revisions, task digests and attempt count matter. Missing rewards remain visible, including on failed Jobs that produced some scores. See Evaluation for metric definitions and submission constraints.
There is no automatic synchronization between this project board and a public benchmark. Project membership and access remain necessary to read project results.
Browse public research
Open Research from the website. Each study links its available paper, repository, Dataset or model resources. A detail page provides:
- Abstract and a study figure to explain the question and setup.
- Reported results, where provided, with the metric and model names.
- What the study covers, describing the experiment scope.
- What remains open, identifying limits of the evidence.
- Cite this work, including a Copy BibTeX button.
Read the paper and its limitations before applying a number to a different model, environment or dataset split. Reported research results describe that study; they do not certify a new Job you run yourself. If copying a citation fails, select the visible BibTeX manually. External resource links may open another site or tab.
Read a public benchmark board
- Open Leaderboard and choose a benchmark.
- Read the benchmark facts, primary metric and resource links.
- Inspect Ranking and any dimension breakdown. Column arrows indicate whether higher or lower is better.
- Read How it is scored and Environment before comparing models.
- Use the linked research and released evaluation resources to inspect the underlying protocol.
Each benchmark has its own primary metric, units, task distribution and verifier. Scores from different benchmark boards are not directly comparable. A rank is a position among the published rows, not a statement about every model or every possible agent setting. Check the displayed configuration and limitations when reusing a reported number.
Explore cu-bench task results and replays
The public cu-bench board includes Per-task results and replays. Its curated Example replays show recorded desktop trajectories; choose a card, play or step through frames, adjust speed, and click a frame to zoom. These examples illustrate selected runs, not every task in the ranking.
Switch to Results matrix to filter by Environment, Tier, Dimension, Model and Track. Only disagreements selects pass/fail differences between visible models within a track. Paired set only, when present, starts enabled and selects the published paired comparisons used by the headline ranking; turning it off explores additional reported runs. The summary switches between environment, tier and difficulty dimension and recalculates for the visible filters. It includes pass/fail cells only; unrun, pending and invalid cells are excluded.
Hover or focus a cell for available composite, steps/budget, failure cause and judge attribution. A cell may open a curated replay, a Console Trial that requires sign-in/access, or only its inspector when no linked evidence is available. Auxiliary composite and judge scores do not replace the strict verifier result. Keep the paired-set and filter settings with any quoted subset statistic.
If the result bundle cannot load, use Retry. A frame that remains at Loading frame… or a missing caption is unavailable evidence, not a newly computed failure. The public board reads frozen published assets; browsing or replaying them does not submit a new evaluation or disclose private project artifacts.
Submit a model for public consideration
The public site's Submit a model action opens the listed research contact by email. The current website does not provide a self-service upload form, a public submission API, or an automatic “publish this Job” action. Submitting is a request for evaluation/review rather than immediate inclusion.
Follow the selected board's Add a model instructions. Depending on its protocol, provide frozen predictions or a model that can be evaluated with the released verifier. A useful submission package identifies:
- The benchmark, exact revision/split and evaluation settings.
- Model/checkpoint version and the code or prediction format needed to reproduce it.
- Runner/agent configuration, attempts and budget/timeout settings.
- Score coverage, failed/missing cases and relevant result artifacts.
- Whether the result is independently verified, self-reported or awaiting review.
These are preparation recommendations; the board's current protocol and contact instructions determine what is accepted. Do not send credentials, signed download URLs or private project data. Use a shareable, appropriately licensed artifact and arrange access with the review contact when necessary.
Common questions
Why is my Console Job absent from the public board? Project submissions remain scoped to your project. Public inclusion uses the separate benchmark review workflow.
Why do two Jobs not appear on the same project board? Inspect their fixed Dataset/Runner references, Task digests and requested attempt counts. They must describe the same comparison protocol.
Can I claim improvement from a higher score? First check matching inputs and budgets, score coverage, repeatability and the benchmark's reporting rules. A higher partial score or a changed test set is insufficient.
Where should I start to reproduce a result? Open the study's paper and resource links, then use Datasets, Profiles, Jobs and Trials to preserve the inputs and evidence for your own evaluation.