Model Loops
Model Loops shows the improvement loops (RSI loops) that were started in your project and how far each has come. Open RSI LOOPS → Model in the Console sidebar. The RSI LOOPS group sits between Overview and Experiment. The pages are read-only: opening them does not start, steer or stop a loop and does not allocate training resources. To start or steer a loop, use the API, MCP or CLI described below.
Improvement targets in RSI LOOPS
The navigation uses the short child label Model, while the page title remains Model Loops. Model Loops track improvements to model capabilities and training methods: compare baseline and candidate versions using evidence from a fixed evaluation protocol. The page shows versions, scores, gates and audits actually reported by the current model research service. Missing evaluation is not a zero score, and successful training does not prove improvement.
Future navigation children may be Infra (kernels and training or inference system efficiency), Harness (agent prompts, tools, memory and orchestration) and Data (data methods and versions). Child labels do not repeat the parent’s “Loops”. These pages are not available yet. Classify a loop by its primary improvement target; a model loop may generate data as part of its work.
The canonical project page is /rsi/model-loops; old /agent-runs page addresses redirect. API/MCP paths and metadata fields below remain compatible. CLI el job run list/get aggregates tagged training work; it is distinct from the Console model research service projection.
Open the Console section
- Select the workspace and project where the loop runs.
- Open RSI LOOPS → Model in the sidebar.
- Review each run's state, round out of its target, best version, spend and last update, newest first. Use Refresh for a current view, and All columns to see every column on a narrow screen.
- Open a run name for its details.
A run appears in this list when either a training Job in the selected project carries the run's rsi_run label, or the run was started by a caller in your workspace and project. The second rule means a freshly started run, for example one started through MCP, is listed before its first training Job exists. Runs started before owners were recorded appear through their tagged training Jobs only.
If the loop service is not connected to the deployment, the page says so. If it cannot be reached, the page says that agent runs are unavailable and offers Retry. Neither case means that there are no runs, zero spend or a stopped loop.
Read a run
The run page has two parts.
- Loop progress is the live view: state, current round out of its target, phase, the role and tool in use, a feed of recent activity (newest first), the steering messages sent so far with their delivery status, the jobs the loop started, and spend against any cap. It refreshes about every three seconds while the tab is visible and stops once the loop has finished, failed or stopped.
- Loop dashboard is what the loop reports about its own results:
- Overview: reported state, current and target round, best version, spend and cap, GPU spend, machine, and the steps of the classic example (data, checkpoint, benchmark, beats base) with their detail.
- Scores by version: trained rows and evaluation suites/splits, with mean, interval and sample size when supplied.
- Final gate: candidate score and change relative to the baseline for a reported gate.
- Recent audits: audit type, version, approval or rejection and first reason.
- Recent rounds: decision and summary; Details opens that round's plan and interventions.
- Owner directives: guidance already submitted to the run, shown as text.
Values shown as — were not provided. A "Best" label, gate or audit is evidence reported by the loop; inspect its evaluation coverage and protocol before making an improvement claim. Training rounds, loop spend and training estimates may have different reporting scopes.
A training Job created by an RSI loop also has a Loop progress tab in Experiment → Jobs, with a link to the Agent Run page.
Inspect tagged training runs from the CLI
el job run list
el job run get RUN_NAME
el job run get RUN_NAME --json
Replace RUN_NAME with the exact run label returned by the list. These commands group visible training Jobs and show rounds, their Jobs and recorded checkpoints. The default output is a table; --json preserves the full response. There are no --cursor or --limit flags on this complete-list command, and it does not create or stop a loop. Loop-specific scores, audits, directives and live progress come from the separate RSI API.
The same CLI group also starts and follows loops, calling that RSI API (see Start a loop):
el job run start arith-carry-1 --brief "Improve multi-digit carries." --model-usd-cap 5 --dry-run
el job run live arith-carry-1 --follow
el job run steer arith-carry-1 "Prefer held-out families."
el job run stop arith-carry-1
start takes the fields of rsi_start_run as options (--brief or --brief-file, --model-usd-cap, --usd-cap, --domain, --rounds, --dry-run, --student-model). live prints the live view as JSON, or with --follow one seq kind: text line per new event until the run ends; Ctrl-C leaves the run running. stop needs training.stop as well. A refused command prints a fixed explanation and the error code, not the service's message; the API shows that message.
Start a loop
A loop is started with the API or MCP, never from the Console. Starting a loop spends money: model spend up to the cap you set, and GPU training on EnvLoop for a live run. Start one only when you mean to.
You need a project API key that holds the training.run capability (the Project automation preset has it, and training.stop), or a signed-in Developer or Admin session. Read-only keys and Viewers receive 403 ACCESS_DENIED.
From a coding agent (MCP)
Connect Claude Code or Codex to the RSI MCP endpoint with your API key, then ask it to start and follow a run.
# Claude Code
claude mcp add --transport http envloop-rsi https://api.envloop.ai/v1/agent-runs/rsi/mcp \
--header "Authorization: Bearer $ENVLOOP_API_KEY"
# Codex: ~/.codex/config.toml (the key stays in the environment)
[mcp_servers.envloop-rsi]
url = "https://api.envloop.ai/v1/agent-runs/rsi/mcp"
bearer_token_env_var = "ENVLOOP_API_KEY"
Any MCP client that supports Streamable HTTP and a bearer header works the same way. The server is stateless and answers JSON, and tools/list returns every tool with its exact input schema. The tools:
| Tool | What it does | Needs |
|---|---|---|
rsi_list_runs | The runs of your project: state, round, target, best version, spend, owner. | read |
rsi_run_status | One run in detail: summary line, steps of the classic example, versions with scores, audits, rounds, spend. compact: true returns only the summary fields (cheap to poll). | read |
rsi_status | What a run does right now, as in the live view below. since is the previous next_cursor. | read |
rsi_round | One round: decision, plan, interventions, audits, summary (round is 1 or higher). | read |
rsi_start_run | Starts a new run (see below). | training.run |
rsi_add_directive | Appends a standing instruction (1 to 4000 characters) to the run's owner directives. | training.run |
rsi_steer | Messages the running operator now (1 to 4000 characters). | training.run |
rsi_control | extend (set the target rounds), stop (after the current round) or resume. extend and resume start paid work. | training.run; stop needs training.stop |
rsi_start_run takes:
| Field | Meaning |
|---|---|
run (required) | A new run id: 1 to 64 lower-case letters, digits and dashes, for example arith-carry-1. An id that exists returns a conflict. |
brief (required) | The goal, up to 4000 characters, added to every round of the run. |
limits.model_usd_cap (required) | The spend cap in USD of the strong-model API: the hard stop. |
limits.usd_cap | Cap in USD for model plus GPU training together. Set it to bound a live run's whole cost. |
limits.model_request_cap | Cap on model requests. |
other limits keys | Optional positive-number tuning keys (counts are integers): request_cap, student_request_cap, diag_per_family, dev_n, collect_tasks, collect_k, admission_n, blind_n, max_rows_per_family, dev_use_cap, eval_limit, calib_n. There is no separate GPU cap: GPU spend is usd_cap minus model spend. An unknown key is rejected. |
domain | The task family. One of arith, code, spatial or finance; default arith. |
rounds | Target rounds, 1 to 1000; default 3. A very high target is effectively unbounded: the spend caps then stop the run. |
dry_run | true starts a GPU-free run: nothing is trained and the student is answered through the model gateway, so it exercises the loop and does not improve a model. It still spends model budget. |
student_model | The gateway student of a dry run; only with dry_run: true. |
The finance domain asks the student to pick an option ID and grades it with judge finance-option-v2. An answer counts only as a bare option ID, or as reasoning whose last non-empty line is exactly one marker ANSWER: <ID>. A hedge, a second option, two markers or a marker that is not last names no option and scores 0.
A start returns state: "starting", the run's mode (live or dry), its owner and the caps it will enforce. The run appears in the Console list at once. Follow it with rsi_status or the Console page, and stop it with rsi_control.
From a script (REST)
The same operations are plain HTTP routes under https://api.envloop.ai/v1/agent-runs/rsi:
curl -X POST -H "Authorization: Bearer $ENVLOOP_API_KEY" -H "Content-Type: application/json" \
-d '{"run": "arith-carry-1", "brief": "Improve multi-digit carries.", "limits": {"model_usd_cap": 5}, "dry_run": true}' \
"https://api.envloop.ai/v1/agent-runs/rsi/runs"
curl -X POST -H "Authorization: Bearer $ENVLOOP_API_KEY" -H "Content-Type: application/json" \
-d '{"action": "stop"}' "https://api.envloop.ai/v1/agent-runs/rsi/runs/arith-carry-1/control"
POST /runs returns 201 and takes the fields of rsi_start_run. POST /runs/RUN_NAME/control takes action (extend, stop or resume) and, for extend, rounds (1 to 1000). A rejected request returns 422 INVALID_RSI_REQUEST, or 409 RSI_CONFLICT when the run id exists, the run is in the wrong state or no machine is free, and the message says which field or condition. GET /runs returns an empty items list with a start hint when your project has no runs.
The el CLI wraps these routes: el job run start, live, steer and stop (see the CLI section).
Steer a running loop with the API or MCP
The Console shows the loop but cannot send steering text, pause or stop a loop; the Console is read-only for every role. Use the API, the RSI MCP endpoint or the CLI for those actions.
The live view is available through the API with an EnvLoop API key that can read training Jobs in the project that owns the run:
curl -H "Authorization: Bearer $ENVLOOP_API_KEY" \
"https://api.envloop.ai/v1/agent-runs/rsi/runs/RUN_NAME/live"
curl -H "Authorization: Bearer $ENVLOOP_API_KEY" \
"https://api.envloop.ai/v1/agent-runs/rsi/runs/RUN_NAME/live?since=43"
The response contains state, round, rounds_target, phase, current (role, tool and start time), events, next_cursor, steers, jobs, spend and updated_at. events are ordered oldest to newest, at most 200 per response, each with a seq, a kind (message, tool_call, tool_result, steer, round, job or error) and a short summary text. Pass the previous next_cursor as since to receive only newer events; since must be a non-negative integer. An unknown run, or one that is not visible to your project, returns 404 RSI_RUN_NOT_FOUND.
A run is visible to you when either of these is true: a training Job in your project carries the run's rsi_run label, or the run was started by a caller in your workspace and project (the RSI service records that owner when the run starts). The second rule makes a freshly started run, for example one started through MCP, visible before its first training Job exists. Runs started before owners were recorded are visible through their tagged training Jobs only. The same rule applies to the run list, detail, rounds, live progress, steering and directives; steering and directives still need the permissions described below.
To steer a loop that is starting, running or paused, send text of 1 to 4000 characters. This needs the Developer or Admin role in a signed-in session, or an API key with the training.run capability; read-only keys and Viewers receive 403 ACCESS_DENIED.
curl -X POST -H "Authorization: Bearer $ENVLOOP_API_KEY" -H "Content-Type: application/json" \
-d '{"text": "Prefer held-out families when choosing data."}' \
"https://api.envloop.ai/v1/agent-runs/rsi/runs/RUN_NAME/steer"
The request returns 202 with the steering message's id, status (queued) and time. The loop delivers queued messages at its next turn boundary, so the message appears in steers first as queued and later as delivered, or as dropped if it could not be delivered. The author is recorded from your authenticated identity, not from the request body. A run that is not active returns 409 RSI_RUN_NOT_ACTIVE, and empty or oversized text returns 422 INVALID_STEER.
External agents use the same operations through the RSI MCP endpoint (see Start a loop): the rsi_status tool (run and optional since) and the rsi_steer tool (run and text). Steering through MCP follows the same permission rules.
How Jobs join a run
The training Job's user_metadata supplies the grouping information. Generic agent_* fields take precedence over corresponding rsi_* fields when both are present.
A Job uses its session user_metadata for grouping; if it has no session metadata and exactly one training model, that model’s metadata is used. Per-model metadata remains available in the Job’s training-run detail. Recover a specific Job from a trainer by matching trainer.model_id against training.model_ids in the paginated client.jobs.list(type="train") response; the run label is a grouping tag, not a Job ID.
{
"agent_run": "example-evaluation-loop",
"agent_round": "2",
"agent_purpose": "train candidate",
"agent_method": "sft",
"agent_checkpoint": "candidate-2",
"agent": "example-agent"
}
This is a metadata example, not a standalone submission. Supply it through the training workflow supported by your client; see Training. agent_run / rsi_run names the group, agent_round / rsi_round names the round, and the purpose, method and checkpoint keys label the corresponding training. Metadata labels do not independently verify that a training step, checkpoint or evaluation succeeded. A managed Qwen/Qwen3.8-27B Job takes these keys as its labels field (options) and returns them as training.labels and training.user_metadata, which is where the run lists read rsi_run and rsi_round.
Inspect individual resources
Use el job get JOB_ID or the Console Jobs page for an individual training Job. Inspect Checkpoints for saved weights and Inference for serving. Metadata labels and successful training status do not prove an evaluation improvement; check the recorded evaluation evidence.