Skip to main content

Task format reference

The current user-side Task directory must contain exactly UTF-8 runner.toml and its declared main.py or main.sh. Entrypoints cannot be symlinks; extra input files and paths outside the directory are rejected. Artifact declarations do not upload input files.

A Runner is an execution package; a Task in a Dataset is an evaluation problem. Directories read runner.toml by default; legacy task.toml / envhub.task/v1 two-file execution packages remain supported, but including both configurations is rejected. Native Harbor task.toml in Datasets keeps its original format.

Minimal Python task​

runner.toml:

schema_version = "envhub.runner/v1"

[runner]
name = "examples/hello"
version = "1"

[execution]
entrypoint = "main.py"
timeout_sec = 10
outputs = ["result.txt"]

main.py:

from pathlib import Path

Path("result.txt").write_text("hello\n", encoding="utf-8")
print("hello")
FieldExampleMeaning
schema_versionenvhub.runner/v1Runner schema
runner.name / runner.versionexamples/hello / 1Runner definition name and version, not trial_id
execution.entrypointmain.py / main.shMatching entrypoint in the directory
execution.commandbash main.shOptional shell script; defaults to bash main.sh or python main.py according to the entrypoint
execution.timeout_sec10Execution deadline in seconds; differs from SDK wait timeout
execution.outputs["result.txt"]Relative artifact paths collected after execution

This page shows the current minimal runnable format; the platform validates specific fields. CLI --timeout-sec / SDK execution={"timeout_sec": ...} can be passed as execution overrides.

Shell task​

Change entrypoint to main.sh and remove main.py from the directory:

#!/usr/bin/env bash
set -eu
printf 'hello\n' > result.txt
printf 'hello\n'

The execution environment provides local Python / Bash interpreters; installing the client does not automatically create a cloud runtime.

Command override​

For normal submissions, omit command / --command. The platform derives the command from runner.toml: main.sh runs as bash main.sh, and main.py as python main.py. A declared execution.command takes precedence over this inference; an explicit submission command overrides it. The client does not insert a default string, so each Runner can use its own entrypoint.

el task submit ./my-task --wait

Task / Job submission command (CLI --command) overrides the Runner's default script and runs via bash -c in the Runner working directory. Text preserves multiple lines, quotes, and shell variables; it must be a nonempty UTF-8 string without NUL and at most 65536 bytes. Whitespace-only scripts are also rejected.

el task submit ./my-task --command 'printf "custom startup\n"; python main.py' --wait

In Python, use client.tasks.submit(files, command="printf custom; python main.py") or client.jobs.submit(dataset, runner="envhub/harbor@0.23.0", command="printf custom; bash main.sh"). Job overrides apply to all members. The API supports the compatibility spelling Command; including both spellings in one request is rejected. Freezing and idempotency use normalized command; changing the script on retry returns a conflict.

Profile input​

Tasks read the Profile JSON frozen on submission from ENVLOOP_TASK_PROFILE, defaulting to {}. The entrypoint validates and interprets specific parameters; Profiles contain no platform credentials.

Returned result​

tasks.result(trial_id) returns a dictionary with stdout, stderr, and outputs. Each output includes the filename, text content, byte size, and SHA-256 of raw bytes. Text content cannot be guaranteed to reconstruct arbitrary binary files without loss.

Task completion status and evaluation reward are separate; this example has no benchmark verifier and does not produce a verified reward=1.