Skip to content

feat(cli): simplify eval output surface#1336

Merged
christso merged 1 commit into
mainfrom
feat/av-eval-output-config-surface
Jun 9, 2026
Merged

feat(cli): simplify eval output surface#1336
christso merged 1 commit into
mainfrom
feat/av-eval-output-config-surface

Conversation

@christso

@christso christso commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Simplifies the agentv eval output surface around one canonical run directory:

  • --output <dir> / config output.dir writes the canonical run directory, always including <dir>/index.jsonl plus run/test artifacts.
  • --export <file> writes optional flat copies such as JSONL, JUnit XML, YAML, or HTML without changing the canonical run directory.
  • Removed --out, --artifacts, --output-format, and config output.format with migration errors instead of silent reinterpretation.
  • Preserves -o as the short alias for --output <dir> and rejects file-like --output results.xml with guidance to use --export results.xml.
  • Keeps feat(cli): rerun captured task bundles #1335 rerun-captured-task-bundle behavior by preserving source metadata propagation during the post-feat(cli): rerun captured task bundles #1335 rebase.

Migration Note

Breaking prerelease cleanup: callers using the old flat/result flags should migrate as follows:

  • --out results.jsonl -> --output <run-dir> --export results.jsonl
  • --artifacts artifacts -> --output artifacts
  • -o junit.xml --artifacts artifacts -> --output artifacts --export junit.xml
  • --output-format jsonl|yaml|html -> remove the flag; the canonical run directory always writes index.jsonl, and extra formats use --export <file>
  • config output.format -> remove it; use config output.dir only for the run directory

Known consumer audit:

  • WiseTechGlobal/sdd used --artifacts; migration is --output <dir>.
  • WiseTechGlobal/WTG.AI.Prompts already uses --output .agentv/results/artifacts and remains aligned with the canonical directory contract.

Rebase Status

After #1335 merged, this branch initially appeared clean only because the local origin/main was stale at 35263cd707a9a89c95728ae86beb7271b76f2358. git fetch origin --prune updated origin/main to f1162312cb7aa645653b51756acfbbed42426929, and this PR was rebased onto that commit.

Conflict resolution:

Verification

Post-#1335 rebase verification on head 14352fd914f24e9c4f4450ed9681472b4bd7e9f8:

  • bun run build
  • bun run typecheck
  • bun run lint
  • bun run validate:examples
  • bun test apps/cli/test/eval.integration.test.ts packages/core/test/evaluation/config.test.ts apps/cli/test/commands/eval/artifact-writer.test.ts apps/cli/test/commands/eval/task-bundle.test.ts apps/cli/test/commands/runs/rerun.test.ts
  • CLI smoke:
    • --out exits 1 with migration guidance
    • --artifacts exits 1 with migration guidance
    • --output-format exits 1 with migration guidance
    • file-like --output results.xml exits 1 with --export guidance
    • --output <dir> --export export.jsonl exits 0 and writes <dir>/index.jsonl plus the flat export
  • git diff --check origin/main...HEAD
  • Public diff scan for private/local path strings was clean outside Beads; Beads delta is limited to this Bead and follow-up av-33j.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 9, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 14352fd
Status: ✅  Deploy successful!
Preview URL: https://8ea05d83.agentv.pages.dev
Branch Preview URL: https://feat-av-eval-output-config-s.agentv.pages.dev

View logs

@christso christso force-pushed the feat/av-eval-output-config-surface branch from 97bf1a3 to 3c15564 Compare June 9, 2026 02:51
@christso christso force-pushed the feat/av-eval-output-config-surface branch from 3c15564 to b14ea2c Compare June 9, 2026 03:28
@christso christso force-pushed the feat/av-eval-output-config-surface branch from b14ea2c to 14352fd Compare June 9, 2026 03:58
@christso christso merged commit 8bed679 into main Jun 9, 2026
10 checks passed
@christso christso deleted the feat/av-eval-output-config-surface branch June 9, 2026 03:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant