Product specification

Product

| Field | Value |
| --- | --- |
| Name | RLViz |
| npm package | `rlviz` |
| CLI | `rlviz` |
| Tagline | Visualize and compare agent rollouts. |

Problem

Researchers and engineers building agent environments, evaluations, and post-training systems inspect trajectories constantly. Those trajectories are commonly stored as JSON, JSONL, database rows, directories of artifacts, or harness-specific logs.

The available options usually require one of the following:

The immediate use case is simpler. An engineer is debugging an environment with a coding agent. The agent reports that a test trajectory failed at a particular step. The engineer asks it to open that trajectory, and a fast local viewer appears with the relevant events and artifacts.

Product boundary

RLViz provides visualization, navigation, comparison, and a plugin boundary. It does not provide an LLM, execute the agent, train the model, or require a hosted control plane.

Coding agents act as operators and extension authors:

Primary users

Environment engineer

Needs to understand whether a failure came from the policy, tool contract, seed state, environment implementation, grader, permissions, or infrastructure.

Post-training researcher

Needs to inspect rollout groups, compare successful and failed trajectories, understand reward composition, identify behavioral divergence, and evaluate checkpoint changes.

Agent engineer

Needs to inspect tool sequences, retries, subagents, artifacts, termination behavior, latency, tokens, and final outcomes without instrumenting the agent again.

Core concepts

Run

A source-native execution or experiment with shared configuration such as model, checkpoint, environment, harness version, and timestamp.

Case

A task or input evaluated within a run. A stable case ID allows the same task to be compared across runs.

Rollout group

Trajectories generated together for the same case and sampling condition. For example, a GRPO sampling group of eight trajectories.

Trajectory

One episode or path through the agent-environment interaction.

Event

One ordered unit in a trajectory. Initial event kinds are:

Signal

A numeric, categorical, or textual assessment attached to a trajectory or event. Examples include reward components, pass/fail, advantage, grader labels, latency, and token counts.

Comparison set

A viewer-created selection of trajectories, potentially across runs, checkpoints, models, or configurations. This is distinct from a source-native rollout group.

MVP user experience

Installation

Users download a release binary or install it through a package manager.

brew install <tap>/rlviz

Open a supported trajectory

rlviz open ./artifacts/task-184.jsonl

Expected behavior:

Open an unsupported trajectory

The CLI returns a machine-readable diagnostic and scaffold command:

{
  "code": "unsupported_format",
  "path": "/absolute/path/trajectory.jsonl",
  "suggested_command": "rlviz plugin init --type adapter --lang python --from /absolute/path/trajectory.jsonl .rlviz/plugins/local-adapter"
}

A coding agent can then create, validate, and use a local adapter:

rlviz plugin init --type adapter --lang python --from ./trajectory.jsonl ./plugins/customer-x
rlviz plugin trust ./plugins/customer-x
rlviz plugin validate ./plugins/customer-x ./trajectory.jsonl
rlviz open ./trajectory.jsonl --adapter ./plugins/customer-x

Single-trajectory viewer

The initial UI has three panes:

The viewer must support:

Initial keybindings:

| Key | Action |
| --- | --- |
| `j` / `k` | Next or previous event |
| `Enter` | Inspect selected event |
| `Space` | Expand or collapse |
| `/` | Search |
| `e` | Next error |
| `r` | Next reward or grader event |
| `x` | Toggle raw payload |
| `o` | Open selected artifact |
| `m` | Show trajectory metadata |
| `?` | Show keybindings |

Rollout-group viewer

Group support follows the single-trajectory viewer but is represented in the canonical schema from the start.

The first group view includes:

Divergence

RLViz distinguishes textual difference from behavioral divergence. Different reasoning text that leads to the same action should not automatically be treated as the first meaningful divergence.

The deterministic baseline algorithm will:

Adapters may provide optional alignment keys and state hashes. Domain-specific analyzer plugins may enrich alignment later. LLM-based semantic alignment is optional and never required for core viewing.

Compact path aggregation

For independent trajectories in a group, RLViz may aggregate equal behavioral prefixes into a compact path tree. This is a derived visualization, not a claim that the source execution literally branched.

Real parent-child branches recorded by a harness remain separate and preserve source-native relationships.

Non-goals for the first release

Privacy and security

MVP success criteria