Skip to main content
The quickstart and Find & fix deviations drive the loop from the dashboard. This page does the same loop agent-first: you stand up an agent from nothing, then hand the wheel to Claude Code, which reads the agent’s own policy, finds wrong-but-not-errored answers your dashboards can’t see, and drives the fix. No cloud account, no LLM key to run the agent.
What you’ll need. Node 20.6+, and Claude Code (or any coding agent) open in the folder. Everything runs locally. We’ll use support-bot - a simulated customer-support agent for a fictional store that ships with Coach as examples/support-bot. Its replies are canned, so it runs deterministically with no API key - but the instrumentation is the real @glassray/tracing SDK, exactly what a production agent ships.

Start Coach

In its own terminal - leave it running:
That’s the whole setup: a local server, an embedded database, and the dashboard in one process. Sent traces before? npx @glassray/coach reset --yes for a clean slate.

Stand up the agent

From a clone of the repo the SDK is already installed - send a day of traffic:
Or copy the folder out and run it as your own project. Either way, thirty-four support tickets land as traces. Instrumenting your own agent is the same handful of lines - wrap the entrypoint, then each step:
support-bot’s policy is three lines in its own system prompt: ground order answers in lookup_order, escalate refunds over the $100 limit, and never echo a full card number. But the buggy corpus quietly violates all three: 5 order-status answers are invented (no lookup_order call), 3 over-limit refunds are auto-issued, and 3 replies echo the customer’s full card number - 11 wrong answers, zero errors.

Look around - everything looks healthy

Open http://127.0.0.1:5899. Overview: 34 traces, ~97% success, and exactly one flagged error. Open it in Traces: get_invoice timed out - a real crash, and the harmless kind of failure, because the customer saw an error and retried. That one red row is all your monitoring sees. While you’re there, open any handle-support-ticket waterfall (full inputs and outputs on every span), scan the tokens column for the 18k-token outlier (a customer pasted an entire email thread), and find the trace where search_kb timed out but the agent recovered gracefully. Everything else is green. Eleven of those green replies are wrong.

Hand it to Claude Code

Install the Glassray skill into the repo, then ask your agent to drive the loop:
In Claude Code:
Set up Glassray flows and evals for this agent from its policy, then run discovery and tell me what’s broken.
Claude reads support-bot.mjs, sees the three policy lines in SYSTEM_PROMPT, and works through the glassray-coach CLI on its own: it inventories durable state, scopes the behaviours as flows (order status, refunds, card updates), turns each policy line into a flow-scoped rule (which fails on this traffic, as it should), and runs deviation discovery (glassray-coach deviations discover) - Coach’s judge reads the actual conversations and clusters what it finds. Expect the three planted failure modes to surface as recurring deviation types, each pointing at its offending traces. None of the eleven threw an error.
Discovery needs a model. On the zero-config ~/.claude subscription a run over this corpus takes a few minutes (each judge call is a full turn) - Claude kicks it off and polls. A metered key (GLASSRAY_LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY) is ~a minute. See providers.
The dashboards said “fine.” Claude - reading the traces, not the metrics - found eleven confidently-wrong answers and named all three patterns.

Let Claude fix the code

Run glassray-coach fix on the ungrounded-order-status deviation and apply the fix.
glassray-coach fix <id> returns a markdown instruction doc - the target behaviour, a repo search plan, and the ordered edits. Executing the search plan lands Claude on the planted bug branches in support-bot.mjs (each buggy ticket takes a shortcut around the policy - answering from memory instead of calling lookup_order, auto-issuing an over-limit refund, echoing the raw card number). Claude removes the shortcuts so every reply follows the policy. (In a hurry? The demo agent also ships the corrected behaviour behind a flag - --fixed.)

Prove it - hands-free

Send fresh traffic and don’t touch anything else:
New traces classify into the flows in the background, and each flow-scoped rule reruns on its own once enough land. Watch it - glassray-coach runs list shows the classify sweep and the autorun eval runs appear; glassray-coach evals list shows the pass rates climb without you touching anything. Green (failed: 0, regressionCount: 0) → glassray-coach deviations resolve <id>.
Empty folder → self-improving loop. Claude wrote the rules from the agent’s own policy, found eleven silent failures your error rate never saw, fixed the code, and the checks now rerun themselves - flagging a regression if any of the three failures ever comes back.

​
The three prompts

The whole run, once Coach is up and traffic has landed, is three messages to Claude Code:
  1. Set up Glassray flows and evals for this agent from its policy, then run discovery and tell me what's broken.
  2. Run glassray-coach fix on the ungrounded-order-status deviation and apply the fix.
  3. I re-ran the agent - check the evals reran and the pass rates climbed, then resolve the deviations.

​
Next steps

CLI & coding agents

Every command, provider config, and the skill that drives this loop.

Find & fix deviations

The same loop, worked from the dashboard, with the full anatomy of a generated fix.

Send traces

Instrument your own agent - the SDK, raw OTLP, and capturing inputs and tokens.

Overview

What Coach is and how it fits alongside hosted Glassray.