$ python3 -m venv .venv && . .venv/bin/activate $ pip install -q tracerazor $ tracerazor --version tracerazor 1.1.0 $ tracerazor audit support-agent-run-2847.json --hermetic TRACERAZOR REPORT ------------------------------------------------------ Trace: support-agent-run-2847 Agent: customer-support-v3 Framework: langgraph Steps: 11 Tokens: 14280 Analysed: 1ms ------------------------------------------------------ TRACERAZOR SCORE: 83 / 100 [GOOD] (raw structural: 86, task value: 0.90) VAE SCORE: 0.77 MVTG: 33.8% (trace is 33.8% above minimum viable token count) Note: TAS is an *ordinal* heuristic score — compare runs within one project over time, not as an absolute efficiency percentage. ------------------------------------------------------ METRIC BREAKDOWN Code Metric Score Target Status SRR Step Redundancy Rate 0.0% <15% PASS LDI Loop Detection Index 0.000 <0.10 PASS TCA Tool Call Accuracy 83.3% >85% FAIL RDA Reasoning Depth Approp. 0.917 >0.75 PASS ISR Info Sufficiency Rate 100.0% >80% PASS TUR Token Utilisation Ratio 0.959 >0.35 PASS CCE Context Carry-over Eff. 0.613 >0.60 PASS CCR Caveman Compression Ratio 0.384 <0.30 FAIL OBS Observation Token Share 0.377 ≥0.30 PASS -- Diagnostics (no default composite weight) ---------- DBO Decision Branch Optimality 0.833 >0.70 PASS [cold] VDI Verbosity Density Index 0.775 >0.60 PASS SHL Sycophancy/Hedging Level 0.219 <0.20 FAIL GAR Goal Advancement Ratio 0.403 ≥0.40 PASS (goal proxy: step 10) CSD Cross-Step Semantic Drift 0.438 ≥0.60 FAIL [drifting pairs: 3→6] ------------------------------------------------------ SUMMARY The customer-support-v3 agent (langgraph) scored 83/100 [GOOD] — it is reasonably efficient with minor inefficiencies. The biggest efficiency gap is OBS (observation share) (score 0.38/1.0). Issues found: 1 tool misfire(s) wasted ~580 tokens on failed calls. Applying the recommended fixes is estimated to save ~4827 tokens per run ($0.0145/run; ~$724/month projected at an assumed 50000 runs — a heuristic estimate, not a measured re-run). ------------------------------------------------------ PER-STEP ANNOTATIONS # Type Tokens Flags 1 reasoning 820 - 2 tool_call 340 Compress context: 100% duplicated input context 3 reasoning 1450 - 4 tool_call 580 Misfired: wrong params for check_refund_eligibility, retried at step 5 5 tool_call 620 Compress context: correction of step 4 6 reasoning 2100 Compress context: 67% duplicated input context 7 tool_call 380 - 8 reasoning 2840 Compress context: 60% duplicated input context 9 tool_call 920 - 10 reasoning 1680 Compress context: 52% duplicated input context 11 tool_call 2550 - ------------------------------------------------------ OPTIMAL PATH RECOMMENDATION Suggested: 10 steps (vs 11 actual) | Est. tokens: 9453 (vs 14280) KEEP Step 1 reasoning Parse the user request. The customer wants a refun ~ TRIM Step 2 tool_call Call get_order_details [Compress context: 100% duplicated input context] KEEP Step 3 reasoning Parse the user request again. The customer wants a - DEL Step 4 tool_call Call check_refund_eligibility [Misfired: wrong params for check_refund_eligibility, retried at step 5] ~ TRIM Step 5 tool_call Call check_refund_eligibility [Compress context: correction of step 4] ~ TRIM Step 6 reasoning Now I need to think deeply about whether this is t [Compress context: 67% duplicated input context] KEEP Step 7 tool_call Call process_refund ~ TRIM Step 8 reasoning Let me re-evaluate whether the refund was processe [Compress context: 60% duplicated input context] KEEP Step 9 tool_call Call get_order_details ~ TRIM Step 10 reasoning Re-evaluating whether the refund was correct. The [Compress context: 52% duplicated input context] KEEP Step 11 tool_call Call send_confirmation ------------------------------------------------------ AUTO-GENERATED FIXES Fix 1: [tool_schema] → check_refund_eligibility Patch: Tool "check_refund_eligibility" failed at step 4 with: "missing required parameter: order_id". Mark the missing parameter(s) as required in the tool schema so the model cannot omit them. Est. savings: 580 tokens/run Fix 2: [context_compression] → system_prompt Patch: Before each tool call, summarise the conversation to the last three relevant facts. Do not re-include information that has already been established earlier in this session. Est. savings: 5524 tokens/run Fix 3: [goal_anchor] → system_prompt Patch: Add to system prompt: "Keep the task objective established at the start of the trace as your working objective. Before acting, check that the action moves measurably closer to it; if it does not, skip it and take the next concrete action instead. Do not restate the objective or summarise progress unless explicitly asked." (Detected drift: trajectory path entropy 1.00 / focus 0.25 (scattered).) Est. savings: 445 tokens/run ------------------------------------------------------ PATH ENTROPY (staying-on-path diagnostic, not part of TAS) Path entropy: 1.000 (0 = directed, 1 = random walk) Focus score: 0.250 [scattered] target ≥ 0.50 Trajectory: 1 advance / 1 stall / 1 regress (vs final step (no task goal in trace)) note: lexical backend — re-run with --enhanced for embedding-based drift ------------------------------------------------------ SAVINGS ESTIMATE (heuristic projection from flagged waste, not a measured re-run) Tokens saved: 4827 (33.8% reduction) Cost saved: $0.0145 per run Projected/month: $724.05 (at an ASSUMED 50000 runs/month — illustration, not your bill; size it with `tracerazor cost --runs `) Latency saved: ~3.2s per run ------------------------------------------------------ EXECUTIVE SUMMARY customer-support-v3 scores 83/100 [GOOD]. Biggest issue: OBS (observation share). Est. ~$724/month at an assumed 50000 runs (heuristic projection). ------------------------------------------------------ [exit 0]