Quality Scoring
The Quality Scoring dashboard (/analytics/quality) provides operators with a systematic view of how well agents are performing beyond basic pass/fail security checks.
Current readiness
| Area | Status | Notes |
|---|---|---|
| 4-dimension quality scoring | Live | Efficiency, safety, accuracy, and completion scores per session. |
| Quality trends and aggregation | Live | Per-agent trends over configurable time windows. |
| SLA definitions and tracking | Live | Define thresholds, target compliance rates, and evaluation windows. |
| User satisfaction signal ingestion | Planned | Thumbs up/down, CSAT signals not yet supported. |
What the page is for
Use the Quality dashboard to answer:
- Which agents are consistently delivering high-quality sessions?
- Are any agents degrading in quality over time?
- Are we meeting our quality SLAs?
- Which quality dimensions (efficiency, safety, accuracy, completion) need attention?
Typical workflow
- Review the summary strip for fleet-wide quality metrics.
- Check the SLA compliance table for any breaches.
- Sort the quality trends table to find agents with declining scores.
- Click through to individual session scores for investigation.
Dimensions
| Dimension | What it measures |
|---|---|
| Efficiency | Tool call count, token usage, and latency vs configurable baselines |
| Safety | Inverse of threat findings and policy violations, weighted by confidence |
| Accuracy | Aggregated faithfulness verdicts from the deception detection system |
| Completion | Whether the agent accomplished the user's apparent goal |
The composite score uses smart weight redistribution when a dimension has no data for a given session.
SLA configuration
Quality SLAs can be created directly from the dashboard. Each SLA specifies a metric (composite or individual dimension), a minimum threshold, a target compliance percentage, and an evaluation window in days.