Human judgment. Measured results.
Manual Evaluation
A shared corpus. Historical two-class comparison.
Historical two-class benchmarks: Human / Machine. Review every recording again in Gold standard for the three-class taxonomy.
Current batch
Loading the latest batch…
Now evaluating
No active recording.
Playback is optional and does not affect evaluation.
Call volume
Labeled recordings in the current batch
Verdict mix
What each engine decided
Verified accuracy
Against your human or machine judgment
Detection latency
Median audio-to-decision time · p95 and sample count
These are legacy two-class judgments. All old labels need a new review in Gold standard.
Recording corpus
Loading recording metadata…
| Recording | Transcript | Judgment | Current batch |
|---|
This list retains legacy binary judgments. Use Gold standard to review Human, Machine, or Screener. Historical batches use the legacy labels.
Add a recording
Upload a WAV to the shared corpus. Evaluation starts only when you run a batch.
BlandAMD is not applicable to uploads without an original provider call.
Three-class review
Gold standard
Label the initial answer. All old labels must be reviewed again.
- Human
- A person answers directly.
- Machine
- Voicemail, IVR, or a recorded greeting.
- Screener
- An automated call-screening assistant, even if a human joins later.
- Total
- —
- Confirmed
- —
- Pending
- —
- Needs review
- —
- Human
- —
- Machine
- —
- Screener
- —
Loading three-class review progress…
Your unsaved reviews are preserved in this page.
All corpus recordings
Loading recording metadata…
| Call ID | Filename | Transcript | Gold label | Review status | Revision | Reviewed at |
|---|---|---|---|---|---|---|
| Loading recordings… | ||||||
Three-class direct audio
Audio research
Sign in to load audio research.
No winner: review counts and gates have not loaded.
The exploratory gate requires 50 independent reviewed examples per class. Snapshot counts alone do not certify a model or winner.
Candidate readiness
Availability reflects server-reported artifacts and dependencies. A candidate name does not establish readiness.
| Candidate | Kind | Readiness | Reason | Members | Inputs |
|---|---|---|---|---|---|
| Readiness has not loaded. | |||||
Frozen snapshots
Current confirmations are frozen with their audio identity and review revision. A snapshot can be created before the training gate is met.
| Snapshot | Created | Dataset hash | Human | Machine | Screener | Gate |
|---|---|---|---|---|---|---|
| Snapshots have not loaded. | ||||||
Research results
Newest first. Execution uses the offline CLI. Insufficient data establishes no winner.
| Run | Created | State | Snapshot | Candidates | Reason |
|---|---|---|---|---|---|
| Results have not loaded. | |||||
Research results
| Candidate | Coverage | Macro-F1 | Balanced accuracy | Human → Machine | Human → Screener | Machine → Human | Machine → Screener | Screener → Human | Screener → Machine | Audio p50 / p95 | Wall p50 / p95 | Queue p50 / p95 |
|---|
A dash means the measurement is unavailable. Zero is shown only when reported. Coverage, class errors and latency are separate evidence; zero observed errors does not prove zero population risk.
Error recordings
Choose a run to inspect its errors.
The CSV contains the complete run. Recording links open protected Gold review filtered to that corpus ID.
| Recording | Gold label | Prediction | Direction | Reason | Audio latency | Wall latency |
|---|
Evidence notes
Historical binary benchmarks remain separate. Review, materialization, exploratory replay, GPU capacity and real-call qualification are distinct evidence.
Historical two-class benchmarks. Three-class review progress has not loaded. All old labels require review in Gold standard.
Streaming tournament
Experiments
Compare the registered shadow-only lineup on one frozen corpus and transcript profile.
Current experiment
No tournament run yet.
Observed evidence
Current leader — provisional
No eligible score yet.
| Candidate | State | Completed | Failed | Reason |
|---|---|---|---|---|
| No candidate execution state yet. | ||||
Historical two-class benchmarks. Three-class review progress has not loaded. All old labels require review in Gold standard.
Tournament evidence
Winners
Observed rankings, matched denominators, timing evidence and unresolved limitations.
Primary evidence
No eligible leader yet
—No balanced five-second score reported.
No matched denominator reported.
Complete horizon
Best accuracy at 15 seconds
—No measured 15-second score.
Comparable timing
Fastest eligible method
—No comparable timing reported.
Leading composition
The backend-declared leader and its registered decision process.
No eligible composition yet.
No decision-process explanation has been supplied.
Accuracy versus time
Balanced correct-by-deadline on matched denominators. Missing values stay missing.
Per-class error comparison
Leading method against current Kooya and available Bland classification evidence.
| Method | Matched | Human → machine | Machine → human | Unknown | Errors | Decision-only p95 |
|---|
Full leaderboard
Completed, pending, failed and blocked candidates remain in the same table.
| Matched | Human recall | Machine recall | Unknown | Errors | Decision-only p95 |
|---|
Paired uncertainty
Paired uncertainty unavailable for the leading method versus K.
Exploratory retrospective 95% intervals on paired groups. Holm adjusts for testing multiple methods. Selection on this corpus requires a separate independent confirmation corpus.
Recording results
Current attempts, ten results per page. Run gold is frozen; review shows the current editable judgment. Refresh to reload results.
| Case | Method | Run gold | Engine outcome | Comparison | Review |
|---|
Human → machine review
Play recordings where the selected method called a gold-labeled human a machine. Your revalidation is stored separately from the frozen benchmark label.
Select a method to load its human-to-machine results.
| Recording | Method | Run gold | Engine outcome | Revalidation | Review |
|---|