# Synth Lab Scoring Methodology

## Rubric v1.0

The Synth Lab benchmark scores keyboard synthesizers with an automated 5-check rubric totaling 100 points.

### 1. Soundcheck (20 points)

**What it measures:** Does the build load and make sound?

**Method:**
- Page loads without crashing: **5 pts**
- Zero console errors: **10 pts**
- AudioContext detected and available: **5 pts**

**Rationale:** If a synth doesn't make sound, nothing else matters. This is the gate check.

### 2. In Tune? (20 points)

**What it measures:** Does it produce the right pitches?

**Method (v1.0 - measured via audio tap):**
- **Audio tap:** Patch `AudioContext` and `AudioNode.prototype.connect` before page load
- When any node connects to `ctx.destination`, also connect it to an `AnalyserNode` (fftSize 8192)
- **Test 3 keys:** A4 (440 Hz), C4 (261.63 Hz), E4 (329.63 Hz)
- **Measure:** Press each key, wait 150ms, capture FFT frequency data
- **Pitch detection:** Find peak bin, convert to Hz: `bin * sampleRate / fftSize`
- **Cents error:** `1200 * log2(measured / expected)`
- **Pass:** Within ±25 cents
- **Score:** `20 * (inTuneKeys / 3)` - full credit if all 3 keys in tune

**Evidence stored:** Raw Hz measured, cents error, in-tune boolean per key

**Not measured:** If audio tap fails (build doesn't use AudioContext destination), mark that specific build with reason.

### 3. Feel (20 points)

**What it measures:** Playability and responsiveness

**Method (v1.0 - measured via audio tap):**
- **Latency test:**
  - Record `performance.now()` at keydown
  - Poll `analyser.getFloatTimeDomainData()` every 5ms
  - Calculate RMS: `sqrt(sum(sample²) / length)`
  - Latency = time until RMS > 0.01 threshold
  - **10 pts** if latency < 100ms, **5 pts** if measured but slow
- **Polyphony test:**
  - Press 3 keys simultaneously (S, D, F)
  - Wait 150ms, measure RMS
  - **10 pts** if 3+ voices detected, **5 pts** if some polyphony
- **Total:** up to 20 pts (10 latency + 10 polyphony)

**Evidence stored:** Latency in ms, polyphony voice count

**Not measured:** If audio tap fails, mark that specific build with reason.

### 4. Chops (25 points)

**What it measures:** Feature completeness

**Method:**
- Feature count from project metadata × 3 points each (max 25)
- Features checked: Waveforms, ADSR, Filter, Delay, Reverb, MIDI, Presets, LFO, Polyphony, Sequencer, Recording, etc.

**Rationale:** More features = more synthesis capabilities. Capped at 25 to prevent feature bloat from dominating the score.

### 5. Stage Presence (15 points)

**What it measures:** UI quality and accessibility

**Method (v1.0 - basic):**
- Mobile-friendly layout (viewport ≥ 320px, no horizontal overflow): **10 pts**
- Responsive scaling: **5 pts**

**Note:** Full Lighthouse performance and accessibility scoring planned for v2.0.

---

## Ranks

| Score | Rank |
|-------|------|
| 90-100 | Legend |
| 75-89 | Headliner |
| 55-74 | Session Player |
| 1-54 | Garage Band |
| 0 | Broke a string |

**Special case:** Failing Soundcheck (0 points) = "Broke a string" regardless of other scores.

---

## Scoring Process

1. **Load:** Open build in headless Chromium via Playwright
2. **Wait:** 1 second for audio context initialization
3. **Test:** Run 5 checks sequentially
4. **Record:** Capture scores, errors, and metadata
5. **Rank:** Calculate total and assign rank

**Skipped builds:**
- `broken: true` - Build errors or missing files
- `sourceOnly: true` - Requires dev server (no static HTML)

These receive rank "Broke a string" with skip reason documented.

---

## v1.0 Limitations

This is an **honest, achievable v1.0** scorer. It provides:
- ✅ Real page loading and error detection
- ✅ AudioContext verification
- ✅ Feature detection from metadata
- ✅ Basic mobile checks

**Not yet implemented (planned for v2.0):**
- FFT pitch accuracy (In tune?)
- Web Audio worklet latency measurement (Feel)
- ADSR envelope verification (Feel)
- Polyphony voice counting (Feel)
- Lighthouse performance/a11y (Stage presence)
- Screenshot and audio clip capture

**Why v1.0 first?**
- Establishes the rubric framework
- Scores all 30 archive builds
- Provides baseline data for v2.0 comparison
- Honest about methodology vs. aspirational perfection

---

## Future Enhancements (v2.0)

- **Pitch detection:** OfflineAudioContext + FFT analysis of C4-C6 range
- **Latency measurement:** Performance.now() timestamps + Web Audio clock
- **Envelope verification:** Capture waveform, check attack/release curves
- **Polyphony counting:** Peak detection on rendered audio
- **Lighthouse integration:** Performance, accessibility, best practices
- **Audio/video capture:** 8-second demo clips via MediaRecorder

---

## Running the Scorer

```bash
cd synth-lab/bench
npm install
node score.mjs
```

Results written to `synth-lab/results.json`.

---

## Reproducibility

- **Browser:** Chromium via Playwright (headless)
- **Viewport:** 1200×800 for desktop checks
- **Timeout:** 15s page load, 1s audio init
- **Rubric:** v1.0 (this document)
- **Suite:** v1.0 (see challenge-suite-v1.0.txt)

---

**Last updated:** 2026-11-15  
**Rubric version:** 1.0  
**Scorer version:** 1.0
