Draft test handbook · methodology 0.2

Repeatable tests before publishable scores.

This handbook turns the public CoffeeScore principles into testable actions. It is deliberately marked draft until equipment, tolerances and reviewer training are validated in pilot tests.

Draft—not a score0 completed test sessions4 category protocolsRaw observations retained

Shared controls

Rules every machine test follows.

Photograph and record the retail unit, included accessories, model identifier and firmware where visible.
Run the manufacturer’s initial setup, rinse and water-hardness procedure before timed tests.
Record room temperature, starting water temperature and water mineral content for every session.
Use the same weighed coffee lot or capsule lot within a comparison group and record its identity privately.
Discard and repeat a run when the recipe, equipment or measurement process deviates from protocol—and retain the reason.

espresso

Manual espresso machines

Measure repeatable espresso and milk performance while preserving the influence of normal user technique.

Five valid espresso runs and three valid milk runs after setup.

Preparation

Use an appropriate dialled-in grinder and record burrs, setting, dose and shot basket.
Primary recipe: 18g dry coffee to 36g beverage, adjusted only where the manufacturer specifies a different basket dose.
Purge and preheat exactly as instructed; record every added step required to reach the target recipe.
For milk testing, use 150ml chilled semi-skimmed dairy milk from a recorded starting temperature.

Measurements retained

Ready timeCold start to manufacturer-ready indication and first permitted brewseconds

Brew ratioWeigh dry dose and beverage outputgrams

Shot timeButton or pump activation to target outputseconds

TemperatureRecord beverage temperature using the declared measurement setup°C

RepeatabilityCalculate spread across valid beverage weights, times and temperaturesvariation

Milk time and finishTime to target temperature; record texture against the published rubricseconds / rubric

SoundA-weighted maximum and equivalent level at fixed distancedBA

bean to-cup

Bean-to-cup machines

Test the default one-touch experience as well as the best result achievable without specialist intervention.

Five default coffee runs, five optimised runs and three runs for the primary milk recipe.

Preparation

Record factory defaults before making any adjustment.
Use one documented bean lot and allow the grinder the manufacturer’s stated bedding-in or adaptation cycles.
Test espresso or short coffee at default settings, then one documented optimised setting.
Where automatic milk is included, use the same milk type, volume and starting temperature for every run.

Measurements retained

First drink timeCold start to completed default drinkseconds

Beverage outputWeigh every completed drinkgrams

TemperatureMeasure immediately after the programme completes°C

Dose proxyWeigh bean hopper and grounds waste before and after controlled runsgrams

Milk performanceTime, output temperature and texture rubricseconds / °C / rubric

Daily cleaningTime the complete prompted and manual routineminutes

SoundSeparate grinding and brewing observations at fixed distancedBA

pod

Pod and capsule machines

Measure speed, consistency, capsule constraints and realistic ownership effort without treating pressure claims as drink quality.

Five valid short drinks and five valid long drinks.

Preparation

Use capsules from one recorded lot and the manufacturer’s default drink buttons.
Run the specified rinse or warm-up process only where the manual requires it.
Test the most common espresso-size drink and one longer supported drink.
Record capsule-system exclusivity, recycling route and used-capsule capacity as ownership facts.

Measurements retained

Ready timeCold start to ready indicationseconds

Drink timeButton press to flow stopseconds

Beverage outputWeigh each completed drinkgrams

TemperatureMeasure immediately after flow stops°C

RepeatabilityCalculate output and temperature spreadvariation

Capsule costUse normal undiscounted UK price per capsule with dateGBP

SoundA-weighted level at fixed distancedBA

filter

Filter coffee machines

Test full- and partial-batch consistency, brew time and holding performance using a transparent brew recipe.

Three valid full batches and three valid partial batches.

Preparation

Use 60g coffee per litre unless the manufacturer’s certified recipe requires another ratio.
Record filter shape, paper, grind setting, water mass and coffee mass.
Test both the common full batch and the machine’s supported half or partial batch.
For hotplate models, measure holding temperature and taste observations at fixed intervals; thermal carafes remain closed between samples.

Measurements retained

Total brew timeStart control to final drawdownminutes

Brew temperatureRecord water or slurry temperature using the declared setup°C

Beverage yieldWeigh brewed outputgrams

Batch consistencyCompare time, yield and temperature across runsvariation

Holding performanceTemperature at 0, 30 and 60 minutes°C

Daily cleaningTime basket, carafe and required wipe-downminutes

SoundA-weighted level during brew cycledBA

Raw-data contract

Every number must keep its context.

A future test session stores the machine and protocol version, retail-unit status, environment, measurement device, run number, timestamp, observation and limitations. Invalidated runs remain visible instead of disappearing.

Test sessionMachine + unitProtocol versionEnvironmentRun observationsDevices usedLimitationsPublication status

What the pilot tests still need to settle

Measurement equipment, acceptable tolerances, sensory rubric wording, blind tasting design, water specification and the minimum reliability evidence for a non-provisional score.

Open the testing queue →Return to CoffeeScore