Usability Testing

What it is

Usability testing observes real people trying to complete real tasks with your product, and studies where they succeed and where they struggle. It measures effectiveness (did they do it), efficiency (how much effort) and satisfaction (how did it feel).

1
  task ──► user attempts ──► observe ──► measure ──► iterate
2
              │                 │            │           │
3
          think-aloud        notes,      success rate,  fix the
4
          (what they say)    recordings  time, SUS      top issue

It is qualitative first: five participants usually surface most major problems, long before a statistically large sample is worth the cost.

When to use it / when to avoid it

Use it for

Before writing more code for a feature people may not understand
When analytics show drop-off but not why
Validating a design with the people who will actually use it
Comparing two design options with the same tasks

Avoid or reduce it when

You ask users what they want instead of watching what they do
You recruit only teammates or power users and call them representative
You test a prototype so rough that the feedback is about the mockup, not the idea
You run it once and never close the loop by fixing the top issues

Frameworks & tools

Tool	Ecosystem	Notes
Maze	Web	Remote, unmoderated task studies at scale
Lookback	Multi	Moderated sessions with recording and notes
UserTesting	Web / mobile	Large participant panel on demand
Hotjar	Web	Heatmaps, session replays, on-site survey
System Usability Scale	Method	Ten-item questionnaire producing a 0-100 score
Nielsen heuristics	Method	Ten heuristics for expert review
Optimal Workshop	Web	Card sorting and tree testing for IA
Playwright	JS/TS	Capture flows and screenshots for annotation

Quick setup

1
npm install -D @playwright/test
2
npx playwright install

1
// record a task flow for later review with participants
2
await page.screenshot({ path: 'flow-step-1.png', fullPage: true });

1
pip install pandas matplotlib
2
# analyse SUS scores and task success rates

1
go get golang.org/x/text/language
2
# localise locale-specific usability content

1
<!-- export survey data for analysis -->
2
<dependency>
3
  <groupId>org.apache.commons</groupId>
4
  <artifactId>commons-csv</artifactId>
5
  <version>1.11.0</version>
6
</dependency>

1
dotnet add package CsvHelper
2
# export and score usability session data

Example

1
import { test } from '@playwright/test';
2

3
test('signup task flow for usability review', async ({ page }) => {
4
  await page.goto('/signup');
5
  await page.screenshot({ path: 'step-1-email.png' });
6
  await page.getByLabel('Email').fill('user@example.com');
7
  await page.getByRole('button', { name: 'Continue' }).click();
8
  await page.screenshot({ path: 'step-2-confirm.png' });
9
});

1
import pandas as pd
2

3
def sus_score(responses):
4
    # responses: 10 ints 1-5, odd items positive, even items reversed
5
    total = 0
6
    for i, r in enumerate(responses, start=1):
7
        total += (r - 1) if i % 2 else (5 - r)
8
    return total * 2.5
9

10
df = pd.read_csv("sus_responses.csv")
11
df["sus"] = df.iloc[:, :10].apply(sus_score, axis=1)
12
print(df["sus"].mean())

1
func successRate(completed, attempted int) float64 {
2
  if attempted == 0 {
3
    return 0
4
  }
5
  return float64(completed) / float64(attempted) * 100
6
}

1
enum Heuristic {
2
  VISIBILITY_OF_STATUS, MATCH_TO_REAL_WORLD, USER_CONTROL,
3
  CONSISTENCY, ERROR_PREVENTION, RECOGNITION_OVER_RECALL,
4
  FLEXIBILITY, AESTHETIC_MINIMALISM, ERROR_RECOVERY, HELP
5
}
6
record Finding(Heuristic heuristic, String evidence, Severity severity) {}

1
using CsvHelper;
2
using System.Globalization;
3

4
using var writer = new StreamWriter("usability_sessions.csv");
5
using var csv = new CsvWriter(writer, CultureInfo.InvariantCulture);
6
csv.WriteRecords(sessions);

Anti-patterns & best practices

Watch behaviour, not opinions. What users do is more reliable than what they say they would do.
Give real tasks, not leading questions. “Book a return flight to Da Nang”, not “Do you like the booking screen?”
Stay quiet during the task. Defer help; a moderator who rescues participants hides the very problem you are looking for.
Recruit representative users. Five real target users beat fifty colleagues.
Report observed problems and evidence. Tie each finding to a clip or quote, then prioritise by severity.
Close the loop. A study whose top issues never get fixed is theatre.