CI & Synthetic Monitoring

What it is

CI (continuous integration) runs your test suite automatically on every change and gates the merge on the result. Synthetic monitoring runs the same kinds of checks against production on a schedule, from outside the system, so you learn about a broken login flow before your users do.

1
  Push / PR
2
     │
3
     ▼
4
  [ CI pipeline ]  ← unit → integration → E2E on every change
5
     │ green
6
     ▼
7
  [ Deploy ]
8
     │
9
     ▼
10
  [ Synthetic monitor ]  ← scheduled E2E + API checks in production
11
     │ failure
12
     ▼
13
  Alert (pager / Slack)

CI and synthetic monitoring share tests, assertions and reporting; they differ in where and when they run. Keep them consistent so a test you trust on PRs means the same thing in production.

When to use it / when to avoid it

Use it for

Gating merges on automated tests (CI)
Catching production-only failures: DNS, TLS, third-party outages, region issues
Watching critical user journeys (login, checkout) around the clock
Trending performance budgets and availability over time

Avoid or reduce it when

Synthetic checks duplicate every unit test — monitor the critical paths, not all code
Flaky tests gate the pipeline; quarantine and fix them instead of rerunning blindly
Monitoring runs from a single region only, giving a false picture of global health

Frameworks & tools

Tool	Ecosystem	Notes
GitHub Actions	CI	YAML workflows; huge marketplace of actions
GitLab CI	CI	`.gitlab-ci.yml`; stages, artifacts, environments
Jenkins	CI	Self-hosted, plugin-driven, very flexible
Playwright	CI/E2E	Runs in CI; trace viewer and reporters
Checkly	Synthetic	Playwright-based checks + API monitors, alerting
Grafana Synthetic Monitoring	Synthetic	Probe-based checks, dashboards and alerts in Grafana

Quick setup

1
on: [push, pull_request]
2
jobs:
3
  test:
4
    runs-on: ubuntu-latest
5
    steps:
6
      - uses: actions/checkout@v4
7
      - uses: actions/setup-node@v4
8
        with: { node-version: 20 }
9
      - run: npm ci
10
      - run: npm test

1
test:
2
  image: node:20
3
  script:
4
    - npm ci
5
    - npm test

1
// Jenkinsfile
2
pipeline {
3
  agent any
4
  stages {
5
    stage('Test') {
6
      steps { sh 'npm ci && npm test' }
7
    }
8
  }
9
}

1
npm install -D checkly
2
npx checkly login
3
npx checkly test       # run checks locally once
4
npx checkly deploy     # push checks to the cloud

1
# Install the synthetic-monitoring-agent and point it at your Grafana
2
docker run -d --name sm-agent \
3
  -e SM_API_HOSTNAME=your-instance.grafana.net \
4
  grafana/synthetic-monitoring-agent

Example

1
name: ci
2
on:
3
  push: { branches: [main] }
4
  pull_request:
5

6
jobs:
7
  test:
8
    runs-on: ubuntu-latest
9
    steps:
10
      - uses: actions/checkout@v4
11
      - uses: actions/setup-node@v4
12
        with: { node-version: 20, cache: npm }
13
      - run: npm ci
14
      - run: npx playwright install --with-deps chromium
15
      - run: npm run test:unit
16
      - run: npm run test:e2e
17
      - uses: actions/upload-artifact@v4
18
        if: always()
19
        with:
20
          name: playwright-report
21
          path: playwright-report/

1
import { defineConfig, devices } from '@playwright/test';
2

3
export default defineConfig({
4
  retries: process.env.CI ? 2 : 0,
5
  workers: process.env.CI ? 2 : undefined,
6
  reporter: [['html'], ['list']],
7
  use: { trace: 'on-first-retry' },
8
  projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],
9
});

1
import { test, expect } from '@playwright/test';
2

3
test('login flow', async ({ page }) => {
4
  await page.goto('https://app.example.com');
5
  await page.getByLabel('Email').fill('monitor@example.com');
6
  await expect(page.getByRole('heading', { name: 'Dashboard' }))
7
    .toBeVisible();
8
});

Anti-patterns & best practices

Fail the build on red. A pipeline people ignore is worse than no pipeline.
Cache dependencies and split slow suites into parallel jobs to keep CI fast.
Quarantine flaky tests with an owner and a deadline; never auto-rerun forever.
Upload reports and traces as artefacts so a failure is reproducible after the run.
Monitor critical journeys, not everything. Synthetic checks are a smoke alarm, not a substitute for a full test suite.
Alert on symptoms (checkout failing) rather than raw internals, and set thresholds to avoid alert fatigue.