QE showcase
How I shape quality like a product
A concise view of how I approach strategy, automation architecture, CI quality gates, and defect analysis — grounded in work at Squiz, Xero, and consulting engagements, with sample artifacts you can mirror on your own programmes.
Test strategy
Shift-left without pretending risk disappears at the unit layer.
I start from user-critical flows and integration seams: what must never break silently, where data and identity make tests expensive, and which contracts broker real coupling between services.
Automation is not a volume contest — it is a portfolio decision across E2E, integration, and fast feedback layers, owned with the same discipline as production code.
Quality checklist
A practical gate list before merging risky change.
- ✓Risk-based scope for the change and blast radius
- ✓Data and environment strategy defined before automation
- ✓Contracts or schema checks at integration boundaries
- ✓CI stages ordered for fail-fast economics
- ✓Flake budget and ownership for unstable tests
- ✓Observability: logs, artifacts, and human-readable summaries
Example test summary (sample copy)
The kind of Slack-friendly narrative I like pipelines to emit.
platform-e2e-nightly · main · a4f91c2 Status: PASSED (with 1 flaky contract stage) Duration: 14m 02s • chromium: 42/42 passed • webkit: 41/41 passed (1 retry on package-registry fixture) • contract: PASSED after retry — broker latency threshold exceeded once Next: watch package-registry p95; consider pinning fixture version.
Promptfoo evals
Second ingest path: signed POST /api/promptfoo/webhook after CI eval.
Supabase connected — awaiting evals
Run the Promptfoo workflow or POST signed results to /api/promptfoo/webhook.
In-repo demonstration (rubric API)
A real test pyramid in this repo — each layer maps to a named GitHub Actions job and can feed the CI dashboard when ingest is configured.
apps/ci-demo hosts a tiny HTTP service: POST /v1/assess-change takes surfaces touched, blast radius, and migration flag, and returns recommended test lanes (unit, contract, integration-matrix, smoke-e2e, etc.) with rationale — the same risk-based triage thinking as above, as a stable JSON contract.
From the repo root: npm run test:ci-demo runs all layers; .github/workflows/ci-demo.yml runs three jobs unit, integration, and contract so run metadata in GitHub matches the pyramid narrative. When SUPABASE_SERVICE_ROLE_KEY and GITHUB_WEBHOOK_SECRET are set on Vercel, completed runs POST to /api/github/webhook and appear on the CI dashboard.