GimmeJob
Sign in
QA metrics & estimation · Chapter 07 / 08

QA Metrics & Estimation

Risk-based allocation & forecasting

An estimate answers 'how much or how long under assumptions'; a forecast answers 'given current evidence, what outcomes are plausible?'. Risk-based allocation adds a second question: where does another unit of testing effort buy the most risk reduction?

Risk & demand → prioritize by impact/likelihood + dependencies → choose test allocation → observe throughput/cycle time → simulate or project outcomes → communicate probability/range → update with new evidence

Allocating test effort by impact and likelihoodCommon

Risk-based allocation directs more testing effort toward failures with greater plausible impact and likelihood while preserving enough breadth to detect unknowns. It is a prioritization mechanism, not a claim that low-risk areas need no testing.

Practical use: Create a small risk register with evidence, impact, likelihood, existing controls and planned test response. Re-rank it when product or operational evidence changes.

Caveat: Multiplying arbitrary 1–5 scores can create false precision. Use the score to structure discussion, then retain the underlying rationale.

Critical-path and dependency constraints on allocationCommon

Risk priority is not the only scheduling constraint. Critical dependencies, environment availability, sequential migrations and specialist bottlenecks can determine when testing can actually happen.

Practical use: Map dependencies before turning effort into dates. Separate tasks that can run in parallel from those gated by a specific delivery, environment or decision.

Caveat: Adding effort to a non-bottleneck does not shorten a schedule controlled by another dependency.

Deliberate de-scoping and communicating what remains untestedCommon

When time is cut, a responsible plan reduces scope explicitly instead of pretending the original plan still fits. De-scoping should identify what evidence will not be collected and the residual risk accepted.

Practical use: Publish a short decision record: retained coverage, removed coverage, reason, risk owner, compensating controls and what would trigger additional testing.

Caveat: Calling de-scoping 'optimization' without naming the missing evidence hides the actual release decision.

Forecast vs estimateCommon

An estimate usually expresses expected effort, duration or size under assumptions. A forecast uses current observations or a probabilistic model to describe possible future outcomes. In practice the terms overlap, so define what your team means.

Practical use: For stakeholder communication, phrase forecasts as outcomes and confidence/ranges: 'based on the last 20 comparable items, the current backlog is likely to finish within…'.

Caveat: Do not change a forecast into a commitment simply by removing the probability language.

Throughput-based forecastingLess common

Throughput-based forecasting uses the observed count of completed comparable work items per time period to project how long a remaining set may take. The Kanban Guide defines throughput as the exact number of work items finished per unit of time.

Practical use: Use a stable item definition and a sufficiently relevant historical window. Plot the distribution rather than relying only on an average.

Caveat: If item mix, team, workflow or definition of finished changes materially, historical throughput may no longer be representative.

The Monte Carlo concept and its assumptionsLess common

Monte Carlo forecasting repeatedly samples from an empirical or modeled distribution to produce a distribution of possible outcomes. For delivery forecasting, common inputs are historical throughput or cycle-time observations.

Practical use: Define the population, historical window, sampling method, number of remaining items and forecast question. Report percentiles/ranges rather than a single simulated average.

Caveat: Simulation does not repair non-representative data. Structural change, strong seasonality, dependent work or a new item class can invalidate the historical sample.

Updating a forecast as evidence changesCommon

Forecasts should become more informative as work produces actual cycle time, throughput, defect discovery and dependency evidence. Reforecasting is a feature of an empirical process, not an admission that the first forecast failed.

Practical use: Set a cadence or event triggers for refresh and keep previous forecasts so calibration can be reviewed later.

Caveat: Continuously moving the date without preserving assumptions and old forecasts prevents learning about systematic bias.

Summary

  • This chapter covers 7 required concepts while keeping tool/formula details tied to a practical decision.
  • Definitions, scope, assumptions and caveats matter more than a number or a tool name by itself.
  • Claims that depend on a standard or product are grounded in the source registry below.

Source registry

Verified 16 Aug 2026
ISTQB Certified Tester Foundation Level Syllabus v4.0.1

Testing vocabulary, estimation techniques, coverage and test-management foundations

ISTQB · verified
Source ↗
The Kanban Guide

WIP, throughput, work item age and cycle time definitions for flow measurement

Kanban Guides / ProKanban.org · verified
Source ↗
NASA Cost Estimating Handbook v4.0

WBS, analogy, uncertainty, risk, documenting assumptions and estimate calibration

NASA · verified
Source ↗
NIST/SEMATECH e-Handbook of Statistical Methods

Distributions, percentiles, variation and statistical interpretation

NIST · verified
Source ↗