40-question original mock exam

Set a 60-minute timer and answer all 40 before opening the key. These are original practice questions written for this learning path; they are not ISTQB sample questions and do not reproduce the official scoring distribution. Use the official v2.2 sample exam as the authoritative example of format and difficulty.

0 / 40 answered

Questions 1–10

1. A receipt application has deterministic upload validation followed by an ML classifier. Which test most directly targets the AI component?

AReject a 30 MB file over the API limit.BVerify OAuth expiry.CMeasure category recall across representative receipt types.DVerify the database foreign key.

2. Which statement best describes a locked model?

AIt cannot be retrained.BIts learned behavior does not update during normal production use.CIt always returns the same output.DIt runs without network access.

3. A model achieves 99% accuracy on a dataset containing 99% negative cases but detects none of the positives. What is the best conclusion?

AThe model is production-ready.BAccuracy is sufficient because it is above 95%.CClass-specific metrics are needed because class imbalance hides failure.DThe test dataset is necessarily too small.

4. What is the primary purpose of a validation set?

AFinal unbiased evaluation after all tuning.BModel selection/tuning decisions during development.CProduction monitoring only.DReplacing training data.

5. Which situation is the clearest example of data leakage?

ATraining data contains nulls.BThe same patient appears in training and test sets through different visits, allowing identity-specific patterns to leak.CA class is rare.DThe model uses a GPU.

6. For TP=80, FP=20 and FN=20, what is precision?

A0.50B0.67C0.80D0.90

7. A cancer-screening model must minimize missed positive cases. Which metric is most directly aligned?

ARecall.BPrecision.CSpecificity only.DInference throughput.

8. Which statement about RAG is correct?

ARetrieval automatically updates model weights.BRAG supplies retrieved information as runtime context and should be tested separately for retrieval and generation quality.CRAG eliminates hallucinations.DRAG is a form of hardware acceleration.

9. Why might a tester use a metamorphic relation?

ATo avoid defining any expected behavior.BTo test relations between outputs when exact expected outputs are difficult to specify.CTo replace all performance metrics.DTo prove a model is unbiased.

10. A traffic-sign image is made 5% brighter within the supported operating range. The class should stay unchanged. This is primarily:

AA/B testing.BMetamorphic testing.CLoad testing.DStatic analysis.

Questions 11–20

11. Which is the strongest acceptance criterion?

A“The model should be fair.”B“The model should be highly accurate.”C“Recall shall be ≥0.97 on acceptance dataset v5 and no defined subgroup shall be more than 0.03 below overall recall.”D“Users should trust the model.”

12. A model’s training performance is high and validation performance is substantially lower. What should be investigated first?

AUnderfitting only.BOverfitting and data/split issues.CNetwork bandwidth.DUI accessibility.

13. Training and validation performance are both poor. Which condition is plausible?

AUnderfitting.BPerfect generalization.CA/B assignment bias only.DConcept drift after deployment only.

14. Which test most directly checks label quality?

AVerify container CPU limit.BReview a stratified sample with qualified annotators against labeling guidance.CIncrease API timeout.DCompare CSS snapshots.

15. Why can a very large dataset still be inadequate?

ALarge datasets cannot train models.BSize does not guarantee representativeness of the intended operating population.CLarge datasets always create overfitting.DISTQB requires small datasets.

16. Which is a data-pipeline defect?

AThe training path converts cents to dollars but the serving path sends cents unchanged.BRecall is chosen as the primary metric.CA human reviewer rejects an output.DA model is versioned.

17. What is an important limitation of an LLM-as-judge?

AIt cannot process text.BIt is automatically ground truth.CIt can have its own bias/variance and needs validation for the judging task.DIt can only be used for exact strings.

18. Which is the best exploratory-test charter?

A“Test the chatbot.”B“Try random prompts.”C“Explore whether the support assistant reveals another customer’s account data when identity claims change across a long conversation.”D“Make the LLM fail.”

19. What makes red teaming different from ordinary happy-path testing?

AIt requires production access.BIt deliberately explores adversarial, unsafe, insecure or policy-violating behavior.CIt only measures latency.DIt cannot be repeated.

20. A GenAI assistant produces several different but correct summaries for the same source. Which assertion is weakest?

AEvaluate factual consistency with the source.BEvaluate required-content coverage.CRequire every output to equal one exact reference string.DEvaluate safety and instruction following.

Questions 21–30

21. What is the best distinction between model-level and system-level testing?

AModel testing is always manual; system testing is automated.BModel testing evaluates learned behavior; system testing also evaluates preprocessing, integration, postprocessing, controls and product behavior.CThey are synonyms.DSystem testing excludes the model.

22. A production population changes from desktop images to mostly low-light mobile images. Which risk is most immediate?

AInput/data drift relative to the evaluated population.BSQL injection.CSource-code branch coverage.DDeadlock.

23. Which statement about drift is correct?

AAny drift proves the model is wrong.BDrift means source code changed.CDrift can invalidate assumptions and trigger renewed evaluation even when model code is unchanged.DDrift affects only adaptive models.

24. Two candidate models receive the same frozen regression inputs and their outputs are compared. This is:

AA/B testing.BBack-to-back testing.CMutation testing.DChaos testing.

25. Two model versions are randomly assigned to different live-user groups and product outcomes are compared. This is:

AA/B testing.BBack-to-back testing.CMetamorphic testing.DStatic testing.

26. Which evidence is most useful before deploying a converted/quantized model to a mobile device?

AOnly the original server model’s accuracy.BRe-evaluation of functional performance plus latency/resource behavior on representative target devices.CThe model file name.DA screenshot of the training notebook.

27. Which item most improves ML-release reproducibility?

ARecording code, data, model, dependency and configuration versions used to build the artifact.BRenaming the model “final-final.”CIncreasing the number of UI tests.DRemoving random seeds from logs.

28. The acceptance-tested model is v18, but production silently loads v17. Which test is most directly designed to catch this?

AArtifact/version identity check during deployment.BDataset balancing.CLabel review.DMetamorphic image transformation.

29. Which is the strongest reason to monitor production data distributions?

ATo replace acceptance tests.BTo detect changes in operating inputs that can make previous evaluation evidence stale.CTo guarantee no defects exist.DTo increase training accuracy automatically.

30. A recommendation model is safe only if low-confidence cases are routed to human review. Where should this requirement be tested?

AOnly inside the model notebook.BAt system/integration level, including threshold interpretation and fallback routing.COnly by checking source code coverage.DOnly through a dataset schema check.

Questions 31–40

31. Which statement about neural-network coverage is most accurate?

A100% neural coverage proves the model is correct.BNeural coverage can indicate exercised internal behavior but is not a proof of quality or complete input coverage.CIt is identical to statement coverage.DIt removes the need for representative data.

32. A perceptron has x=[1,0], w=[0.8,-0.3], b=-0.2 and outputs 1 when the score is ≥0. What is the output?

A0 because -0.2 is negative.B0 because one input is zero.C1 because the score is 0.6.DCannot be determined.

33. Which is the clearest example of an adversarial test?

ARandomly sample normal invoices.BDeliberately craft small input changes intended to make a classifier evade detection while remaining valid to the user.CReview requirements.DCheck database backups.

34. What should a tester do when two human experts disagree frequently about “correct” labels?

AIgnore disagreement and trust the first label.BInvestigate labeling guidance, ambiguity and adjudication because the oracle itself may be unreliable.CDelete all disputed records automatically.DUse accuracy only.

35. Which is a valid risk of a third-party hosted AI API?

AModel/provider changes can alter behavior even if your application code is unchanged.BHosted models cannot be tested.CHosted models are always adaptive.DThey eliminate privacy risk.

36. Which action best supports testing an adaptive production model?

ARun a one-time pre-release test and disable monitoring.BAdd monitoring, controlled update rules, renewed evaluation and rollback/override mechanisms.CAssume new data always improves quality.DRemove version tracking.

37. In a risk-based ML test strategy, what should drive technique selection?

AThe tester’s favorite framework.BProduct risks, quality characteristics, lifecycle stage and consequences of failure.CAlphabetical order of tools.DThe metric that gives the highest score.

38. Which statement about safety is best?

ASafety is proven once model accuracy exceeds 95%.BSafety should be evaluated across the full system, including controls, fallback and operational context.CSafety applies only to robotics.DSafety and security are always identical.

39. Which practice is most likely to keep a final test set unbiased?

ARepeatedly inspect it while tuning until results are good.BFreeze it and avoid using its results to make iterative model-selection decisions.CCopy hard test examples into training after each failed run and continue calling it the same test set.DUse only training data for final evaluation.

40. You have one hour before the exam. Which revision is highest value?

AMemorize a list of AI product brands.BRework weak learning objectives, metric calculations, distinctions between test techniques, and errors from the official sample exam.CLearn a new ML framework API.DRead unrelated AI news.

Ready to evaluate?Your answer key stays hidden until you submit all 40 answers.

Source registry

Reviewed 2026-09-06 · 3 chapter references

Certified Tester AI Testing Syllabus v2.0

Primary exam authority. Learning objectives, terminology, chapter scope, recommended training time, and hands-on objectives in this guide are mapped to this syllabus.

ISTQB · official syllabus

Source ↗

CT-AI v2.0 Sample Exam Questions v2.2

Official reference for question style and difficulty. Use after completing the learning chapters; do not memorize the questions.

ISTQB · official sample exam

Source ↗

CT-AI v2.0 Sample Exam Answers v2.2

Official explanations for the v2.2 sample exam. Review reasoning, not only the correct option.

ISTQB · official sample exam answers

Source ↗