Defect Escapes and False Rejects: How to Set Acceptance Metrics for AI Inspection

11/11/2026

An AI inspection station can show high accuracy while letting defective units pass and sending good units to rework. For APAC manufacturers, those errors create different costs and need separate acceptance limits. The World Economic Forum’s Global Lighthouse Network reported a 96% reduction in false detection rate for an AI-enabled quality inspection initiative at LG Electronics’ Changwon site in South Korea. Use that site-specific result to frame the opportunity, then measure the errors and review workload on your own line. [1][2]

Set separate limits for missed defects and false rejects, then verify both against independently confirmed product condition at the point where the factory makes its release decision. Include review workload and line timing so the accepted threshold works in production. [2][3]

1. Choose the evaluation unit and positive class

Choose the unit the line acts on. A camera may inspect individual solder joints, while automated optical inspection (AOI) flags a circuit and the operator holds an entire panel. If the plant releases circuits, calculate circuit-level acceptance metrics and keep joint-level results for diagnosis. Define how multiple findings roll up to a single disposition. [2][3][4]

For this guide, positive means defective under the approved product specification. Have quality engineering define the reference label and how ambiguous cases are resolved. Preserve the original AOI call, the AI recommendation, and the final disposition as separate records. State whether an “escape” means a defect passed the AI station, passed final inspection, or reached the customer. Each boundary answers a different question. [2][3][4]

2. Explain the four possible outcomes

A confusion matrix compares the inspection decision with independently verified condition. In this binary example, flag means the unit is held for review or rejected at the station. If the workflow has a separate review outcome, report its volume and track its final disposition as well. [2][3]

OutcomeVerified conditionStation decisionOperational effect
True positive (TP)DefectiveFlagA genuine defect enters review or rejection
False negative (FN)DefectivePassA defect escapes this station
False positive (FP)ConformingFlagA good unit enters unnecessary review or rejection
True negative (TN)ConformingPassA good unit proceeds

Keep the station result separate from the end-to-end result. A good unit flagged by AI and cleared by an operator is a false call at the station. It becomes a false reject at final disposition only if the workflow incorrectly rejects it. Track both when manual review sits between detection and release. [3][4]

3. Define rates before setting targets

Put the denominator beside every reported rate. Under the binary station definition above, use these calculations. [2][3][4]

  • Defect miss rate: FN ÷ (TP + FN). This is the share of confirmed defective units that passed. Defect recall is its complement.
  • False-flag rate: FP ÷ (FP + TN). This is the share of conforming units flagged. Use final rejection decisions in the matrix when reporting the final false-reject rate.
  • Escape rate among passed units: FN ÷ (FN + TN). This is the defective share of station passes. Multiply by one million to express it in parts per million (ppm).
  • Review workload: Units routed to manual review divided by inspected units. Report review time as well when sizing the station.

If the pilot reviews only boards flagged by existing AOI, its results describe that queue. To estimate escapes among all station passes, independently audit a representative sample of passed units. A published production AOI dataset illustrates the boundary: it contains measurements from boards referred for manual inspection. [4]

4. Calculate a transparent hypothetical example

Suppose a trial independently verifies 10,000 units, including 100 defective units. At the selected threshold, the station flags 95 defective units and 99 conforming units. It passes the remaining five defective units and 9,801 conforming units. Applying the definitions above gives the following results. [2][4]

MetricCalculationResult
Overall accuracy(95 + 9,801) ÷ 10,00098.96%
Defect recall95 ÷ 10095%
Defect miss rate5 ÷ 1005%
False-flag rate99 ÷ 9,9001%
Escape rate among passes5 ÷ 9,806About 510 ppm
Review workload if every flag is reviewed194 ÷ 10,0001.94%

A production buyer can now see what the headline accuracy conceals: five confirmed defective units passed, and 99 good units required an unnecessary hold. For rare defects, report the number of independently verified opportunities and a suitable confidence bound. A trial with zero observed escapes still has statistical uncertainty, particularly when few relevant units were tested. [2][5]

5. Agree on thresholds and trade-offs

Quality engineering should set escape limits by defect severity. Operations should set review capacity and allowable capture-to-disposition time. Choose a threshold on validation data that meets both constraints, then lock it before testing on later production lots. If the acceptance run misses a limit, investigate by product revision, defect class, and capture conditions before approving any change. Research on industrial AOI highlights the risk of tuning thresholds after viewing evaluation results or overlooking changes over time. [2][3][6]

Lessons Learned from the Field

TMA developed an AI inspection solution for a Korean flexible printed circuit board (FPCB) manufacturer using images from its existing AOI equipment and integrating the results into the production workflow. TMA Innovation’s FPCB offering separately publishes 95% defect-detection accuracy and six seconds per board. The public figures leave the defect mix, accuracy denominator, and timing boundaries unspecified. Request those details before treating either figure as an acceptance criterion. [7][8]

Use that integration example when comparing TMA with a low-cost offshore development centre scoped mainly to model training. Ask each bidder to replay a good board flagged by AOI and a defective board passed by the proposed workflow. Require them to show how board identity, source image, model decision, operator review, and final disposition stay linked. TMA’s published architecture covers image acquisition, edge inference, factory integration, and model lifecycle management. [3][7]

Technical deployment pipeline

  • Agree on acceptance. Record the evaluation unit, reference labels, severity limits, and review capacity. [2][3]
  • Build the baseline. Sample normal production, audit passed units, and evaluate rare defects in a separately reported challenge set. [2][4]
  • Validate the threshold. Group images by physical unit, reserve later lots, and lock the operating point before final testing. [2][6]
  • Run in shadow mode. Record AI decisions alongside the existing release process and adjudicate disagreements. [2][3]
  • Qualify the line. Test timing, review queues, missing images, and rollback before authorising live disposition. [2][3]

TMA reports 29 years of engineering experience, 4,000 engineers serving clients in 30 countries, and mission-critical programmes involving hundreds of engineers. Its telecom R&D centres provide a precedent for long-term engineering partnerships. For a regional inspection programme, name the owners of optics, line interfaces, validation, and model releases. TMA lists ISO/IEC 27001 and reports GDPR checks in software quality assurance. Verify certification scope and agree on access and retention for records that identify personnel. [9][10][11]

FAQ

What is the difference between a defect escape and a false reject?

A defect escape is a defective unit that passes the specified inspection boundary. A false reject is a conforming unit the workflow rejects. A good unit flagged and cleared by review is a false call. [3][4]

What denominator should we use for defect escapes?

Use confirmed defective units to calculate the miss rate. Use all passed units to estimate defective output among passes. State the inspection boundary and denominator with the result. [2][4]

Can zero observed escapes justify release?

Compare a suitable upper confidence bound with the approved limit. Check that the tested units cover the relevant defects and operating conditions before approving release. [2][5]

How should we choose an AI inspection threshold?

Choose it on validation data against the approved miss, false-reject, and review limits. Freeze it before testing on reserved production lots. [3][6]

How should buyers evaluate an AI inspection vendor?

Ask for a comparable line reference, a reproducible confusion matrix, and a demonstration linking each disputed result to final disposition. Confirm who owns line integration, threshold changes, and support. [2][3][7]

Scope an acceptance trial with TMA Innovation

Bring one product family, its defect catalogue, representative AOI records, and the station’s cycle requirement to a TMA Innovation inspection review. Agree on the evaluation boundary, the audit sample, and the limits quality and operations will sign off before deployment. [3][7][8]

References

[1] World Economic Forum — The Global Lighthouse Network Playbook for Responsible Industry Transformation, LG Electronics Changwon case.

[2] NIST — AI Risk Management Framework, Valid and Reliable.

[3] TMA Solutions — AI Visual Inspection for Manufacturing QA/QC.

[4] Pfab et al. — Data of automated optical inspection of surface-mounted technology electronic production, Data in Brief, 2024.

[5] NIST — Exact Binomial Confidence Limits.

[6] Pfab and Rothering — Towards Improved Research Methodologies for Industrial AI: A Case Study of False Call Reduction, IEEE COMPSAC, 2025.

[7] TMA Solutions — Custom AI development and Korean FPCB inspection example.

[8] TMA Innovation — Defect Detection on Electronic Board.

[9] TMA Solutions — Company Overview and Engineering Capabilities.

[10] TMA Solutions — Telecommunications Engineering and R&D Centres.

[11] TMA Solutions — CSR Report 2024, Security and Compliance.

TMA Innovation
Author: TMA Innovation
Table Of Content
1. Choose the evaluation unit and positive class
2. Explain the four possible outcomes
3. Define rates before setting targets
4. Calculate a transparent hypothetical example
5. Agree on thresholds and trade-offs
Lessons Learned from the Field
Technical deployment pipeline
FAQ
What is the difference between a defect escape and a false reject?
What denominator should we use for defect escapes?
Can zero observed escapes justify release?
How should we choose an AI inspection threshold?
How should buyers evaluate an AI inspection vendor?
Scope an acceptance trial with TMA Innovation
References
Start your projectRight now!
Contact Us
Share: