ASF Standards · Laboratory · Standard 12

Standard 12 — Digital Care and Artificial Intelligence

5 criteria · 3 core · 2 standard-level · Version 1.0 · Aligned to ISQua EEA Principle 7, 6th Edition

Report content, delivery, and turnaround workflow are addressed in Standard 8. This standard addresses the specific governance of artificial intelligence used in diagnostic result interpretation.

Criteria in this standard

12.1 — AI-Assisted Diagnostic Tools Are Genuinely Validated Before Clinical Use
12.2 — A Qualified Professional Genuinely Reviews Every AI-Assisted Result
12.3 — AI-Assisted Tool Performance Is Genuinely Monitored on an Ongoing Basis
12.4 — Laboratory Staff Are Genuinely Consulted and Trained Before an AI Tool Is Introduced
12.5 — Accountability for AI-Assisted Diagnostic Decisions Is Explicitly Defined
12.6 — The Ordering Clinician Is Genuinely Informed When a Result Involved AI-Assisted Interpretation

12.1

AI-Assisted Diagnostic Tools Are Genuinely Validated Before Clinical Use

Core

Any AI-assisted tool used in result interpretation — automated image analysis, pattern recognition in laboratory data — is genuinely validated against the laboratory’s own real patient population and testing conditions before clinical use, not deployed on the strength of a vendor’s general performance claims alone.

In plain terms: Before an AI tool is used to help interpret real results, the lab has actually tested it on its own patients and conditions — not just trusted the vendor’s general performance numbers.

Facility category	Crisis	Transition	Small	Standard
Applicability	Adapted	Full	Full	Full

Why this matters

This is marked Core because a diagnostic tool’s published performance figures are genuinely based on the population and conditions it was trained and tested on — which may differ meaningfully from a specific laboratory’s own real patient population, equipment, and sample preparation practices. Local validation is what actually confirms the tool performs as claimed in this specific laboratory’s real conditions, not just in the vendor’s own testing environment.

What good looks like

A tool is genuinely validated against the lab’s own population.
Validation is genuinely documented with real figures.
Re-validation is genuinely triggered by meaningful change.

Common failure modes

A tool is deployed based solely on the vendor’s published accuracy figures, with no genuine local validation.

Worked example

In practice

A laboratory adopting an AI-assisted image analysis tool.

BeforeThe vendor’s published accuracy figure was taken as sufficient, with no genuine testing against this laboratory’s own sample set.

ActionA local validation study was run against a real sample set from this laboratory’s own patient population, confirming comparable performance before clinical rollout.

AfterThe Monitor reviewed the local validation study documentation. Verified.

If you are starting from zero — do this first

Run a local validation study against this laboratory’s own sample set before any clinical use.

The most common mistake: Trusting a vendor’s general performance claim as sufficient, without genuine local validation against this laboratory’s own real conditions.

Self-assessment questions

1. Is a tool genuinely validated against the lab’s own population before clinical use? — A real, local validation.
Evidence: Validation study documentation

2. Is validation genuinely documented with real performance figures? — A real, documented record.
Evidence: Validation report

3. Is re-validation genuinely triggered by a meaningful change? — A real, triggered re-validation process.
Evidence: Re-validation protocol

Common reasons for a PARTIAL answer

A validation study was conducted but never actually repeated after a significant equipment change.

Implementation plan

When	What
Before any clinical use	Run a local validation study and document results.

How the Monitor verifies this

Method	What	Detail
DOCUMENT	Validation study review	Reviews the local validation documentation and figures.

Supervisor tips

Ask for the actual local validation data, not just the vendor’s marketing material.

Evidence base

US Food and Drug Administration. Artificial Intelligence and Machine Learning in Software as a Medical Device. Silver Spring: FDA; 2023.

ASF training courses on GMJ Academy →

Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.

12.2

A Qualified Professional Genuinely Reviews Every AI-Assisted Result

Core

Every result genuinely involving AI-assisted interpretation is genuinely reviewed by a qualified laboratory professional before release — not an automated interpretation released directly to the clinician with no real human verification step.

In plain terms: A qualified person always actually checks an AI-assisted result before it goes out — the AI never has the final say on its own.

Facility category	Crisis	Transition	Small	Standard
Applicability	Adapted	Full	Full	Full

Why this matters

This is marked Core because a diagnostic error reaching a clinician and patient genuinely carries real, direct harm potential — AI-assisted interpretation, however well-validated, is a support to professional judgement, not a replacement for it. Genuine, meaningful human review is the actual safeguard against the AI tool’s own real limitations and failure modes.

What good looks like

Every result is genuinely reviewed before release.
The reviewer can genuinely override the AI output.
A real instance shows a genuine override.

Common failure modes

Review has become a formality — the reviewer habitually approves AI output without genuine, independent scrutiny.

Worked example

In practice

A laboratory whose AI review step had become routine.

BeforeReviewing professionals had come to approve AI-generated interpretations almost automatically, with genuine independent scrutiny rarely actually happening.

ActionA structured review checklist was introduced requiring the reviewer to genuinely document their own independent assessment before approval.

AfterThe Monitor reviewed a sample of completed checklists showing genuine, substantive review. Verified.

If you are starting from zero — do this first

Introduce a structured review checklist requiring documented, independent assessment.

The most common mistake: Review becoming a routine, automatic approval rather than genuine, independent professional scrutiny.

Self-assessment questions

1. Is every AI-assisted result genuinely reviewed before release? — Real, genuine verification.
Evidence: Review records

2. Can the reviewer genuinely override the AI output? — A real, functioning override.
Evidence: Override capability documentation

3. Is there a real instance of a professional genuinely overriding an AI interpretation? — A concrete, real example.
Evidence: Override record

Common reasons for a PARTIAL answer

Review happens but has genuinely become a rubber-stamp rather than independent scrutiny.

Implementation plan

When	What
Week 1-2	Introduce a structured, documented review checklist.

How the Monitor verifies this

Method	What	Detail
DOCUMENT	Review record sample	Reviews a sample of completed reviews for genuine, substantive scrutiny.

Supervisor tips

Ask for a real, specific example of an AI interpretation being genuinely overridden.

Evidence base

Topol EJ. High-Performance Medicine: The Convergence of Human and Artificial Intelligence. Nature Medicine. 2019;25(1):44-56.

ASF training courses on GMJ Academy →

Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.

12.3

AI-Assisted Tool Performance Is Genuinely Monitored on an Ongoing Basis

Standard

The accuracy of an AI-assisted diagnostic tool is genuinely monitored on an ongoing basis after deployment, with real tracking of discrepancies between AI-assisted and professional interpretation — not initial validation treated as sufficient for the tool’s entire operational lifetime.

In plain terms: The lab keeps actually checking whether the AI tool stays accurate over time, not just trusting the one-time validation done before it was first used.

Facility category	Crisis	Transition	Small	Standard
Applicability	Adapted	Full	Full	Full

Why this matters

An AI tool’s real performance can genuinely drift over time as reagent batches, equipment, or patient population characteristics change — initial validation, however thorough, is a snapshot, not a permanent guarantee. Ongoing monitoring is what actually catches a genuine performance drift before it causes real harm.

What good looks like

Accuracy is genuinely monitored on an ongoing basis.
Discrepancies are genuinely tracked.
A genuine pattern triggers a real response.

Common failure modes

Initial validation is treated as a one-time check, with no genuine ongoing monitoring afterward.

Worked example

In practice

A laboratory whose AI tool had never been monitored post-deployment.

BeforeInitial validation data was treated as sufficient indefinitely, with no genuine ongoing tracking of discrepancies between AI and professional interpretation.

ActionA monthly discrepancy log was introduced, genuinely tracking every case of disagreement between AI output and the reviewing professional’s final interpretation.

AfterThe Monitor reviewed the discrepancy log and confirmed it was genuinely being used. Verified.

If you are starting from zero — do this first

Introduce a discrepancy log tracking AI-versus-professional disagreement.

The most common mistake: Treating a one-time initial validation as sufficient for the tool’s entire operational lifetime.

Self-assessment questions

1. Is accuracy genuinely monitored on an ongoing basis? — Real, continuous monitoring.
Evidence: Monitoring records

2. Are discrepancies genuinely tracked? — A real, documented log.
Evidence: Discrepancy log

3. Does a genuine pattern trigger a real response? — A real, usable escalation process.
Evidence: Escalation protocol

Common reasons for a PARTIAL answer

A discrepancy log exists but is never actually reviewed for genuine patterns.

Implementation plan

When	What
Week 1-2	Introduce a discrepancy log and a regular review schedule.

How the Monitor verifies this

Method	What	Detail
DOCUMENT	Discrepancy log review	Reviews the log and evidence it is genuinely analysed over time.

Supervisor tips

Ask when the discrepancy log was last actually reviewed, not just whether it exists.

Evidence base

US Food and Drug Administration. Artificial Intelligence and Machine Learning in Software as a Medical Device. Silver Spring: FDA; 2023.

ASF training courses on GMJ Academy →

Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.

12.4

Laboratory Staff Are Genuinely Consulted and Trained Before an AI Tool Is Introduced

Standard

Laboratory staff who will actually use or review AI-assisted tools are genuinely consulted before introduction, with real training on the tool’s genuine limitations — not a tool deployed with staff expected to trust its output without real understanding of where and how it can genuinely be wrong.

In plain terms: Before a new AI tool is introduced, staff get a real say and genuinely understand its specific limitations — not just trained to click “approve.”

Facility category	Crisis	Transition	Small	Standard
Applicability	Adapted	Full	Full	Full

Why this matters

Staff who will genuinely review AI-assisted output need real, substantive understanding of the tool’s specific, known failure modes to actually catch an error — training limited to “how to operate the software” misses this genuine, more important understanding of when and how the tool can actually be wrong.

What good looks like

Staff are genuinely consulted before introduction.
Training genuinely covers specific, known limitations.
Staff can genuinely describe a real limitation.

Common failure modes

Training covers only how to operate the software, with no genuine understanding of the tool’s actual failure modes.

Worked example

In practice

A laboratory introducing an AI image-analysis tool.

BeforeTraining covered only how to operate the software interface, with no genuine discussion of specific scenarios where the tool was known to underperform.

ActionTraining was revised to genuinely include documented failure-mode scenarios from the validation study, with staff input incorporated beforehand.

AfterThe Monitor interviewed a staff member who could correctly describe a genuine, specific limitation. Verified.

If you are starting from zero — do this first

Build training content around the tool’s actual, documented failure modes, not just its interface.

The most common mistake: Training that covers software operation but never genuinely conveys the tool’s actual, specific limitations.

Self-assessment questions

1. Are staff genuinely consulted before introduction? — Real, prior consultation.
Evidence: Consultation record

2. Does training genuinely cover specific, known limitations? — Real, substantive training.
Evidence: Training materials

3. Can staff describe a genuine limitation of the tool they use? — Tests actual, conveyed understanding.
Evidence: Staff interview

Common reasons for a PARTIAL answer

Training happened but genuinely covered only interface operation, not limitations.

Implementation plan

When	What
Before any launch	Build and deliver limitation-focused training.

How the Monitor verifies this

Method	What	Detail
ASK	Staff interview	Asks staff to describe a specific limitation of the AI tool.

Supervisor tips

Ask a staff member what the AI tool is known to get wrong, specifically.

Evidence base

European Commission. Ethics Guidelines for Trustworthy Artificial Intelligence. Brussels: European Commission; 2019.

ASF training courses on GMJ Academy →

Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.

12.5

Accountability for AI-Assisted Diagnostic Decisions Is Explicitly Defined

Core

The laboratory has genuinely considered and documented accountability for a diagnostic decision informed by an AI-assisted tool — who is responsible when the tool is wrong — not leaving this as an unexamined question until an actual misdiagnosis forces an answer.

In plain terms: Everyone knows, in advance, who’s actually responsible if an AI-assisted diagnostic result turns out to be wrong.

Facility category	Crisis	Transition	Small	Standard
Applicability	Adapted	Full	Full	Full

Why this matters

This is marked Core because a genuine diagnostic error carries real, direct patient harm potential — leaving accountability genuinely unexamined until an actual misdiagnosis occurs means the question gets answered under real pressure, potentially poorly, exactly when clarity matters most. Explicit, documented accountability, established before any incident, is what ensures a genuine, prompt, and appropriate response.

What good looks like

Accountability is explicitly documented.
Reviewing professionals genuinely understand their own accountability.
A defined process exists for reviewing accountability in an error.

Common failure modes

Accountability has never been genuinely examined until an actual diagnostic error forces the question.

Worked example

In practice

A laboratory reviewing its AI governance after a near-miss.

BeforeAccountability for an AI-assisted diagnostic error had never been explicitly documented, leaving real ambiguity when a near-miss was reviewed.

ActionA clear policy was developed explicitly confirming the reviewing professional retains diagnostic accountability regardless of AI assistance.

AfterThe Monitor reviewed the policy and interviewed a professional who could clearly describe their own accountability. Verified.

If you are starting from zero — do this first

Document accountability for AI-assisted diagnostic decisions explicitly, before an incident forces the question.

The most common mistake: Leaving accountability genuinely unexamined until an actual diagnostic error forces an answer under pressure.

Self-assessment questions

1. Is accountability for an AI-assisted diagnostic decision explicitly documented? — A real, clear answer.
Evidence: Documented accountability policy

2. Do reviewing professionals genuinely understand their own accountability? — Real, demonstrated understanding.
Evidence: Staff interview

3. Is there a defined process for reviewing accountability in a diagnostic error? — A real, usable process.
Evidence: Incident review protocol

Common reasons for a PARTIAL answer

A policy exists but staff genuinely haven’t been told about it.

Implementation plan

When	What
Week 1-2	Document accountability explicitly and communicate it to all reviewing staff.

How the Monitor verifies this

Method	What	Detail
ASK	Staff interview	Asks a reviewing professional to describe their accountability for an AI-assisted result.

Supervisor tips

Ask a reviewing professional directly: “If the AI was wrong and you approved it, who’s responsible?”

Evidence base

European Commission. Ethics Guidelines for Trustworthy Artificial Intelligence. Brussels: European Commission; 2019.

ASF training courses on GMJ Academy →

Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.

12.6

The Ordering Clinician Is Genuinely Informed When a Result Involved AI-Assisted Interpretation

Standard

Where AI genuinely assisted in interpreting a diagnostic result, the ordering clinician is genuinely informed of this on or alongside the report — not left to assume every result was generated by human interpretation alone — so that the clinician can, in turn, inform the patient as part of normal clinical practice.

In plain terms: The lab makes sure the doctor who ordered the test actually knows when AI helped interpret the result, so the doctor can tell the patient — since the lab itself usually has no direct relationship with the patient.

Facility category	Crisis	Transition	Small	Standard
Applicability	Adapted	Full	Full	Full

Why this matters

A laboratory genuinely sits one step removed from the patient — the ordering clinician is the actual point of contact who explains results. A patient’s genuine right to know when AI was involved in their care can only be honoured here if the laboratory first ensures the clinician genuinely knows, since the lab has no direct channel to disclose to the patient itself. This criterion is the laboratory’s own half of an obligation it shares with the ordering clinician.

What good looks like

The report or an accompanying note genuinely flags AI involvement.
This disclosure is genuinely clear enough for the clinician to pass along.
A clinician can genuinely confirm they noticed it.

Common failure modes

AI involvement is technically logged in an internal system but never genuinely appears on or alongside the report the clinician actually receives.

Worked example

In practice

A laboratory using AI-assisted image analysis for a pathology result.

BeforeAI involvement was logged internally for quality tracking but genuinely never appeared anywhere on the report the ordering clinician actually saw.

ActionA brief standard line was added to the report template: “This result includes AI-assisted image analysis, reviewed by [pathologist name].”

AfterThe Monitor interviewed an ordering clinician who could genuinely confirm they noticed and understood the note. Verified.

If you are starting from zero — do this first

Add a standard, brief disclosure line to the report template for any result involving AI-assisted interpretation.

The most common mistake: AI involvement tracked internally for quality purposes but never actually reaching the report the clinician sees.

Self-assessment questions

1. Does the report genuinely indicate AI involvement? — A real, standard disclosure on the report.
Evidence: Report template

2. Is this disclosure genuinely clear enough for the clinician to pass to the patient? — A real, usable note.
Evidence: Report wording sample

3. Can an ordering clinician confirm they genuinely noticed this on a recent report? — A real, concrete confirmation.
Evidence: Clinician interview

Common reasons for a PARTIAL answer

A disclosure line exists in the template but clinicians genuinely don’t notice it among other report text.

Implementation plan

When	What
Week 1-2	Add a standard AI-disclosure line to the report template for any AI-assisted result.

How the Monitor verifies this

Method	What	Detail
ASK	Clinician interview	Asks an ordering clinician whether they noticed AI disclosure on a recent report.

Supervisor tips

Ask to see a real, recent report with AI involvement and check the disclosure is genuinely visible, not buried.

Evidence base

World Health Organization. Ethics and Governance of Artificial Intelligence for Health: Guidance on Large Multi-Modal Models. Geneva: WHO; 2024.

ASF training courses on GMJ Academy →

Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.

← Laboratory Standards — all standards
← Standard 11 — Sustainable Care
Standard 13 — Supporting the Care Workforce →