ASF Standards · Hospital · Standard 19
Standard 19 — Digital Care and Artificial Intelligence Systems for Care
Criteria in this standard
19.1 — Digital Care Systems Follow a Genuine Assessment and Management Process
19.2 — Digital Care Never Disadvantages a Patient Who Cannot Use It
19.3 — Genuine Technical Expertise Supports Digital Care Systems
19.4 — AI Systems Are Introduced in Line With Law and Best-Practice Guidance
19.5 — AI Systems Are Genuinely Monitored for Unintended Consequences
19.6 — Staff Are Genuinely Consulted Before an AI System Is Introduced
19.7 — Accountability for AI-Assisted Care Is Explicitly Defined
19.8 — Patients Are Genuinely Informed When Their Care Involves AI
19.2 — Digital Care Never Disadvantages a Patient Who Cannot Use It
19.3 — Genuine Technical Expertise Supports Digital Care Systems
19.4 — AI Systems Are Introduced in Line With Law and Best-Practice Guidance
19.5 — AI Systems Are Genuinely Monitored for Unintended Consequences
19.6 — Staff Are Genuinely Consulted Before an AI System Is Introduced
19.7 — Accountability for AI-Assisted Care Is Explicitly Defined
19.8 — Patients Are Genuinely Informed When Their Care Involves AI
19.1
Digital Care Systems Follow a Genuine Assessment and Management Process
Standard
Before any digital care system — telehealth platform, remote monitoring tool, patient-facing app — is introduced, the hospital genuinely assesses its costs and benefits, manages its implementation, and continues to manage it afterward — not adopted because a vendor made a compelling pitch, with no real internal evaluation.
In plain terms: Before the hospital adopts a new digital tool, someone has actually weighed up whether it’s worth it, checked it will work with what the hospital already has, and thought through what could go wrong — not bought it because the demo looked impressive.
| Facility category | Crisis | Transition | Small | Standard |
|---|---|---|---|---|
| Applicability | Adapted | Full | Full | Full |
Why this matters
Digital care systems are often introduced through enthusiasm, vendor pressure, or a genuine desire to modernise — all understandable, none of them a substitute for real evaluation. A hospital that adopts a new telehealth platform or remote monitoring tool without checking whether it actually integrates with existing systems, without weighing its real costs against its real benefits, and without considering what could go wrong, risks ending up with a system that creates more problems than it solves: data trapped in a system that doesn’t talk to the patient record, clinical staff working around a tool rather than with it, or patient safety risks nobody anticipated because nobody looked.
What good looks like
- A genuine cost/benefit analysis happens before implementation, not after.
- Interoperability with existing systems is genuinely assessed and risks mitigated.
- Potential unintended consequences are considered proactively, not discovered in hindsight.
Common failure modes
- A system is adopted based on a vendor demonstration with no independent evaluation.
- Interoperability problems only surface after the system is already in clinical use.
- Unintended consequences are never considered until they actually occur.
Worked example
In practice
A hospital considering a new remote patient monitoring platform.
BeforeA vendor presented a remote monitoring platform at a conference. The CMO was impressed and initiated procurement directly, with no formal evaluation of cost, benefit, or compatibility with the hospital’s existing electronic health record.
ActionA structured evaluation was introduced: IT assessed interoperability with the existing EHR, finance modelled real cost against projected benefit in reduced readmissions, and clinical staff were asked to identify likely unintended consequences, such as alert fatigue from excessive monitoring notifications.
AfterThe Monitor reviewed the completed evaluation document, including the interoperability assessment and the identified alert-fatigue risk with its mitigation plan. Verified.
If you are starting from zero — do this first
- Build a simple evaluation template covering cost, benefit, interoperability, and risk.
- Require this template be completed before any digital system procurement proceeds.
- Involve both IT and the clinical staff who will actually use the system in the evaluation.
The most common mistake: Letting enthusiasm for a new technology substitute for a genuine, structured evaluation of its costs, interoperability, and risks.
Self-assessment questions
1. Is a genuine cost/benefit analysis conducted before any digital care system is implemented, not adopted on vendor assurance alone? — A real, documented evaluation, not a decision made purely on a sales pitch.
Evidence: Digital system evaluation documentation
Evidence: Digital system evaluation documentation
2. Is interoperability between the new system and existing hospital systems genuinely assessed, with risks from non-interoperability actually mitigated? — Real risk assessment, not an assumption that systems will simply work together.
Evidence: Interoperability assessment
Evidence: Interoperability assessment
3. Are potential unintended consequences of introducing the system genuinely considered before implementation, not discovered only afterward? — Real, proactive consideration, not a system deployed and its effects observed only in hindsight.
Evidence: Pre-implementation risk review
Evidence: Pre-implementation risk review
Common reasons for a PARTIAL answer
- A cost/benefit analysis exists but interoperability was never formally checked. — Financial evaluation alone doesn’t confirm the system will actually integrate safely.
- Evaluation happens for major systems but is skipped for smaller, seemingly low-risk tools. — A consistent process should apply regardless of how minor a new system initially seems.
- Unintended consequences are discussed informally but never actually documented.
Implementation plan
| When | What |
|---|---|
| Week 1 | Build a structured evaluation template for digital system procurement. |
| Week 2 | Require the template’s completion as a formal procurement gate. |
| Ongoing | Apply the process consistently to every new digital care system, regardless of scale. |
How the Monitor verifies this
| Method | What | Detail |
|---|---|---|
| DOCUMENT | Evaluation document review | Reviews a recent digital system evaluation for genuine cost/benefit and interoperability analysis. |
| ASK | IT/clinical interview | Asks whether unintended consequences were genuinely considered before a recent system’s launch. |
Supervisor tips
- Ask to see the evaluation for the most recently introduced digital care system. — A real, recent example is the clearest evidence this process genuinely functions.
- Ask IT staff directly whether interoperability was checked before go-live, not after. — Timing matters — a check performed after deployment doesn’t meet the intent.
Evidence base
World Health Organization. Global Strategy on Digital Health 2020-2025. Geneva: WHO; 2021.
ASF training courses on GMJ Academy →
Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.
19.2
Digital Care Never Disadvantages a Patient Who Cannot Use It
Core
A genuine process ensures that patients who cannot use digital devices, lack internet access, or are otherwise unable to engage with digital approaches are never disadvantaged by the hospital’s use of digital care — not digital convenience for most patients purchased at the cost of real access for the patients least able to adapt.
In plain terms: A patient who can’t use a smartphone or doesn’t have internet access still gets exactly the same quality of care as everyone else — a real, working alternative, not a second-class pathway nobody actually tested.
| Facility category | Crisis | Transition | Small | Standard |
|---|---|---|---|---|
| Applicability | Adapted | Full | Full | Full |
Why this matters
Digital care genuinely improves access and convenience for many patients — but the patients least likely to benefit are often exactly the patients most vulnerable to begin with: older people, those with lower digital literacy, those in poverty without reliable internet access, people with disabilities that make digital interfaces harder to use. This is marked Core because the risk here isn’t inconvenience — it’s a genuine equity failure where a hospital’s digital modernisation effort quietly degrades care for precisely the patients who can least absorb that degradation. A real, tested, equally functional alternative isn’t optional; it’s what makes digital care an addition to patient access rather than a subtraction from it.
What good looks like
- A genuine, defined alternative exists for patients who cannot use a digital approach.
- New digital services are genuinely pre-tested with representatives of affected patient groups.
- Real evidence shows patients using the alternative received equivalent, undiminished service.
Common failure modes
- A digital-only pathway exists with no real, functioning alternative for patients who can’t use it.
- A new digital service launches without ever being tested with patients who might struggle.
- An alternative exists on paper but patients using it experience a genuinely worse service.
Worked example
In practice
A hospital that moved appointment booking to an app-only system.
BeforeOutpatient appointment booking moved to a smartphone app, with the old phone-booking line quietly deprioritised and often leaving callers on hold for 20 minutes or more. Older patients and those without smartphones found booking appointments had become significantly harder.
ActionThe phone-booking line was restored to equal staffing priority alongside the app, with wait times actively monitored to confirm genuine parity. The app rollout for future digital services was changed to include pre-testing with a panel of older patients before launch.
AfterThe Monitor called the phone-booking line directly and reached a human within two minutes, matching the app’s typical booking time, and reviewed pre-testing documentation for the hospital’s most recent digital service launch. Verified.
If you are starting from zero — do this first
- Identify every current digital-only patient-facing process and check for a real alternative.
- Test the alternative yourself — call it, try it — to confirm it’s genuinely equivalent, not degraded.
- Build pre-testing with affected patient groups into how future digital services are launched.
The most common mistake: Assuming an old, non-digital process still functions as a real alternative without ever actually testing whether it’s been quietly left to degrade.
Self-assessment questions
1. Is there a genuine, defined alternative for patients who cannot or do not want to use a digital care approach? — A real, equally functional alternative, not a digital-only pathway with no real fallback.
Evidence: Documented alternative access pathway
Evidence: Documented alternative access pathway
2. Was the introduction of a new digital care service genuinely pre-tested with representatives of patients who might struggle to use it? — Real, prior testing with the affected group, not an assumption that accessibility will work out.
Evidence: Pre-launch testing documentation
Evidence: Pre-launch testing documentation
3. Is there evidence that a patient unable to use digital care actually received equivalent, undiminished service through the alternative route? — A real, demonstrated instance, not a theoretical alternative that has never actually been tested in practice.
Evidence: Service comparison data or direct test
Evidence: Service comparison data or direct test
Common reasons for a PARTIAL answer
- An alternative exists but is genuinely slower or lower-quality than the digital route. — Nominal equivalence isn’t the same as a service that’s actually equally good.
- Pre-testing happened for one digital service but isn’t a consistent practice. — Genuine protection against disadvantage should apply to every new digital service, not just one.
- The alternative is known to staff but not actively communicated to patients who need it.
Implementation plan
| When | What |
|---|---|
| Week 1 | Audit every current digital-only patient process for a genuine alternative. |
| Week 2 | Test each alternative directly to confirm real equivalence, not just nominal existence. |
| Ongoing | Require pre-testing with affected patient groups for every new digital service. |
How the Monitor verifies this
| Method | What | Detail |
|---|---|---|
| ASK | Direct alternative test | Personally tests the non-digital alternative pathway for a genuine, equivalent experience. |
| DOCUMENT | Pre-testing review | Reviews pre-launch testing records for recent digital service introductions. |
Supervisor tips
- Personally try the non-digital alternative, not just confirm it exists on paper. — Direct testing reveals whether an alternative is genuinely equivalent or quietly degraded.
- Ask frontline staff whether patients ever complain about struggling with a digital-only process. — Real staff experience often surfaces equity gaps before they appear in any formal record.
Evidence base
World Health Organization. Ethics and Governance of Artificial Intelligence for Health. Geneva: WHO; 2021.
ASF training courses on GMJ Academy →
Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.
19.3
Genuine Technical Expertise Supports Digital Care Systems
Standard
The hospital has genuine access to technical expertise — in-house, vendor-provided, or contracted — to support the effective use of its digital care systems, including testing and quality checks before full implementation, not digital systems left running with no real technical support behind them.
In plain terms: When a digital system breaks or a clinician needs help using it properly, there’s a real person to call — not a system left to run itself with nobody actually responsible for keeping it working.
| Facility category | Crisis | Transition | Small | Standard |
|---|---|---|---|---|
| Applicability | Adapted | Full | Full | Full |
Why this matters
A digital care system is only as reliable as the support behind it. Software updates, integration issues, unexpected errors, and the ordinary friction of real-world clinical use all require genuine technical capacity to resolve — and a hospital that adopts digital systems without securing that capacity first is setting staff up to work around problems rather than have them properly fixed. This isn’t about having a large in-house IT department; a small hospital can meet this through a solid vendor support contract or a shared regional technical resource, as long as that support is genuinely accessible when actually needed.
What good looks like
- Genuine, accessible technical expertise exists for every digital care system in use.
- Testing and quality checks genuinely occur before full implementation, not after.
- A real, known escalation path exists when a digital system issue arises.
Common failure modes
- A digital system is in use with no clear technical support arrangement at all.
- Testing happens informally, if at all, with issues discovered only after patients are affected.
- Staff don’t know who to contact when a digital system malfunctions.
Worked example
In practice
A small hospital running a telehealth platform with no formal support arrangement.
BeforeThe telehealth platform had been set up by a staff member who has since left. When it malfunctioned, nobody currently on staff knew who to contact, and clinical sessions were cancelled for two days while someone tracked down the original vendor contact.
ActionA formal support contract was signed with the platform vendor, including a defined response-time guarantee. A named internal staff member was designated as the primary point of contact, with vendor support details posted visibly for all clinical staff using the system.
AfterThe Monitor confirmed the support contract existed with a genuine response-time commitment, and interviewed a clinical staff member who could correctly describe who to contact for a technical issue. Verified.
If you are starting from zero — do this first
- List every digital care system currently in use and confirm whether each has a real support arrangement.
- For any gap, secure a vendor support contract or designate an internal technical lead.
- Make the support contact genuinely visible and known to the staff who actually use each system.
The most common mistake: Technical support knowledge living entirely with one staff member, with no documented arrangement that survives their departure.
Self-assessment questions
1. Does the hospital have genuine, accessible technical expertise — in-house or contracted — for its digital care systems? — Real, available support, not systems left to run unsupported once installed.
Evidence: Technical support contract or arrangement
Evidence: Technical support contract or arrangement
2. Were new testing and quality checks genuinely performed before full implementation of a digital care system? — A real pre-launch verification process, not a system deployed directly to patients untested.
Evidence: Pre-implementation testing records
Evidence: Pre-implementation testing records
3. When a digital system issue arises, is there a real, known route to technical support, not staff left to troubleshoot alone? — A genuine, known escalation path, not an assumption that someone will eventually sort it out.
Evidence: Staff interview, posted support contact
Evidence: Staff interview, posted support contact
Common reasons for a PARTIAL answer
- Support exists for major systems but not for smaller, department-level digital tools. — Genuine coverage should extend to every digital care system in clinical use.
- A support contract exists but staff don’t actually know it exists or how to use it. — Support that nobody knows how to access doesn’t function in practice.
- Testing occurred before initial launch but not for subsequent updates or changes.
Implementation plan
| When | What |
|---|---|
| Week 1 | Inventory all digital care systems and their current support status. |
| Week 2-3 | Secure or formalize support arrangements for any system lacking one. |
| Week 4 | Communicate support contacts clearly to all relevant staff. |
How the Monitor verifies this
| Method | What | Detail |
|---|---|---|
| DOCUMENT | Support arrangement review | Reviews the technical support contract or arrangement for each digital system. |
| ASK | Staff interview | Asks a staff member who they would contact for a digital system issue. |
Supervisor tips
- Ask a frontline clinical user, not IT staff, who they’d call if the system failed mid-shift. — The end user’s actual knowledge is the real test of whether support is genuinely accessible.
- Check whether support knowledge depends entirely on one individual staff member. — A single point of failure in support knowledge is a real, specific risk worth probing.
Evidence base
NHS Digital. Technology Assessment Criteria for Digital Health Products. London: NHS Digital; 2022.
ASF training courses on GMJ Academy →
Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.
19.4
AI Systems Are Introduced in Line With Law and Best-Practice Guidance
Standard
Any artificial intelligence system used to support diagnosis, triage, or care decisions is introduced and managed in accordance with applicable national or regional law and regulation, or in their absence, available best-practice guidance such as WHO’s ethics and governance guidance for AI — not adopted with no reference to any external standard of responsible use.
In plain terms: Before using an AI tool in patient care, the hospital has actually checked what the law requires, or if there’s no specific law, has grounded its approach in real published guidance — not made it up as it went along.
| Facility category | Crisis | Transition | Small | Standard |
|---|---|---|---|---|
| Applicability | Adapted | Full | Full | Full |
Why this matters
AI regulation in healthcare is evolving rapidly and unevenly across jurisdictions — some regions have specific, binding requirements; many do not yet. In the absence of binding law, a hospital is not free to simply do whatever it wants with clinical AI; credible, published guidance exists from bodies like WHO, the US FDA, and national health systems, and grounding AI governance in one of these sources is what distinguishes responsible adoption from an ungoverned experiment run on real patients. This is about having an actual, citable basis for how AI is governed in this hospital, not a vague sense that AI is probably being used responsibly.
What good looks like
- AI systems are genuinely introduced in line with applicable law, where it exists.
- Where no specific law exists, introduction is genuinely informed by recognized best-practice guidance.
- The hospital can identify the specific regulatory or guidance basis for a given AI system.
Common failure modes
- No reference to law or regulation has been made at all before adopting an AI tool.
- Guidance is vaguely cited without the hospital actually having reviewed or applied it.
- Nobody can say, when asked, what the governance basis for a given AI system actually is.
Worked example
In practice
A hospital that adopted an AI-assisted radiology triage tool with no formal governance review.
BeforeThe radiology department adopted an AI triage tool to flag urgent imaging findings, based on a colleague’s recommendation at another facility. No one checked national regulatory requirements or reviewed any published governance guidance before go-live.
ActionA governance review was retrospectively conducted, confirming the jurisdiction had no AI-specific medical device regulation yet in force, and formally adopting WHO’s AI ethics and governance guidance as the hospital’s reference framework, with a documented checklist applied to this and future AI tools.
AfterThe Monitor reviewed the governance checklist and confirmed the radiology lead could specifically name WHO’s guidance as the basis for the tool’s oversight. Verified.
If you are starting from zero — do this first
- Check whether any national or regional AI-specific healthcare regulation currently applies.
- Where none exists, formally adopt a recognized guidance source — WHO, FDA, or similar — as the hospital’s reference.
- Apply this basis as a documented checklist to every AI system currently in use.
The most common mistake: Assuming AI tools fall under general IT or medical device policy with no AI-specific governance basis actually identified or applied.
Self-assessment questions
1. Is any AI system in clinical use genuinely introduced in line with applicable law or regulation on AI, where this exists? — Real, verified compliance, not an assumption that general IT policy already covers AI-specific requirements.
Evidence: Regulatory compliance review
Evidence: Regulatory compliance review
2. Where no specific legislation exists, is the AI system’s introduction genuinely informed by recognized best-practice guidance, such as WHO’s AI ethics and governance guidance? — A real, referenced guidance source, not an ad hoc decision with no external grounding.
Evidence: Documented guidance reference
Evidence: Documented guidance reference
3. Can the hospital identify, when asked, the specific regulatory or guidance basis for how a given AI system is governed? — A real, specific answer, not a vague assurance that AI use is “handled appropriately.”
Evidence: Staff interview
Evidence: Staff interview
Common reasons for a PARTIAL answer
- Guidance was reviewed once at adoption but hasn’t been revisited as AI regulation evolves. — This is a fast-moving regulatory area; periodic review matters.
- A guidance source is named but hasn’t actually been applied to this hospital’s specific AI tools. — Citing a source isn’t the same as genuinely working through its application here.
- Different departments have adopted different, uncoordinated governance approaches.
Implementation plan
| When | What |
|---|---|
| Week 1 | Research applicable AI-specific regulation in the hospital’s jurisdiction. |
| Week 2 | Formally adopt a recognized guidance source if no binding regulation exists. |
| Week 3 | Apply this basis as a checklist to every current AI system in clinical use. |
How the Monitor verifies this
| Method | What | Detail |
|---|---|---|
| DOCUMENT | Governance basis review | Reviews the documented regulatory or guidance basis for AI governance. |
| ASK | Staff interview | Asks the department using an AI tool to name its governance basis specifically. |
Supervisor tips
- Ask for the name of the specific law or guidance document governing a given AI tool. — A vague answer reveals the governance basis isn’t genuinely established, just assumed.
- Check whether governance was reviewed at adoption only, or is genuinely kept current. — AI regulation changes quickly; a static, one-time review may already be outdated.
Evidence base
World Health Organization. Ethics and Governance of Artificial Intelligence for Health: Guidance on Large Multi-Modal Models. Geneva: WHO; 2024.
ASF training courses on GMJ Academy →
Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.
19.5
AI Systems Are Genuinely Monitored for Unintended Consequences
Standard
The hospital genuinely monitors and evaluates its AI systems — auditing results for over-diagnosis or missed diagnosis, tracking patient complaints related to AI-assisted care, and checking for other unintended consequences — not deploying an AI tool and assuming it works as intended with no real, ongoing scrutiny.
In plain terms: The hospital actually keeps checking whether its AI tools are working as expected — catching real problems like missed diagnoses or bias — not deploying a tool once and simply trusting it from then on.
| Facility category | Crisis | Transition | Small | Standard |
|---|---|---|---|---|
| Applicability | Adapted | Full | Full | Full |
Why this matters
AI systems can fail in ways that aren’t obvious at the point of use — a diagnostic tool trained on a different population than the one it’s now serving can systematically miss certain presentations or over-flag others, and these patterns often only become visible through deliberate, ongoing monitoring rather than any single clinical encounter. A hospital that treats AI output with the same unquestioning trust as a simple calculator is missing the genuine, documented risk that these systems carry: they can be confidently, consistently wrong in ways a single clinician reviewing a single case won’t necessarily catch.
What good looks like
- AI output is genuinely, regularly audited for over-diagnosis or missed diagnosis.
- Patient complaints related to AI-assisted care are genuinely tracked and reviewed.
- A documented instance exists of monitoring catching a real issue, with a genuine response.
Common failure modes
- AI output is trusted with no ongoing audit of its actual accuracy.
- Patient concerns about AI-assisted care are absorbed into general complaints with no distinct tracking.
- No monitoring process has ever actually caught or flagged an issue.
Worked example
In practice
A hospital using an AI tool to flag abnormal lab results for urgent review.
BeforeThe AI flagging tool had been running for a year with no formal audit of its accuracy. Nobody had checked whether it was correctly flagging genuinely abnormal results or missing anything it should have caught.
ActionA quarterly audit was introduced, comparing a sample of AI-flagged and non-flagged results against independent clinical review. The audit identified a pattern of missed flags for a specific result range, which was reported to the vendor and addressed in a subsequent system update.
AfterThe Monitor reviewed the quarterly audit reports, including the documented missed-flag issue and the vendor’s resulting fix. Verified.
If you are starting from zero — do this first
- Identify every AI system currently influencing clinical decisions.
- Establish a periodic audit comparing AI output against independent clinical review for each.
- Create a distinct channel for tracking patient concerns specifically related to AI-assisted care.
The most common mistake: Treating AI output as reliably correct by default, with no ongoing audit process ever established to actually test that assumption.
Self-assessment questions
1. Is the output of AI-assisted diagnosis or care decisions genuinely audited for over-diagnosis or missed diagnosis, not assumed reliable without checking? — Real, ongoing audit, not an assumption that AI accuracy requires no verification.
Evidence: AI output audit records
Evidence: AI output audit records
2. Are patient complaints or concerns related to AI-assisted care genuinely tracked and reviewed? — A real, specific tracking mechanism, not AI-related concerns absorbed into general feedback with no distinct visibility.
Evidence: AI-specific complaint tracking log
Evidence: AI-specific complaint tracking log
3. Is there a documented instance where monitoring identified an issue with an AI system’s performance, and a genuine response followed? — A real, concrete example, not a monitoring process that has never actually caught anything in practice.
Evidence: Issue identification and response record
Evidence: Issue identification and response record
Common reasons for a PARTIAL answer
- Auditing happens but at too infrequent an interval to genuinely catch emerging patterns. — Audit frequency should be genuinely matched to the clinical stakes of the AI system.
- Monitoring exists but has never been genuinely independent of the AI system’s own output. — A real audit requires comparison against independent clinical judgment, not the system checking itself.
- No issue has ever been identified, raising the question of whether monitoring is genuinely rigorous.
Implementation plan
| When | What |
|---|---|
| Week 1 | Identify every AI system currently influencing clinical decisions. |
| Week 2-3 | Establish a periodic, independent audit process for each system. |
| Week 4 | Create a distinct tracking mechanism for AI-related patient concerns. |
| Ongoing | Conduct audits on a genuine, defined schedule matched to clinical risk. |
How the Monitor verifies this
| Method | What | Detail |
|---|---|---|
| DOCUMENT | Audit record review | Reviews periodic AI output audits for genuine, independent comparison. |
| DOCUMENT | Issue response review | Looks for a real, documented instance of monitoring catching and addressing an issue. |
Supervisor tips
- Ask to see the most recent AI audit and whether it found anything worth flagging. — A genuinely rigorous audit occasionally finds something; one that never does merits scrutiny.
- Confirm audits are conducted independently of the AI vendor, not solely by the vendor itself. — Independent review is what gives the audit genuine credibility.
Evidence base
US Food and Drug Administration. Artificial Intelligence and Machine Learning in Software as a Medical Device. Silver Spring: FDA; 2023.
ASF training courses on GMJ Academy →
Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.
19.6
Staff Are Genuinely Consulted Before an AI System Is Introduced
Standard
Staff who will actually deliver care using a new AI system are genuinely consulted before it is introduced, to understand the practical implications and identify real training needs — not an AI tool rolled out to clinical staff with no engagement beyond a brief notification that it now exists.
In plain terms: Before a new AI tool goes live, the staff who will actually use it have had a real say — not just been told it’s coming, with their actual concerns and training needs never genuinely asked about.
| Facility category | Crisis | Transition | Small | Standard |
|---|---|---|---|---|
| Applicability | Adapted | Full | Full | Full |
Why this matters
AI systems are genuinely disruptive to established clinical workflows in ways that are often invisible from outside the workflow itself — the staff actually doing the work know where friction will arise, what training will genuinely be needed, and what practical problems a system’s designers never anticipated. A hospital that introduces an AI tool through top-down announcement rather than real consultation loses this knowledge entirely, and frequently ends up with a tool staff quietly work around rather than genuinely adopt — undermining the very benefit the system was meant to deliver.
What good looks like
- Staff are genuinely consulted before, not after, an AI system’s introduction is finalized.
- Consultation genuinely identifies training needs, which are then actually delivered.
- Staff asked directly can describe having been genuinely consulted.
Common failure modes
- Staff learn about a new AI system only once the decision to adopt it is already final.
- Training needs are identified but never actually addressed before go-live.
- Staff describe the AI system as something that was simply imposed on them.
Worked example
In practice
A hospital introducing an AI-assisted clinical documentation tool.
BeforePhysicians were informed by email that a new AI documentation assistant would go live the following month, with a link to a vendor-produced video. No consultation occurred, and when the tool launched, physicians found it didn’t fit how they actually structured clinical notes, leading many to simply stop using it.
ActionA relaunch was planned with genuine physician consultation first — focus groups to understand actual documentation workflows, identifying specific friction points the tool needed to address, and physician-led training sessions rather than a vendor video.
AfterThe Monitor interviewed physicians who could describe having been genuinely consulted before the relaunch, with specific examples of changes made based on their input. Verified.
If you are starting from zero — do this first
- Before finalizing any AI system decision, hold a genuine consultation session with the staff who will use it.
- Ask directly what training they believe they’ll need, not assume this centrally.
- Deliver that training before go-live, not as an afterthought once problems emerge.
The most common mistake: Treating a notification email or vendor demo video as equivalent to genuine staff consultation.
Self-assessment questions
1. Are staff who will actually use a new AI system genuinely consulted before its introduction, not informed only after the decision is finalized? — Real, prior consultation, not a rollout announcement to staff with no genuine input sought.
Evidence: Consultation session records
Evidence: Consultation session records
2. Does this consultation genuinely identify practical training needs, which are then actually addressed before go-live? — Real, identified needs matched by real, delivered training, not a consultation that produces no follow-through.
Evidence: Training needs assessment and delivery record
Evidence: Training needs assessment and delivery record
3. Can staff asked directly describe having been genuinely consulted about a recent AI system introduction? — Tests whether consultation actually reached and registered with staff, not just whether it technically occurred.
Evidence: Staff interview
Evidence: Staff interview
Common reasons for a PARTIAL answer
- Consultation happened with department leads but not the staff actually doing the work. — Genuine consultation should reach the actual end users, not only their supervisors.
- Training needs were identified but delivery was rushed or incomplete before go-live. — Identification without genuine, adequate delivery leaves staff underprepared regardless.
- Consultation occurred for one AI system but isn’t yet a consistent practice.
Implementation plan
| When | What |
|---|---|
| Before any launch | Hold genuine consultation sessions with the actual staff who will use the system. |
| Following consultation | Document identified training needs and build a real delivery plan. |
| Before go-live | Complete training delivery, confirmed by staff feedback, not assumed sufficient. |
How the Monitor verifies this
| Method | What | Detail |
|---|---|---|
| DOCUMENT | Consultation record review | Reviews records of genuine staff consultation before a recent AI system’s introduction. |
| ASK | Staff interview | Asks frontline staff directly whether they were genuinely consulted and how. |
Supervisor tips
- Ask a frontline user, not a department head, whether they felt genuinely consulted. — The actual end user’s experience is the real test of whether consultation reached them.
- Ask for a specific example of something changed because of staff input. — A concrete example distinguishes genuine consultation from a box-ticking exercise.
Evidence base
Topol EJ. High-Performance Medicine: The Convergence of Human and Artificial Intelligence. Nature Medicine. 2019;25(1):44-56.
ASF training courses on GMJ Academy →
Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.
19.7
Accountability for AI-Assisted Care Is Explicitly Defined
Standard
The hospital has genuinely considered and documented accountability arrangements for care and treatment delivered with AI support — who is responsible when an AI-assisted decision is wrong — not leaving this as an unexamined question until the first time it actually matters.
In plain terms: Everyone knows, in advance, who is actually responsible if an AI-supported decision turns out to be wrong — not a question nobody has thought through until it comes up for real, with a patient already affected.
| Facility category | Crisis | Transition | Small | Standard |
|---|---|---|---|---|
| Applicability | Adapted | Full | Full | Full |
Why this matters
AI-assisted decisions sit in genuinely new territory for clinical accountability — a tool contributed to a decision, but a human clinician remains responsible for the care actually delivered, and that relationship needs to be explicit rather than assumed. Without clear, documented accountability, a safety incident involving AI-assisted care risks becoming a confused dispute about where responsibility actually lay, exactly when clarity matters most — for the affected patient, for the clinician involved, and for the hospital’s own ability to learn genuinely from what happened.
What good looks like
- Clinical accountability for AI-supported decisions is explicitly, clearly documented.
- Clinicians using AI-supported tools genuinely understand their own accountability.
- A defined process exists for reviewing accountability when an incident involves AI-assisted care.
Common failure modes
- Accountability for AI-assisted decisions has never been explicitly addressed or documented.
- Clinicians assume the AI system itself bears some responsibility, diffusing genuine accountability.
- No defined process exists for reviewing an incident where AI contributed to the decision.
Worked example
In practice
A hospital using an AI-assisted sepsis prediction tool with no documented accountability policy.
BeforeAn AI tool flagged sepsis risk scores to nursing staff, who were expected to escalate high-risk patients. No policy explicitly stated whether responsibility for a missed escalation sat with the nurse, the ordering physician, or was assumed to be shared with the AI system itself.
ActionA clear accountability policy was documented: the AI tool is a decision-support input only, and the clinician reviewing the flag retains full clinical accountability for the escalation decision, with this explicitly taught in onboarding for the tool and confirmed through a staff sign-off.
AfterThe Monitor reviewed the documented policy and interviewed a nurse who could clearly state her own accountability for escalation decisions, independent of the AI flag. Verified.
If you are starting from zero — do this first
- For each AI system in clinical use, explicitly document where clinical accountability sits.
- Communicate this clearly to the staff who use the system, not leave it implicit.
- Build a specific step into incident review processes for events involving AI-assisted care.
The most common mistake: Leaving accountability for AI-assisted decisions genuinely ambiguous, assuming it will become clear if and when it’s ever actually tested by a real incident.
Self-assessment questions
1. Is clinical accountability for a decision made with AI support explicitly documented, not left genuinely ambiguous between clinician and system? — A real, clear answer, not an assumption that responsibility will sort itself out if something goes wrong.
Evidence: Documented accountability policy
Evidence: Documented accountability policy
2. Do clinicians using AI-supported tools genuinely understand their own accountability for the final care decision? — Real, demonstrated staff understanding, not a policy nobody using the tool has actually internalized.
Evidence: Staff interview, training sign-off
Evidence: Staff interview, training sign-off
3. Is there a defined process for reviewing accountability when an AI-assisted decision contributes to a safety incident? — A real, usable process, not a question left entirely open until an actual incident forces it.
Evidence: Incident review protocol
Evidence: Incident review protocol
Common reasons for a PARTIAL answer
- A policy exists but hasn’t actually been communicated to or understood by frontline staff. — A documented policy nobody using the tool actually knows about doesn’t meet the real intent.
- Accountability is clear for one AI system but undefined for others in use. — Genuine coverage should extend consistently to every AI system influencing care.
- General incident review exists but has no specific step addressing AI involvement.
Implementation plan
| When | What |
|---|---|
| Week 1 | Document explicit accountability for each AI system currently in clinical use. |
| Week 2 | Communicate this clearly to all relevant staff, with sign-off confirmation. |
| Week 3 | Add an AI-specific step to the hospital’s incident review protocol. |
How the Monitor verifies this
| Method | What | Detail |
|---|---|---|
| DOCUMENT | Accountability policy review | Reviews the documented accountability policy for AI-assisted care. |
| ASK | Staff interview | Asks a clinician to describe their own accountability when using an AI-supported tool. |
Supervisor tips
- Ask a clinician directly: if the AI flagged something wrong, who is responsible? — A confident, specific answer reveals genuine understanding, not assumed clarity.
- Check whether the incident review process has an actual AI-specific step, not a generic one. — A specific step signals the hospital has genuinely thought this through in advance.
Evidence base
European Commission. Ethics Guidelines for Trustworthy Artificial Intelligence. Brussels: European Commission; 2019.
ASF training courses on GMJ Academy →
Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.
19.8
Patients Are Genuinely Informed When Their Care Involves AI
Standard
A patient whose care involves an aspect delivered with AI system support is genuinely informed of this — not left to assume every decision was made by a human clinician alone, with disclosure treated as optional rather than a genuine, standard part of informed care.
In plain terms: If AI is actually involved in part of a patient’s care, they’re genuinely told — not left to assume a human made every decision alone.
| Facility category | Crisis | Transition | Small | Standard |
|---|---|---|---|---|
| Applicability | Adapted | Full | Full | Full |
Why this matters
A patient’s genuine right to understand their own care includes knowing, in plain terms, when a decision affecting them was shaped by an AI system rather than a clinician’s judgement alone. This isn’t a formality — a patient who genuinely knows AI was involved can ask informed questions, seek a second opinion with that context, or simply understand their care more completely. Disclosure buried in a general consent form a patient never actually reads does not meet this bar.
What good looks like
- Patients are genuinely informed when AI is involved in their care.
- Disclosure is genuinely understandable, in plain language.
- A patient can genuinely confirm they were told.
Common failure modes
- AI disclosure is buried in a general consent form, technically present but never genuinely read or understood.
Worked example
In practice
A hospital using an AI-assisted imaging triage tool.
BeforeAI’s role was mentioned only in a lengthy general consent document, which patients signed without genuine awareness that an AI tool had actually been involved in their specific case.
ActionA brief, plain-language note was added to the patient’s own results discussion: “An AI-assisted tool helped flag areas for the radiologist to review closely.”
AfterThe Monitor interviewed a patient who could genuinely confirm they understood AI had been involved. Verified.
If you are starting from zero — do this first
- Add a brief, plain-language disclosure point at the moment results or care decisions involving AI are actually discussed with the patient.
The most common mistake: Treating a line buried in a general consent form as genuine disclosure, when the patient never actually registers it.
Self-assessment questions
1. Is a patient genuinely informed when AI is involved in their care? — Real, standard disclosure.
Evidence: Disclosure protocol
Evidence: Disclosure protocol
2. Is this disclosure genuinely understandable, not buried in technical language? — A real, plain-language explanation.
Evidence: Disclosure wording sample
Evidence: Disclosure wording sample
3. Can a patient asked directly confirm they were genuinely told? — A real, concrete confirmation.
Evidence: Patient interview
Evidence: Patient interview
Common reasons for a PARTIAL answer
- Disclosure exists in a consent form but patients genuinely cannot recall or explain it when asked directly.
Implementation plan
| When | What |
|---|---|
| Week 1-2 | Draft plain-language disclosure wording for each AI-assisted care point and train staff to deliver it. |
How the Monitor verifies this
| Method | What | Detail |
|---|---|---|
| ASK | Patient interview | Asks a patient whether they were genuinely told AI was involved in their care. |
Supervisor tips
- Ask a patient directly rather than relying on the consent form’s existence alone.
Evidence base
World Health Organization. Ethics and Governance of Artificial Intelligence for Health: Guidance on Large Multi-Modal Models. Geneva: WHO; 2024.
ASF training courses on GMJ Academy →
Foundation courses A-00 to A-03 are live. Criterion-specific modules are being developed and will link here when published.