1. Orientation and Learning Outcomes
AI is not a disease, so this chapter adapts the usual clinical sequence. "Pathophysiology" becomes how AI works and fails. "Presentation" becomes where AI appears in physiotherapy. "Assessment" becomes how to appraise a tool. "Management" becomes safe, staged implementation.
By the end you should be able to:
- Define AI, machine learning (ML), deep learning, and large language models (LLMs).
- Interpret sensitivity, specificity, PPV, and AUC for an AI tool.
- Identify common failure modes, including bias and dataset shift.
- Apply a staged, safety-first approach to adopting AI in a clinical setting.
2. Core Concepts and Terminology (the "Mechanism")
| Term | Working definition | Physiotherapy example |
|---|---|---|
| Artificial intelligence | Systems performing tasks that normally require human cognition | Automated exercise feedback |
| Machine learning | Algorithms that learn patterns from data rather than following fixed rules | Predicting low back pain (LBP) chronicity from baseline variables |
| Supervised learning | Learns from labelled examples | Classifying gait as normal or pathological from labelled trials |
| Unsupervised learning | Finds structure in unlabelled data | Clustering patients into movement phenotypes |
| Deep learning | Multi-layer neural networks, strong with images, signals, and video | Pose estimation (e.g., OpenPose, MediaPipe) from smartphone video |
| Computer vision | Extracting information from images or video | Estimating knee valgus during a single-leg squat |
| Natural language processing / LLM | Models trained on text that generate or interpret language | Drafting patient education or summarising notes |
| Training / validation / test set | Data used to fit, tune, and finally evaluate a model | Test data must be unseen |
| Overfitting | Model memorises noise and performs poorly on new data | High accuracy in the development clinic, poor accuracy elsewhere |
| Explainability | Ability to understand why a model produced an output | Highlighting which inputs drove a risk score |
Most clinical AI is narrow AI: it does one task in one context. It does not "understand" the patient. It outputs a statistical estimate conditioned on its training data.
3. How AI Fails (the "Pathology")
- Data bias. Obermeyer et al. (2019) showed that a widely used US algorithm underestimated the needs of Black patients because it used healthcare cost as a proxy for need. Proxy outcomes can encode inequity.
- Dataset shift. Performance falls when the population, equipment, or workflow differs from the training data. Examples are different camera angles, older adults, or neurological populations when the model was trained on healthy young adults.
- Automation bias. Clinicians over-trust an output and under-weight contradictory findings.
- Hallucination (LLMs). Fluent but fabricated content, including invented references or dosages.
- Lack of calibration. A "70% risk" does not correspond to a true 70% event rate.
- Privacy breach. Entering identifiable data into non-compliant tools.
4. Presentation: The AI Landscape in Physiotherapy
- Movement analysis: markerless pose estimation, inertial measurement unit (IMU) wearables, and automated gait or posture metrics.
- Digital musculoskeletal (MSK) programmes and tele-rehabilitation: real-time repetition counting, range-of-motion (ROM) estimation, and adherence tracking.
- Clinical decision support: prognostic models and risk stratification, such as chronicity prediction in LBP, falls risk, and post-arthroplasty recovery trajectories.
- Imaging support: automated reading of radiographs, MRI, and ultrasound (e.g., muscle thickness segmentation).
- Administrative AI: ambient scribing, letter drafting, scheduling, and coding.
- Robotics and neuro-technology: exoskeletons, adaptive robotic gait trainers, and brain-computer interface rehabilitation.
- Patient-facing chatbots and LLMs: patients increasingly arrive with AI-generated advice that you must be able to evaluate.
5. Assessment: Appraising an AI Tool
Treat an AI tool like a diagnostic test or outcome measure. Ask what it measures, in whom, against what reference standard, and with what error.
Key metrics
- Sensitivity: proportion of true positives detected. A high value is needed for screening tools (SnNout).
- Specificity: proportion of true negatives correctly identified (SpPin).
- PPV/NPV: these depend heavily on prevalence.
- AUC (ROC): commonly interpreted as 0.5 = chance, 0.7–0.8 = acceptable, 0.8–0.9 = excellent, >0.9 = outstanding. Treat these bands as conventions, not guarantees of clinical usefulness.
- Calibration: agreement between predicted and observed risk.
- Agreement metrics for measurement tools: intraclass correlation coefficient (ICC; <0.5 poor, 0.5–0.75 moderate, 0.75–0.9 good, >0.9 excellent, per Koo & Li), Bland–Altman limits of agreement, standard error of measurement (SEM), and minimal detectable change (MDC95 = 1.96 × √2 × SEM).
Benchmarks for validating AI movement outputs. Compare against established clinical measures, for example:
- Goniometric knee flexion, where a clinically tolerable error is often cited as within about 5° (inter-rater goniometry variability is itself roughly 5–6°).
- Timed Up and Go (TUG), where ≥12 s is the CDC STEADI falls-risk threshold.
- 30-second Chair Stand and gait speed (<0.8 m/s signals elevated risk in older adults).
An AI-derived value should agree with these measures at least as well as two trained clinicians agree with each other.
6. Worked Example: Why Prevalence Matters
An AI falls-risk screen has sensitivity 90% and specificity 80%. Screen 1,000 community-dwelling adults with a 10% fall prevalence:
With 90% sensitivity and 80% specificity, a falls screen in community adults with 10% prevalence has a PPV of only 33%. On a rehabilitation ward with 40% prevalence, the same tool reaches about 75%.
- True positives = 90; false negatives = 10
- False positives = 180; true negatives = 720
- PPV = 90 / 270 = 33%; NPV = 720 / 730 ≈ 99%
Two of three "high-risk" flags are false alarms. The tool is useful for ruling out risk but must be confirmed with clinical assessment (TUG, Berg Balance Scale, orthostatic blood pressure, medication review) before committing resources. In a rehabilitation ward with 40% prevalence, the same tool has a much higher PPV (about 75%).
7. Evidence and Reporting Frameworks
Look for these when reading AI studies:
- TRIPOD+AI: reporting of prediction model development and validation.
- PROBAST: risk-of-bias assessment for prediction models.
- CONSORT-AI / SPIRIT-AI: trials of AI interventions.
- DECIDE-AI: early-stage live clinical evaluation.
Hierarchy of confidence: internal validation → external validation (different site and population) → prospective evaluation → randomised trial showing patient-relevant outcomes. Most physiotherapy AI tools currently sit at the first two levels. Accuracy against a reference does not prove that patients do better.
8. Management: Staged Implementation Protocol
The timings below are pragmatic suggestions, not evidence-based standards.
Stage 1: Define and screen (weeks 0–2). State the clinical problem and the intended users and population. Check regulatory status (e.g., FDA clearance, UK MHRA or EU MDR CE/UKCA marking as software as a medical device) and data governance (GDPR/HIPAA, data processing agreement, local storage).
Stage 2: Appraise (weeks 2–4). Check external validation, subgroup performance (age, sex, ethnicity, disability), and the reference standard. Request the vendor's intended-use statement and known limitations.
Stage 3: Silent pilot (about 4–8 weeks). Run the tool in the background while clinicians decide independently. Compare outputs against clinician judgement and track disagreements, ideally auditing a meaningful sample (e.g., ≥100 cases or all cases in a small service).
Stage 4: Supervised use. Clinician-in-the-loop with documented override rights. Record when the AI output changed the plan and why.
Stage 5: Monitor and audit (ongoing). Review performance at least quarterly for drift, adverse events, and equity. Set clear criteria for withdrawal.
Consent and communication. Tell patients when AI contributes to their care, in plain language, and document it.
9. Red Flags and Contraindications to Use
Do not use or rely on an AI output when:
- It conflicts with clinical red flags (e.g., cauda equina signs, suspected fracture, cardiovascular instability, neurological deficit). Clinical examination overrides algorithm output.
- The patient falls outside the validated population (children, amputees, severe cognitive impairment, atypical gait aids).
- The tool is unregulated but makes diagnostic or treatment claims.
- Input data quality is poor (low light, occluded joints, loose wearables).
- Identifiable patient data would be entered into a public LLM or non-compliant platform.
Don't rely on AI output when the patient is outside the validated population (children, amputees, severe cognitive impairment, atypical gait aids) or input quality is poor.
Warning signs of a poor vendor: no external validation, accuracy reported only on training data, no subgroup analysis, no named reference standard, and "black box" claims with no documentation.
10. Clinical Pearls
- Ask "compared with what?" Every AI claim needs a reference standard.
- Prevalence changes PPV. Never quote PPV without the setting.
- Verify every LLM-generated reference, dose, and contraindication against primary sources.
- Keep your own skills sharp. Continue to palpate, observe, and measure. Over-reliance degrades clinical reasoning.
- Check who is missing from the data. Older, rural, disabled, and non-English-speaking patients are frequently under-represented.
- Document your reasoning, including when you disagree with the tool.
- Use AI to free time for the therapeutic relationship, not to replace it.
11. Summary
AI in physiotherapy is a family of narrow, data-driven tools for movement analysis, prediction, imaging, and administration. Its outputs are probabilistic and context-dependent, and they can fail through bias, dataset shift, and over-trust. Appraise AI as you would any test: sensitivity, specificity, prevalence-dependent PPV, calibration, agreement with clinical measures, and external validation. Adopt it in stages (screen, appraise, silent pilot, supervised use, audit) within regulatory and privacy limits. The clinician remains accountable for every decision.
References
- Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25:44–56.
- Rajpurkar P, Chen E, Banerjee O, Topol EJ. AI in health and medicine. Nat Med. 2022;28:31–38.
- Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366:447–453.
- Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378.
- Wolff RF, Moons KGM, Riley RD, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. 2019;170:51–58.
- Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med. 2020;26:1364–1374.
- Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med. 2022;28:924–933.
- World Health Organization. Ethics and Governance of Artificial Intelligence for Health: WHO Guidance. Geneva: WHO; 2021.
- Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15:155–163.