AI safety and limitations
CareFlow Kids includes optional AI clinical assistance to help clinicians review structured medical information.
AI support is designed as a decision‑support tool, not an automated medical decision system.
The clinician always remains responsible for the final medical interpretation and clinical decisions.
When AI is used
The AI system is only used when the clinician explicitly requests it, for example by clicking Ask AI during a consultation.
AI is not automatically running in the background and does not continuously analyze patient data.
This means that clinical information is only shared with the AI provider when the clinician chooses to request assistance.
What information the AI receives
When AI assistance is requested, CareFlow Kids generates a structured JSON representation of the visit.
This structured information may include:
- patient age
- patient sex
- symptoms and complaints
- vital signs
- physical examination findings
- growth information
- relevant history (for example perinatal history or past medical history)
- problem listing generated during the visit
This structured format helps the AI analyze the clinical context more reliably.
What information is NOT sent
Direct personal identifiers are not included in the AI request.
The following information is not transmitted to the AI provider:
- patient name
- family name
- alias
- MRN (medical record number)
This reduces the exposure of possible identifiable patient information.
Local guideline checks
Before contacting an online AI model, CareFlow Kids can run local guideline rules.
These rules operate entirely on the clinician's device and may:
- detect red flags
- suggest guideline-based actions
- highlight potential diagnoses
Because these rules run locally, no patient information is transmitted externally when they are executed.
AI limitations
AI systems have important limitations.
AI responses may depend on:
- the completeness of the clinical information entered
- the configuration of the physician prompt
- the knowledge available to the AI model
AI suggestions may therefore be:
- incomplete
- overly general
- occasionally incorrect
For this reason, AI output should always be interpreted as supporting information, not a definitive diagnosis.
Clinical responsibility
Medical decisions must always remain under the responsibility of the clinician.
The AI system cannot:
- make medical diagnoses
- prescribe treatments
- replace clinical judgment
Its role is limited to helping the clinician review and organize clinical information.
Privacy considerations
When AI is used, structured clinical information is transmitted to the configured AI provider.
Clinicians should therefore:
- review the information before requesting AI assistance
- use AI only when clinically appropriate
- follow applicable medical privacy regulations
