Does the AI that drafts your notes train on patient consultations?

Training retains consultation content as material used to change how a model behaves, so the product performs differently in future for every practice using it. Copies of the audio, transcript and draft persist either way.

A no-training commitment leaves untouched quality assurance, evaluation and regression testing against stored consultations, human review of samples, prompt and template tuning, fine-tuning for a single customer, and aggregate analytics.

Ask, use by use, what copies exist once the clinician signs, who can read each, and for how long:

• Producing this practice's notes, letters and summaries.
• Fault diagnosis by support staff, whether that access is logged, and whether the practice is told.
• Human review of samples for quality, who performs it, and whether the practice can decline.
• Training, fine-tuning or adapting any model, the supplier's own or a third party's.
• Anything described as aggregated, statistical or de-identified, with the method named.

The Privacy Act 1988 covers an individual who is reasonably identifiable, so a de-identification claim asserts that limb no longer holds. Removing the name, the date of birth and the Medicare number is only the start. The OAIC's guidance on developing and training generative AI models states that de-identification is context dependent, may be difficult to achieve, and that developers should treat data as personal information where there is doubt.

A recorded voice is a biometric characteristic, and automated redaction is a statistical model with an error rate, so ask how it was tested and against what set of consultations, and expect a method and a recall figure.

Under the Notifiable Data Breaches scheme, unauthorised disclosure of health information likely to result in serious harm is an eligible data breach, and the practice that collected it carries the notification obligation.

Health information is sensitive information under the Privacy Act, and the small business exemption does not reach an organisation that provides a health service, whatever its turnover. Practices in New South Wales, Victoria and the Australian Capital Territory also sit under state and territory health records legislation.

Care and the clinical record are the primary purpose, and any other use is a secondary use governed by Australian Privacy Principle 6, which for sensitive information requires consent or a use the patient would reasonably expect that is directly related to that purpose.

Section 16B permits handling health information without consent for research relevant to public health or public safety, following guidelines approved under section 95A of the Act and with the approval of a human research ethics committee. Supplier product development does not enter that route.

A term accepted by a practice manager at signup is not consent from the patient. The OAIC's generative AI guidance directs a developer that cannot establish a secondary use as within reasonable expectations to seek consent, to offer a meaningful ability to opt out, or both.

Where a practice permits the use, Australian Privacy Principle 5 requires patients to be told at collection what their information will be used for and who receives it, and Australian Privacy Principle 1 requires the privacy policy to describe it.

A no-training statement covers one company, and a scribe is usually several.

Ask which company performs each step, which tier of the model API the supplier holds, whether that tier excludes customer content from training by default, and whether inputs are retained for abuse monitoring. Ask whether those positions are contract terms or settings.

An opt-out that starts switched on has already contributed every consultation captured before someone found it. Parameters fitted using earlier consultations do not change when the source records are deleted, so ask in writing what data from this practice has already been used for training, fine-tuning, evaluation or human review, and when.

A public page, a questionnaire answer and a product setting can each change in a release. Ask for the commitment in the agreement, with the artefacts that let the practice check it: access logs, a maintained subprocessor list, retention periods per data type, and a deletion certificate.

Each item belongs in the agreement, not in a policy the supplier can vary alone:

• Customer data, including consultation audio, transcripts, drafted notes and derived artefacts, will not be used to train, fine-tune, adapt, evaluate or benchmark any model, whether identified, pseudonymised, de-identified or aggregated.
• The supplier processes customer data only on the practice's documented instructions and for no purpose of its own, and no subprocessor may use it otherwise.
• The supplier maintains a current subprocessor list and notifies the practice before it changes.
• Human access is limited to named purposes, is logged, and the log is available to the practice on request.
• Retention periods are stated separately for audio, transcripts and finished notes, with the deletion method and the treatment of backups.
• On termination, all customer data and derived artefacts are deleted within a stated period and the deletion is certified in writing.
• These terms cannot be varied by a change to the product, an online policy or a default setting without the practice's written agreement.