Plain Language
Health facilities, clinics, physician's offices, and group practices that use or deploy AI or clinical decision support systems in patient care must provide a comprehensive disclosure to any licensed health care professional or other person who uses the tool or views its outputs. The disclosure must cover twelve categories of information: developer identity and funding; intended use and patient population; out-of-scope uses and known risks; inputs and output generation methods; training data details including demographic representativeness and bias; validation processes; performance measures; ongoing maintenance and update procedures; a notice of health care entity and developer liability; and a notice that the worker may override the tool's output based on professional judgment. The disclosure must be provided at the time of tool use, in plain language, linked in the patient's health record, and with sufficient time for the professional to make informed decisions. This functions as a detailed model card requirement specific to health care AI tools, directed at the deploying health care entity rather than the AI developer.
Statutory Text
(a) A health facility, clinic, physician's office, or office of a group practice that uses or deploys a covered tool for patient care shall disclose required information, described in subdivision (b), to any licensed health care professional or other person using a covered tool or viewing outputs from a covered tool. (b) Required information under subdivision (a) shall include all of the following: (1) Details on the covered tool, including developer, funding source, any foundation model used, and description of output. (2) Intended use of the covered tool, including intended patient population, intended users, and intended decisionmaking role. (3) Cautioned out-of-scope use of the covered tool, including known risks and limitations. (4) List of the inputs into the covered tool. (5) Description of how the covered tool generates outputs. (6) Development details of the covered tool, including, but not limited to, all of the following: (A) Description of the training set or clinical research underlying recommendations, including demographic representativeness and known biases based on protected characteristics. (B) Description of the relevance of training data to deployed setting. (C) Process used to ensure fairness in development of the intervention. (7) Description of the validation process. (8) Qualitative measures of performance. (9) Description of ongoing maintenance of intervention implementation and use. (10) Description of updates and continued validation or fairness assessment process. (11) Notice that health care entities and developers are liable for harm that results from the use of artificial intelligence in patient care. (12) Notice that a worker providing direct patient care is permitted to override the output of a covered tool if, in the judgment of the worker acting in their scope of practice, such an override is appropriate for the patient, or as necessary to comply with applicable law, including civil rights law. (c) (1) A disclosure made pursuant to this section shall be provided at the time the licensed health care professional or other person uses the covered tool or views any recommendation or output generated by the covered tool. (2) The disclosure shall be provided in plain language to, and linked in the health record of, any patient whose care was affected by the output of the covered tool or whose health information was used as an input to the covered tool. (3) The disclosure shall be provided with ample time for the licensed health care professional or other person to review and make reasoned decisions based on their professional judgment on whether and how to use the covered tool.