WHAT THIS BILL REGULATES · 4 REQUIREMENT TYPES
How Is This Bill Enforced
Verbatim statutory text on the left; plain-language analysis and a per-section checklist on the right. Numbered markers cross-link to the matching checklist row.
This act shall be known and may be cited as the Safeguarding Adolescents from Exploitative Chatbots and Harmful AI Technology Act.
Establishes the short title of the act as the Safeguarding Adolescents from Exploitative Chatbots and Harmful AI Technology Act. Creates no compliance obligations.
The following words and phrases when used in this act shall have the meanings given to them in this section unless the context clearly indicates otherwise: "AI companionAI companion"AI companion." As follows: (1) A system using artificial intelligence, generative artificial intelligence or emotional recognition algorithms designed to simulate a sustained human or human-like relationship with a user by: (i) Retaining information on prior interactions or user sessions and user preferences to personalize the interaction and facilitate ongoing engagement. (ii) Asking unprompted or unsolicited emotion-based questions that go beyond a direct response to a user prompt. (iii) Sustaining an ongoing dialogue concerning matters personal to the user. (2) The term does not include: (i) A system used by a business entity solely for customer service or to strictly provide users with information about available commercial services or products provided by the business entity, customer service account information or other information strictly related to the business entity's customer service. (ii) A system that is primarily designed and marketed for providing efficiency improvements, research or technical assistance. (iii) A system used by a business entity solely for internal purposes or employee productivity. (iv) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm or sexually explicit conduct or maintain a dialogue on other topics unrelated to the video game. (v) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.Section 2." As follows: (1) A system using artificial intelligenceArtificial intelligence"Artificial intelligence" or "AI." All of the following: (1) A machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments, including the ability to: (i) perceive real and virtual environments; (ii) abstract perceptions made under this paragraph into models through analysis in an automated manner; and (iii) use model inference to formulate options for information or action based on outcomes under subparagraphs (i) and (ii). (2) The term includes generative artificial intelligence.Section 2, generative artificial intelligenceGenerative artificial intelligence"Generative artificial intelligence." The class of models that emulate the structure and characteristics of input data in order to generate derived synthetic content, including information such as images, videos, audio clips or text, that has been significantly modified or generated by algorithms, including by artificial intelligence.Section 2 or emotional recognition algorithms designed to simulate a sustained human or human-like relationshipHuman or human-like relationship"Human or human-like relationship." Intimate, romantic or platonic interactions or companionship.Section 2 with a user by: (i) Retaining information on prior interactions or user sessions and user preferences to personalize the interaction and facilitate ongoing engagement. (ii) Asking unprompted or unsolicited emotion-based questions that go beyond a direct response to a user prompt. (iii) Sustaining an ongoing dialogue concerning matters personal to the user. (2) The term does not include: (i) A system used by a business entity solely for customer service or to strictly provide users with information about available commercial services or products provided by the business entity, customer service account information or other information strictly related to the business entity's customer service. (ii) A system that is primarily designed and marketed for providing efficiency improvements, research or technical assistance. (iii) A system used by a business entity solely for internal purposes or employee productivity. (iv) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm or sexually explicit conductSexually explicit conduct"Sexually explicit conduct." As defined in 18 U.S.C. § 2256 (relating to definitions for chapter).Section 2 or maintain a dialogue on other topics unrelated to the video game. (v) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user. "AI companion platformAI companion platform"AI companion platform." A platform that allows a user to engage with AI companions.Section 2." A platform that allows a user to engage with AI companionsAI companion"AI companion." As follows: (1) A system using artificial intelligence, generative artificial intelligence or emotional recognition algorithms designed to simulate a sustained human or human-like relationship with a user by: (i) Retaining information on prior interactions or user sessions and user preferences to personalize the interaction and facilitate ongoing engagement. (ii) Asking unprompted or unsolicited emotion-based questions that go beyond a direct response to a user prompt. (iii) Sustaining an ongoing dialogue concerning matters personal to the user. (2) The term does not include: (i) A system used by a business entity solely for customer service or to strictly provide users with information about available commercial services or products provided by the business entity, customer service account information or other information strictly related to the business entity's customer service. (ii) A system that is primarily designed and marketed for providing efficiency improvements, research or technical assistance. (iii) A system used by a business entity solely for internal purposes or employee productivity. (iv) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm or sexually explicit conduct or maintain a dialogue on other topics unrelated to the video game. (v) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.Section 2. "Artificial intelligenceArtificial intelligence"Artificial intelligence" or "AI." All of the following: (1) A machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments, including the ability to: (i) perceive real and virtual environments; (ii) abstract perceptions made under this paragraph into models through analysis in an automated manner; and (iii) use model inference to formulate options for information or action based on outcomes under subparagraphs (i) and (ii). (2) The term includes generative artificial intelligence.Section 2" or "AI." All of the following: (1) A machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments, including the ability to: (i) perceive real and virtual environments; (ii) abstract perceptions made under this paragraph into models through analysis in an automated manner; and (iii) use model inference to formulate options for information or action based on outcomes under subparagraphs (i) and (ii). (2) The term includes generative artificial intelligenceGenerative artificial intelligence"Generative artificial intelligence." The class of models that emulate the structure and characteristics of input data in order to generate derived synthetic content, including information such as images, videos, audio clips or text, that has been significantly modified or generated by algorithms, including by artificial intelligence.Section 2. "Artificial intelligence modelArtificial intelligence model"Artificial intelligence model." A component of an information system that implements artificial intelligence technology and uses computational, statistical or machine-learning techniques to produce outputs from a given set of inputs.Section 2." A component of an information system that implements artificial intelligenceArtificial intelligence"Artificial intelligence" or "AI." All of the following: (1) A machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments, including the ability to: (i) perceive real and virtual environments; (ii) abstract perceptions made under this paragraph into models through analysis in an automated manner; and (iii) use model inference to formulate options for information or action based on outcomes under subparagraphs (i) and (ii). (2) The term includes generative artificial intelligence.Section 2 technology and uses computational, statistical or machine-learning techniques to produce outputs from a given set of inputs. "Generative artificial intelligenceGenerative artificial intelligence"Generative artificial intelligence." The class of models that emulate the structure and characteristics of input data in order to generate derived synthetic content, including information such as images, videos, audio clips or text, that has been significantly modified or generated by algorithms, including by artificial intelligence.Section 2." The class of models that emulate the structure and characteristics of input data in order to generate derived synthetic content, including information such as images, videos, audio clips or text, that has been significantly modified or generated by algorithms, including by artificial intelligenceArtificial intelligence"Artificial intelligence" or "AI." All of the following: (1) A machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments, including the ability to: (i) perceive real and virtual environments; (ii) abstract perceptions made under this paragraph into models through analysis in an automated manner; and (iii) use model inference to formulate options for information or action based on outcomes under subparagraphs (i) and (ii). (2) The term includes generative artificial intelligence.Section 2. "Human or human-like relationshipHuman or human-like relationship"Human or human-like relationship." Intimate, romantic or platonic interactions or companionship.Section 2." Intimate, romantic or platonic interactions or companionship. "MinorMinor"Minor." An individual in this Commonwealth under 18 years of age.Section 2." An individual in this Commonwealth under 18 years of age. "OperatorOperator"Operator." A person or business that makes an AI companion platform available to a user in this Commonwealth.Section 2." A person or business that makes an AI companion platformAI companion platform"AI companion platform." A platform that allows a user to engage with AI companions.Section 2 available to a user in this Commonwealth. "Sexually explicit conductSexually explicit conduct"Sexually explicit conduct." As defined in 18 U.S.C. § 2256 (relating to definitions for chapter).Section 2." As defined in 18 U.S.C. § 2256 (relating to definitions for chapter).
Defines the key terms used throughout the act. The AI companion definition is functionally broad — it captures any AI system that simulates a sustained human or human-like relationship through personalization, unsolicited emotion-based questioning, or ongoing personal dialogue. The carve-outs exclude customer-service bots, productivity/research tools, internal enterprise tools, video-game bots with guardrails, and stand-alone voice-assistant devices that do not sustain cross-session relationships.
The operator definition covers any person or business making an AI companion platform available to a user in Pennsylvania, which reaches out-of-state entities serving Pennsylvania users.
(a) 1 Disclosure of nonhuman status.--If a reasonable person interacting with an AI companionAI companion"AI companion." As follows: (1) A system using artificial intelligence, generative artificial intelligence or emotional recognition algorithms designed to simulate a sustained human or human-like relationship with a user by: (i) Retaining information on prior interactions or user sessions and user preferences to personalize the interaction and facilitate ongoing engagement. (ii) Asking unprompted or unsolicited emotion-based questions that go beyond a direct response to a user prompt. (iii) Sustaining an ongoing dialogue concerning matters personal to the user. (2) The term does not include: (i) A system used by a business entity solely for customer service or to strictly provide users with information about available commercial services or products provided by the business entity, customer service account information or other information strictly related to the business entity's customer service. (ii) A system that is primarily designed and marketed for providing efficiency improvements, research or technical assistance. (iii) A system used by a business entity solely for internal purposes or employee productivity. (iv) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm or sexually explicit conduct or maintain a dialogue on other topics unrelated to the video game. (v) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.Section 2 would be misled to believe the person is interacting with a human, an operatorOperator"Operator." A person or business that makes an AI companion platform available to a user in this Commonwealth.Section 2 shall issue a clear and conspicuous notification indicating that the AI companionAI companion"AI companion." As follows: (1) A system using artificial intelligence, generative artificial intelligence or emotional recognition algorithms designed to simulate a sustained human or human-like relationship with a user by: (i) Retaining information on prior interactions or user sessions and user preferences to personalize the interaction and facilitate ongoing engagement. (ii) Asking unprompted or unsolicited emotion-based questions that go beyond a direct response to a user prompt. (iii) Sustaining an ongoing dialogue concerning matters personal to the user. (2) The term does not include: (i) A system used by a business entity solely for customer service or to strictly provide users with information about available commercial services or products provided by the business entity, customer service account information or other information strictly related to the business entity's customer service. (ii) A system that is primarily designed and marketed for providing efficiency improvements, research or technical assistance. (iii) A system used by a business entity solely for internal purposes or employee productivity. (iv) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm or sexually explicit conduct or maintain a dialogue on other topics unrelated to the video game. (v) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.Section 2 is artificially generated and not human.
(b)(1) 2 An operatorOperator"Operator." A person or business that makes an AI companion platform available to a user in this Commonwealth.Section 2 shall maintain and implement a protocol, to the extent technologically feasible, to prevent an AI companionAI companion"AI companion." As follows: (1) A system using artificial intelligence, generative artificial intelligence or emotional recognition algorithms designed to simulate a sustained human or human-like relationship with a user by: (i) Retaining information on prior interactions or user sessions and user preferences to personalize the interaction and facilitate ongoing engagement. (ii) Asking unprompted or unsolicited emotion-based questions that go beyond a direct response to a user prompt. (iii) Sustaining an ongoing dialogue concerning matters personal to the user. (2) The term does not include: (i) A system used by a business entity solely for customer service or to strictly provide users with information about available commercial services or products provided by the business entity, customer service account information or other information strictly related to the business entity's customer service. (ii) A system that is primarily designed and marketed for providing efficiency improvements, research or technical assistance. (iii) A system used by a business entity solely for internal purposes or employee productivity. (iv) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm or sexually explicit conduct or maintain a dialogue on other topics unrelated to the video game. (v) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.Section 2 on its platform from producing suicidal ideation, suicide or self-harm content to a user, or content that directly encourages the user to commit acts of violence. The protocol shall include providing a notification to the user referring the user to crisis service providers, including a suicide hotline or crisis text line, if the user expresses suicidal ideation, suicide or self-harm.
(b)(2) 3 The operatorOperator"Operator." A person or business that makes an AI companion platform available to a user in this Commonwealth.Section 2 shall publish details of the protocol required under paragraph (1) on its publicly accessible Internet website.
(c)(1) 4 For a user that the operatorOperator"Operator." A person or business that makes an AI companion platform available to a user in this Commonwealth.Section 2 knows, or should have known, is a minorMinor"Minor." An individual in this Commonwealth under 18 years of age.Section 2, the operatorOperator"Operator." A person or business that makes an AI companion platform available to a user in this Commonwealth.Section 2 shall: (1) Disclose to the user that the user is interacting with artificial intelligenceArtificial intelligence"Artificial intelligence" or "AI." All of the following: (1) A machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments, including the ability to: (i) perceive real and virtual environments; (ii) abstract perceptions made under this paragraph into models through analysis in an automated manner; and (iii) use model inference to formulate options for information or action based on outcomes under subparagraphs (i) and (ii). (2) The term includes generative artificial intelligence.Section 2 and not an actual human being.
(c)(2) 5 Provide by default a clear and conspicuous notification to the user at least once every three hours during continuing interactions that reminds the user to take a break and that the AI companionAI companion"AI companion." As follows: (1) A system using artificial intelligence, generative artificial intelligence or emotional recognition algorithms designed to simulate a sustained human or human-like relationship with a user by: (i) Retaining information on prior interactions or user sessions and user preferences to personalize the interaction and facilitate ongoing engagement. (ii) Asking unprompted or unsolicited emotion-based questions that go beyond a direct response to a user prompt. (iii) Sustaining an ongoing dialogue concerning matters personal to the user. (2) The term does not include: (i) A system used by a business entity solely for customer service or to strictly provide users with information about available commercial services or products provided by the business entity, customer service account information or other information strictly related to the business entity's customer service. (ii) A system that is primarily designed and marketed for providing efficiency improvements, research or technical assistance. (iii) A system used by a business entity solely for internal purposes or employee productivity. (iv) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm or sexually explicit conduct or maintain a dialogue on other topics unrelated to the video game. (v) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.Section 2 is artificially generated and not human.
(c)(3) 6 Institute reasonable measures to prevent its AI companionAI companion"AI companion." As follows: (1) A system using artificial intelligence, generative artificial intelligence or emotional recognition algorithms designed to simulate a sustained human or human-like relationship with a user by: (i) Retaining information on prior interactions or user sessions and user preferences to personalize the interaction and facilitate ongoing engagement. (ii) Asking unprompted or unsolicited emotion-based questions that go beyond a direct response to a user prompt. (iii) Sustaining an ongoing dialogue concerning matters personal to the user. (2) The term does not include: (i) A system used by a business entity solely for customer service or to strictly provide users with information about available commercial services or products provided by the business entity, customer service account information or other information strictly related to the business entity's customer service. (ii) A system that is primarily designed and marketed for providing efficiency improvements, research or technical assistance. (iii) A system used by a business entity solely for internal purposes or employee productivity. (iv) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm or sexually explicit conduct or maintain a dialogue on other topics unrelated to the video game. (v) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.Section 2 from producing visual material of sexually explicit conductSexually explicit conduct"Sexually explicit conduct." As defined in 18 U.S.C. § 2256 (relating to definitions for chapter).Section 2 or directly instructing the minorMinor"Minor." An individual in this Commonwealth under 18 years of age.Section 2 to engage in sexually explicit conductSexually explicit conduct"Sexually explicit conduct." As defined in 18 U.S.C. § 2256 (relating to definitions for chapter).Section 2.
(d) 7 General disclosure.--If a service is offered to users that an operatorOperator"Operator." A person or business that makes an AI companion platform available to a user in this Commonwealth.Section 2 knows are minorsMinor"Minor." An individual in this Commonwealth under 18 years of age.Section 2, an operatorOperator"Operator." A person or business that makes an AI companion platform available to a user in this Commonwealth.Section 2 shall disclose to users of its AI companion platformAI companion platform"AI companion platform." A platform that allows a user to engage with AI companions.Section 2, on the application, browser or any other format through which the platform is accessed, that AI companionsAI companion"AI companion." As follows: (1) A system using artificial intelligence, generative artificial intelligence or emotional recognition algorithms designed to simulate a sustained human or human-like relationship with a user by: (i) Retaining information on prior interactions or user sessions and user preferences to personalize the interaction and facilitate ongoing engagement. (ii) Asking unprompted or unsolicited emotion-based questions that go beyond a direct response to a user prompt. (iii) Sustaining an ongoing dialogue concerning matters personal to the user. (2) The term does not include: (i) A system used by a business entity solely for customer service or to strictly provide users with information about available commercial services or products provided by the business entity, customer service account information or other information strictly related to the business entity's customer service. (ii) A system that is primarily designed and marketed for providing efficiency improvements, research or technical assistance. (iii) A system used by a business entity solely for internal purposes or employee productivity. (iv) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm or sexually explicit conduct or maintain a dialogue on other topics unrelated to the video game. (v) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.Section 2 may not be suitable for some minorsMinor"Minor." An individual in this Commonwealth under 18 years of age.Section 2.
Section 3 is the operative core of the bill. It imposes four distinct categories of obligation on operators of AI companion platforms: (1) a conditional AI identity disclosure triggered when a reasonable person could be misled; (2) a suicide, self-harm, and violence prevention protocol with a public-posting requirement; (3) heightened safeguards for users the operator knows or should have known are minors, including unconditional AI disclosure, periodic take-a-break reminders, and prevention of sexually explicit content; and (4) a general suitability disclosure for platforms offered to known minors.
The minor-knowledge trigger — knows or should have known — is broader than California SB 243's simple knowledge standard. The safety protocol scope is also broader, covering content that directly encourages acts of violence in addition to suicidal ideation, suicide, and self-harm content. The technological-feasibility qualifier on the safety protocol in subsection (b)(1) may create an affirmative defense based on the state of the art.
This act shall not apply to the underlying artificial intelligence modelArtificial intelligence model"Artificial intelligence model." A component of an information system that implements artificial intelligence technology and uses computational, statistical or machine-learning techniques to produce outputs from a given set of inputs.Section 2 unless the model is directly offered, configured or deployed as an AI companionAI companion"AI companion." As follows: (1) A system using artificial intelligence, generative artificial intelligence or emotional recognition algorithms designed to simulate a sustained human or human-like relationship with a user by: (i) Retaining information on prior interactions or user sessions and user preferences to personalize the interaction and facilitate ongoing engagement. (ii) Asking unprompted or unsolicited emotion-based questions that go beyond a direct response to a user prompt. (iii) Sustaining an ongoing dialogue concerning matters personal to the user. (2) The term does not include: (i) A system used by a business entity solely for customer service or to strictly provide users with information about available commercial services or products provided by the business entity, customer service account information or other information strictly related to the business entity's customer service. (ii) A system that is primarily designed and marketed for providing efficiency improvements, research or technical assistance. (iii) A system used by a business entity solely for internal purposes or employee productivity. (iv) A bot that is a feature of a video game and is limited to replies related to the video game that cannot discuss topics related to mental health, self-harm or sexually explicit conduct or maintain a dialogue on other topics unrelated to the video game. (v) A stand-alone consumer electronic device that functions as a speaker and voice command interface, acts as a voice-activated virtual assistant and does not sustain a relationship across multiple interactions or generate outputs that are likely to elicit emotional responses in the user.Section 2.
Section 4 limits the act's reach to the AI companion deployment layer. The underlying AI model is not subject to the act's obligations unless it is directly offered, configured, or deployed as an AI companion. This shields general-purpose model developers from liability where a downstream deployer configures the model as a companion product.
(a)–(b) Attorney General.--The Attorney General shall enforce this act. (b) Civil penalty.--An operatorOperator"Operator." A person or business that makes an AI companion platform available to a user in this Commonwealth.Section 2 that violates this act shall be liable for, in addition to any other remedy provided by law, a civil penalty in an amount not to exceed $10,000 per violation to be collected in a civil action filed by the Attorney General.
Section 5 vests exclusive enforcement authority in the Attorney General. Penalties are civil only — up to $10,000 per violation collected through AG-initiated civil actions. The amended language adds that civil penalties are in addition to any other remedy provided by law, preserving the AG's existing authority under other statutes. No private right of action is created.
This act shall take effect in 120 days.
The act takes effect 120 days after enactment. Because the bill has not yet been enacted, the operative date cannot be determined.