By CIO Insider Team
 

OpenAI has ensured that conversations on ChatGPT that suggest a possibility of grave bodily harm to others can be examined by human moderators and in extreme case situations, reported to the police.

The company described these steps in one of its most recent blogposts describing how the AI processes delicate interactions and can be a safety risk.

Stating that ChatGPT is capable of offering compassionate assistance to those users that feel distressed, OpenAI emphasized that its precautionary measures distinguish between self-harm and harm to others. In the case of users who are suicidal, the AI refers them to mental health services like 988 in the US or Samaritans in the UK, however, it does not directly report such instances to the police to ensure privacy.

Also, when a user expresses intent to harm someone else, the conversation is routed to a specialised review pipeline. Human moderators trained in the company’s usage policies examine the chat, and if they identify an imminent threat, OpenAI may alert authorities. Accounts involved in such incidents can also be banned.

The company acknowledged that its safety mechanisms are more reliable in short exchanges. In long or repeated conversations, the safeguards can degrade, potentially allowing responses that conflict with safety protocols. OpenAI said it is strengthening these protections to maintain consistency across multiple interactions and to prevent gaps that could increase risk.

Also Read: Microsoft Launches in-house AI models

In addition to managing threats of harm, OpenAI is working on ways to intervene earlier for other risky behaviours, such as extreme sleep deprivation or unsafe stunts, by grounding users in reality and guiding them toward professional help. The company is also developing parental controls for teen users and exploring mechanisms to connect users to trusted contacts or licensed therapists before crises escalate.

Also Read: Has the Generation of Generative AI Begun?


OpenAI’s blogpost highlights that conversations on ChatGPT are not entirely private in certain situations. Users should be aware that if their messages indicate potential danger to others, they could be reviewed by trained moderators and may trigger real-world interventions, including police involvement.