The UK’s data watchdog has hit Snapchat owner Snap with a preliminary enforcement notice over "potential" failure to properly assess the privacy risks posed by its generative AI chatbot ‘My AI’.
Whilst at this stage a regulatory warning shot, the Information Commissioner Office (ICO)’s notice is a wake up call for organisations looking to embed ChatGPT-powered chatbots into their systems.
Snap launched the ‘My AI’ feature for UK Snapchat+ subscribers in February 2023, rolling it out to its wider Snapchat user base in the UK in April 2023. The feature, powered by OpenAI, marked the first example of generative AI embedded into a major messaging platform in the UK.
“The ICO’s investigation provisionally found the risk assessment Snap conducted before it launched ‘My AI’ did not adequately assess the data protection risks posed by the generative AI technology, particularly to children. The assessment of data protection risk is particularly important in this context which involves the use of… the processing of personal data of 13 to 17 year old children” the ICO said on October 6.
Snap's AI chatbot warning
John Edwards, UK Information Commissioner, added in a short notice: “The provisional findings of our investigation suggest a worrying failure by Snap to adequately identify and assess the privacy risks to children and other users before launching ‘My AI’. We have been clear that organisations must consider the risks associated with AI, alongside the benefits. Today's preliminary enforcement notice shows we will take action in order to protect UK consumers' privacy rights" he said.
Calling out a company for a “worrying failure” whilst simultaneously, as the ICO has, emphasising that “no conclusion should be drawn at this stage that there has, in fact, been any breach of data protection law or that an enforcement notice will ultimately be issued” will be frustrating for Snap.
A Snap spokesperson said in an emailed comment: “We are closely reviewing the ICO’s provisional decision. Like the ICO we are committed to protecting the privacy of our users. In line with our standard approach to product development, My AI went through a robust legal and privacy review process before being made publicly available.
"We will continue to work constructively with the ICO to ensure they’re comfortable with our risk assessment procedures.”
Snap says it baked in additional safeguarding features
The company added “on background” (not something The Stack had agreed to: PRs, please don't do this) that ChatGPT is a “a publicly available service, which has been integrated into Snapchat and programmed with additional moderation and safeguarding features [that aim to] avoid responses that are violent, hateful, sexually explicit, or otherwise offensive.
The Snap MyAI feature “considers a Snapchatter’s age group, so My AI aims to keep conversations age appropriate” and includes parental safeguards that “give parents and guardians the ability to know whether their teens have been communicating with My AI in the past seven days through our Family Center” it said, requesting that this background detail not be attributed to the unnamed spokesperson its official comment was.
The release of the preliminary enforcement notice follows an ICO reminder to companies developing or using generative AI that they should be considering their data protection obligations from the outset.
The ICO said it has issued advice to developers and users of generative AI on the issues that they must consider and will continue to scrutinise the compliance of products and services introduced to market.
The ICO emphasised that “the Commissioner’s findings in the notice are provisional. No conclusion should be drawn at this stage that there has, in fact, been any breach of data protection law or that an enforcement notice will ultimately be issued. The ICO will carefully consider any representations from Snap before taking a final decision” it added.
Snap notes of its AI chatbot that "in a chat conversation, My AI can answer a burning trivia question, offer advice on the perfect gift for your BFF's birthday, help plan a hiking trip for a long weekend or suggest what to make for dinner. We’re constantly working to improve and evolve My AI, but it’s possible My AI’s responses may include biased, incorrect, harmful, or misleading content. Because My AI is an evolving feature, you should always independently check answers provided by My AI before relying on any advice, and you should not share confidential or sensitive information."
