Strategic Communications and Marketing News Bureau

Illinois researchers examine teens’ use of generative AI, safety concerns

CHAMPAIGN, Ill. — Teenagers use generative artificial intelligence for many purposes, including emotional support and social interactions. A study by University of Illinois Urbana-Champaign researchers found that parents have little understanding of GAI, how their children use it and its potential risks, and that GAI platforms offer insufficient protection to ensure children’s safety.

The research paper by information sciences professor Yang Wang, the co-director of the Social Computing Systems Lab, and doctoral student Yaman Yu is one of the first published sources of data on the uses and risks of GAI for children. Wang and Yu will present their findings in May 2025 at the IEEE Symposium on Security and Privacy, the flagship computer security conference.

Wang and Yu said teens often use GAI platforms but little is known about how they use them, and their perceptions of its risks and how they cope has not previously been explored by scholars.

The researchers analyzed the content of 712 posts and 8,533 comments on Reddit that were relevant to teenagers’ use of GAI. They also interviewed seven teenagers and 13 parents to understand their perceptions of safety and how parents attempted to mitigate risk.

They found that teenagers often use GAI chatbots as therapy assistants or confidants to provide emotional support without judgment and help them cope with social challenges. AI chatbots are embedded into social media platforms such as Snapchat and Instagram, and teens incorporate them into group chats, use them to learn social skills and sometimes treat them as romantic partners. They use GAI for academic purposes such as essay writing, rephrasing text and generating ideas. Teens also posted on Reddit about requesting sexual or violent content and bullying AI chatbots.

“It’s a very heated topic, with a lot of teenagers talking about Character AI and how they are using it,” Yu said, referring to a platform for creating and interacting with character-based chatbots.

Wang and Yu reported that both parents and children had significant misconceptions about generative AI. Parents had little to no understanding of their children’s use of GAI, and their exposure to the tools was limited. They were unaware of their children’s use of tools such as Midjourney and DALL-E for image generation and Character AI. They viewed AI as a tool for homework and as functioning like a search engine, while children primarily used it for personal and social reasons, the researchers said.

Teenagers reported their concerns included becoming overly dependent or addicted to chatbots to fill a void in personal connections, the use of chatbots to create harassing content, unauthorized use of their personal information and the spread of harmful content, such as racist remarks. They also were concerned about AI replacing human labor and about intellectual property infringement.

Parents perceived that AI platforms collect extensive data, such as user demographics, conversation history and browser history, and they were concerned about children sharing personal or family information. However, parents “did not fully appreciate the extent of sensitive data their children might share with GAI … including details of personal traumas, medical records and private aspects of their social and sexual lives,” the researchers wrote. Parents also were concerned about children inadvertently spreading misinformation and worried that overreliance on AI would lead their children to avoid critical thinking.

Parents said they want child-specific AI that is trained only with age-appropriate content or a system with embedded age- and topic-control features. Children said their parents don’t advise them on specific uses of GAI and they want parents to discuss its ethical use rather than restrict it, the researchers reported.

GAI platforms provide limited protection to children, focus on restricting explicit content and do not offer parental control features tailored to AI. Wang and Yu said both the risks to children and strategies to mitigate it are more complex and nuanced than just blocking inappropriate content. One key challenge for identifying and preventing inappropriate content on GAI platforms is its dynamic nature of generating unique content in real time, compared to static online content, they said.

The researchers said it is critical that the platforms provide transparent explanations of their security and privacy risks identified by experts and recommended they offer content filters that can be tailored to individual families’ needs and their children’s developmental stages.

However, safety strategies can’t be purely technical and need to go beyond filtering and restrictions, recognizing the tension between children’s autonomy and parental control of managing online risks. Wang and Yu said adults first need to understand the motivations behind children’s behaviors on GAI. They suggested a support chatbot that could provide a safe environment to explain potential risks, enhance resilience and offer coping strategies to teenage users.

“AI technologies are evolving so quickly, and so are the ways people use them,” Wang said. “There are some things we can learn from past domains, such as addiction and inappropriate behavior on social media and online gaming.”

Wang said their research is a first step in addressing the problem. He and Yu are creating a taxonomy of risk categories that can be used to have conversations about the risks and interventions to help mitigate them. It also will help identify early signals of risky behavior that include the amount of time spent on a GAI platform, the content of conversations and usage patterns, such as the time of day that children are using the platforms, Wang said.

He and Yu are working with Illinois psychology professor Karen Rudolph, the director of the Family Studies Lab whose research focuses on adolescent development, to establish age-appropriate interventions.

“This is a very cross-disciplinary topic, and we’re trying to solve it in cross-disciplinary ways involving education, psychology and our knowledge of safety and risk management. It has to be a technical and a social interaction solution,” Yu said.

Editor's note:

To contact Yang Wang, email yvw@illinois.edu. To contact Yaman Yu, email yamanyu2@illinois.edu.
The paper “Exploring Parent-Child Perceptions on Safety in Generative AI: Concerns, Mitigation Strategies, and Design Implications” is available online. DOI: 10.48550/arXiv.2406.10461

Read Next

Humanities Diptych image with book cover of "The New Internationals" and a headshot of English professor David Wright Faladé

English professor’s novel tells of love triangle in post-WWII Paris, based on his family history

CHAMPAIGN, Ill. — A new novel by University of Illinois Urbana-Champaign English professor David Wright Faladé tells the story of three people in a love triangle in post-World War II Paris. The characters in “The New Internationals” — a young French woman who has survived the Holocaust, a university student from West Africa and a […]

Life sciences Portrait of the research team posing together.

Minecraft players can now explore whole cells and their contents

CHAMPAIGN, Ill. — Scientists have translated nanoscale experimental and computational data into precise 3D representations of bacteria, yeast and human epithelial, breast and breast cancer cells in Minecraft, a video game that allows players to explore, build and manipulate structures in three dimensions. The innovation will allow researchers and students of all ages to navigate […]

Arts Photo of seven dancers onstage wearing blue tops and orange or yellow flowing skirts. The backdrop is a Persian design.

February Dance includes works experimenting with live music, technology and a ‘sneaker ballet’

CHAMPAIGN, Ill. — The dance department at the University of Illinois Urbana-Champaign will present February Dance 2025: Fast Forward this week at Krannert Center for the Performing Arts. February Dance will be one of the first performances in the newly renovated Colwell Playhouse Theatre since its reopening. The performances are Jan. 30-Feb. 1. Dance professor […]

Strategic Communications and Marketing News Bureau

507 E. Green St
MC-426
Champaign, IL 61820

Email: stratcom@illinois.edu

Phone (217) 333-5010