Online issues and worries: Generative AI

Even though AI has a lot of amazing benefits, we know that there are also risks. One of these risks is scams.

Because AI will do what humans programme it to do, some criminals use AI to scam people out of money. They do this in different ways. Explore these scams below and get advice on keeping yourself safe from them.

Messaging scams

Criminals might programme AI to send messages to thousands of people all at once. These messages might be sent to people as emails, text messages or direct messages (DMs) in other apps or platforms.

Usually, these messages seem to be from someone you know at first glance like a friend, parent or carer. Or it might seem like it’s from a government organisation or somewhere similar.

The message could ask you to open an attachment, click a link or respond, but the request is sometimes strange. And unless the person or organisation sending it was hacked, the message will come from an email address, number or profile that isn’t one you recognise.

How to tell if it’s a scam 

Check the source

If the email, number or profile doesn’t match what you know about the person or organisation, don’t open any attachments, click any links or respond.

Consider the information

Is the message strange in any way? This could include spelling errors from professional organisations or weird words that your friend would never use, for example. The message might ask you for strange things like gift cards or offer you a prize of some sort too. If anything feels off, take time to investigate rather than acting right away.

Contact the real person or organisation

If you’re not sure if the source is real, contact your friend or family member via their usual email, number or profile. For official organisations, you can use a search engine to find their website and contact information. Check if the suspicious message is from them. If it wasn’t from them, forward emails to report@phishing.gov.uk and texts to 7726. Then, report the message in the platform that was used.

It's important to remember that not all messaging scams will look fake right away. Some will be very convincing, so always check carefully before responding and never send personal information like passwords or financial information over messages.

Voice-cloning scams

Audio deepfakes are possible with the use of AI. This is where someone has taken audio of a person’s voice from somewhere online like social media videos and has used it to teach an AI tool what the person sounds like. The AI can then generate words using that person’s voice.

Once criminals have a person’s voice cloned, they can use it to call their loved ones and scam them out of money.

For example, imagine someone has cloned your voice. They then call your uncle and pretend to be you. The voice tells your uncle that you’re in trouble: you’re on a school trip and got separated from the group on the way to the airport. You can’t reach your teacher or parents and think the plane might have left. Can he send you money for a new ticket home? To your uncle, you sound scared and panicked. He wants to help but is panicked too, so might not be thinking clearly. The voice gives him the instructions for where to send the money and he does. It’s not until later that he learns that you’re safe at home, but he’s already sent a large sum of money to the scammer.

Criminals who use voice-cloning technology to scam people are counting on the panic and emotions a person might feel if a loved one is in trouble.

How to prevent voice-cloning scams

Just like other scams, you always need to think critically about messages you receive. With voice-cloning scams, though, there are a couple other things to think about to stay safe.

Keep your content private

If you post videos or share voice notes, make sure your privacy settings are strong so that only family and friends you know from offline spaces like school can watch or listen. To clone your voice, an AI tool needs source material; if your voice isn’t online, scammers don’t have anything to teach it with.

Set up a code word

It’s a good idea to set up code words with your friends and family members. You can use this for any kind of scam, but it’s particularly useful if you need to check whether a person calling you is who they say they are. So, choose a word (that’s unique for each person) that you can easily remember, maybe a shared favourite character or phrase from an inside joke. If a friend or family member calls you in a panic (or ‘you’ call them), simply ask them what the code word is. A scammer probably won’t know what you’re talking about.

Sexual extortion (sextortion)

Sextortion is a combination of the words ‘sexual’ and ‘extortion’. It’s when someone threatens to share sexual or naked images of a person with their friends, family, school or community unless the victim pays money or sends other nude images.

How it works

Criminals might use AI for sextortion by creating deepfakes of the victim. They can do this by finding real images of the victim across social media or similar platforms. The images don’t need to be nude images. They then add the victim’s face to the body of a nude person, maybe real or maybe AI-generated.

If a criminal has a lot of skill, uses a high-quality AI tool or has a lot of source images, these deepfaked nudes can seem very realistic.

They can then share the images with the victim as ‘proof’ and demand money or real nude images to keep it secret. Most teens in the UK say that it would be worse to have a deepfake nude of them created and shared around than a real nude. So, a lot of people might feel pressure to give into criminals’ demands to avoid others seeing those images.

Preventing and tackling sextortion

Remember that whether the image of you is real or a deepfake, the only person to blame is the criminal who is extorting you. It is illegal to:

• create deepfake nudes of children and teens
• share any kind of nude image of children and teens with anyone
• extort anyone with any threat in the UK

If someone threatens to share nudes of you unless you meet their demands, report them to police and tell a trusted adult such as a parent, carer or teacher.

Things you can do to make it harder for criminals to target you

• Make any online profile private so only close friends and family can see your videos and images.
• In app settings, review who can contact you. You can set your accounts up so that only friends can message you privately and can require any new contacts to go through an approval process.
• Only let close friends and family contact you privately. Any communication with strangers should be in public spaces rather than DMs to limit your risk.
• If discussing things in public, only share public information. Avoid real names or locations and never invite someone to add you on a different platform.
• Be honest about your age. If you’re too young to use a platform, don’t lie about your age. Those limits are there to protect you. Platforms with age restrictions give special privacy and protection to people at different ages. If you lie about your age, you won’t get the protection that’s right for you.

What to do if someone does try to sexually extort you

1. Stop responding to them.
2. Tell a trusted adult and show them the messages.
3. Together, screenshot and save the messages as evidence.
4. Block the user and report them to the platform.
5. Report them to police on the CEOP website or, if there’s immediate danger, call 999.
6. Use the Report Remove tool to get any sexual images (whether real or deepfaked) of you taken off of the internet.
7. Talk about how you feel. Some people might feel really anxious, embarrassed or humiliated even though it’s not their fault. It’s harmful to your mental health to keep it all bottled in, so make sure you speak with:

• a trusted adult like a parent, carer or teacher
• a helpline counsellor such as those on Meic or Childline. You can call them on your phone or send them messages if that’s more comfortable
• others your age on safe message boards like the ones on Childline or on Ditch the Label
• a licensed therapist through the NHS that your parents or carers have helped you find

Criminals will try to make you feel weak and embarrassed so that you don’t report them or reach out for help. But what they’re doing is illegal and wrong. If they’re doing it to you, they might have done the same to someone else, so make sure to take action to protect yourself and others.