Voice Cloning

iasparliament Logo
January 23, 2024

Why in News?

Voice clone fraud has been on the rise in India.

What is voice cloning?

  • AI voice cloning – It is the process of creating a synthetic replica of a person’s voice through machine learning and speech synthesis technology.
  • It is called as voice deepfakes or audio deepfakes.
  • Objective – To achieve a high level of naturalness that sounds exactly like a person’s voice.

Usage of Voice Deepfakes

  • Voice assistants – Creation of personalized voice assistants, chatbots, video game characters, animated film avatars, custom call centre voices, and much more.
  • Language translationsMeta’s SeamlessM4T, can understand nearly 100 languages from speech or text and generate translations in real-time.
  • Protecting original voicesApple introduced a voice cloning feature to help people who may be in danger of losing their voice say to a degenerative disease.
  • Creating new songsYouTube’s Dream Track partners with creators in the U.S. to allow them to create song clips featuring AI vocals with permission from pop stars.
  • AI Voice Scams – Easy access to AI voice clones also spawned disinformation.
  • Rallying for votes in Election – A Pakistan based political party used an AI-generated speech from the now imprisoned leader in an attempt to rally for votes virtually.
  • Generating hate speech – Harry Potter actress Emma Watson voice reading out a portion of the Mein Kampf.

How are voice clones done?

  • It is a complex process that involves audio data, an algorithm to train on the data, and finally fine-tuning your cloned AI voice.
  • Requirement – AI model will require audio data for the machine learning process to trigger.
  • Ways to share audio data – Uploading an audio file or recording voice samples in the app.


  • The recorded or uploaded audio data is then analyzed by our model to extract various acoustic features, such as pitch, tone, and rhythm.
  • Speech synthesis – The analysed audio is used to train a speech synthesis model, such as a neural network.
  • The model is trained on this data to pick up on the nuances and acoustic features of the user’s voice.
  • Speech perfection – Once the voice is cloned, the user has the option to continue refining the AI voice with various voice augmenting variables such as prosody, phoneme, and emotions.

A report by Market US has revealed that the global market for these applications stands at $1.2 billion in 2022 and is estimated to touch almost $5 billion in 2032 with a CAGR above 15-40%.

What is the scenario of AI voice clone scams in India?

  • ‘The Artificial Imposter’ Report– India topped the list with the maximum number of victims.
    • 47% of surveyed Indians have either been a victim or knew someone who had fallen prey to the scam.
    • Scam in India is almost twice the global average of 25%.
  • McAfee Report – 66% of Indian participants admitted that they would respond to a voice call or a phone call that appeared to be from a friend or family member in urgent need of money.
    • The most effective excuses used by the sender were that they had been robbed (70%) and involved in a car accident (69%).
    • 86% Indians were prone to sharing their voice data online or via voice notes at least once a week.

What can be done?

  • Starting voice cloning challenges – Public can be asked to send in their ideas to detect, evaluate and monitor cloned devices.
  • Adopting impersonation rules – It will help deter deceptive voice cloning
  • Quickening regulators response – The control measures be taken at the pace at which generative AI releases are moving.
  • Digitally signed videos - To verify that content can be trusted.
  • AI voice clone detectors – Advancing their development and promoting its active usage.
  • Cracking down social media platforms to control the spread.
  • 4 dangerous signs to watch out for
    • Unexpected calls
    • Urgent requests
    • Unnatural speech patterns
    • Requests for money or personal information



  1. The Hindu| Voice Cloning as Scams
  2. Resemble.AI| Process of Voice Cloning
  3. Times of India| Measures to control Voice Cloning Scams
Login or Register to Post Comments
There are no reviews yet. Be the first one to review.



Free UPSC Interview Guidance Programme