While producing voice clones and artificial intelligence-based visual effects for Indian cinema and television, Divyendra Singh Jadoun started receiving calls from politicians asking whether he could produce AI films, or deepfakes, for their campaign.
His startup, The Indian Deepfaker, has a great chance ahead of it, with a national election scheduled for May of this year and a fiercely contested local election in his home state of Rajasthan last November. Jadoun, though, was hesitant.
“The technology to create deepfakes is so good now; it can be done almost instantaneously, with very little effort, and people cannot tell if it’s real or fake,” Jadoun, 30, said.
“There are no guidelines on deepfakes, and that’s worrying, as it has the potential to influence how a person votes,” he said.Recent popular content includes Instagram reels with Indian Prime Minister Narendra Modi singing in regional languages and TikTok videos featuring Prabowo Subianto and Anies Baswedan, two Indonesian presidential contenders, speaking Arabic well.
However, they were all made by AI and shared without a label.
Misinformation is common on social media platforms ahead of the upcoming elections in Bangladesh, India, Indonesia, and Pakistan. Tech experts and officials say deepfakes, which are videos or audio produced using artificial intelligence and broadcast as real, are especially worrying.
Modi has called deepfake videos a “big concern” in India, where over 900 million people are able to cast ballots.
Deepfakes of all three presidential candidates and their running mates are circulating online in Indonesia, where more than 200 million voters are expected to cast ballots on February 14. These fakes have the potential to sway election results, according to Nuurrianti Jalli, a social media misinformation specialist.
“From microtargeting of voters with disinformation to spreading false narratives at a scale and speed unachievable by human actors alone, these AI tools can significantly influence voter perceptions and behaviour,” she stated.
“AI-generated content can further skew public perception and influence voting behavior in environments where misinformation is already prevalent,” said Jalli, an assistant professor at Oklahoma State University’s media department.
There have been increasing worries about the impact of deepfake images and videos produced by generative AI tools like Midjourney, Stable Diffusion, and OpenAI’s Dall-E on US presidential polls in November after they appeared before elections in Turkey, Argentina, and New Zealand last year.
AI speeds up, lowers the cost, and increases the effectiveness of misinformation production and dissemination, according to a new analysis from US nonprofit Freedom House.
Following polls on January 7, Prime Minister Sheikh Hasina of Bangladesh was declared to be serving a fourth consecutive term. However, films featuring female opposition politicians Rumin Farhana in a bikini and Nipun Roy in a swimming pool have surfaced, which are deepfakes.
Even low-quality deepfake content is deceiving people, even if it is swiftly disproved, according to Sayeed Al-Zaman, an assistant professor of journalism at Jahangirnagar University in Bangladesh who specializes in social media studies.
“Given the low levels of information and digital literacy in Bangladesh, deepfakes can be potent carriers of political propaganda if crafted and deployed effectively,” added the politician.
“But the government does not appear concerned.”
A request for a response from the Ministry of Information was not answered.
Imran Khan, the former prime minister of Pakistan who is currently serving a prison sentence for violating the Official Secrets Act, addressed an online electoral rally in December with the help of an artificial intelligence (AI)-created image and voice clone. The event attracted a large number of participants.
Although Pakistan has proposed an AI law, advocates for digital rights have criticized the absence of safeguards against misinformation and to protect women and other vulnerable areas.
“It is imperative to emphasize the danger that misinformation presents to Pakistan’s democratic process and election system,” stated Nighat Dad, a co-founder of the nonprofit Digital Rights Foundation.
“In the past, misinformation distributed online has been able to affect party support, voting patterns, and even legislative changes. This will be made simpler by synthetic media,” she continued.
“WARNING SIGN”
DeepMedia, a business that creates tools to identify synthetic media, estimates that in 2023, at least 500,000 video and voice deepfakes will be shared globally on social media platforms.
Platforms have had difficulty keeping up.
Facebook, Instagram, and WhatsApp parent company Meta stated that it seeks to eliminate artificial intelligence-generated content when it “may mislead, especially when it comes to video content.”
YouTube’s parent company, Google, stated in November that in order to use the platform, “creators must disclose altered or synthetic content that is realistic, including using AI tools, and we’ll inform viewers about such content through labels.”.
Raman Jit Singh Chima, Asia policy director at advocacy group Access Now, claimed that platforms are “holding their punches” because nations like Bangladesh, Indonesia, and India have recently passed laws to more strictly regulate online content and penalize social media sites for content deemed misinformation.
According to these nations, “this election cycle is actually worse than the last one because platforms are not responsive and proactive enough, nor are they set up to handle problems.” And that’s a pretty bad indication,” he remarked.
“There is a danger that the world’s attention is only on the US election, but the standards being applied there and the effort being made there should be duplicated everywhere,” he stated.
Jadoun, who had declined to create deep-fake campaign films for the state polls, is preparing to do so for the general election in India, where Modi is expected to win a third term.
These will be customized WhatsApp video messages from politicians to party members, not to voters.
“They can really have an impact because there are hundreds of thousands of party workers, and they will, in turn, forward them to their friends and family,” he stated.
To avoid any confusion, we will include a watermark indicating that artificial intelligence was used in its creation. That is crucial.
