In an era where reality and fabrication blur with unsettling ease, the rise of deepfake technology poses an unprecedented challenge to our understanding of truth and authenticity.
Not long ago a Tik Tok account Ugyen_Tech of an Artificial Intelligence Company in Bhutan named NoMindBhutan posted three videos replicating the face and voice of Kinley Wangchuk of Azha Pasa, Passang Tshering of Passu Diary fame and singer Sonam Wangchen all promoting the AI company and its services.
The only problem was that none of the three famous figures have given consent to share their face or name to the company.
Bhutanese singer Sonam Wangchen expressed his thoughts on the deepfake video featuring him, commenting, “I came across the videos through my friends who showed it to me, and I didn’t really mind as it is very creative. However, it would be nice to have proper consent and permission, since it is very risky and scary at the same time, if it were to be misused.”
Passang also expressed concern saying that Deep Fake features are easily accessible with AI.
Ugyen Dendup, currently a 2nd year student in Gyelpozhing College Information Technology (GCIT), is the co-founder and CEO of the first AI startup company in the country called NoMindBhutan.
When questioned about obtaining consent for the deepfake videos posted on their TikTok account, he explained that they attempted to contact certain influencers, but did not receive their responses. However, they noted that these influencers acknowledged the videos by liking the comments they mentioned them in.
“As an AI research and development firm, we’re actively exploring the latest open-source models such as LLaMA2, Falcon 7b, and deepfake models, adapting them to suit our specific needs. Given our status as a small company, creating foundational AI models from scratch is beyond our current capacity, so we primarily leverage open-source options and customize them accordingly. Our experimentation with deepfake technology reflects our commitment to innovation. Our primary objective at NoMindBhutan is to develop user-friendly AI solutions and products that are accessible to all. Our initial offering, a personalized AI Chatbot, has already been successfully deployed for clients including BNB, NLCS, and Bhutan NDI.”
Ugyen Dhendup explained the process behind their creation of deepfake videos on TikTok, stating, “We utilize a selection of open-source models obtained from Hugging Face, which we operate locally on our devices. Hugging Face functions as an online AI community platform where individuals exchange their AI and machine learning models.”
While it is admirable that college students can form an AI company there are also concerns on the use of AI technology for deepfakes.
As deepfake technology advances at an alarming pace, the potential for its malicious use raises concerns about the integrity of information, the erosion of trust, and the impact on society as a whole.
Dr Tshering Cigay, ICT domain lead of DeSuung Skilling Program and a former CEO of Tech Park shared, “Ethical use of AI is a big concern today. While AI has a lot of positive uses, the potential for negative use is also huge. Deepfake, for instance, can be used to make engaging educational videos or movies, but it can also be used to make real looking fake videos to blackmail people. All consumers of information and media have to learn to be skeptical about everything they see online and try to differentiate the fake from the real.”
A TikTok user named Sonumbtopguy recently shared a troubling incident in a video. He recounted how he followed a person, thinking they were Bhutanese, as they shared images of their country. Appreciating the content, he reciprocated by posting videos about Bhutan, which he missed while living abroad. However, his experience took a dark turn when the person shared a link titled “the greatest chunk of all times.”
Upon clicking the link, Sonumbtopguy’s TikTok app froze. Despite restarting, he received an email shortly after. Since TikTok accounts are often linked with Facebook, scammers exploit background details from Facebook to execute their cons. Sonumbtopguy became a victim of deepfake technology, wherein videos are manipulated to appear genuine.
Despite dismissing the blackmail threats initially, the scammers sent the manipulated videos to some of his contacts. The experience has been distressing for him and highlights the swift advancements of AI, which can have alarming implications in a short span of time.
Another TikTok video by a user named Thukten Wangmo Gyeltshen shared her bitter experience on the deepfake video which has been traumatizing. In the video, she shared about STOPNCII.org which STOP Non-Consensual Intimate Image-based-abuse is a website where we can stop our deepfake videos circulating online without our concern.
Last year, a viral deepfake video featuring a Bollywood actress heightened concerns regarding the misuse of AI technology.
The world where anyone can be effortlessly manipulated into saying or doing things they never did—a realm where the line between reality and deception becomes increasingly difficult to discern is here in Bhutan.