The United States’ Special Operations Command is looking for companies to help create deepfake internet users so convincing that neither humans nor computers will be able to detect they are fake.
The plan, mentioned in a new 76-page wish list by the Department of Defense’s Joint Special Operations Command, or JSOC, outlines advanced technologies desired for the country’s most elite, clandestine military efforts. Special Operations Forces (SOF) are interested in technologies that can generate convincing online personas for use on social media platforms, social networking sites, and other online content.
The document specifies they want the ability to create online user profiles that appear to be a unique individual that is recognizable as human but does not exist in the real world, with each featuring “multiple expressions” and “Government Identification quality photos.”
In addition to still images of faked people, the document also notes that the solution should include facial & background imagery, facial & background video, and audio layers, and JSOC hopes to be able to generate “selfie video” from these fabricated humans. These videos will feature more than fake people: Each deepfake selfie will come with a matching faked background, to create a virtual environment undetectable by social media algorithms.
The Pentagon has already been caught using phony social media users to further its interests in recent years. In 2022, Meta and Twitter removed a propaganda network using faked accounts operated by U.S. Central Command, including some with profile pictures generated with methods similar to those outlined by JSOC. A 2024 Reuters investigation revealed a Special Operations Command campaign using fake social media users aimed at undermining foreign confidence in China’s Covid vaccine.
Last year, Special Operations Command, or SOCOM, expressed interest in using video “deepfakes,” for “influence operations, digital deception, communication disruption, and disinformation campaigns.” Such imagery is generated using a variety of machine learning techniques, generally using software that has been “trained” to recognize and recreate human features by analyzing a massive database of faces and bodies. This year’s SOCOM wish list specifies an interest in software similar to StyleGAN, a tool released by Nvidia in 2019 that powered the globally popular website “This Person Does Not Exist.”
Within a year of StyleGAN’s launch, Facebook said it had taken down a network of accounts that used the technology to create false profile pictures. Since then, academic and private sector researchers have been engaged in a race between new ways to create undetectable deepfakes, and new ways to detect them. Many government services now require so-called liveness detection to thwart deepfaked identity photos, asking human applicants to upload a selfie video to demonstrate they are a real person and in some cases, they require a live audio and video feed to prove you’re actually real.
This more detailed listing shows that the United States pursues the exact same technologies and techniques it condemns in the hands of our foes. National security officials have long described the state-backed use of deepfakes as an urgent threat — that is, if they are being done by another country.
In September of last year, we talked about a joint statement made by the NSA, FBI, and CISA that warned “synthetic media, such as deepfakes, presents a growing challenge for all users of modern technology and communications. It described the global proliferation of deepfake technology as a “top risk” for internet users.
In a briefing to reporters, U.S. intelligence officials cautioned that the ability of foreign adversaries to disseminate “AI-generated content” without being detected — exactly the capability the Pentagon now seeks — represents a “malign influence accelerant” from the likes of Russia, China, and Iran. Earlier this year, the Pentagon’s Defense Innovation Unit sought private sector help in combating deepfakes with an air of alarm: This technology is increasingly common and credible, posing a significant threat to the Department of Defense, especially as U.S. adversaries use deepfakes for deception, fraud, disinformation, and other malicious activities.
Both Russia and China have been caught using deepfaked video and user avatars in their online propaganda efforts, prompting the State Department to announce an international “Framework to Counter Foreign State Information Manipulation” in January. Foreign information manipulation and interference is a national security threat to the United States as well as to its allies and partners,” a State Department press release said.
Let’s file this one under… What Could Possibly Go Wrong!!!!
Framework to Counter Foreign State Information Manipulation
https://www.state.gov/joint-statement-from-the-united-states-united-kingdom-and-canada-on-countering-foreign-information-manipulation/
This Person Does Not Exist
https://thispersonnotexist.org/
StableDiffusion
https://stablediffusionweb.com/
Thanks to Security Now for this information
https://twit.tv/shows/security-now
For more than 20 years, David Snell’s Tech Talk has been a regular spot on The South Shore’s Morning News on 95.9 WATD fm. At 8:11, David chats with show hosts Rob Hakala and Beth Foster about what’s happening in IT today. The subjects range from computer viruses, scams and cybercriminals to what Amazon, Apple or Microsoft are planning next.
He often shares new product information and reviews software that may help you, especially when there is a free version to try!
On this blog, he provides links, sources and other necessary information. And, on the Tuesday before Christmas, you can expect his annual NORAD Santa report!
If you have a question that you’d like him to answer on the show, please email him.
