The vast majority of the synthetic images do not indicate in the posts or on the pages that they are AI-generated, even though Meta requires users to label AI-generated content on its platforms while it is working toward methods to automatically detect such content.
Meta did not respond to a request for comment.
As generative AI technology makes it easier to propagate misleading content and outright disinformation, information researchers have grown increasingly concerned about the implications of unchecked AI images’ flooding social platforms.
Though certain hallmarks of AI-generated art can make it easier to tell real images from synthetic ones, the lack of systematic labeling has some users — especially older ones — falling for falsified content, according to comments on many of the posts. Those Facebook pages have not exhibited clear motives for their AI spam, but users are already pointing out the potential of a scam operation.
The Stanford study, which has not yet been peer-reviewed, also found that some of the images have been posted from Facebook pages that were stolen from other people or organizations, including a church in Georgia and a windmill seller, and then repurposed into AI spam pages.
One such page, now filled with AI images of Jesus and flight attendants, has kept the name of a high school band from North Carolina. Karen Jarvis, a spokesperson for Davie County Schools, confirmed in an email that the page Davie High School War Eagle Bands is now unaffiliated with the school, which was forced to create a new page.
“In fact, it was the original DCHS Band’s Facebook page, but was hijacked from our high school band, and has since become what you see today,” Jarvis wrote. “There have been countless attempts (from the band director, school officials, current band members, and alumni) to regain control, report the page/images, etc., but Facebook has been non-responsive.”
The phenomenon has also caused some observers to speculate that the pages are baiting gullible users to identify potential scam targets.
“AI-generated content appears to be a boon for spam and scam actors because the images are easy to generate, often visually sensational, and attract engagement,” the Stanford researchers wrote in their preprint paper.
They wrote that the pages use “batches of inauthentic followers” to give themselves a more legitimate appearance, as well as to engage with real commenters, and that the scam accounts would sometimes seek personal information from commenters or try to sell them fake products.
A search by NBC News found multiple replies from accounts asking to befriend commenters, each one using a similar script. One commenter who wrote, “Love it!! So cute!!” on an AI-generated image of a toddler snuggled up in a basket of kittens, received a reply within hours from a recently created account under the name Stephen Townsend. The account, which did not respond to a request for comment, displayed no personal information or posts aside from a profile and a cover photo, both of which were uploaded on the same day.
“Hello, Honestly I’m really impressed with your profile and personality. I also admire your good sense of humor here. I don’t normally write in the comment section, but I think you deserve this compliment,” the profile replied. “I would like to be your friend. Kindly send me a friend request please. If you don’t mind. Thank you.”
It’s the same style of comment found across Facebook, usually left as replies to comments on popular AI posts, as well as directly on users’ profile pages. Some users suspicious that the pages are driving potential scam campaigns are also leaving their own comments to warn those who may fail to distinguish certain images as AI-generated.
Hazel Thayer, a Facebook user who shared several of the bizarre images on TikTok after she noticed them in her feed a few weeks ago, said she now gets AI images like those maybe every 10 posts: “I’m scrolling right now. I just got one — that was four posts in.”
And from her perspective, it certainly looks like some fake activity.
“Because if you see the comments of these friggin shrimp Jesuses, it’s all people going ‘Amen,’” Thayer said. “I don’t think anybody is going ‘Hallelujah’ for our shrimp lord.”
Source: | This article originally belongs to Nbcnews.com
