Ads narrated by the AI-created voices of country stars like Combs, Lainey Wilson, and Tim McGraw are popping up online and preying upon a passionate fan base
It looks like Luke Combs and sounds like Luke Combs, but the words the country star is “speaking” in an online ad peddling “miracle” weight-loss gummies sure aren’t his.
“I was at an all-time low and then God blessed me with the miracle root gummies. My good friend Lainey Wilson had recommended them to me,” a voice that resembles Combs’ says in the video, which began circulating on Facebook and Instagram last month.
It’s no surprise that it’s all bullshit: Wilson’s recommendation, Combs’ testimonial of how he “shaved off 46 pounds,” and, especially, the voice — a meticulous AI facsimile of Combs’ twangy North Carolina baritone.
Combs and Wilson are just the latest country singers, joining names like Tim McGraw, Garth Brooks, and Miranda Lambert, to have their likenesses hijacked online by AI scammers. The frightening thing is that some are falling for it, because, at least in the case of Combs, the AI voice is just that good. And people in the country-music industry are pissed.
“The first version of it, you can tell it definitely wasn’t Luke. But the second and third versions, where they kept tweaking the voice, got closer and closer,” Chris Kappy, Combs’ manager, tells Rolling Stone. “I know Luke’s speaking voice like the back of my hand, but people I know were falling for it. Then I started having fans email me saying, ‘Hey, I haven’t gotten my gummies yet,’ or asking, ‘Is this real?’ I was like, ‘Oh, my God, I have to say something.”
Kappy posted about the fake ads on Instagram to warn fans and sound an AI alarm for the industry. “To other managers out there, AI is a scary thing and they’re using it against us,” he wrote. “If you see this happening, please report it as quickly as possible.”
“Scams have been going on in the country-music fan base for a long time, and it’s because the fan base is so passionately loyal about the artists that they can prey on that,” Kappy says. “I’ve been talking to different managers and saying, ‘You need to pay extra attention to this because they’re going to come for everybody.’ We need to keep our heads on a swivel right now because it’s only going to get worse. It’s our job to keep educating the fan base about how it’s happening and what we’re doing.”
Editor’s picks
Unfortunately, like anyone who has been impersonated online knows (including Rolling Stone journalists), shutting down imposters can be, at best, a game of Whack-a-Mole. Technically, Instagram has policies in place to prevent impersonation, but there’s been difficulties in enforcing them, Combs’ manager says: “Instagram is overrun by fake accounts.”
Both Combs’ and Wilson’s teams have explored legal action, but against whom, exactly, has been hard for them to ascertain. According to Kappy and a report on Snopes.com debunking the singers’ endorsements, the ads are originating from somewhere in India.
“There’s no real way to get in front of it because, sure, you could go after the creators — if you can find them,” says Mandelyn Monchick, Wilson’s manager. “You don’t know who to go after. Do you sue the person who created the fake ad? Or the software company? Who’s the target here?”
While her team worked to shut down the ads behind the scenes, Wilson, who leveled up as country’s newest star with the release of her album Bell Bottom Country, a string of CMA awards, and a prime spot opening for Combs on his stadium tour, posted a personal video online cautioning fans about the scam. “I’m sure a lot y’all have seen some ads about me losing weight, being hospitalized, and that I started taking some weight-loss gummies and blah blah blah that saved my life. Well, surprise, it ain’t true,” she said. “I just don’t want y’all spending money on something that ain’t real.”
Related
“It bothered Lainey because she stands for people being authentic and feeling comfortable in their own skin,” Monchick says. “She said, ‘I don’t want people thinking I’m a phony or people feeling bad about themselves because they think they have to lose weight.’ It’s ironic that authenticity is the thing that drives [country singers] to the top. It’s what people are attracted to and gravitate to — and it’s the perfect storm for AI to come into the picture.”
It’s a storm fueled by country fans who relate to the singers and the availability of cheap technology. Zach Wener, the founder of AI voice startup Uberduck, says re-creating a voice is among the most simple of AI feats. “There’s commercial products where you can go out and sign up for $10 a month, take a clip of somebody’s voice, and generate a voice that sounds kind of like them. Then type whatever you want into a text box and generate audio output,” he says.
It’s even more successful if there’s preexisting interviews and audio — and for celebrities, there’s a ton. For the Combs fake, the scammers likely culled words and phrases from the singer’s appearance on Joe Rogan’s podcast. “With podcasts, like Joe Rogan’s, the audio is obviously super clean, they use high-quality mics, and there’s no background noise whatsoever,” Wener says, “so that data would make the job of someone who wanted to re-create a voice a lot easier.”
It also gives the AI scammers a leg up if the person they’re mimicking has spoken about the specific topic in the past. For instance, Combs has talked openly about body image, and Wilson acknowledged to fans that she’s recently gotten thinner, not from weight-loss gummies but from a hectic schedule of performing.
“Lainey’s known for being curvier, and then she lost weight. People were starting to comment on that,” Monchick says. “So there is that authentic part again.”
The authenticity and openness inherent to country music is arguably what also makes its stars — and their fan bases — so exploitable: Fans want to believe. Even Wilson’s own bus drivers were nearly fooled by the ads.
“When the gummy stuff started, we were talking about it on the bus,” Monchick says, “and Lainey has two ladies who drive, and one of them goes, ‘Wait a second, are those the same gummies that Miranda Lambert had been using?’”
Jennifer Grygiel, associate professor in communications at Syracuse University and an expert in memes, says it may look like the scammers are zeroing in on country fans specifically, but it’s actually more nuanced than that.
“It’s about audience, and that’s nothing new,” Grygiel says. “But if you’re talking about those two country stars you mentioned, there’s going to be a slightly different demographic. Whoever is selling these weight-loss gummies has essentially said that this is who they think will be attracted to this product. American culture has put forth this idea that you’re supposed to look a certain way, and it’s really unhealthy.”
While any weight-loss gimmicks can have serious side effects, Kappy worries about the possibility of the AI fakes one day selling more malicious products. “What happens when they start saying, ‘Invest in this or do this’?” he says. “That’s the scary part to me, when they begin actively using this against the fan.”
Trending
“It can dilute and wreck the consumer confidence in artists,” says Monchick. “I hate to doomsday it, but I think some bad shit is going to happen before they do something about it.”
Combs meanwhile has acknowledged the ads, but in a subtle self-deprecating way: “PS,” he captioned a recent Instagram post of photos of himself, “the weight loss gummies ain’t workin’.”
