Open Rates Aren’t Reliable Anymore. Can You Still Trust Your Click Rates?

These stories are presented thanks to Who Sponsors Stuff, which gives you and your team the tools to quickly find and reach out to relevant sponsors for your newsletter. They track 350+ newsletters, have direct contact information for 6,000+ sponsors, and keep you on the cutting edge of who’s spending money in the email advertising space. Find out how their Sales Pro product can supercharge your ad sales operation today.

In fall 2023, four different Inbox Collective clients all reported a similarly peculiar thing: Readers seemed to be clicking on links in highly unusual ways.

Some saw higher-than-normal clicks on the first few links in their newsletter — click rates were five or 10 times what the clients expected to see. Others saw exceptionally high clicks on nearly every link in their emails.

None could figure out what was happening. Were readers clicking on the “View in Browser” link at high rates because there was an issue with how the email displayed in the inbox? Had Apple or Gmail rolled out some secret change that affected their click data? Were their email dashboards broken?

The actual answer was something that’s long been an open secret in the email space: The clicks in their newsletters — and yours, too — don’t all come from real readers. 

Many come from bots.

How big an issue is this? The answer varies based on who you talk to. Omeda, whose clients send over eight billion emails per year through their platform, told Inbox Collective that 63% of all clicks they see in emails come from bots. AWeber, one of the longest-running email service providers, shared their internal data: Only about 5% of clicks on their platform come from bots.

There’s a reason for the big gap between those numbers — it has to do with the types of email addresses each platform typically sends to. (We’ll get to that in a moment.)

But both of those numbers might be eye-opening, even for savvy newsletter operators. On nearly every email you send, it’s safe to say that at least some of the clicks come from bots, not people.

Who’s behind these bots? Can you actually trust the click data you see in your email service provider’s reports? And what does this mean for anyone with a newsletter ad strategy built around driving clicks?

Why bots might be clicking on your newsletters

These clicks aren’t the result of bots operating with malicious intent. To the contrary, they’re coming from tools that are trying to keep inboxes safe.

Barracuda, one of the world’s biggest digital security companies, launched their first anti-spam firewall 20 years ago. But Barracuda’s former chief technology officer, Fleming Shi, told Inbox Collective that email security has changed significantly in just the past year — largely due to the rise of artificial intelligence. (Shi left Barracuda a few weeks after our interview for a new role.)

“I think the data is the sort of the oil for the bad actors, and generative AI becomes the tool,” he said. “Generative AI is going to be able to create new phishing attacks faster.” 

The global metrics around cybersecurity back that up. In October, SlashNext, an email security platform, reported a 1,265% increase in phishing attacks over the previous 12 months. Vade, another email security platform, saw more than 493 million phishing attacks in the third quarter of 2023, a jump of 173% just from Q2. SlashNext researchers even discovered online forums with tutorials about how to use ChatGPT to write phishing emails. As Shi explained, these AI tools allow bad actors to move quickly, easily creating emails in multiple languages, or adjusting the tone or voice of an email to fit their target.

The platforms have countered with their own AI-based solutions. In October, one major security company, Proofpoint, acquired Tessian, which deploys AI to stop phishing attempts. Barracuda has their own AI tools to protect inboxes, Shi said. Deploying AI for security is the only way they can keep up with the volume of attacks.

What are these hackers after? They may aim to steal passwords and confidential business data. They could be trying to install ransomware, holding a company’s systems hostage until a ransom is paid. Or they could attempt what’s known as conversation hijacking, where a cybercriminal gains access to your accounts but lurks in the inbox until they gain information that could cause damage. With that sort of hijacking attempt, Shi said, “they don’t attack you until they figure out the best way to attack you.”

The FBI reported that businesses lost more than $2.7 billion via email in 2022 — and cyberattacks have only increased in the past year.

In response, email security platforms have ramped up defenses to protect clients. One common tactic is deploying bots to check links deemed to be suspicious. 

“Those systems are making sure that you are not sending anything sketchy,” said Tony Napoleone, vice president of client experience at Omeda, which publishes a quarterly report on email data, including bot activity. “They’re able to discern that by reading the various HTML bits in the email itself, or by clicking on those links — going through and making sure that wherever you are taking them is not going to wind up in some sort of sketchy place.”

Shi said the models that Barracuda uses to protect the inbox continue to get more sophisticated. “[The email] doesn’t even have to have a malicious file or link,” he said. “If the conversation looks different from how you usually converse, we will flag it.”

Barracuda works closely with Microsoft 365, and integrations with that platform allow them to build models to protect businesses based on their normal email behavior. “We can actually get into the backend and be able to build a model based on one year’s worth of your email corpus,” he said. “Now we know who talks to who, how they talk to each other.” From there, Barracuda can create a unique strategy to secure each inbox. Shi said that even within a business, the model that protects your inbox might be different from the one that protects your colleague’s.

But in the process of identifying and stopping bad actors, there’s a side effect: Your newsletter might end up seeing an unusual number of clicks. Some bots might stop clicking after a certain number of links pass a security check — which explains why some newsletters see bot-affected link clicks drop off for links lower down in an email. Jennifer Nespola Lantz, vice president of industry relations and deliverability at Kickbox, said she’s seen cases where timestamps show that an email appears to have been opened before it was officially delivered to the inbox.

Even though these bots only click to try to protect user data, many email service providers (ESPs) don’t do anything to filter out those clicks. When a newsletter operator looks at their data, they’re likely to see at least some bot clicks mixed in alongside clicks from real people.

“Those click metrics are, in some cases, a victim of this,” Napoleone said. “If your ESP is not removing those clicks from your top-line reports, you are going to be over-inflating or overstating your perceived clicks to either your own internal teams or to advertisers.”

It can be frustrating for newsletter operators to hear that these bots might be affecting their data. But Nespola Lantz noted that these security tools play a crucial role in keeping inboxes secure.

“The bots make it even harder” to figure out what readers are really clicking on, Nespola Lantz said. “But on the other end of it, they’re very much needed for anti-abuse.”

Which types of email addresses are most affected?

Three types of recipients tend to be most affected by bot clicks. The first are business email addresses, as large businesses try to defend themselves against business email compromise, or BEC, attacks. “Generally, where you see [bots] more is in B2B-type scenarios where you’re going to corporate domains,” said Tom Kulzer, CEO of AWeber.

The second type are email addresses for government employees. The third are .edu email addresses — these could be emails for readers at prominent universities, but they also come from K-12 institutions, too.

“If you’re at a .gov or .edu receiving domain, your [newsletter is] going to have increased scrutiny,” Napoleone said.

Readers using domains like Gmail, Outlook, or Yahoo tend to be less affected by these sorts of bots, Kulzer said. That’s part of the reason AWeber reported seeing far fewer bot clicks than Omeda — newsletters sent through AWeber usually go to a more general audience, whereas Omeda’s customers are more likely to be publishers or B2B senders who target business, .gov, or .edu addresses.

Can you still trust your email data?

The numbers provided by Omeda certainly caught my attention. This past August, 58% of all clicks through Omeda were tied to bots. And that was a relatively low month for 2023. In June, the number was 77% — meaning that fewer than one out of every four clicks came from a human.

AWeber’s data, which reflects an audience based primarily on more common domains, presents a far different picture.

“When I look at our click rates over the last five years, they don’t tangibly change that much,” Kulzer said. “There’s some seasonal ebbs and flows. But if bots were a major thing, you would see click rates change significantly.” AWeber’s data suggests that bot-affected clicks exist — they’re just not something that the average newsletter operator needs to panic about.

Even if you’re sending to a general audience, and bot clicks are far less of an issue for your newsletter, Kulzer noted other problematic bots that operators need to pay attention to. AWeber data shows that more than half of all email sign-ups are fake, which means that any newsletter needs to have a clear strategy for protecting its sign-up forms.

And then there’s the issue with open rates. In September 2021, Apple rolled out a new feature called Mail Privacy Protection (MPP), which tries to protect user data from email marketers. But MPP doesn’t really protect your data — it mostly makes it harder to understand if your audience is actually engaged. When users turn on MPP, Apple opens emails on their behalf.

MPP has been so widely adopted by consumers that, according to a November report from Litmus, more than 65% of all emails opened are connected to MPP. (Outlook, for comparison’s sake, represented about 3% of global opens.) 

All that’s led to significant inflation across the industry in terms of open rates. Among Inbox Collective clients, many have seen open rates rise by 10-20% since MPP rolled out.

Email data, Nespola Lantz said, can be messy. Even before MPP, many email metrics had flaws. What your email provider records as an open, she pointed out, is actually the result of a tiny image at the bottom of your newsletter being loaded in a reader’s inbox. 

“There’s still an education problem of what’s going on with these metrics,” she said. “That’s a larger problem, even outside of non-human interactions. What does this metric truly mean? Because people see opens and they think, ‘Oh, [a human] read it?’ No, it just means an image was downloaded.”

Here’s a general rule: When it comes to email, the data you see in your email service provider tends to be directionally accurate — it points in the direction of what’s actually happening. Let’s say your newsletter has an average open rate of 60%. Does that mean exactly 60% of readers are opening? I’d tell you that your actual engagement is probably lower than that, but a high open or click rate is still an indicator that your audience is engaged.

What does this mean for anyone with an ad strategy?

If you’re selling ads within a newsletter, try to be proactive. When sharing estimated performance metrics, like opens and clicks, with an advertiser in advance of their first newsletter ad, try to filter out bot-affected campaigns.

When it comes time to place the ad, make sure to ask your advertiser to share unique URLs with tracking data, or UTMs, appended to the end of the URL so they can track how many clicks actually came through from your newsletter instead of just relying on your internal data. The more data you can share, the better. And if you’re selling ads on a cost-per-click (CPC) basis, discuss in advance whether you’ll be charging them based on clicks shown in your ESP or the visits they track on their site.

Nespola Lantz said she recommends that clients download their data and sort through it in Microsoft Excel or Google Sheets to understand the scale of the bot issue. She also encourages newsletter operators to sit down with their advertisers and walk them through what’s actually happening with these bots.

“You have to educate your partner,” she said. “And with that education, as you start to look closer at your numbers, reset the expectation of what your agreements are.” In some cases, she’s recommended that companies rewrite their contracts to better reflect the data both parties have access to.

Napoleone said many Omeda clients have been able to use these conversations around bot clicks as a springboard towards long-term deals with partners.

“Our clients, once they realize what’s going on, they see this as a huge area of opportunity. They say, ‘Great, I have a chance here to be a market leader to go out proactively to my advertisers and say, ‘Let me tell you what’s actually going on out there,’’” he said. “They take what could be viewed as a negative and really spin it into a positive because at that point, it calls into question what their competitors are doing.”

What should you do to keep these bots from clicking on your emails?

There is no guaranteed way to prevent these bots from clicking — even highly-reputable brands will still see some bot clicks on their emails. But here are a few best practices to try to minimize the impact of bot clicks, according to the ESPs and experts I spoke to:

• Make sure you’ve set up all the necessary forms of authentication — Setting up the three main forms of authentication, SPF, DKIM, and DMARC, are musts for anyone sending out email. Implementing these tools is one of the best ways to prove that your emails are legitimate.
• Large organizations should consider setting up BIMI — Brands that have taken the step of validating their domain through BIMI are likely to be viewed as trustworthy, both by bots and human readers.
• If you send from a custom IP, make sure that IP is healthy — If you drive high engagement and low spam complaints on that IP, you’re in good shape. You should also set up tools like Gmail Postmaster to monitor your IP reputation.
• Be consistent with email sending patterns — Suddenly ramping up the volume of emails sent might appear suspicious to a security tool, as would regularly changing the email address you send from. Suddenly changing the type of content you send might also be a red flag. These are the kinds of patterns that spammers tend to deploy, and if you act like a spammer, you’re going to attract additional security attention.
• Be extra careful with the types of links you share in the newsletter — Using a link shortener, like a bit.ly, or a QR code might catch the attention of a security tool. If you want a reader to download a file, you may want to drive readers first to your site — an unexpected email attachment will probably be screened for safety reasons.

All of this data around bots might scare someone away from the email space. But Napoleone said he still believes email can be a powerful channel, both for newsletter operators and the brands that want to advertise in them.

“In the end, having that solid relationship with your audience, and being able to connect quality content with a quality audience, is going to continue to be the thing that really sets you apart,” he said. “The newsletter operators that can really nail that are going to be miles ahead of their competitors.”

Update: This story was originally published on December 27, 2023, and updated on April 2, 2024, to include an interview with Barracuda’s former chief technology officer, Fleming Shi.

Thanks to our sponsor

The stories you’re reading on inboxcollective.com are made possible thanks to the generous support of our fall sponsor, Who Sponsors Stuff, which gives you and your team the tools to quickly find and reach out to relevant sponsors for your newsletter. They track 350+ newsletters, have direct contact information for 6,000+ sponsors, and keep you on the cutting edge of who’s spending money in the email advertising space. Find out how their Sales Pro product can supercharge your ad sales operation today.