Three Questions Everyone Should Ask in a Survey

These stories are presented thanks to Who Sponsors Stuff, which gives you and your team the tools to quickly find and reach out to relevant sponsors for your newsletter. They track 350+ newsletters, have direct contact information for 6,000+ sponsors, and keep you on the cutting edge of who’s spending money in the email advertising space. Find out how their Sales Pro product can supercharge your ad sales operation today.

Unsure what to do next with your newsletter? Run a survey of your audience.

Surveys are one of the best ways to quickly get answers to the big questions that any newsletter operator struggles with, like:

• How can I improve the content in my newsletter?
• How can I grow my newsletter more effectively?
• How can I better understand who reads my newsletter and what makes them unique?
• How can I increase conversions to a paid product, like a subscription or course?

Newsletters are powerful because they can be a two-way conversation between you and your readers. But if you don’t make time to ask questions and truly listen to what readers have to say, how will you build a strategy around their needs?  

You can’t read your subscribers’ minds. Even individual reader feedback can only help you to a limited extent. However, you can ask your readers questions and hear what they have to say to help you create stuff they’ll love even more.

I’ll give you an example. When I got to the New Yorker, we had a weekly newsletter that highlighted what happened that week in arts and culture. However,  the product seemed a little undefined — clicks were low, and the list wasn’t growing, which suggested that we hadn’t built a newsletter yet that readers loved. 

So we asked the audience: What do you want from us in your inbox? The feedback was clear: Readers had trouble figuring out what books to read, what TV shows and movies to watch, and what podcasts to listen to. They wanted New Yorker writers to reveal what they enjoyed reading, watching, and listening to.

The result? We rebranded and relaunched our culture newsletter as The New Yorker Recommends — and it quickly became one of the biggest and most engaged newsletters in our portfolio.

I’ve worked with hundreds of newsletter operators on their survey strategy, and I’ve learned there are three big questions that can help you unlock great insights from readers. Let’s get into those three questions, figure out what else you might want to ask, and then discuss next steps so you can get your survey out into the world.

Always start your survey with these three questions

Whenever you run a survey about your newsletter, you should start with three types of basic questions:

1. Something numeric
2. Something about the value of your newsletter
3. Something open-ended

But why these kinds of questions specifically? Let’s get into it.

Ask something numeric

It’s so helpful to have readers give you some sort of score. This can be a straightforward question, like, “On a scale of 1 to 10, how useful is this newsletter to you?”

You could ask the same slightly different way: “How valuable is this newsletter to you?” or “How important is this newsletter to you?”

I’ll sometimes ask this question using a different scale: 1 to 4 so that a reader has to lean more positively or negatively. (A 5 on a scale of 1 to 10 doesn’t tell you anything.)

Larger teams often ask the numeric question using what’s known as Net Promoter Score, or NPS. This question was designed so large brands could compare their results to others in their sector. The NPS question always follows the same format: “On a scale of 1 to 10, how likely are you to recommend our product to a friend or colleague?”

If you use NPS and want to calculate the score, add up all the 9s and 10s you get and subtract the total number of 1s through 6s. The result is your NPS score, and the higher the number, the better the score.

So let’s say you get 100 responses, and they break down like this:

10s — 40

9s — 20

8s — 10

7s — 10

6s — 4

5s — 4

4s — 4

3s — 3

2s — 3

1s — 2

For the sake of NPS, the 7s and 8s actually don’t matter. What NPS really looks at are your superfans (the 9s and 10s) and your detractors (the 1s through 6s). So for this, we’d add up the 9s and 10s (a total of 60) and subtract it from the 1s through 6s (a total of 20). The result is an NPS score of 40 — a very positive result!

(You can use a tool like this one from SurveyMonkey to quickly calculate your results.)

What does a good score look like? In 2023, BlueLena surveyed 40 newsrooms and found the average NPS score for a media company was 25. (I’ve yet to see NPS benchmarks just for the newsletter space.)

Any of these numeric questions could be a good first question for your survey. Your goal is to get a score to measure your favorability at the moment. Once you’ve collected that score, that’s something you can measure against when you do another survey next year. Save the results — and see whether readers feel that your newsletter is actually improving year after year. 

Ask something about the value of your newsletter

With this type of question, you’re asking, What does this newsletter do for you, the reader? Why do you make time for it? Why is it valuable for you?

Come up with a list of reasons why someone might choose to read your newsletter. For instance, for a daily newsletter covering local news, reasons might include:

• I want to be informed about what’s happening in my area
• I want to be the first to know about breaking news
• I want more context to understand what’s happening locally
• I trust this newsletter to provide me with the information I need to know
• I want to stay connected to my community
• I want to discover new things happening nearby
• I want to read stories I won’t find anywhere else
• I don’t want to miss an important story

Come up with your own answers and see which ones readers choose. You’ll find that some answers will be far more popular than others. Take note — readers might be telling you that you need to emphasize certain types of content in your newsletter.

This question can also help understand your audience’s motivations or behaviors. Below is a great example from The Flip Side, a daily newsletter that provides readers with perspectives on the day’s biggest topics. They don’t tell you what happened — they assume you’ll get the actual news elsewhere — but they try to surface viewpoints from the political left and right to make sure you hear ideas you might otherwise miss.

There are a lot of different reasons why someone might be interested in a newsletter like theirs. So, they came up with a few different options for their survey. Maybe you read The Flip Side because you’re trying to get smarter or win arguments, or because you’re using it as a teaching device in your classroom.

What could they do with these results? Learning that many readers are also teachers might lead them to try advertising in publications aimed at educators. If they learned that readers were trying to become more open-minded, they could adjust the marketing language in paid ads to emphasize that as a benefit of subscription.

Here’s a great example of a publication using this type of question to identify the right marketing language. In 2019, the London Free Press launched a newsletter all about things to do in their community for the upcoming weekend. But there was still a big question: What did readers most want to see in the newsletter? They asked readers and randomized the answers — the choices showed up in a different order for every reader to reduce the chance that the first choice was the most-clicked one.

The result? Readers overwhelmingly said they wanted to get the most out of living in London. The Free Press then updated its landing page copy to reflect the message that resonated with readers.

You don’t have to be a genius marketer to come up with a slogan for a newsletter.  Just ask the audience, then use their own words to attract new readers to your newsletter.

Something open-ended

Make sure you always give readers the chance to share feedback in their own words. Readers will often have things they want to share with you directly — good or bad — as long as you give them the opportunity to do so.

You can ask this question a few different ways, depending on the type of feedback you seek.

If you’re looking for more positive feedback, you might ask something like: 

• What do you love most about our newsletter?
• What do you find most useful about our newsletter?
• What was one of your favorite issues of our newsletter?

Whereas if you want constructive feedback, you might ask for something like:

• Is there anything we could do to make this newsletter better for you?
• What would you like to see changed about our newsletter?
• If you could add one thing to our newsletter, what would it be?

As Sarah Ebner and Michael Hoole of the Financial Times wrote for Inbox Collective, these open-ended replies can be “incredibly valuable.”

Not only do we get some brilliant (and very amusing) comments along the lines of “X writer is a genius, give him a pay rise,” but we have also had useful remarks suggesting that: 

• Certain parts of a newsletter are confusing. 
• The email is too long or too short.
• Some readers would prefer more insight and fewer links. 
• Promoting premium content in standard newsletters is extremely frustrating for readers who can’t access this exclusive content.
• We can implement better signposting — for example, ending a weekly newsletter by calling out that it will be sent again at the same time the next week, so people know when to expect it and can build up a regular reading habit.

From this feedback, we’ve been able to make several changes to improve our newsletters for all readers.

Make sure you ask at least these three questions — something numeric, something about the value of your newsletter, and something open-ended — to understand how well you’re doing, what you’re doing well, and what you might be missing.

Ask additional questions

Your survey might just need three questions, and that’s OK! But most surveys seek to go a little deeper than just three questions — there’s a lot more you can learn from readers if you ask.

Most newsletter operators choose to keep their surveys relatively short, something you can finish in 3-5 minutes since shorter surveys typically get a better response rate. But others intentionally go longer, like Virginia Sole-Smith, the writer behind Burnt Toast, who ran a 50-question survey of her readers. Why ask so many questions? She told Inbox Collective’s Claire Zulkey:

I only want the people who are willing to spend 15 minutes slogging through this thing to fill it out because they’re the ones who have put the most into the community. They’re probably the most active in the comments. That’s who I want to hear from because they’re the ones who are in the community anyway and should be driving. If I’m going to make any changes, they would be who I would listen to.

You’re free to ask as many questions as you want in the survey. But make sure you’re asking something to either:

1. Learn something
   or
2. Act on something

For instance, maybe you’re curious about demographic data — you want to know about the gender, age, or household income of the average reader. That’s fine. But I’ll ask you: What will you do with this information?

Maybe you’ve got a good answer already: You’ll use it to build out your media kit for advertisers or to identify lookalike audiences to target with paid ads. That’s great — go ahead and ask the question!

However, some teams ask the question without any clear next step. In that case, I might encourage you to cut it from your survey, since every unnecessary question will lower your response rate.

With that caveat out of the way, here’s what else you might want to include in your survey.

Ask about the types of content they want to see more of

What do readers want to read more of from you? You could ask them to select from a list of specific types of stories. This is a great way to get feedback on a new idea — if you’re not sure if readers will be interested in a new story format or topic, start by asking them directly!

You could also offer them a handful of topics and ask them to choose from that list. If you go this route, make sure you’re as specific as possible. For instance, for the example below, Boulder Reporting Lab could have asked if readers would be interested in more stories about “Arts & Culture” — but what did that really mean? Were readers interested in coverage of local theater? Live music? Restaurant news? Instead of listing a broad topic, they asked for more specific answers that led to a better understanding of their audience’s needs.

Ask for a testimonial

Why ask for a testimonial on top of another open-ended question? By asking for a testimonial, you’re asking readers to give you feedback in a different way — you’re asking them to say, as succinctly as possible, why they love the work you do.

I asked this question once when working on a survey with a large local newsroom. We’d already asked, “Is there anything we could do to make the newsletter more useful for you?” But the testimonial question led to an interesting discovery: Dozens of readers told us they loved the newsletter because it helped them stay in touch with their hometown, even though they’d long since moved away. This opened our eyes to the possibility that there were lots of other readers who used to live in the area and who might like this newsletter. That led to a successful paid campaign on social media targeting readers who met that criteria.

Once you’ve collected the testimonials, you can re-use them anywhere you market your newsletter — like on a pop-up on your website or on the ads you run on channels like Facebook.

Ask about your paid offering

If you have a subscription, membership, or donation strategy, you can pose this question one of two ways:

1. For readers who already pay to support your work, ask them why they chose to pay or what benefits they use on a regular basis.
2. For readers who do not pay, ask them what might entice them to do so.

If you sell products, like a course or an ebook, you could ask similar questions about why someone bought the product or chose not to.

The goal of a question like this is to figure out if the reasons people do (or do not) pay match up with how you promote your paid products. For instance, maybe you’ll get feedback that many paying members paid specifically to get access to your members-only community, but you aren’t promoting the community when talking about the value of becoming a member. That’s great — you’ve just identified a strategy tweak to implement! Go update your CTAs to reflect the benefit that other readers said was so valuable to them.

Ask about how they discovered your newsletter

Yes, you can track the conversion rates on things like your sign-up page. But how did readers make it to that page? Your analytics dashboard might show the traffic as “Direct” or “Unknown,” leaving you in the dark about how a reader actually found you.

So, you might want to ask readers how they found you. Here’s a great example of this from a case study Federica Cherubini wrote for the Membership Puzzle Project about Outride.rs, a Polish news site, and their flagship newsletter, Brief: 

Follow-up surveys sent to new subscribers revealed a powerful acquisition pathway that quantitative data alone didn’t show: 15 percent of their readers said they subscribed as a result of a friend’s recommendation. When the team started paying more attention to sharing and recommendation behavior, they noticed that many readers took screenshots of the Brief when it arrived every Friday and shared it on their Instagram Stories.

The result? Outride.rs adjusted their strategy to share more content from their newsletter directly through Instagram Stories and included CTAs to sign up within those posts.

Ask for demographic data 

Who reads your newsletter? You can probably get some of the data from your ESP or from a tool like Google Analytics, but you can learn a lot more about readers just by asking them.

If your newsletter runs ads, getting that demographic data can be hugely useful when pitching advertisers on the value of your audience. This sort of information might be shared with potential advertisers in your media kit.

If your newsletter is connected to a non-profit, demographic data can also be helpful — particularly when writing proposals for grants. (Many large foundations will ask for audience data as part of the grant-writing process.)

Or maybe you just want to learn a little more about the audience to understand who does or doesn’t read so you can target future readers.

If you’re asking demographic questions, you might ask for:

• A reader’s age (based on an age range, like 25-34)
• Their location
• Level of education
• Household income (based on a range like $75,000-$100,000)

Ask about the quality of your advertisers

Do readers like the advertisers in your newsletter? Have they purchased something based on the ads they see in your newsletter? Ask readers — and then be sure to share the results in your media kit. Positive feedback might help you convince new businesses to spend money on ads in your newsletter.

Ask if they’d be open to answering additional questions in person or over the phone

Surveys can reveal a lot about your audience, but sometimes, you need to spend more time with readers to fully understand their needs. For instance, in 2021, the team at the Malaysia-based newsroom New Naratif ran a survey that revealed an issue with their membership strategy: Since their content and events were open to all readers, there wasn’t a compelling reason to become a member. Why pay for membership if there wasn’t any sort of exclusive benefit?
So they went back and talked to more than a dozen existing members over video calls. The results were striking, wrote Deborah Augustin, the then-membership engagement manager for New Naratif:

My video calls with members also revealed that members were enjoying our events and wanted more events where they could hear from the experts and contributors in our network. This led us to the decision to make events exclusive to members from May onwards. And just like that, we finally solved our exclusivity problem! 

If you need to ask additional questions in person or on the phone, one suggestion: Make sure you compensate respondents for their time. That might mean paying them directly in cash or offering them something in exchange — anything from a gift card to complimentary access to your product.

Ask readers to sign up for your newsletter

The reader who completes a survey is an engaged reader — so while they’re taking your survey, ask them if they’d like to hear even more from you.

If you’ve got multiple newsletters, ask a reader to sign up for an additional newsletter. You’re probably already collecting their email address as part of the survey, so signing up for more newsletters should take just a single click.

If you’re collecting feedback from readers who typically engage with you via your site, social media, or podcasts, ask them to sign up for your main newsletter at the end of the survey.

You’ll find that surveys, done well, can be a surprisingly good way to convert readers to a newsletter.

What if I only want to ask one question?

That’s absolutely fine — but if you want to go that route, think about running an in-newsletter poll, not a full survey.

Any of the topics I’ve already covered, from demographic questions to questions about content strategy, could be asked in a one-click survey like this:

Or you could ask for simple feedback, like asking for a thumbs up or down at the end of your newsletter.

With either strategy, you can always direct the reader to a form afterward to ask additional questions.

Many email platforms, like Beehiiv or Substack, offer the ability to run polls within a newsletter. Polling tools, like CloseAlert or Feedletter, can also be used to collect feedback. 

Some teams will keep things even simpler: In that 1440 example above, their team created two different surveys via Google Forms. Readers who clicked on the thumbs-up emoji were sent to a survey where they could answer why they liked the newsletter; readers who clicked the thumbs-down were taken to a form where they could say more about what they disliked. Clicks were tracked manually via their email platform.

When to run a survey

There isn’t a bad time to run a survey — any time you want clarity, you should think about running one!

But there are two times that are particularly common for running surveys:

1. At the end of the year — Many newsletters run an annual survey in November or December. Ephraim Gopin wrote in his guide to building an end-of-year survey, “My annual subscriber survey helps me improve what I provide and decide which new products or services I want to launch in the coming year. I can see what’s missing, what the needs are, and meet those needs.”
2. As part of a welcome series — Try asking readers about your newsletter, maybe a month or two after they sign up, to make sure you’re delivering a product that new readers are enjoying.

If you do run something as part of the welcome series, make sure you regularly make time to check the data. As I wrote in my guide to welcome emails:

Because survey responses tied to a welcome series will come in on a rolling basis, most teams don’t check the data every day. If you have a survey in your welcome series, check the data every few weeks — or at least once a quarter — to see if you can spot any trends or ideas to implement.

If you suspect that you may have a hard time remembering to check this data, set a recurring reminder on your calendar or to-do list to help you stay on top of this.

Tools for running a survey

There are a lot of tools that can help you run a survey — in my work with Inbox Collective clients, I’ve worked on more than a dozen different survey platforms. Truthfully, I haven’t seen a huge difference between most of these tools.

If your email platform offers its own survey tool, you might want to start with that for simplicity’s sake. For Mailchimp clients, for instance, I often recommend using the Mailchimp survey tool since it can allow teams to tag readers based on their newsletter responses.

If you’re using an outside tool, start with free options, like Google Forms, which should give you the flexibility to ask a variety of questions and collect the information you need. (Plus, you can easily move that data into a spreadsheet to slice and dice the data.)

Two others to explore: Typeform allows you to design forms with different backgrounds and images and includes lots of out-of-the-box survey templates, in case you’re not interested in spending time on the design of the survey but still want it to look good. SurveyMonkey also offers plenty of templates, as well as sophisticated dashboards for analyzing your survey response data. Both have free and paid plans.

How to get readers to take your survey

Once you’ve created your survey, there’s one big thing left to do: Convince readers to take it and start providing feedback. However, since most transactions these days are followed by a request to participate in a survey, it may feel difficult to ask your readers to fill out one more form. I’ve got a few tips that can help.

Start by sending an email about the survey to your intended audience, asking them to fill it out. Your audience might be your full email list, or it might be a targeted segment of the list, like paying members or highly-engaged readers.

Two tips for sending the survey email:

• Say how long it’ll take to complete the survey — Be honest with your readers. Don’t say it’ll take 2-3 minutes if it’ll take 5-10. I often recommend to clients that they ask their co-workers to take the survey and time themselves while doing it. See how long it actually takes and use that as a benchmark.
• Be direct with your subject line — If readers feel you’ve tricked them into opening the email and find that you’ve just sent them a survey instead of the usual newsletter content, they might be upset. So be direct with them — a subject line like, “We need your feedback to make our newsletter better” sets the right expectations up front.

You should also include asks in your regular newsletters. Add a link in the note at the top of a newsletter, or create a house ad that you run within the email to drive readers back to the survey.

Try promoting the survey in your newsletter footer, as the Financial Times does, and run that promotion for a few days to drive as many survey responses as you can.

You may want to promote the survey in other places, too, like your website or social media. (If you do this, you’re probably going to want to ask more general questions about your content strategy, not specific questions about your newsletter.) If you choose to promote the survey to a non-newsletter audience, make sure to ask them to sign up for your newsletter at the end of the survey.

Should you offer a reward for taking the survey?

A reward can boost your response rate, but think carefully about what you’re offering readers. For local newsrooms, for instance, many Inbox Collective clients have found that it’s better to offer gift cards to local restaurants than to a vendor like Amazon. 

I wouldn’t recommend giving every reader a gift card — tell readers that a handful of gift cards will be randomly given away to anyone who responds to the survey. (This also gives them a compelling reason to share their email address, since that’s how you’ll contact them if they win the prize.)

How many responses should you aim for?

This will vary based on the segment you’re targeting, but a general rule of thumb is that if you can get 2-5% of your audience to respond to a survey, that’s a representative sample size.

Do keep in mind: The people who respond to surveys tend to be your most opinionated readers — these are readers who both love what you’re doing and who hate it. You’ll probably get a lot of answers on the extremes, and that’s OK.

If you ask for demographic data, you may find that the responses tend to come from an older audience. You may want to make an extra effort to promote your audience in other places — like on social media or via a podcast — to try to pull in a representative sample of the full audience.

How to utilize your survey results

Once you’ve got the data from the survey, what do you do with all of it?

First, go through the responses at the aggregate level. What score did you get from the audience? Did readers tell you that they read the newsletter for the reasons you expected? What was the general audience sentiment from the open-ended feedback? 

Then, dig into individual responses. A warning: A percentage of these will be super useful, but you’ll also get a lot of answers that don’t tell you much. (“Nothing, it’s great!” is a common reply to an open-ended question about what you can improve in your newsletter.) You may have to sort through a lot of answers to find the useful stuff — but what you find might be gold. 

Then, start to think about how heavily you want to weigh the survey responses as you figure out your next steps. I always tell my clients that a survey should be a guide, not an edict. It’s something that can help you triangulate your way to the right decision. Think about what you want to do, how readers behave in your newsletters (or on your site), and then what they tell you via surveys. If all three sources point in the same direction on a particular idea, that’s a great sign.

Every team uses survey data in different ways. At the Atlantic, reader feedback helped their team craft a list of five core audience needs. One, wrote Emily Goligoski, the magazine’s former executive director of audience research, was the idea that readers needed the chance to “take a meaningful break.” Since readers wanted a break from heavy stories, editors made an effort to add lighter pieces to their editorial calendar:

As an editor, [Isabel] Fattal said this need for a break has also been useful for counter-programming and planning purposes. “It’s just as valuable for our readers to have a way to decompress, to follow us down new paths of curiosity, and to have a reading experience that’s fun but not vacuous.” She continued, “Some of the best Atlantic articles are the ones that make you both laugh and cry, and that’s what I think a meaningful break can do for our readers.”

Some teams, like the Financial Times, use reader surveys to identify newsletters that are underperforming and need significant tweaks. In an article for Inbox Collective, the FT’s Sarah Ebner and Michael Hoole wrote:

The newsletter at the bottom of our reader survey is a middling performer in terms of open rate and click through rate, but we’ve realized from this survey that, obviously, readers don’t feel passionate about it. As a result, we’ll be making some changes to this newsletter to see if we can improve it for readers.

How else might a survey help you figure out how to improve a newsletter? Documented, a newsroom covering immigration news, provides a useful case study. Documented’s Fisayo Okare wrote about an audience needs survey their team used to decide what content to prioritize within the newsletter. It led to some unexpected discoveries — and specific changes that needed to be made to their daily newsletter, Early Arrival:

We continually heard that for some readers in New York — which is where the majority of readers are based — their first point of contact for announcements about events — such as city council hearings, rallies, protests, and town halls — is Early Arrival. “Sometimes, I hear about rallies, and I hear about them after the fact. If the information was given on time, I could participate in it,” a reader said….

Now, we’ll have a section dedicated to events and job opportunities relevant for our audience. Readers will now receive a timely list of upcoming events or job opportunities, in-house, and from third parties. We will be experimenting with this category to feature ads and sponsored posts as an alternative avenue to generate revenue.

How else might you utilize survey results?

• You could add new content or sections to your newsletter to fill unmet needs identified during the survey.
• You could utilize testimonials in paid ads promoting the newsletter.
• You could adjust your sign-up page to emphasize specific reasons why current readers subscribe.
• You could take often-repeated phrases about the value of a paid product and weave those into sales emails or on-site CTAs.
• You could share survey data with advertisers to remind them of the value of your audience.

Give yourself time to really dig into the survey results, and use the data to steer your decision-making process going forward.

A few final best practices for running a survey

Make sure you regularly run surveys — Even if you only send out an end-of-the-year survey, that’s fine! Reader opinions may change from year to year, so you need to have a way to continually gauge feedback and update your strategy when things do shift.

When possible, keep your survey short — Most surveys don’t need to be longer than five or six questions, and the longer the survey, the lower the response rate. (Readers may not make it through a survey that has 10+ questions.) If you’re going to ask a lot of questions, consider offering a compelling incentive to those who take it.

Ask something to learn or do something — Don’t just ask a question to ask a question. Lots of newsletters, for instance, ask readers to state their gender as part of a survey. My question would be: What do you plan to do with that data? If you find the audience skews a bit more female than male, will you adjust your newsletter in any way? In most cases, the answer is, “No, we’re just asking because that’s what we’ve seen other newsletters do.” Instead, focus on questions that can help you better understand your audience or that lead to an action item.

Aim for a 2-5% response rate — If you don’t hit that goal right away, that’s OK — keep asking readers, in newsletters or on social media, to complete the survey until you’ve gotten enough responses to have a meaningful sample of the audience.

Make time to review the data — Don’t just give the NPS score a cursory glance and move on. Really dig into the responses at the aggregate and individual levels to see if you can spot trends or opportunities. You may need to spend significant time reviewing the results before you find the responses that unlock something great for your newsletter.

Thanks to our sponsor

The stories you’re reading on inboxcollective.com are made possible thanks to the generous support of our fall sponsor, Who Sponsors Stuff, which gives you and your team the tools to quickly find and reach out to relevant sponsors for your newsletter. They track 350+ newsletters, have direct contact information for 6,000+ sponsors, and keep you on the cutting edge of who’s spending money in the email advertising space. Find out how their Sales Pro product can supercharge your ad sales operation today.