Too much feedback is toxic (how to prevent an overdose)

Written by Laura Bosco on November 08, 2018

Intercom co-founder Des Traynor says feedback on your product is like oxygen for your business. But here's something most folks don't know—too much oxygen is toxic

The same is true for feedback on your product. 

An overdose will paralyze your team. It'll leave them confused about what to do, where to go, and who to listen to. But if you're intentional about why and how you gather product feedback, this is something you can avoid.

The importance of some product feedback

Too much feedback is a problem but, just like oxygen, you can't go totally without it. (I suppose you could, but your business won't survive long.) Some level of feedback from customers is still important because it plays a big role in your marketing, sales, and churn:

  • Marketing: Understanding what your customers need helps you craft compelling messages about meeting that need. 
  • Sales: Surveys give you insight into perceived value and price points for your product. 
  • Churn: Knowing why customers cancel today helps you identify ways to keep that from happening in the future. 

The goal, then, is to find a sweet spot. You want to gather actionable product feedback, but you don't want to get too bogged down in miscellaneous stats and information.  

#informationoverload

This is tough when you're a startup. When your product or business is just taking off, you're really hungry for user feedback, and you typically don't have enough of it. But as you gain momentum and users, a drought can quickly turn into a flood; you can go from too little information to too much information. 

If you build good feedback habits from the start, though, you'll stay afloat and efficient as you grow. 

Step 1: Start with a really good product feedback question (stupid questions do exist)

When you collect product feedback, have a really good question in mind

A good question is one that is specific, answerable, and relevant to your success. That last part seems obvious, but it's so easy to chase vanity and non-essential questions. They look easy and fun. But when you have limited time, resources, and manpower, those questions are distractions you don't need. 

So if you're spending time and energy on a question, make sure it matters to your business.

What bad feedback questions look like

Here are a couple to avoid:

  • How can we improve? A really vague question like this one is a poor starting point for gathering customer feedback. That’s because vague questions get you vague answers. A specific question would give you more actionable information. If you’re struggling to come up with a specific question, try focusing on: one feature, one process, one timeframe, or a specific customer group. 
  • Where should we go next year? This question is a double violation—it’s vague and unanswerable. It doesn’t give much direction on what to ask your customers, and chances are you’re not even sure what you mean by it. Run away! 
  • Should our mascot be a dog? Sure, this is specific. And it’s about connecting with your audience, so it loosely relates to sales and marketing. But is this really the most pressing question for your product right now? Does it make a big impact on your bottom line? I doubt it. 

This guy has some questions about your question.

If the question doesn't impact your business in a big way, find a better question. 

Example of a good feedback question

Say you have a product that gives working Moms a better way organize their to-dos. Hurray! You think one of the most valuable parts of your software is a kick-ass calendar. You're convinced this calendar can 10x your customers' productivity and help make the product super sticky. However, new signups aren't using it. You want to know why. 

Your Really Good Question is, "Why aren't new signups using the calendar?" 

Note that it’s specific, answerable, and relevant to your success.

Also keep in mind that a Really Good Question acts as your North Star for gathering feedback. The question is not (usually) something you ask customers; it informs what you do ask your customers. It’s your guide throughout the whole feedback process. 

How this shakes out

If you do a good job collecting product feedback (we'll talk about that below), you'll identify potential answers to your Really Good Question. Maybe customers aren't using the calendar because:

  • It's really confusing and clunky
  • Their existing calendars do a much better job at calendar-ing
  • They don't know it exists 
  • Unless they can share it with family, it's useless to them

Feedback will help you determine which of these potential answers—if any—are correct. 

Step 2: Gather the right product feedback, in the right way, from the right customers 

Once you have a Really Good Question, it's time to round up actionable product feedback. To do that, try using the who / what / how / when framework. Identify: 

  • Who: the specific type of customer you want to engage
  • What: a set of questions that relate back to your Really Good Question
  • How: the feedback method you'll use to prompt customers
  • When: the timeframe most relevant to your question

Image from Intercom's very helpful "3 rules for customer feedback."

This strategic approach is how you get the right feedback, in the right way, from the right people. 

Let's circle back around to our example question, "Why aren't new signups using the calendar?" For this question you could work with the following who / what / how / when

  • Who: paid new sign ups 
  • What: questions like, "did you know we offer a calendar?" or "what would make our calendar more useful for you and your family?"
  • How: questions in an email onboarding series or in-app surveys  
  • When: within 7 days of signing up

Keep in mind that the more feedback channels you use (the more Hows you employ), the more work you'll create for yourself later! Stick to a few channels at first—just 1 or 2. Trust me, your future self with thank you for reducing information overload. 

Please don’t waste anyone’s time 

Remember that you don't want to waste your customers' time with nonsense questions. Prompts such as, "do you like us?" may boost your ego, but the feedback won't provide actionable, meaningful data. And if you're using a survey format, adding in a lot of these vanity questions (aka lengthening the survey) reduces the likelihood that your respondents will answer. So please, don’t do it

Don’t bore customers with unnecessary questions

P.s. You won't always pick the right who / what / when / where but, if you're asking good questions, the feedback you gather could still be useful. Store whatever you collect. You never know what questions it could answer later.

Step 3: Dig out your x-ray goggles

Once you’ve collected responses, read through everything once at face-value. Then put on your x-ray goggles, and read through it all again. Find the stuff between the lines, and try and uncover why customers said something; see through the feedback to the core issues.

Okay, fine, these aren’t x-ray goggles. But we bet they look pretty similar.

If you struggle to do this, try using the jobs-to-be-done framework.

This framework considers what job a customer hires your product to do. It's helpful for overall marketing strategy, and it can be applied at a more granular level as well. 

With feature requests and feedback, this framework can help you uncover what a customer is trying to accomplish. With our calendar question, our Mom customers hire both the app and the calendar within the app to do specific jobs. Knowing what those jobs are can help us get much closer to answering our Really Good Question.

Keep in mind that you may need to follow-up with customers on this if it isn't clear. Make sure you ask permission to re-engage with them in case you get stuck.

Optional Step 4: Segment that ish

Sometimes, first-round feedback gets really messy. This is especially true if you get a lot of responses, or if you didn’t who/what/how/when very well. When that happens, feedback looks chaotic—like an abstract painting that's intriguing but totally confusing. 

Hmm, I see icicles. Maybe our next feature should be an icy-pop?

Instead of guessing at what all it means, use segmentation to help yourself out.  

The main benefit of segmentation is that it helps you organize feedback and find hidden patterns. For example, let’s say we surveyed every type of new signup in our app example. Feedback look like a mess, so we segment by customer type below. We break out freemium and paid tiers. When we do that, we find that freemium members have a different pain point than paid members. That’s a good starting point for figuring out our next steps or next Really Good Question! 

If you’re continually collecting feedback (via live chat, tickets, etc.), established segments can also help you compare responses and expose trends as you build a feedback library.

There are a lot of different ways you can parse responses, but here are three big buckets to get you started: 

1. Segmenting by Customer 

  • Customer type: What product level or payment tier does the customer subscribe to? A free user may have different opinions, or less valuable opinions, than a paid one. If you're trying to strategically grow a specific tier of your product, perhaps you want to prioritize feedback from that tier. 
  • Customer stage: Where is the customer in the product journey? They could have just signed up, upgraded, downgraded, or canceled. Jason Fried from Basecamp notoriously listens to customers that just signed up or left. He believes they provide the most valuable feedback. What stage best addresses your current question? 

2. Segmenting by Submission 

  • Submission type: Depending on your question, one submission type may be more valuable than the other. The feedback you collected was either: 
    • Given: initiated by a customer—Intercom heavily weights given feedback because it's unprompted
    •  Requested: initiated by you 
    • Observed: involved no direct interaction with the customer, such as analytics, search queries, or other data  
  • Feedback channel: How did the customer deliver the feedback? This could be: phone, app, email survey, on-site survey, social media, support ticket or any other method the customer used to submit their feedback or response. 
  • Volume: Did one customer request this? Or have more than 10 mentioned it? Tracking volume can help you see whether this is a one-off request or something many customers consider valuable.

3. Segmenting by Response Content 

  • Category: What are some broad ways you can organize responses? Common starter categories include: feature requests, pain points, and education issues. A request for color-coded calendar events would go under feature requests. Something like, "I don't know how to add events" is an education issue (and probably a UI one, too). If you need a more robust system, Intercom has some other really great ideas for expanding your categories. 
  • Product area: What product area or process does the feedback relate to? For example: onboarding, account setup, payment processing, or log in. If you're conducting multiple rounds of product feedback, you may want to remember some other things like date, customer ID, and whether you can follow up with the customer. 

Don't start from scratch!

Get a free feedback template that'll help you start organizing and segmenting feedback today. 

Remember that you really need to listen to feedback (don’t forget those x-ray goggles!) before you’re ready to consider an action plan. But if you overdo it with the feedback, or get too much of the wrong kind, you may end up more confused than when you started. 

How you create an action plan is a whole other topic...but follow the steps in this guide and rest assured you're off to a non-toxic start!

Laura Bosco is a web copywriter and people person. She helps small businesses explain what they do and create fun messages. Ping her on Twitter to start a conversation, or just say hi.