How many questions is too many questions in an NPS Survey?

Will Pagden

Morning all,

I am currently creating our companies NPS strategy and looking to get this live. I am a big fan of the NPS survey being quite too the point with "Would you recommend us to a friend or colleague?" and keeping it quite short. Giving the option for free text feedback.

To give context I am also implementing a VoC program for product improvement feedback and CSAT surveys at lifecycle milestones.

 

I am keen to know what others have done previously that has worked well/hasnt worked well.

 

Thanks in advance!

Tanuj Diwan

In my opinion, survey length totally depends on the objective of the survey, some examples of types of surveys used

1.  If it just a relationship NPS survey that you send every quarter to the same customer it should not have more than 3-4 questions in total. And if someone is a promoter you can ask them to rate your Product on App stores/G2 etc or ask them for a referral. 

2. If it is a survey that is after a support interaction it can be a 2 question CSAT, CES gauging the overall satisfaction/effort, and an open-ended one.

3. If the survey is about onboarding or a custom project delivery then we can use NPS/CSAT question and also add some key drivers questions like Quality of Delivery, Timeline, Support, etc

4. If the survey is about a new feature than we can use CSAT with some key drivers as well.

And as you mentioned about VOC program, the last thing we need to make sure is to map the timing and channel of the survey for better response rates and reach out to customers who actually give a bad rating ( Close the Loop)

Let me know if this helps. 

Alex Farmer

Two minutes or less is required length in my view.  For B2B, I'm a fan of including:

1. Confidence Question (created by Lincoln Murphy): In the next 12 months how confident that your team will achieve its goals with our solution? Answers: Not at all confident, slightly confident, somewhat confident, confident, very confident.
2. Product-market fit question: Product market fit: How would you feel if you could no longer use our software/product? Answers: N/A - we no longer use, Not disappointed, somewhat disappointed, very disappointed
3. If you give me a 8, 9, or 10 in NPS question, we show a reference question - asking if they'd be willing to be a reference in the future.

Also found "Tell me one thing you'd like to see us start or stop doing?" useful at getting long-form feedback.

Will Pagden

Thanks @Alex Farmer  really useful. I like those questions, whether I will use them in the NPS survey or voice of customer surveys will need to be decided!

Brian Hartley

Paging @Steve Bernstein 

Daniel Sonnenschein

I strongly recommend adding the Sean Ellis test question:

How would you feel if you could no longer use our product?

For me, it has been the most informative to identify advocates and clients at risk, even if not NPS detractors

Steve Bernstein

You've been given some good advice by @Alex Farmer and @Tanuj Diwan .  I would add that there's really no "right" number of questions but generally we shoot for no more than ~3 minutes. That is, time it for the different persona and timing, and set expectations around how long the questionnaire was designed to take, but buffer that "ask" with your commitment to address what you are hearing so there's a strong WIIFM. 

Having done this sort of work in B2B for over 20 years we've found (contrary to popular opinion) the length of your questionnaire is NOT the key driver of customer participation in B2B. Long questionnaires could drive abandon rates, but you can obtain high customer participation (think 60-80% and higher) with a longer questionnaire by ensuring that your customers have a "WIIFM" to participate. Don't throw a survey over the wall with the common, "Your feedback is important..." because that tactic doesn't work. Instead position your listening/engagement program with a commitment to follow-up and *address* what you are hearing. You may not be able to do everything for everyone, but you can certainly follow-up in some way (1:1, 1:Account, in QBRs, 1:Many, etc) and at least address what they tell you by setting appropriate expectations and/or obtaining root cause along with a resolution. The better you can do that, the stronger customer participation will be. Imagine what you can do with 80% response rates (deep insight! opportunity to strengthen relationships!) from the right people (all key stakeholders!) at the right time.

A few other "top of mind" thoughts regarding the design:

1. The "recommend" question often fails in B2B because people can take it literally and you'll get comments like, "I don't know anyone to recommend to..." or "I'm to allowed to..."  (remember the B2C origins of NPS!!) so we'll often add, "... if you were to be in a position to do so..."
2. Although some of the questions you ask will change based on where the account is in the lifecycle (e.g. how you determine perceived value will be far different right after onboarding compared to before renewal), be sure you have a ways to longitudinally track the sentiment from the same contacts and accounts over time.. are they trending in the right direction?
3. One of my favorite questions, which can also drive expansion, is the "relationship ladder."  Present options like Vendor, Solution Provider, and Business Partner, and ask how the account perceived the provider's posture vs. what posture they want from the provider. The follow-up should drive detailed discussions about what a relationship upgrade" would look like.  Just be sure to only ask this question of the right persona, in the right customer segments, and at the right time (a good VoC platform makes this easy!) because otherwise you're liable to drum up interest for small accounts that may not make sense for your business.
4. Don't ask questions that attempt to isolate sentiment around your internal departments, e.g. "What do you think about our technical support?" or "How's your CSM?"  Those questions are inside-out, and also create friction internally. Instead, consider, "What do you think about the way way in which we manage the relationship with your company?" or similar... the root-cause process will undoubtedly uncover any gaps in customer expectations and where those expectations came from.

I'll avoid writing a book here so please reach out to me directly if I can help with templates, tools, processes, etc...!

Steve Bernstein

One other thought... PLEASE consider the rating scale on your questions. If you're asking "semantic differential" questions with text anchors you'll most certainly need to convert them to "top box" ratings and/or numeric scales in order to do any quantified analysis.  Here's an article that explains why you might want to use the standard 0-10 scale instead: https://waypointgroup.org/why-a-0-10-scale-is-your-best-option/

Will Pagden

This is great! Really good insight @Steve Bernstein. Thank you very much. Being in week 3 at a new place have 1 million and 1 things to do but this is high on my agenda so will be sure to use that wisdom.

Thomas Seelbinder

I've always approached this question with how I feel as a consumer. 

Personally, I find myself rarely completing surveys that are more than a handful of questions. 

our NPS is 3 core questions and two additional questions that are conditional based on previous answers. If we rate below a certain number, we ask what we can do to improve. If we rate above a certain number, we ask what they'd love to see from us in the future. 

 

 

Will Pagden

That’s the way I’m thinking. No more than 5 in total, 1 or 2 of which are conditional based. Thanks! 

Matt Myszkowski

Boom! Great response @Alex Farmer!

Only thing I will add - questions need to be "in the moment" - not a week or month later. Only then do you get relevant & actionable insights driving continuous improvements.

Matt Vadala

Just some further resources @Will Pagden  - this is a great read on VoC resources. May not be applicable now but great to get this insight

 

https://blog.frame.ai/redefining-voice-of-customer-in-2020

Will Pagden

Thanks @Matt Vadala - Will check it out.

David Ellin

@Will Pagden, while many people classify surveys as ‘NPS Surveys’, the only real NPS questions is the one about recommending to friends and families.

From my experience, if a survey can’t be completed in 3 minutes or less, it’s too long. IMO, it’s better to do a few short surveys (1-2 minutes) than 1 long survey. I don’t respond to long surveys.

I agree with my friend, @Steve Bernstein, that the length is not a driver of participation and is a driver of abortion. I also think that if your survey program includes Pulse and Transaction surveys in addition to NPS, short works better for continued participation.

Will Pagden

Thanks @David L Ellin - I have kept it really short and sweet, well under 3 minutes and also have a few CSAT surveys to complement along the customer journey which are also really short and easy to answer!

Thanks all for the ideas and feedback.