Actionable & Meaningful Feedback

Matt Myszkowski

Hey all - back from a period of annual leave and catching up on some posts, but one of my own....

Currently we (SAP) use a twice yearly wave of surveys that go to our customers with multiple questions regarding their experience using the Customer Success offering (it is a paid engagement) and the experience using our software, ending with a CES (Customer Effort Score) defining question.

In my experience (working in CS for 12 years & for a VOC software provider previously) is that we should dramatically reduce the number of questions, increase the number of surveys but make them very specific to a particular touchpoint while asking "in the moment". So for example, you deliver a QBR and when that touchpoint is closed in your CRM/CS software and ideally within 24 hours  a survey is issued either via email/SMS asking no more than 3 questions around that very specific touchpoint - something like:

1. Did the QBR help you understand the value of "XYZ"? (+ optional comment field)
2. Did the QBR satisfy all your needs? (+ optional comment field)
3. Did your CSM facilitate the QBR to the quality you demand from SAP? (+ optional comment field)

What are your thoughts? What are you all doing? How does the two (or more) ways of issuing surveys change your response rate and ability to drive actionable change/improvements?

Anita Toth

Matt -- Is anyone doing customer interviews to back the survey data and get deeper, more meaningful responses?

Rohan Sheth

Hi Matt,

This is a great question. I completely agree with your thought here to increase the number of surveys while reducing the number of questions. Not only does it help keep a pulse on the customer sentiment more frequently but also increases the response rates since they don't need to spend too much time completing the survey.

We have been encouraging and supporting our customers to trigger contextual surveys. For example, send out a short survey to all users who have accessed a new feature for 'x' days after it was launched or a survey to all those users who successfully completed their onboarding process, and so on. We've noticed a considerable spike in the response rates since these take only a few seconds to a couple of minutes to complete.

In addition, they do still continue to trigger an NPS/CSAT/CES survey once or twice a year.

Anita Toth

@Rohan Sheth  I'm being a pain here -- same question as to Matt --  Is anyone doing customer interviews to back the survey data and get deeper, more meaningful responses?

Rohan Sheth

Hi Anita - We work mostly with SaaS businesses and have seen this being managed in a couple of different ways; the first is when the survey is triggered within the product, based on the response type a follow-up question is enabled to get more details. The thing to keep in mind is that these are personalized to the individual user based on their use of the system and not a generic blast that goes out to all users.

For example, if the question was about the utility and experience of the new feature and the response was negative i.e. a thumbs down or sad smiley or radio button that indicates unsatisfactory experience, a follow-up question pops-up asking for more information. This question is shown only to these set of customers who have had a poor experience. 

Once the survey is completed, the customer success teams export the responses to analyze the data and reach out to their customers with specific questions during the next touchpoint. This again is possible since the survey was directed to solicit their feedback of a specific touchpoint or experience rather than a generic one.

The intent is still to listen to the customer's feedback, acknowledge it, and take appropriate action. It may not always be possible to immediately resolve the issue but critical to understand the problem and communicate with the customer.

Matt Myszkowski

Hi @Anita Toth

Yes, but in a relatively unstructured way currently. We are in a "proof of concept" phase with this as I try to take the argument to a wider, more senior audience trying to drive change in our business. in our entire VOC programme.

David Jackson

Matt,

Like you, part of my background is with a VoC provider.  This e-book explains how we used to advise our customer on VoC program design.

Cheers

Dj

Steve Bernstein

Hi @Matt Myszkowski -- good question, and while I don't mean to rain on your parade there are a few things we've learned over the 20+ years...
1. CRITICAL definition: There's thinking that fewer questions ==> higher response rate. Not accurate. Fewer questions MAY lead to lower abandon rates (where someone starts and doesn't press the "submit" button).

2. After doing this for so long, our clients see very low abandon rates. That is, our approach is to explicitly communicate something to the effect of, "We've tailored these questions specifically to your role in working with us, and have ensured the questionnaire will take no more than 3 minutes." Assuming you do that, I can confidently say that you will see only ~1-2% of contacts will abandon. The bigger lever for response is the relationship with those contacts and the extent to which they understand "what's in it for me" to expose themselves? You may have a different viewpoint, but in B2B it's NOT about the 3 minutes, it's about whether there's some payoff for the risk in exposing themselves.  No one wants to give feedback if it's going into  black hole, and no one wants to give "candid" feedback that might negatively impact working relationships. So your questionnaire and your communications need careful thought an planning to avoid those land mines.

3. So how do you get customers to want respond? There needs to be clear communications around the WIIFM, and the better you can demonstrate listening the more you'll see response rate climb. Our customers often see 80%+ response rates, which not only gives CSMs great insight into the health and symptoms (what's working well  not-working well) in an individual account, but also allows for trustworthy (representative) analysis at the cohort (aggregate) level so you can pinpoint the right root-causes to stomp out.

4. Specific events are rarely the right time to "survey" (we prefer the word "assess"). I know the QBR example was just an example, but the take-away is that you may need feedback more about the outcomes more than "satisfaction." For example, who wants to attend a QBR? If they were "satisfied" with the meeting then maybe you think everything's ok... but the QBR is there to strengthen relationships and drive action, so following up after sufficient time has elapsed for the customer to be able to assess those elements is likely to be more valuable. The point is that the "journey" needs more time for the customer to realize, and they may not be able to fully assess/appreciate a particular experience immediately following the event.

5. Have toxicity/fatigue rules... No one want to respond to 5 different assessments (nee, surveys!) so taking a "journey" perspective while also understanding the impact of the experience on the overall relationship would be key, and then be sure to throttle so that no one gets too many emails...

6. To @Anita Toth's point, customers will only be able to provide you with SYMPTOMS, not root-causes, and you'll need to talk to at least some of them to understand where any gap in expectations came from. Why did they expect something different and where did those expectations come from? What was the impact (to the customer) of not meeting the expectation? Unless you want the customer to write a book that you'll be expected to fully digest, a follow-up process is critical.

As always, I would appreciate everyone's thoughts/comments on my reply... maybe I missed something or misunderstood and if a live call is better then I'd be happy to arrange something for all who are interested.
/Steve

Anita Toth

@Matt Myszkowski    Got it!  I have some data that can help you with that. ? Let me know. Happy to share. 

------------------------------
Anita Toth
Customer Retention/Churn Consultant
------------------------------
-------------------------------------------
Original Message:
Sent: 09-09-2020 06:34
From: Matt Myszkowski
Subject: Actionable & Meaningful Feedback

Hi @Anita Toth

Yes, but in a relatively unstructured way currently. We are in a "proof of concept" phase with this as I try to take the argument to a wider, more senior audience trying to drive change in our business. in our entire VOC programme.

------------------------------
Matt Myszkowski
VP, Customer Success Management, EMEA
SAP
------------------------------

Original Message:
Sent: 09-08-2020 11:46
From: Anita Toth
Subject: Actionable & Meaningful Feedback

Matt -- Is anyone doing customer interviews to back the survey data and get deeper, more meaningful responses?

------------------------------
Anita Toth
Customer Retention/Churn Consultant

Original Message:
Sent: 09-07-2020 07:07
From: Matt Myszkowski
Subject: Actionable & Meaningful Feedback

Hey all - back from a period of annual leave and catching up on some posts, but one of my own....

Currently we (SAP) use a twice yearly wave of surveys that go to our customers with multiple questions regarding their experience using the Customer Success offering (it is a paid engagement) and the experience using our software, ending with a CES (Customer Effort Score) defining question.

In my experience (working in CS for 12 years & for a VOC software provider previously) is that we should dramatically reduce the number of questions, increase the number of surveys but make them very specific to a particular touchpoint while asking "in the moment". So for example, you deliver a QBR and when that touchpoint is closed in your CRM/CS software and ideally within 24 hours  a survey is issued either via email/SMS asking no more than 3 questions around that very specific touchpoint - something like:

1. Did the QBR help you understand the value of "XYZ"? (+ optional comment field)
2. Did the QBR satisfy all your needs? (+ optional comment field)
3. Did your CSM facilitate the QBR to the quality you demand from SAP? (+ optional comment field)

What are your thoughts? What are you all doing? How does the two (or more) ways of issuing surveys change your response rate and ability to drive actionable change/improvements?


------------------------------
Matt Myszkowski
VP, Customer Success Management, EMEA
SAP
------------------------------

Anita Toth

@Rohan Sheth  This sounds like a great way to personalize the experience and collect the right data based on key activities by the users, realizing that not all issues can immediately be solved.

I'm really curious:

Have you heard from customers how they like this system?

Do they find value in only being surveyed at key points with direct follow-up from their CSMs?

What do they think? ?

Anita Toth

@Steve Bernstein  Great points you raised here. As always, businesses have to tailor what they're doing to their customers, not simply doing something because that's (presumably) the 'best' way to do it. 

I agree that the biggest focus should be on value for the customer for having taken their precious time to answer a survey.

Let them know what that value will be if they fill out the survey. Close the loop for them too by following up. Make them feel that the time they took to respond will benefit them in some way. That really goes far with helping your customers feel valued; it increases loyalty; and increases the likelihood they'll fill out another survey in the future.

It's all a learning process but if the focus is on the giving value to the customer in exchange for their time to answer the survey, you're on the right track. ??

Rohan Sheth

@Anita Toth The increase in response rates that our customers have seen in the last few months definitely suggests that they are happy to share their feedback when relevant. And the direct follow-up from their CSMs shows that they are listening and care enough to reach out.

Anita Toth

@Rohan Sheth This is great to hear. ? Sound like your customers are appreciating the outreach, if response rates have increased.

It's that self-perpetuating circle where customers feel more valued (CSMs reaching out) so they're more likely to respond to the survey, which makes them feel more valued when the CSM reaches out, leading them to want to fill out the survey. ?

Thanks for sharing this. Sounds like this system is working well. 

Matt Myszkowski

So many great points @Steve Bernstein

I would love to discuss this further with you and others (I know I am due to get time with @Anita Toth!).

@Jeff Breunsbach - One for Thursdays?

Anita Toth

@Matt Myszkowski  +1 on the idea of having this as a Thursday topic