How do you measure CSM sentiment in health scorecards?

Will Pagden

We have just rolled out a new Customer Health score right before I joined the business. There has been lots of back and forth around whether CSM sentiment should be included or not?

Right now it has around a 10% weighting and is dependant on the CSMs last interaction which they rate Red, Amber or Green. 
I want to update this and put more of a matrix in place that is more granular so we can really understand why the sentiment is that way and an aggregation of all key stakeholders within our customers.

Keen to know what others are doing in regards to this? 

John Bosch

Hey Will,

We just went through this as well.  In Version 1 of our health score, we had their sentiment in at 20% so that it was mathematically weighted enough to counteract bad product usage reporting that surfaced due to the health score.  In Version 2 we fixed the accuracy of the product reporting and added a category for "Customer Outcomes Status" which replaced CSM sentiment.  I am a big fan of giving CSMs the ability to report on how easy or difficult it is to work with a certain customer but we have now done that through effort and complexity reporting rather than sentiment.  Hope that helps.  

Will Pagden

@John Bosch, thank for the response, can you expand a bit more on how you're measuring effort and complexity?

Britt Hall

We trust CSM pulse/sentiment above all other indicators. Every operational metric can show green, but if the CSM has marked the pulse as red, it's a red account.

There are plenty of risk factors that our scorecard might not cover that will greatly impact the health of the customer. Allowing the pulse to override our calculated score gives us a chance to flag those issues and make better forecasts.

And of course, the pulse often/typically matches our calculated health score. We trust the system until it fails us, then always default to trusting our instincts.

John Bosch

The effort is reflective of the time spent.  So in our world, 20% of your time in a given week on 1 customer is extremely high but happens,  10% is high, 5% is probably still too high, etc....

Complexity: How complex is it to achieve our customer's desired outcomes?  

5 - Extremely Complex

4 - Complex

3 - Average

2 - Kind of Easy

1 - Easy

N/A - We haven't agreed on what the outcomes are or should be yet.  

We then report on time by customer, time by CSM, and time by complexity.  The last one is interesting because it will reflect when we are investing a ton of time with a customer to achieve a standard outcome.  This helps me to understand the difference between "Demanding" customers who aren't prioritizing the actions they need to take to be successful vs. "Demanding" because what we are trying to do is hard and justifiably requires more time from our team to figure out.

Will Pagden

Wow @John Bosch this is comprehensive and I love it. When my brain can cope with it, I may need to drill deeper into that with you!

Will Pagden

Wow thats a brave move @Britt Hall but if it works thats great! I have done it previously and it didn't work for me.

Where I am new into a company, for me its important to make data driven decision until I am fully up to speed with our customer base and CSMs

Britt Hall

Ha. Totally fair! We do a lot based on intuition and gut feel. Certainly doesn't work for everyone and will not work for us forever, but it's serving us well for now.

David Jackson

Will,

I have always kept CSM sentiment out of health scores but tracked it as a separate field.  That way you get a data view and a sentiment view.  This approach also works where you have both CSM and self-serve customers; in the latter you have no sentiment element.

Ronald Krisak

In the absence of a score early on, the majority of our health score was gut feel.  Over time, we have moved more & more to every category is an actual quantitative metric....and leaving the gut feel behind.  After reviewing this thread, I might re-think that on our next revision of our health score to add some of that back in. 

Will Pagden

I think it’s a tough one @Ronald Krisak, in my scenario we have so many measurable data points where do you start. it was interesting though, when creating the scorecards and weighting’s, I tasked myself with not reviewing the CSM sentiment. It actually came out extremely close, I think there were only 1 or 2 customers who had a different health score than sentiment. 

We have a very quick TTV so we get data very quickly, in addition we’ve built measures in that assess the data based on their lifecycle stage so 2 customers with the same data could have different ratings based on lifecycle.

Santosh Sahoo

In the early days of CS adoption, we had CSM sentiment as a much higher weightage. We created a customer happy path by retrospective analysis of most successful customers, then then agreed on 8 parameters which predicted a customer sentiment. These range from Support data, NPS scores, Executive sponsorship, Value Realized etc. A Weighted score of these 8 parameters gives us the customer sentiment now.