Is anyone using regression analysis to refine their customer health dashboards?

Ed Powers

I'm curious how many people are pursuing the middle ground between mostly subjective health scores and full-on, automated predictive analytics from CS tech vendors. If you are using regression to manually analyze your data, what challenges have you found? If you aren't, what's the reason for that? 

Brian Hartley

Hi @Ed Powers my previous role had an entire department of research analysts conduct regressions analysis for us on our customer experience data.  I learned a ton about marketing research and statistics even though I didn't work on the research team  I would love to do the same at my current company but we are too small to justify an in-house analyst team.

Marijn Verdult

We recently did a Regression Analysis on all our inboud leads to determine a Lead Score: i.e. should the prospect receive and automated demo or should an Account Exec reach out.

We would love to do the same for our Customer Health but unfortunately we dan't have enough data points yet to find significant relationship, the sample size is just too small...

 

Marijn Verdult

@Brian Hartley - do you have enough and especially clean enough data on all your customers? I.e. do you have data on the factors that could be an indicator?

If so, you don't need an in-house analyst team you can even make a start with having justified health scores based on the data using Excel. Of course, you would still need enough samples and data you (and the rest of your company) can trust

Brian Hartley

Hey @Marijn Verdult great question.  I think we have clean enough data.  What data points would you look at, once you have enough?

Marijn Verdult

@Brian Hartley Hard to say without knowing your company/business/customers. The best way is to start with a large number of hypotheses - that's something that should come from within the organisation or from leadership and don't need to be based on anything, can be just something that COULD be. (Nice brainstorming session for the Friday drinks hangout????)

• Does the breadth of adoption influence renewals?
• Does the amount of activities / recorded touches in the CRM influence renewals?
• Does the industry influence renewals?
• Ect...

This number could be large since we will disregards a few of them because some are dependant on each other (like support tickets openend and support tickets closed - we would only need one or the other or a ratio between the two), others will be rejected (no correlation) or others are influenced by external factors (for example, you generally speaking try harder for large deals to renew and safe them so comparing large and small deals is not fair).

Then, testing these hypotheses is relatively easy if you can create an Excel dump with all the data. I will spare you the detail but would be happy to assist.

The result could be that we have a few factors that influence renewal (some stand-alone but sometimes 2 that work together) with an associated weight to them. Almost like, Prio 1 factor, Prio 2  factor, ect..
 

Marijn Verdult

What I have explained here above is called the Scientific Method
 

Brian Hartley

@Marijn Verdult Great stuff, let me take a minute to digest and will holler at you if I have questions.

Marijn Verdult

@Brian Hartley Absolutely! 

Steve Bernstein

Hi Ed -- The predictive capabilities from regression usually are dependent on having enough data, and for typical B2B environments that is often hard to come by.  And, for environments where you can acquire sufficient data to analyze, we also find that there's a non-linear nature to this, i.e. some drivers are more impactful for "promoters" while others are more impactful for "detractors" and non-linear regression (ordinal logistic regression) creates another layer on the thinking.  Here's an example that investigates the slope of the curve, where you can see that while "Technical Support" has the lowest scores, we find that improving Support won't really drive improvements because the data predicts that the Business Value attributes are far more likely to create promoters for this client (green = promoters, yellow = passives, and red = detractors):

That said, we have seen the strongest predictor of "loyalty" is very simple and doesn't require much sophisticated analysis: Does the customer participate in joint planning exercises?  For example, if you're having a QBR with an account then you'd ideally want to acquire "current-state" feedback from all those contacts in an account that should participate... are they giving you the feedback that you need to be able to effectively address the gaps?  If not, then you certainly know that they're not "with" you and our research finds these accounts are up to 14x more likely to churn (for  well-run program, and 3-5x more likely for a low-participant program). 

Andreas Knoefel

I have a Math Ph.D., so data is not scary to me. For many others it is, and the terms and concepts look like witch craft and are thus shunned, or they rely blindly on some OOTB. 

If you don't have a knack for data, you end up garbage-in/garbage out, depending on how clean, complete, and continuous your inputs are. That is why I run some manual analysis with regression and more sophisticated techniques, run simulations on the suggested results with a small team before I roll out a new process.