Bug prioritization framework/process

Nick Maugeri

Hi everyone!

We're revamping our bug prioritization framework/process at the moment but have been struggling with a few key areas. Some of the specific question marks we have are:

• How do you flag bugs and track the number of occurrences?
  • Does CS or eng own the "number of users" impacted?
• How do you communicate with the product team as bugs are resolved or being worked on?
• What do you use as the various tiers of bugs? We currently are looking at % of users impacted but the numbers seem really high (i.e. 50-90% users impacted is currently the middle tier in terms of priority). 
  • Are there ways you tie the bug to current and future revenue?
• What resolution times benchmarks do you use?
• Do you have any shareable resources around how your org defines 'bugs'?

Would love to hear how others approach this and work with their eng teams to resolve issues!

Brian Hartley

Hey @Nick Maugeri - I added my thoughts below.

• How do you flag bugs and track the number of occurrences? We use Salesforce Service cloud from the CS perspective and escalate/triage to Jira for our engineering team to tackle (if necessary)
  • Does CS or eng own the "number of users" impacted? We will make our best attempt to quantify impact once we are made aware of the bug.
• How do you communicate with the product team as bugs are resolved or being worked on? We have daily stand-ups both within the CS team as well as a cross-functional stand-up with product in which we share our top 5 list.  We just hired a production support engineer which will help us tremendously.
• What do you use as the various tiers of bugs? We currently are looking at % of users impacted but the numbers seem really high (i.e. 50-90% users impacted is currently the middle tier in terms of priority).  We loosely use a low, medium, high and critical tier structure.  Critical means the site is inaccessible for all users.  From their the definition is definitely subjective and can create friction between the teams (i.e. CS thinks it is high but engineering thinks it is low)
  • Are there ways you tie the bug to current and future revenue? Great idea, haven't yet.
• What resolution times benchmarks do you use? Nothing formal but we have a lot of opportunity to improve.
• Do you have any shareable resources around how your org defines 'bugs'? We typically subscribe to this definition of a bug.  The system is not functioning as it was designed and built (by definition of the code itself not personal opinion).  This can be frustrating as often times we may categorize something as a bug and the engineering team will tell us "that is how it was designed" and then we have to flip the "bug" to a "feature request".

Chad Horenfeldt

Hi Nick - we're in the process of revamping this as well. It's always an evolving process.

• How do you flag bugs and track the number of occurrences? CH: CX and Product / Eng does this. We use Jira to communicate with Prod/Eng. 
  • Does CS or eng own the "number of users" impacted? CH: No one owns this but Prod / Eng will ultimately determine what gets fixed and when. We bring data to show the impact and need for resolution. 
• How do you communicate with the product team as bugs are resolved or being worked on? CH: Jira and our Support Software (Kustomer) but mostly Jira which is integrated with Kustomer. It's important to have a system that allows a dialog and that the Prod/Engineering team use. At my last company it was Clubhouse. 
• What do you use as the various tiers of bugs? We currently are looking at % of users impacted but the numbers seem really high (i.e. 50-90% users impacted is currently the middle tier in terms of priority). CH: It's currently based on severity and then CX has a "top" label which we use to escalate items that may seem like lower priority. We're revamping this to include other criteria: ARR impact, client health, onboarding blocker etc...
  • Are there ways you tie the bug to current and future revenue? CH: We're in the process of doing this. At the moment, our largest customers do get preferential treatment. 
• What resolution times benchmarks do you use? CH: We don't but Prod does devote 20% of their sprint time to CX items and they track that in a dashboard that is visible. 
• Do you have any shareable resources around how your org defines 'bugs'? CH: Yes