How We Handle Unplanned Changes

What are unplanned changes?

Unplanned changes comes from various sources:

  • Penetration test remediation

  • InfoSec and AppSec may identify vulnerabilities that need to be patched

  • Product support

  • Browser incompatibilities and breaks

  • Sales engineering and support

All of these sources represent critical business needs and often disrupt our plan. Having a plan in place for how and when to interrupt the plan is important. There are three key steps to this: identify & log, prioritize and escalate, and resolve

Identify & Log

We first need to identify and log the reason for the unplanned change. Identifying and logging means understanding what work needs to be done and recording it. Let's take an an example from Customer Solutions to describe this.

Every day our customers reach out to our Customer Solution Specialists for support using our products. Often times, they reach out to tell us about a defect. The Customer Solutions Specialist confirms the defect through replication and logs the issue into our issue management system describing the issue in detail including an estimation of it's impact to the business.

When the unplanned change is a defect remediation the defect should be logged with:

  • A description of the defect

  • A description of the expected functionality

  • Current workarounds

  • Steps to replicate

  • Supporting images, gifs, or videos

  • An estimation of the defects frequency, severity, and scope.

All unplanned changes should be logged similarly. However, not all unplanned change is a defect. So, we can take the Customer Solutions example above and modify it slightly to fit a wider audience:

When logging unplanned changes we should include the following:

  • A description of the change needed

  • Supporting images, gifs, or videos

  • An estimation of impact to the business

Prioritize

After we've identified and logged the needed change we need to prioritize it. We should prioritize a change by categorizing it into one of four categories:

  1. Critical = Needs to be addressed immediately (hot fix)

  2. High = Needs to be addressed soon

  3. Medium = Can be addressed through Product/Project management scheduling

  4. Low = Does not need to be addressed with any specific timeline

The way we determine what category a change should fall into is by assessing it's importance. We do this by looking at it's impact to the business. Impact should be evaluated by considering the frequency, scope, and severity of the affect of the defect or absence of change.

Frequency

How often the issue affects our users or how often users will be impacted by the absence of the change. When thinking about frequency you should consider how often a user will observe the given defect or how often they will benefit from the change.

Scope

How wide is the surface area of the defect or change. You should consider what extent of the product is affected by the defect.

Severity

How significantly does the change effect the user or workflow. We should ask how impactful the change will be to our users experience. When thinking about severity consider the following levels:

For defects:

  • Severity 1 - The feature is broken and there is no workaround

  • Severity 2 - The feature’s most common usage path is broken but there is a workaround

  • Severity 3 - One of the feature’s less common usage paths is broken and there is a more common workaround

For Sales Engineering and Support:

  • Severity 1 - Change is needed to close a deal with a significant price tag or with a significant new customer

  • Severity 2 - Change will unblock a significant opportunity that is in stuck in the pipeline

  • Severity 3 - Change would make a material difference to a substantial prospect that is considering us

It's possible that a defect with high severity and narrow scope is not considered urgent because it may be a path rarely taken by our users. Alternatively, it's possible that an issue with low severity by high frequency is considered urgent because it's seen as a constant nuisance by our users.

Escalate

If during prioritization we determine that a change needs to happen immediately we need to escalate it immediately otherwise we should escalate it during regular prioritization (see below). Department heads may escalate product changes if they feel a change needs to occur.

How do I escalate something?

If your dealing with a defect, the first thing you should do is send the issue to Customer Solutions. They can work with the customer to find a work-around and will replicate the issue as a first step toward resolving it. Otherwise, if you're not dealing with a defect, the first thing you should do is bring it to your manager. Your manager can then evaluate how you've prioritized the change. If they feel it needs to be further escalated they should then notify product engineering.

Any department head can escalate an urgent change to engineering. The most common example of this is Customer Solutions. The manager of customer solutions regularly needs to make calls between those issues that can wait to be scheduled and those that need to get repaired immediately.

To escalate a change post a message in the #escalation Slack channel. Project management and engineering are listening to this channel and can respond quickly during business hours. If it's after hours and you need an immediate response use the #s_o_s channel.

Both the VP of Engineering and VP of Product should be notified on any issue that is escalated as critical.

Resolve

All changes will be reflected in the issue management system and the reporter who logged the request for change will be notified anytime the issue is updated including when the issue is shipped to production. In the event you need to communicate with the person working on an issue, each issue identifies the person who the issue is assigned to in the "assigned to" field. When issues are shipped to production issues are marked "In Release". In cases where the change was related to a customer, the Customer Solutions Manager responsible for the change or relationship will follow up with the customer to let them know that the issue has been resolved.

Regular Prioritization

Department heads have the option to attended a weekly prioritization meeting led by Project Management. Department heads from Product, Engineering, and Project Management will be in regular attendance. However, any department heads that a change in the next development iteration should attend to make sure the change gets planned. To join this meeting speak to the Project Management Team.