Verifying the Effectiveness of Corrective Action
By Craig Cochran - North Metro Atlanta Region Manager, Georgia Tech Enterprise Innovation Institute
When I first got into quality, I really hated verifying the effectiveness of actions taken to correct a problem. After all, I was young and inexperienced. All of the people whose actions I was verifying were older, wiser, and more experienced than I was. Who was I to say that their actions were effective or ineffective? My assumptions were as follows:
- If they said they did something, then they certainly did it.
- Whatever they did was directly related to the problem causes, or they wouldn't have done it.
- The action must have been effective; they would have told me otherwise.
All of these assumptions had to be correct, because I was working with seasoned professionals, right? Ha! Boy, did I learn a lesson.
People just want to get paperwork off their desks or out of their in-boxes as quickly as possible. Taking actions on problems is one of many responsibilities that people have and, unfortunately, it's not always top priority. That's why it's crucial that action be carefully verified. Verification is not an act of suspicion or disrespect; it's simply a necessary part of problem solving.
What exactly is being verified? You are seeking evidence that the causes of the problem have been removed or reduced. In a perfect world, each problem cause would be removed. Poof, it's gone. This is not always possible, though. Sometimes the best you can hope for is a reduction of the causes. The cause is still there, but it manifests itself less frequently or less severely. So the best option is to remove the cause, but the next best option is to at least reduce the cause.
Evidence
The key to verification is evidence. You are seeking objective, factual evidence that your problem causes have been reduced or removed. This evidence usually takes the form of data or records. Another powerful form of evidence is your own first-hand observations. That’s not to say that you can't accept verbal evidence, but records, data, and first-hand observations are certainly better.
The exact amount of evidence depends on the magnitude of the problem. Broader and more severe problems lead to more profound solutions, which in turn require more evidence to verify effectiveness. It is a simple matter of scale. The scale of verification must match the scale of the actions taken.
It's important to note that you're sampling evidence. You're taking a representative subgroup of all the available evidence. A 100-percent investigation of evidence is not necessary or particularly effective. Take what you believe to be a balanced and representative sample of the evidence. Shown in figure 1 are some examples of evidence to sample, all related to a problem with orders being late.
Problem: Late Orders
| Evidence | Type of Sample |
|---|---|
| Service records | Review 10 service records from last week to see if work was performed on time |
| Customers | Contact three of the customers who reported late orders and see if they have noticed an improvement |
| Procedures | Review the service procedures to see if they've been revised to include recently implemented improvements |
| Employees | Interview three employees at random. Make sure they understand what has been done to reduce late orders and their roles in implementing the improvements |
| Training records | Review the training records of these same three employees to see if they received training in the revised methods and procedures |
Figure 1: Examples of evidence
The evidence in figure 1 is a broad survey of indicators related to the "late order" problem. If we positively verify this evidence, then we can logically conclude that the actions were effective. Of course, the specific type of evidence and sample sizes will vary, depending on the nature of the problem and the magnitude of actions.
Verification Method
You don't just show up in a department and start asking for evidence. That's a formula for frustration and ill will. Instead, give people notice that you're coming. If you show up unannounced, there is a chance that nobody will be available to assist you by providing evidence. Providing some notice also removes the "Gotcha!" aspect that sometimes accompanies verification activities. Surprise verifications are not needed, as a broad-based examination of evidence will always reveal the true state of corrective action effectiveness.
Communication about the verification process will remove roadblocks and smooth your path. The following telephone conversation illustrates the type of communication to engage in prior to verification of effectiveness of corrective actions.
You: "Hello, Jill. Do you mind if I drop by your department today and verify the effectiveness of actions you've taken on the late order problem?"
Jill: "Uh, I guess not. How long will it take?"
You: "It shouldn’t take long at all. I just need to sample some evidence related to our actions."
Jill: "Do you suspect that we didn't take action?"
You: "No, of course not. I just can't close-out the issue until we know if our actions have been effective. We're also going to Jim's department tomorrow to do the same thing. You're not being targeted, I can assure you."
Jill: "Okay, I understand. Drop by around 10 a.m. and we'll be happy to show you what we've done."
You: "Thanks for all your cooperation. I’ll see you about 10."
As this discussion indicates, people don't always understand the intent of verifying effectiveness. They may think it's vindictive or personal, and you want to remove this misconception as quickly as possible. Your verification of effectiveness will be objective, factual, and impersonal. Make sure everybody understands that prior to your arrival. Your verification will go much smoother.
Once you're in the department, what exactly are you going to verify? Obviously, the evidence will differ on a case by case basis, but here are some of the most common verification points:
Did the actions address problem causes, instead of just symptoms? - Taking action on symptoms is akin to putting a band-aid on a serious wound: it does nothing to treat the underlying causes. The actions taken must get beyond the superficial symptoms and address the underlying causes of the problem, removing or significantly reducing them. The single biggest reason for problem-solving failure is action on symptoms instead of true causes.
Are the actions fully implemented? - Speak to the people responsible for planning and taking action. Have their plans been fully implemented? Are there steps that are pending? What obstacles exist? You can't verify effectiveness until actions have been fully carried out.
Have procedures been revised or developed? - Improvements don't stick unless they are made the new norm. Make sure that all relevant documentation reflects the new methods put in place by the corrective action.
Are employees aware of and knowledgeable about the changes? - If a process has been improved, employees will typically know about it, especially if they are responsible for implementing the change. Speak to employees in the work area and see if they're familiar with the changes and their roles in implementing them. Awareness of improved methods may come from formal training processes or through informal communications. If formal training is used, then records of training would be another type of evidence that could be verified.
Are products or outcomes improved?- This is the bottom line: Have the products been improved? An improved process should ultimately lead to improved products. Is there evidence that this has happened? What do records and data indicate? Hearsay and verbal affirmations can't be used to prove that products have been improved.
Has measurement or monitoring been established?- The effectiveness of some corrective action can't be known without ongoing measurement or monitoring of the process. In these cases, have the controls been set and put in place? What do the measurements indicate? Does the data indicate the process has improved and stabilized to the new level?
What is the customer's perception of an improvement?- Perceptions are everything. Have customers noticed a change in the quality of goods or services? Keep in mind that these could be internal or external customers. Locate the applicable customers and get their opinions. If customers have not noticed an improvement, it can be logically argued that the actions have not been effective.
Has the problem reoccurred? - If the problem continues to occur at the same level as before, then the corrective action is not effective. Only data and records can be used to prove a lack of recurrence.
Is top management aware of the corrective action?- Top management isn't expected to be aware of every corrective action in the organization, but they should be aware of the large ones and overall trends. Top management awareness would certainly help support a determination of full implementation and communication.
Ineffective Actions
It's unfortunate that in reality not everything you verify will be effective for improvement. The most common reasons for this are because solutions didn't work, or the problem-solving actions were never fully implemented, or the corrective actions were aimed at the problem's symptoms instead of its causes. When you determine that actions are ineffective, be diplomatic and forthright. Tell the process owner why you believe the actions are ineffective and describe the evidence that led you to that conclusion. Get the process owner's perspective on the situation. Through an interactive discussion, you usually arrive at an agreement about effectiveness or, in this case, the lack thereof.
Once an agreement has been reached and the facts are clear, determine the next steps to take. Usually the next steps involve revisiting the identification of the causes, and planning and implementing a new plan of action. It's possible that you may need to facilitate the new corrective action. A little bit of coaching can go a long way, especially when the person taking action has hit a roadblock and isn't clear how to proceed.
If called upon to facilitate a corrective action that was initially ineffective, here are some principles to reinforce:
Planning ensures success. - The better the plan for implementation, the more likely the action is to be successful. Many people will define their plans in broad, sweeping terms without providing adequate details to enable implementation. For example, to say, "Install a new blower above the No. 3 oven," sounds clear, but there is a world of complexity within that single statement. It's often easier to break large actions into bite-sized tasks delegated to employees who can be assigned responsibilities, resources, due dates, and reviews.
Communicate early and often.- When planning action, build frequent communication into the plan. This communication can be in many different forms (i.e., meetings, formal reviews, teleconferences, e-mail updates, written reports, etc.). Frequent communication makes it harder for commitments to fall through the cracks. Visibility and transparency are the allies of effective action.
Stay focused on the causes. - When entering the later phases of problem solving, actions take center stage and the causes tend to fade in significance. Fight this tendency. It's critically important that everyone remember exactly what causes are being removed or reduced. Examine and re-examine the actions to make sure they're affecting the underlying causes of the problem, not just the symptoms.
Get creative. - When actions are ineffective, it's often because what we've chosen to do is tired and stale. They're the same old actions people tried years ago that didn't work then and don't work now. What is needed is a big dose of creativity. One of the easiest ways to trigger creativity is to bring new and more diverse people onto the problem-solving team. An injection of new blood will often make the difference. Another effective creativity technique is doing a second brainstorming session on the causes. Sometimes ineffective actions produce a deeper understanding of what is causing the problem.
Make change happen. - Effective solutions will regularly change the way work is done. Ineffective solutions are often the result from re-training on old methods, re-enforcing flawed procedures, and asking people to try harder. None of these actually change anything. Is it any wonder that the problem persists? If we fail to change the work, we usually fail to reduce or remove the problem.
Your role is to be a thinking coach. Help the team look at the problem and its causes from a new perspective. Injecting a little fun and humor into the process also helps at this point. After all, team members do become frustrated. Humor and fun are brain lubricants, and brains need all the lubrication they can get during problem solving.
© Copyright 2009 Craig Cochran. Republished with permission.
This article first appeared in the June 17, 2009 issue of Quality Digest Daily, available online at www.qualitydigest.com
