Six Sigma Training.ORG

Six Sigma teaches us to view everything as a process. We should take an objective look at the system, measure the values, form a model, enact changes on the process, and observe the effects of these changes. However, we sometimes want to measure objectively something that is intrinsically subjective, the opinions of people. These fall under the category of what I like to call “Human Metrics.” Some common examples of these metrics are customer satisfaction, perception of quality, and ease of use. How do we accurately measure these values?

Surveys are regularly used as the catchall for human metrics. This has several flaws, though. First, people are not consistent graders, and a 7/10 for one person may be an 8/10 for another. Ultimately, though, this is not a problem because it represents the same sort of variation you will see in any measurement. The second problem is self-selection of responders. This is well documented, and can create extreme disparities between real and actual numbers. A good example of this can be found with call-in surveys. In many cases, only those with a chip on their shoulder will be compelled to call, creating a clearly biased sample. To counter this, sometimes incentives are offered to get a higher response rate, but this brings us to the third problem with surveys, non-response. Where some people may choose to simply not respond to a survey, others will purposely subvert the survey itself by ignoring the questions and answering in some sort of pattern. This is common with incentives, like contests for surveys, because it is so easy to simply hit the number one repeatedly without hearing the questions being asked. It can be hard to pick out this group, and one can only hope that over the long term the collective contributions of these unresponders will balance each other out.

How do you deal with these problems? A common solution is to try to address and eliminate the issues inherent to surveys. Select the sample by hand, normalize the scores across respondents, and remove surveys with obvious patterns or very unusual scoring habits. Ultimately though, even if these methods were perfect at eliminating their respective flaws, you would still be left with the fact that people are bad judges of their own opinions. Thus, you cannot use the responses to predict future behavior. This brings us to the best method for retrieving human metrics, using a proxy.

Measurement by proxy is a method that has existed for millennia. If you know the angle of the sun, you can measure the height of a flagpole by measuring the length of its shadow. You can apply this to human metrics and find values from behaviors that reflect certain opinions. Often, these are the values you care about most. Repeat sales, for example, is a good measurement of customer satisfaction. However, while repeat sales a metric that you can find using existing data, some require testing. For example, if you want a metric for ease of use, then you can look at the time taken to use the product. To test this, one might take a group of people with little or no familiarity with a product and ask them to use it. The time taken does not exactly demonstrate ease of use, but the two are related enough to make this a reasonable proxy in certain situations.

That is not to say that surveys cannot be useful, or don’t have a place. Simply put, they are very easy to implement, and consistent surveys allow the tracking of trends in user opinions. Additionally, measurement by proxy is far from perfect. Other factors in your system can seep into your measurement and taint the values. For example, the number of repeat sales may be artificially increased by having a product with a short lifespan. Naturally, the short lifespan is bad for customer satisfaction, but the proxy would insist otherwise.

In conclusion, as a Six Sigma expert, you should look to be quantitative in your assessment of systems. Human metrics are no exception to this rule. You cannot abandon the Six Sigma philosophy just because “Everyone is different.” Instead, look to measure how they are different.