Millennial Web Content Management: Improving usability- how much is enough?

Most people agree that improving web usability is important. But how much money and effort is enough? Using Six Sigma can help you decide.

Several months ago, I attended a two-day workshop at the Usability University in Washington DC. Dr. Robert Bailey, President, Computer Psychology, Inc. was the instructor. He introduced the idea of using the Sigma Six Methodology on websites. We thought it was great idea and decided to post the material. Dr. Bailey is planning on publishing a paper on the subject that will no doubt provide a great more detail than is in this post.

What's Sigma Six

Six Sigma is a highly disciplined process that helps an organization focus on developing and delivering near-perfect products and services. Companies like General Electric, Honeywell and Motorola have embraced the process for years. It is well known throughout the private sector and the world.

The basic idea behind Six Sigma is that if you can measure how many "defects" you have in a process, you can systematically figure out how to eliminate them and get as close to "zero defects" as possible.

Six Sigma is a very agressive goal. An organization can expect the following at various sigma levels:

a) Six Sigma- only 3.4 defects per million opportunities or 99.9997% error-free.

b) Five Sigma only 230 defects per million opportunities or 99.98% error-free.

c) Four Sigma only 6,200 defects per million opportunities or 99.4% error-free.

d) Three Sigma- about 66,810 defects per million- or 93% error-free

e) Two Sigma- about 308,500 defects per million- or 69% error-free

Life in Six Sigma World

Here are some examples of achieving the ultimate level of performance courtesy of the Honeywell Corporation website;

? if your water heater operated at Four Sigma, you would be without hot water more than 54 hours each year. At Six Sigma, you'd be without hot water for less than two minutes a year.

? If your electricity operated at Four Sigma performance, your lights would be out an hour a week. At Six Sigma, you would be without lights about two seconds a week.

? With a Four Sigma design process, six out of every 1,000 design elements relating to a new product are flawed before the product is fully commercialized. With a Six Sigma process, only about three of every one million design elements are flawed.

? If your telephone operated at Four Sigma, you would be without service for more than four hours a month. At Six Sigma, it would be about nine seconds a month.

? At Four Sigma, about six out of every 1,000 invoices will contain incorrect information. At Six Sigma, mistakes will occur only about three times in every 1,000,000 invoices.

? If your car operated at Four Sigma performance, you would spend 37 minutes in the repair shop for every 100 hours you operate the vehicle. (This sounds about right for my car), At Six Sigma, you would have only 1.2 seconds of repair for every 100 hours of operation.

How do websites rate?

Most companies operate at about the Three Sigma level and are 97% error free. That's pretty good. Just making the jump from Four Sigma to Five Sigma requires a 27-fold performance improvement. The move from Five Sigma to Six Sigma requires approximately another 60-fold improvement. We'r talking alot of effort and money here.

According to Dr. Bailey, most websites operate at or below Three Sigma. To achieve Three Sigma, a website would have to employ a heuristic usability evaluation and also undertake perfomance tests with real users.

Don't be thrown by these terms. A heuristic evaluation is just a fancy name for usablity experts evaluating your website against usability guidelines. The evaluation is then used to make improvements to the site. Performance testing is nothing more than giving users real tasks one on one on a computer and observing how successful or unsucessful they are in performing the task. You then use the data to improve the website.

Who needs a Sigma Six website

Most organizations, including government agencies and non profits, probably don't need a Sigma Six website. However, we think that a Three Sigma website is very doable.

Websites that are transactional like Dell, Amazon, eBay can probably justify the costs and effort required to reach Four Sigma and above. However, the changes required to do so deal more with changing business rules, process and the culture of an organization. One can easily understand how a Dell Computer would not want 7 percent of its customers to fail in the online order process of a computer. Ditto for Amazon and eBay.

Web sites that are information rich and provide technical or public information can probably get by at the Three Sigma level. However, by changing some business rules on how content is organized and presented, they can approach Four Sigma.

Whether or not an information rich website should should strive for Four Sigma depends on the organization. For example government agency websites could probably live with Three Sigma since their existence may not be threatened by disaffected users. In contrast, commercial websites that are information rich probably need to at least make the effort to retain users by providing well organized content and highly usable websites.