Software Quality

There’s an old joke in the software business that states, “In the past 40 years, we’ve had two years of experience twenty times.”  Unfortunately, this joke has more truth to it than most of us would care to admit.  Many programmers are under the mistaken impression that the problems that they’re facing are new and haven’t been seen before and/or successfully solved.

 

None of this would be very important if the overall quality of the software we use was not horrible.  If software quality is measured by its reliability (doesn’t crash), being free of errors (bugs), and ease of use, we have been moving backwards.  This, coupled with the fact that software plays an ever increasing role in every aspect of our lives, means that we are depending more and more on things that are getting less and less workable.

 

 

During the course of my career in software engineering, spanning some 50 years, I’ve seen one example after another of people “re-inventing the wheel.”  Even worse, though an effective solution was found for a particular problem in the past, programmers faced with the same problem today are unaware of it and re-solve the problem in a less than optimal way, if at all.  This leads to wasted motion and problems going unsolved.

 

In the early 1970’s, a series of software failures put one of the largest banks in Japan out of business for an entire day.  There were serious repercussions to the Japanese economy and to the reputations of both the bank and the software supplier (my employer).  Consequently, measures were taken in the designs of the mainframe operating system and applications to insure that these types of incidents would no longer occur.  Now, 40 years later we hear story after story about web sites that crash (e.g., www.healthcare.gov), business being suspended due to a computer outage (e.g., the NASDAQ shut down), and errors that make users want to tear out their hair.  One has only to read the user reviews of apps created for smart phones and tablets to appreciate the dismal level of reliability.  Never mind games and apps written by obvious amateurs, the applications associated with billion dollar corporations are often no better. There is software technology to avoid catastrophic failures, but apparently we have learned to live with unstable software and accept it. Users should know that unreliability is not a necessary aspect of using computers and demand better.

 

I am a frequent user of a very popular suite of office applications for text, presentation slides, and spreadsheets.  I also use a variety of web sites. A few months ago, I started to track the number of bugs that I observed in my personal usage.  Not a day goes by when I don’t experience at least one and often multiple bugs with these programs.  I’ve seen inputs incorrectly processed, text files whose format inexplicably changed, lost data, web pages with text overwriting other text, links to nowhere, etc.  – not to mention freezes, crashes, and hangs.

 

Obviously, rigorous testing was not done.  “Beta” testing, where early users shake out about-to-be released software, is inefficient and misleading.  Beta test users are not typical of the general population of users. Moreover, it is not an efficient way to test given that beta usage is atypical and unstructured. The result is that the bugs that are found are encountered multiple times, and many bugs that would be seen by mainstream users are missed.

 

If the goal is to fix every bug found, the most expensive bugs to fix are those found by end users.  This is because many users will encounter the same bug, and the developer will have to screen every error report. This also causes negative hits to the developer’s reputation.  The better alternative is to create a rigorous, complete set of tests that must be successfully run prior to release.

 

Unfortunately, there is considerable evidence that many of the top software companies have no commitment to fixing every bug found.  After many reports have been submitted over a period years, I have encountered the same bugs in multiple versions of the office suite I use every day.  Either the users or the developers don’t care, or both.

Since known bugs are allowed to remain in software, when new function or improvements are developed, they are made on top of these bugs.  It’s like doing surgery in a pigsty.

 

Usability issues are probably the most annoying.  A recent incident occurred when I filled in a web site form requiring my account number and telephone number.  I first put my telephone number in with no spaces or punctuation.  I got an error message saying I needed hyphens.  I corrected the phone number and put the account number in with hyphens, as it was shown on my paper statement from the company. Then, I got an error message saying to put the account number in with no punctuation!  To add insult to injury, everything I had entered was then cleared so that I had to start all over again!  Why should a user have to do anything that a computer can do better (like standardize the format of a phone number)? Unfortunately, error messages and cleared forms, to many programmers, are the appropriate punishment for stupid users.

 

If software developers cared, they would monitor the error messages that are produced when real users use their programs.  They would then see that many of the so-called user errors could be easily corrected by their software.  What this would take is a shift in the mindset of software developers to the position that there are no stupid or careless users, only usability issues.

 

Help line calls also are a great indicator of where usability improvements could be made.  In many places, the reports about each call are summarized with a “closing code.”  While some calls actually report new or already seen bugs, many are closed out with a notation that it was a user error, that the user should have read the manual, or that the software was “working as designed.”  The most positive closing code usually is that the user called with a “suggestion.” Every one of these calls represents data about product usability and should be treated as a usability bug report.  The goal ought to be to eliminate the need for users to call the help line or even read the manual.

 

Every strategy to improve software quality mentioned here has successfully been put into practice in the past.  The fact that most software that we use today seems to be low quality is evidence that users have become less discriminating and demanding of quality, that many software companies are indifferent to the quality of their products, and that programmers are unaware of the techniques that have been used successfully in the past to improve quality.

 

One of the key missing ingredients is the proper education of programmers. As a colleague of mine has pointed out, programming is usually taught as an art rather than an engineering discipline. Missing are proper software engineering education and professionalism, user demand for higher quality, and a longer-term view in software business models.  Rather than continuing in our current unworkable direction,

 

I assert that it is more profitable to produce high quality, usable, reliable software.  But the current trends are disturbing.   I am reminded of the old Chinese proverb: “If we don’t change our direction, we are likely to wind up where we are headed.”