This is an HTML rendering of a working paper draft that led to a publication. The publication should always be cited in preference to this draft using the following reference:
Testing Computer Software, second ed. by Cem Kaner, Jack Falk, and Hung Quoc Nguyen, John Wiley & Sons, New York, 1999, ISBN 0-471-35846-0, 480 pp., US$34.99.
Testing is sometimes regarded as the ugly duckling of software development. Programmers want to write perfect code, development managers prefer to schedule releases without the additional uncertainties testing results bring, and financial officers surely want to do away with additional testing costs. Fortunately, three authors who love this ugly duckling wrote Testing Computer Software, and they successfully transfer their knowledge and-more importantly-their views to the reader.
The authors primarily target testers and test managers involved in producing non safety-critical software. They readily admit that computer science graduates rarely have useful testing knowledge-software testing can be better performed by suitably trained college graduates. The authors thus painstakingly introduce all relevant software engineering and programming concepts needed to perform testing, and they also point to relevant material for further reading. The software life cycle, corresponding testing cycles, glass-box versus black-box testing, regression testing, maintenance, and boundary conditions are some of the topics covered.
Having designed and implemented a bug tracking system for managing a medium-scale development project, I was impressed by the pragmatic attitude the authors advanced; all suggestions in this area are easy to put into practice and are clearly the result of extensive real-life experience. The implementation of a bug-tracking system, which is outlined in a separate chapter, is likely to bring forward political issues. Although many useful metrics can be extracted from such a system, the authors wisely point out that it would be counterproductive to use it for tracking individual performance of programmers or testers. Reports can still help make estimates based on past performance figures and judging project progress.
Another particularly thorny issue concerns the problem report life cycle. A state diagram would help readers better understand the temporal and organizational relationship between problem resolution and status. However, even the authors appear split on the issue of how to treat deferred bugs.
Although the book's second edition was published in 1999, the material concerning the testing of printers, testing tools, localization, and legal aspects needs to be brought up-to-date. Most printers nowadays are not directly driven by end-user software and are seldom operated using simple escape sequences. Testing tools suitable for character input/output are not applicable to graphical user interfaces, character code pages are disappearing in favor of Unicode, and language libraries and operating systems provide internationalization support with radically different testing demands from earlier ad hoc localization approaches. References also cover only material published up to 1991.
The authors devote the last part of the book to testing management, explaining why we should integrate it into the software development life cycle. Important milestones include alpha and beta testing and the user interface freeze. Testing can be performed in-house by a dedicated testing group, or it can be outsourced to an external agency; readers who have read the previous book chapters will probably favor the former approach. The authors' empathy with software testers and managers is commendable. Recognizing that testing is often an entry-level job that can lead to a software development position can help testers work toward professional advancement and help managers constructively deal with the inevitably high staff turnover rate.
Diomidis Spinellis is an assistant professor in the Department of Management Science and Technology at the Athens University of Economics and Business. Contact him at email@example.com.