软件测试的狭义论和广义论——静态和动态的测试软件测试的辨证论——正向思维和反向思维软件测试的风险论——测试是评估软件测试的经济学观点——为盈利而测试软件测试的标准论——验证和确认

英文介绍(The Cost of Bug NOTE Customer Requirement Specification Software Design Document Test Document Software Development Lifecycle Model Waterfall Model Spiral Model Summary)

软件测试(1. 软件测试的狭义论和广义论 2.软件测试的辨证论 3.软件测试的风险论 4.软件测试的经济学观点 5. 软件测试的标准论)

英文介绍

The Cost of Bug

As you will learn in Chapter 2, software doesn’t just magically appear there’s usually a planned, methodical development process used to create it.

Figure 1.2. The cost to fix bugs can increase dramatically over time.

Time When Bug Is Found

The costs are logarithmic that is, they increase tenfold as time increases. A bug found and fixed during the early stages when the specification is being written might cost next finds it, the cost could easily be thousands or even millions of dollars.

As an example of how this works, consider the Disney Lion King case discussed earlier. The root cause of the problem was that the software wouldn’t work on a very popular PC platform. If, in the early specification stage, someone had researched what PCs were popular and specified that the software needed to be designed and tested to work on those configurations, the cost of that effort would have been minimal. If that didn’t occur, a backup would have been for the software testers to collect samples of the popular PCs and verify the software on them. They would have found the bug, but it would have been more expensive to fix because the software would have to be debugged, fixed, and retested. The development team could have also sent out a preliminary version of the software to a small group of customers in what’s called a beta test. Those customers, chosen to represent the larger market, would have likely discovered the problem. As it turned out, however, the bug was completely missed until many thousands of CD-ROMs were created and purchased. Disney ended up paying for telephone customer support, product returns, replacement CD-ROMs, as well as another debug, fix, and test cycle. It’s very easy to burn up your entire product’s profit if serious bugs make it to customer.

What Exactly Does a Software Tester Do?

You’re now seen examples of really nasty bugs. You know what the definition of a bug is, and you know how costly they can be. By now it should be pretty evident what a tester’s goal is:

The goal of a software tester is to find to bugs:

You may run across product teams who want their testers to simply confirm that the software works, not to find bugs. Reread the case study about the Mars Polar Lander, and you’ll see why this is the wrong approach. If you’re only testing things that should work and setting up your tests so they’ll pass, you will miss the things that don’t work. You will miss the bugs.

If you’re missing bugs, you’re costing your project and your company money. As a software tester you shouldn’t be content at just finding bugs you should think about how to find them sooner in the development process, thus making them cheaper to fix.

The goal of a software tester is to find bugs and find them as early as possible.

But, finding bugs, even finding them early, isn’t enough. Remember the definition of a bug. You, the software tester, are the customer’s eyes, the first one to see the software. You speak for the customer and must seek perfection.

This goal of a software tester is to find bugs, find them as early as possible, and make sure they get fixed.

This final definition is very important. Commit it to money and refer back to it as you learn the testing techniques discussed throughout the rest of this book.

NOTE

It’s important to note that “fixing” a bug does not necessarily imply correcting the software. It could mean adding a comment in the user manual or providing special training to the customer. It could require changing the statistics that the marketing group advertise or even postponing the release of the buggy feature. You’ll learn throughout this book that although you’re seeking perfection and making sure that the bugs get fixed, that there are practical realities to software testing. Don’t get caught in the dangerous spiral of unattainable perfection.

What Makes a Good Software Tester?

In the movie star Trek II: the Wrath of Khan, Spock says, “As a matter of cosmic history, it has always been easier to destroy than to create.” At first glance, it may appear that a software tester’s job would be easier than a programmer’s. breaking code and finding bugs must surely be easier than writing the code in the first place. Surprisingly, it’s not. The methodical and disciplined approach to software testing that you’ll learn in this book requires the same hard work and dedication that programming does. It involves very similar skills, and although a software tester doesn’t necessarily need to be a full-fledged programmer, having that knowledge is a great benefit.

Today, most mature companies treat software testing as a technical engineering profession. They recognize that having trained software testers on their project teams and allowing them to apply their trade early in the development process allows them to build better quality software. Unfortunately, there are still a few companies that don’t appreciate the challenge of software testing and the value of well-done testing effort. In a free market society, these companies usually aren’t around for long because the customers speak with their wallets and choose not to buy their buggy products. A good test organization (or the lack of one) can make or break a company.

Here’s a list of traits that most software testers should have:

 They are explorers. Software testers aren’t afraid to venture into unknown situations. They love to get a new piece of software, install it on their PC, and see what happens.

 They are troubleshooters. Software testers are good at figuring out why something doesn’t work. They love puzzles.

 They are relentless. Software testers keep trying. They may see a bug that quickly vanishes or is difficult to re-create. Rather than dismiss it as a fluke, they will try every way possible to find it.

 They are creative. Testing the obvious isn’t sufficient for software testers. Their job is to think up creative and even off-the-wall approaches to find bugs.

 They are (mellowed) perfectionists. They strive for perfection, but they know when it becomes unattainable and they’re okay with getting as close as they can.

 They exercise good judgment. Software testers need to make decisions about what they will test, how long it will take, and if the problem they’re looking at is really a bug.

 They are tactful and diplomatic. Software testers are always the bearers of news. They have to tell the programmers that their baby is ugly. Good software testers know how to do so tactfully and professionally and know how to work with programmers who aren’t always tactful and diplomatic.

 They are persuasive. Bugs that testers find won’t always be viewed as severe enough to be fixed. Testers need to be good at making their points clear, demonstrating why the bug does indeed need to be fixed, and following through on making it happen.

In addition to these traits, having some education in software programming is a big bugs. As you’ll see in Chapter 6. “Examining the Code,” knowing how software is written can give you a different view of where bugs are found, thus making you a more efficient and effective tester. It can also help you develop the testing tools discussed in Chapter 15, “Automated Testing and Test Tools.”

Lastly, if you’re an expert in some non-computer field, your knowledge can be invaluable to a software team creating a new product. Software is being written to do just about everything today. Your knowledge of teaching, cooking, airplanes, carpentry, medicine, or whatever would be a tremendous help finding bugs in software for those areas.

Summary

Software testing is a critical job. With the size and complexity of today’s software, it’s imperative software testing be performed professionally and effectively. Too much is at risk. We don’t need more defective computer chips, crashed system, or stolen credit card numbers.

In the following chapters of Part I “The Big Picture,” you’ll learn more about the big picture of software development and how software testing fits in. This knowledge is critical to helping you apply the specific test techniques covered in the remainder of this book.

Quiz

These quiz questions are provided for your further understanding. See Appendix A, “Answers to Quiz Questions,” for the answers but don’t peek!

1: In the Year 2000 bug example, did Dave do anything wrong?

2: True or False: It’s important what term your company or team calls problem in its software.

3: What’s wrong with just testing that a program works as expected?

4: How much more does it cost to fix a bug found after the product is released than it does from very start of the project?

5: what’s the goal of a software tester?

6: True or False: A good tester relentlessly strives for perfection.

7: Give several reasons why the product specification is usually the largest source of bugs in a software product.

Chapter 2. The Software Development Process

IN THIS CHAPTER

 Product Components

 Software Project Staff

 Software Development Lifecycle Models

To be an effective software tester, it’s important to have least a high-level understanding of the overall process used to develop software. If you write small programs as a student hobbyist, you’ll find methods you use are much different from what big companies use to develop software. The creation of a new software product may involve dozens, hundreds, even thousands of team members all playing different roles an working together under tight schedules. The specific of what these people do, how they interact, and how they make decisions are all part of the software development process.

The goal of this chapter isn’t to teach you everything about the software development process that would take an entire book! The goal is to give you can overview of the all the pieces that go into a software product and a look at a few of the common approaches in use today. With this knowledge you’ll have a better understand of how best to apply the software testing skills you learn in the later chapters of this book.

The highlights of this chapter include

 What major components go into a software product

 What different people and skills contribute to a software product

 How software progress from an idea to a final product

Product Components

What exactly is a software product? Many of us think of it as simply a program that we download from the Internet or install from DVD that runs on our computer. That’s a pretty good description, but in reality, many hidden pieces go into making that software. There are also many pieces that “come in the box” that are often taken for granted or might even be ignored. Although it may be easy to forget about all those parts, as a software tester, you need to be aware of them, because they’re all testable pieces and all have bugs.

What Effort Goes Into a Software Product?

First, look at what effort goes into a software product. Figure 2.1 Identifies a few of the abstract pieces that you may not have considered.

Figure 2.1 A lot of hidden effort goes into a software product.

So what are all these things, besides the actual code, that get funneled into the software? At first glance they probably seem much less tangible than the program listing a programmer creates. And they definitely aren’t something that can be viewed directly from the product’s CD-ROM. But, to paraphrase a line from an old spaghetti sauce commercial, “they’re in there.” At least, they should be.

The term used in the software industry to describe a software product component that’s created and passed on to someone else is deliverable. The easiest way to explain what all deliverables are organize them into major categories.

Customer Requirement

Software is written to fulfill some need that a person or group of people has. Let’s call them the customer. To properly fill that need, the product development team must find out what the customer wants. Some teams simply guess, but most collect detailed information in the form of surveys, feedback from previous versions of the software, competitive product information, magazine reviews, focus groups, and numerous other methods, some formal, some not. All this information is then studied, condensed, and interpreted to decide exactly what features the software product should have.

Specification

The result of the customer requirements studies is really just raw data. It doesn’t describe the proposed product, it just confirms whether it should (or shouldn’t) be created and what features the customers want. The specifications take all this information plus any unstated but mandatory requirements and truly define what the product will be, what it will do, and how it will look.

The format of specifications varies greatly. Some companies specially those developing products for the government, aerospace, financial, and medical industries use a very rigorous process with many checks and balance. The result is an extremely detailed and thorough specification that’s locked down, meaning that it can’t change except under very extreme conditions. Everyone on the development team knows exactly what they are creating.

The most of specifications varies greatly. Some companies especially those developing products for the government, aero space, financial, and medical industries use a very rigorous process with many checks and balances. The result is an extremely detailed and thorough specification that’s locked down, meaning that it can’t change except under very extreme conditions. Every on the development team knows exactly what they are creating.

There are development teams, usually ones creating software for less-critical applications, who produce specifications on cocktail napkins, if they create them at all. This has the distinct advantage of being very flexible, but there’s lot of risk that not everyone is “on the same page.” And, what the product finally becomes isn’t know until released.

Schedules

A key part of a software product is its schedule. As a project grows in size and complexity, with many pieces and many people contributing to the product, it becomes necessary to have some mechanism to track its progress. This could range from simple task lists to Gantt charts (see Figure 2.2) to detailed tracking of every minute task with project management software.

Figure 2.2. A Gantt chart is a bar that shows a project’s tasks against a horizontal timeline.

The goals of scheduling are to know which work has been completed, how much work is still left to do, and when it will be finished.

Software Design Document

One common misconception is that when a programmer creates a program, he simply sits down and starts writing code. That may happen in some small, informal software shops, but for anything other than the smallest programs, there must be a design process to plan out how the software will be written. Think about this book, which required an outline before the first words were typed, or a building, which has blueprints drawn the first concrete is poured. The same planning should happen with software.

The documents that programmers create vary greatly depending on the company, the project, and the team, but their purpose is to plan and organize the code that is to be written.

Here is a list of a few common software design documents:

 Architecture. A document that describes the overall design of the software, including descriptions of all the major pieces and how they interact with each other.

 Data Flow Diagram. A formalized diagram that shows how data moves through a program. It’s sometimes referred to as a bubble chart because it’s drawn by circles and lines.

 State Transition Diagram. Another formalized diagram that breaks the software into basic states, or conditions, and shows the means for moving from one state to the next.

 Flowchart. The traditional means for pictorially describing a program’s logic. Flowcharting isn’t very popular today, but when it’s used, writing the program code from a detailed flowchart is a very simple process.

 Commented Code. There’s an old saying that you may write code once, but it will be read by someone at least 10 times. Properly embedding useful comments in the software code itself is extremely important, so that programmers assigned to maintain the code can more easily figure out what it does and how.

Test Document

Test documentation is discussed in detail in Chapters 1720 but is mentioned here because it’s integral to what makes up a software product. For the same reasons that programmers must plan and document their work, software testers must as well. It’s not unheard of for a software test team to create more deliverables than the programmers.

Here’s a list of the more important test deliverables:

 The test plan describes the overall method to be used to verify that the software meets the product specification and the customer’s needs. It includes the quality objectives, resource needs, schedules, assignments, methods, and so forth.

 Test cases list the specific items that will be tested and describe the detailed steps that will be followed to verify the software.

 Bug reports describe the problems found as the test cases are followed. These could be done on paper but are often tracked in a database.

 Test tools and automation are described in detail in Chapter 15, “Automated Testing and Test Tools.” If you team is using automated methods to test your software, the tools you use, either purchased or written in-house, must be documented.

 Metrics, statistics, and summaries convey the progress being made as the test work progresses. They take the form of graphs, chart, and written reports.

What Parts Make Up a Software Product?

So far in this chapter you ‘re learned about the effort that goes into creating a software product. It’s also important to realize that when the product is ready to be boxed up and shipped out the door, it’s not just the code that gets delivered. Numerous supporting parts go along with it (see Figure 2.3). Since all these parts are seen or used by the customer, they need to be tested too.

Figure 2.3. The software CD-ROM is just one of the many pieces that make up a software product.

It’s unfortunate, but these components are often overlooked in the testing process. You’ve surely attempted to use a product’s built-in help file and found it to be not so helpful it or worse just plain wrong. Or, maybe you’ve checked the system requirements on a sticker on the side of a software box only find out after you bought it that the software didn’t work on your PC. These seem like simple things to test, but no one probably even gave them a second look before the product was okayed for release. You will.

Later in this book you’ll learn about these non-software pieces and how to properly test them. Until then, keep this list in mind as just a sampling of what more there is to a software product than just the code:

Help files User’s manual

Samples and examples Labels and stickers

Product support info Icons and art

Error messages Ads and marketing material

Setup and installation Readme file

Software Project Staff

Now that you know what goes into a software product and what ships with one, it’s time to learn about all the people who create software. Of course, this varies a great deal based on the company and the project, but for the most part the roles are the same, it’s just the titles that are different.

The following lists, in no particular order, the major players and what they do. The most common names are given, but expect variations and additions:

 Project managers, program managers, or produces drive the project from beginning to end. They’re usually responsible for writing the product spec, managing the schedule, and making the critical decisions and trade-offs.

 Architects or system engineers are the technical experts on the product team. They’re usually very experienced and therefore are qualified to design the overall systems architecture or design for the software. They work very closely with the programmers.

 Programmers, developers, or coders design and write software and fix the bugs that are found. They work closely with the architects and project managers to create the software. Then, they work closely with the project managers and testers to get the bugs fixed.

 Testers or QA (Quality Assurance) Staff are responsible for finding and reporting problems in the software product. They work very closely with all members of the team as they develop and run their tests, and report the problems they find. Chapter 21, “Software Quality Assurance,” thoroughly cover the differences between software testing and software quality assurance tasks.

 Technical writers, user assistance, user education, manual writers, or illustrators create the paper and online documentation that comes with a software product.

 Configuration management or builder handles the process of pulling together all the software written by the programmers and all the documentation created by the writers and putting it together into a single package.

As you can see, several groups of people contribute to a software product. On large teams there may be dozens or hundreds working together. To successfully communicate and organize their approach, they need a plan, a method for getting from point A to point B. that’s what the next section is about.

Software Development Lifecycle Model

A running joke in the computer industry is that three things should never be seen in the process of being created: laws, sausage, and software. Their creation process is so messy and disgusting that it’s best to just wait and see the final result. That may or may not be totally true, but with most old sayings, there is a grain of truth behind the words. Some software is developed with the rigor and discipline of a fine craftsman, some software with tightly controlled chaos, and other software is stuck together with duct tape and chewing gum. Usually, in the end, it’s apparent to the customer what process was used. The process used to create a software product from its initial conception to its public release is known as the software development lifecycle model.

As discussed previously, there are many different methods that can be used for developing software, and no model is necessarily the best for a particular project. There are four frequently used models, with most others just variations of these:

 Big-Bang

 Code-and-Fix

 Waterfall

 Spiral

Each model has its advantages and disadvantages. As a tester, you’ll likely encounter them all and will need to tailor your test approach to fit the model being used for your current project. Refer to these model description as you read the rest of this book and think about how you would apply the various testing techniques you learn under each of them.

Big-Bang Model

One theory of the creation of the universe is the big-bang theory. It states that billions of years ago, the universe was created in a single huge explosion of nearly infinite energy. Everything that exists is the result of energy and matter lining up to produce this book, DVDs, and Bill Gates. If the atoms didn’t line up just right, these things might all be just quivering masses of goop.

The big-bang model for software development shown in Figure 2.4 follows much the same principle. A huge amount of matter (people and money) is put together, a lot of energy is expended often violently and out comes the perfect software product…or it doesn’t.

Figure 2.4. The big-bang model is by far the simplest method of software development.

The beauty of the big-bang method is that it’s simple. There is little if any planning, scheduling, or formal development process. All the effort is spent developing the software and writing the code. It’s process that is used if the product requirements aren’t well understood and the final release date is completely flexible. It’s also important to have very flexible customers, too, because they won’t know what they’re getting until the very end.

Notice that testing isn’t shown in Figure 2.4. In most cases, there is little to no formal testing done under the big-bang model. If testing does occur, it’s squeezed in just before the product is released. It’s a mystery why testing is sometimes inserted into this model, but it’s probably to make everyone feel good that some testing was performed.

If you are called in to test a product under the big-bang model, you have both an easy and a difficult task. Because the software is already complete, you have the perfect specification the product itself. And, because it’s impossible to go back and fix things that are broken, your job is really just to report what you find so the customers can be told about the problems.

The downside is that, in the eyes of project management, the product is ready to go, so your work is holding up delivery to the customer. The longer you take to do your job and the more bugs you find, the more contentious the situation will become. Try to stay away from testing in this model.

Code-and-Fix Model

The code-and-fix model shown in Figure 2.5 is usually the one that project teams fall into by default if they don’t consciously attempt to use something else. It’s a step up, procedurally, from the big-bang model, in that it at least requires some idea of what the product requirements are.

Figure 2.5. The code-and-fix model repeats until someone gives up.

A wise man once said, “there’s never time to do it right, but there’s always time to do it over.” That pretty much sums up this model. A team using this approach usually starts with a rough idea of what they want, does some simple design, and then processed into a repeating cycle of coding, testing, and fixing bugs. At some point they decide that enough is enough and release the product.

As there’s very little overhead for planning and documenting a project team can show results immediately. For this reason, the code-and-fix model works very well for small projects intended to be created quickly and then thrown out shortly after they’re done, such as prototypes and demos. Even so, code-and-fix has been used on many large and well-known software products. If your word processor or spreadsheet software has lots of little bugs or it just doesn’t seem quite finished, it was likely created with the code-and-fix model.

Like the big-bang model, testing isn’t specifically called out in the code-and-fix model but does play a significant role between the coding and the fixing.

As a tester on a code-and-fix project, you need to be aware that you, along with the programmers, will be in a constant state of cycling. As often as every day you’ll be given new or updated releases of the software and will set off to test it. You’ll run your tests, report the bugs, and then get a new software release. You may not have finished testing the previous release when the new one arrives, and the new one may have new or changed features. Eventually, you’ll get a chance to test most of the features, find fewer and fewer bugs, and then someone (or the schedule) will decide that it’s time to release the product.

You will most likely encounter the code-and-fix model during your work as a software tester. It’s good introduction to software development and will help you appreciate the more formal methods.

Waterfall Model

The waterfall method is usually the first one taught in programming school. It’s been around forever. It’s simple, elegant, and makes sense. And, it can work well on the right project. Figure 2.6 shows the steps involved in this model.

Figure 2.6. The software development process flows from one step the next in the waterfall model.

A project using the waterfall model moves down a series of steps starting from an initial idea to a final product. At the end of each step, the project team holds a review to determine if they’re ready to move to the next step. If the project isn’t ready to progress, it stays at that level until it’s ready.

Notice three important things about the waterfall method:

 There’s large emphasis on specifying what the product will be. Note that the development or coding phase is only a single block!

 The steps are discrete; there’s no overlap.

 There’s no way to back up. As soon as you’re on a step, you need to complete the tasks for that step and then move on you can’t go back. [1]

[1] Variations of the waterfall model loosen the rules a bit, allowing some overlap of the steps and the ability to back up one step if necessary.

This may sound very limiting, and it is, but it works well for projects with a well-understood product definition and a disciplined development staff. The goal is to work out all the unknowns and nail down all the details before the first line of code is written. The drawback is that in today’s fast moving culture, with products being developed on Internet time, by the time a software product it so carefully thought out and defined, the original reason for its being may have changed.

Form a testing perspective, the waterfall model offers one huge advantage over the other models presented so far. Everything is carefully and thoroughly specified. By the time the software is delivered to test group, every detail has been decided on, written down, and turned into software. From that, the test group can create an accurate plan and schedule. They know exactly what they’re testing, and there’s no question about whether something is a feature or a bug.

But, with this advantage, comes a large disadvantage. Because testing occurs only at the end, a fundamental problem could creep in early on and not be detected until days before the scheduled product release. Remember from Chapter 1, “Software Testing Background,” how the cost of bugs increases over time? What’s needed is model that folds the testing tasks in earlier to find problems before they become too costly.

Spiral Model

It’s not quite utopia, but the spiral model (see Figure 2.7) goes a long way in addressing many of the problems inherent with the other models while adding a few of its own nice touches.

Figure 2.7. The spiral model starts small and gradually expands as the project becomes better defined and gains stability.

The spiral model was introduced by Barry Boehm in 1986 in his Association for Computing Machinery (ACM) paper, “A Spiral of Software Development and Enhancement.” It’s used fairly often and has proven to be an effective approach to developing software.

The general idea behind the spiral model is that you don’t define everything in detail at the very beginning. You start small, define your important feature, try them out, get feedback from your customers, and then move on to the next level. You repeat this until you have your final product.

Each time around the spiral involves six steps:

1. Determine objectives, alternatives, and constraints.

2. Identify and resolve risks.

3. Evaluate alternatives.

4. Develop and test the current level.

5. Plan the next level.

6. Decide on the approach for the next level.

Built into the spiral mode is a bit of waterfall (the steps of analysis, design, develop, test), a bit of code-and-fix (each time around the spiral), and a bit of big-bang (look at it from the outside). Couple this with the lower costs of finding problems early, and you have a pretty good development model.

If you’re a tester, you’ll like this model. You’ll get a chance to influence the product early by being involved in the preliminary design phases. You’ll see where the project has come from and where it’s going. And, at the very end of the project, you won’t feel as rushed to perform all your testing at the last minute. You’re been testing all along, so the last push should only be a validation that everything is okay.

Summary

You now have an understanding of how software products are create both goes into them and the processes used to put them together. As you can see, there’s no definitive approach. The four models presented here are just examples. There are many others and lot of variations of these. Each company, each project, and each team will choose what works for them. Sometimes they will choose right, sometimes they will choose wrong. Your job as a software tester is to work the best you can in the development model you’re in, applying the testing skills you learn in the rest of this book to create the best possible.

Quiz

These quiz questions are provided for your further understanding. See Appendix A, “Answers to Quiz Questions,” for the answers but don’t peek!

软件测试

在G J Myers的经典著作《软件测试之艺术》（The Art of Software Testing）中，给出了软件测试（Software Testing）的定义：“程序测试是为了发现错误而执行程序的过程”。这个定义，被业界所认可，经常被引用。除此之外，G J Myers还给出了与测试相关的三个重要观点，那就是：

测试是为了证明程序有错，而不是证明程序无错误；

一个好的测试用例是在于它能发现至今未发现的错误；

一个成功的测试是发现了至今未发现的错误的测试。

实际上，这里暗示了“软件测试”在不同侧面上的含义，也就决定了对软件测试不同的定义和不同的理解，软件测试的不同视野，概括为如下5类：

软件测试的狭义论和广义论——静态和动态的测试

软件测试的辨证论——正向思维和反向思维

软件测试的风险论——测试是评估

软件测试的经济学观点——为盈利而测试

软件测试的标准论——验证和确认

1. 软件测试的狭义论和广义论

G.J.Myers所给出了测试定义——“程序测试是为了发现错误而执行程序的过程”，实际是一个狭义的概念，因为他认为测试是执行程序的过程，也就是传统意义上的测试——在代码完成后，通过运行程序来发现程序代码或软件系统中错误。但是，这种意义上的测试是不能在代码完成之前发现软件系统需求、发现设计上的问题，把需求、发现设计上的问题遗留到后期，这样就会可能造成设计、编程的部分返工。增加软件开发的成本、延长开发的周期等。需求阶段和设计阶段的缺陷产生的放大效应会加大。这非常不利于保证软件质量。这种狭义论是受软件开发瀑布模型影响。

正是为了更早地发现问题，所以将测试延伸到需求评审、设计审查活动中去，也就是将“软件质量保证”的部分活动归为测试活动。实际上，在软件开发实际操作中，常常将软件测试和质量保证——这两种努力（efforts）合并起来。

延伸后的软件测试，被认为是一种软件测试的广义概念。这就引出软件测试的两个概念“静态测试”和“动态测试”，如 测试方法的辩证统一 （1）所述，这样就由静态测试和动态测试构成一个全过程的、完整的软件测试，而且静态测试显得更为重要。

2.软件测试的辨证论

G.J.Myers的第2个观点“测试是为了证明程序有错，而不是证明程序无错误”，引出了软件测试的另外一个争论，软件测试究竟是证明所有软件的功能特性是正确的呢？还是其反向思维——对软件系统进行各种试探和攻击，找出软件系统中不正常或不工作的地方呢？从我个人理解，这两个方面都有一定道理，前者（证明所有软件的功能特性是正确的）是从质量保证的角度来思考软件测试，后者（证明程序有错）从软件测试的直接目标和测试效率来思考，两者应该相辅相成。在后者的思想背景下，我们认为，测试不是为了证明所有的功能可以正常工作，恰恰相反，测试就是为了找出那些不能正常工作、不一致性的地方。也就是说，测试的一般工作就是发现缺陷 (detect bug)，即在软件开发过程中，分析、设计与编码等工作都是建设性的，而测试是带有“破坏性”的工作。

对于不同的应用领域，两者的比重是不一样的，如国防、航天、银行等软件系统，承受不了任何系统失效，因为一次系统的失效完全有可能导致灾难性的损失，所以强调前者以保证非常高的软件质量。而一般的软件服务应用则不同，强调后者，质量目标设置在“用户可接受水平”，不要国度追求质量，从而可以降低软件开发成本。作者建议，在我们实际操作中，可以分阶段实施不同的测试思想，在早期阶段集中在“证明程序有错”—— 发现Bug，后期集中在验证所有特性是否正常工作——降低风险，见作者的另外一篇讨论：测试执行中非常有效的策略

下面就是这两种观点的基本描述：

验证软件是验证软件是“工作的”，以正向思维，针对软件系统的所有功能点，逐个验证其正确性。其代表人物是软件测试领域的先驱Dr. Bill Hetzel （代表论著《The Complete Guide to Software Testing》)。

证明软件是“不工作的”，以反向思维方式，不断思考开发人员理解的误区、不良的习惯、程序代码的边界、无效数据的输入以及系统的弱点，试图破坏系统、摧毁系统，目标就是发现系统中各种各样的问题。其代表人物就是上面多次提到的G.J.Myers。他强调，一个成功的测试必须是发现Bug Bug的测试，不然就没有价值。

3.软件测试的风险论

测试被定义为“对软件系统中潜在的各种风险进行评估的活动”，这就是软件测试的风险论。软件测试自身的风险性是大家公认的，测试的覆盖度不能做到100％。测试的这种风险定义一方面源于这层含义，另外软件测试的标准有时不清楚，“软件规格说明书（Specification/ Spec）”是其中的一个标准，但也不是唯一的，因为Spec中有些内容完全有可能是错误的。所以，我们常常强调软件测试人员应该站在客户的角度去进行测试，除了发现程序中的错误，还要发现需求定义的错误、设计上的缺陷，可以针对Spec 去报Bug。但是，测试在大多数时间/情况下,是由工程师完成，而不是客户自己来做，所以又怎么能保证工程师和客户想得一样呢？

有人把开发比作打靶，目标明确，就是按照Spec 去实现系统的功能。而把测试比作捞鱼，目标不明确，自己判断哪些地方鱼多，就去哪些地方捞；如果只捞大鱼（严重缺陷），网眼就可以大些、撒网区域相对比较集中（测试点集中在主要功能-major features）。如果想把大大小小的鱼捞上来，网眼就要小、普遍撒网，不放过任何一块区域（测试点遍及所有功能——all features）。

在“风险”论的框架下，软件测试可以被看作是一个动态的监控过程，对软件开发全过程进行检测，随时发现不健康的征兆，发现问题、报告问题，并重新评估新的风险，设置新的监控基准，不断地持续下去，包括回归测试。这时，软件测试可以完全看作是软件质量控制的过程。

对应这种观点，产生基于风险的测试策略，首先评估测试的风险，功能出问题的概率有多大？哪些是用户最常用的20％功能——Pareto原则（也叫80/20原则）？如果某个功能出问题，其对用户的影响有多大？然后根据风险大小确定测试的优先级。优先级高的测试，优先得到执行，一般来讲，针对用户最常用的20％功能（优先级高）的测试会得到完全执行，而低优先级的测试（另外用户不经常用的80％功能）就不是必要的，如果时间或经费不够，就暂时不做或少做。

4.软件测试的经济学观点

“一个好的测试用例是在于它能发现至今未发现的错误”，体现了软件测试的经济学观点。实际上，软件测试经济学问题至今仍是业界关注的问题之一。经济学的核心就是要盈利，盈利的基础就是要有一个清楚的商业性目标。同样，商业性目标是否正确，直接决定了企业是否盈利的结果。多数情况下，软件测试是在公司内的执行。正是公司的行为目的，决定了软件测试含义或定义的经济性一面。正如，对软件质量的定义不仅仅局陷于“和客户需求的一致性、适用性”，而且要增加其它的要求——“预算内、按时发布、易于维护”。

软件测试也一样，要尽快尽早地发现更多的缺陷，并督促和帮助开发人员修正缺陷。原因很简单：平均而言，如果在需求阶段修正一个错误的代价是1，那么，在设计阶段就是它的3～6倍，在编程阶段是它的10倍，在内部测试阶段是它的20～40倍，在外部测试阶段是它的30～70倍，而到了产品发布出去时，这个数字就是 40～ 1000倍。修正错误的代价不是随时间线性增长，而几乎是呈指数级增长的。

5. 软件测试的标准论

如果从标准论来看软件测试，可以定义为软件测试就是“验证（Verification）”和“有效性确认（Validation）”活动构成的整体，即软件测试 = V&V。

“验证”是检验软件是否已正确地实现了产品规格书所定义的系统功能和特性。验证过程提供证据表明软件相关产品与所有生命周期活动的要求（如正确性、完整性、一致性、准确性等）相一致。相当于，以Spec为标准进行软件测试活动，验证软件产品和Spec的一致性。

“有效性确认”是确认所开发的软件是否满足用户真正需求的活动。相当于，保持对软件需求定义、设计的怀疑，一切从客户出发，理解客户的需求，发现需求定义和产品设计中的问题。这主要通过各种软件评审活动来实现。

需要说明的是，软件测试的对象是产品（包括阶段性产品，如市场需求说明书、产品规格说明书、技术设计文档、数据字典、程序包、用户文档等），而质量保证和管理的对象集中在软件开发的标准、流程和方法等。