借地方用一下翻译

Volume Number: 13
Issue Number: 10
Column Tag: Quality Control
Software Test Automation and the Product Life Cycle
by Dave Kelly, Symantec Corporation
Implementing software test in the product life cycle
The PLC and Automated Test
A product's stages of development are referred to as the product life cycle (PLC). There is considerable work involved in getting a product through its PLC. Software testing at many companies has matured as lessons have been learned about the most effective test methodologies. Still, there is a great difference of opinion about the implementation and effectiveness of automated software testing and how it relates to the PLC.
Computers have taken over many functions in our society that were once "manual" operations. Factories use computers to control manufacturing equipment and have cut costs enormously. Electronics manufacturing use computers to test everything from microelectronics to circuit card assemblies. Since automation has been so successful in so many areas, does it make sense that a software program should be used to test another software program? This is referred to as "automated software testing" for the remainder of this article.
Software testing using an automatic test program will generally avoid the errors that humans make when they get tired after multiple repetitions. The test program won't skip any tests by mistake. The test program can also record the results of the test accurately. The results can be automatically fed into a database that may provide useful statistics on how well the software development process is going. On the other hand, software that is tested manually will be tested with a randomness that helps find bugs in more varied situations. Since a software program usually won't vary each time it is run, it may not find some bugs that manual testing will. Automated software testing is never a complete substitute for manual testing.
There has been plenty of debate about the usefulness of automatic software testing. Some companies are quite satisfied with the developer testing his/her own work. Testing your own work is generally thought of as risky since you'll be likely to overlook bugs that someone not so close to the code (and not so emotionally attached to it) will see easily. As soon as the developer says it's done they ship it. The other extreme is the company that has its own automatic software test group as well as a group that tests the software manually. Just because we have computers does that mean that it is cost effective to write tests to test software and then spend time and resources to maintain them? The answer is both yes and no. When properly implemented, automated software test can save a lot of time, time that will be needed as the software approaches shipping.
This is where the PLC comes in. How effectively you make use of the PLC will often be dependent on your programming resources and the length of the PLC. Companies large and small struggle with software testing and the PLC. Hopefully, this discussion of the PLC should help you determine when to use automation and when manual testing is preferred. This should help you answer the questions: "Why should I automate my software testing?" "How can I tell if automation is right for my product?" "When is the best time to develop my test software?".
The Product Life Cycle
As we discuss the use of automated and manual testing, we need to understand what happens in each phase of the product life cycle. The PLC is made up of six major stages, the Design Phase, the Code Complete Phase, the Alpha Phase, the Beta Phase, the Zero Defect Build Phase, and the Green Master Phase. You can think of the PLC as a timeline showing the development of a software product. These are the major milestones that make up the Product Life Cycle. Products that follow these guidelines for implementation of the PLC will have a much better chance of making it to market on time.
The implementation of the PLC varies widely from company to company. You can use this as a guide for future reference to assist you in your automation efforts. Your implementation will vary from the ideal PLC that is discussed here, but your software's success may depend on how well you've implemented its PLC. If your PLC is to include automated testing you should pay attention to which automated tasks are performed during each phase.
For each phase we'll describe it, define its special importance and discuss how to incorporate software automation into your project. Most other discussions of the PLC don't include the lessons learned about test automation. This should be your "one-stop" guide to help you know how and when automation fits into the PLC.
Design Phase
What is the Design Phase? The design phase begins with an idea. Product managers, QA, and Development get together at this point to determine what will be included in the product. Planning is the essence of the design phase. Begin with the end in mind and with a functional specification. Write down all of your plans. What will your product do? What customer problems does it solve?
Incorrectly, some companies don't include Quality Assurance (QA) in the design phase. It is very important that QA be involved as early as possible. While developers are writing code, QA will be writing tests. Even though QA won't really have the total picture of the product, they will want to get as much of a jump on things as possible. Remember, that the primary purpose of QA is to report status. It is important to understand the product's status even early in the Design Phase.
Why is the Design Phase important? If you think you're too short on time to write up a functional description of your product, then consider the extra time involved to add new features later on. Adding features later (especially once the Code Complete Phase has been reached) is known as "feature creep". Feature creep can be a very costly haphazard way to develop your product, and may materially interfere with delivery of the software.
Automation activity during the Design Phase. As soon as the functional specification is written, create all test cases so that they can be run manually. Yes, that's right, manually! These manual tests are step-by-step "pseudo" code that would allow anyone to run the test. The benefits of this approach are:
1. Your test cases can be created BEFORE ever seeing the software's user interface (UI). It is too soon to automate tests at this point in time, but you can create manual tests with only a small risk of changes that will occur . This is a point of great frustration for those who have tried to implement automated test scripts too early. Just as soon as the test script is written changes in the UI are bound to be introduced and all the work on the script is found to be for nothing.
2. When (not if) the code is modified, you will always have manual procedures that can be adapted to the change more quickly than an automated test script. This is a great way to guarantee that you will at least have tests you can perform even if automation turns out to not be feasible. (Note: one of the objections to software test automation is that the tests must be continually updated to reflect changes in the software. These justifiable objections usually stem from the fact that automation was started too early.)
3. Test methods can be thought out much more completely because you don't have to be concerned with the programming language of the automation tool. The learning curve of most automation tools may get in the way of writing meaningful tests.
If you have the resources available, have them begin training on the test tools that will be used. Some members of the team should start writing library routines that can be used by all the test engineers when the start their test coding. Some of these routines will consist of data collection/result reporting tools and other common functions.
After the manual test cases have been created decide with your manager which test cases should be automated. Use the Automation Checklist found later in this article to assist you in deciding what tests to automate. If you have enough manpower you may want to have an test plan team and an automation team. The test plan team would develop tests manually and the automation team would decide which of the manual tests should be run automatically (following the guidelines of the Automation Checklist later in this article). The automation team would be responsible for assuring that the test can be successfully and cost effectively automated.
Sometime during the design phase, as soon as the design is firm enough, you'll select the automation tools that you will need. You don't have to decide exactly which tests need to be automated yet, but you should have an idea of the kinds of tests that will be performed and the necessary capabilities of the tools. That determination is easier as the software gets closer to the code complete phase. Your budget and available resources will begin to come into play here.
For just a moment, let's discuss some of the considerations you should use in selecting the test tools you need. You'll also want to keep in mind the Automation checklist later in this column. It will help you determine if a test should be automated. There are a few good testing tools including Apple Computer's Virtual User (VU) (See the September, 1996 article "Software Testing With Virtual User", by Jeremy Vineyard) and Segue's QA Partner (Segue is pronounced "Seg-way").
Is there a lot of user interface (UI) to test? Software with a lot of UI is well suited for automated black box testing. However, some important considerations are in order here. You need to get involved with development early to make sure that the UI can be "seen" by the automation tool. For example: I've seen programs in which a Virtual User 'match' task (note: a task is what a command is called in Virtual User) couldn't find the text in a text edit field. In those cases, this occurred because the program didn't use standard Macintosh calls, but rather was based on custom libraries that provided UI features their own way.
Will the automated test environment effect the performance or operation of the system being tested? When you're trying to test the latest system software, you don't want the testing system changing the conditions of the test.
Is the speed that the tests run a consideration? If you're trying to measure the performance of a system you'll want to make sure that the conditions are as much like the "real world" as possible. You should consider the amount of network traffic that is present while you're running your tests. Also, the speed of your host processor can effect the time it takes your tests to run. You should schedule your tests so that you minimize the possibility of interfering with someone else on your network. Either isolate your network from others or warn them that you will be testing and that there is a possibility that their network activity may slow down.
What kinds of tests will be performed? The lower the level the testing is, the more likely white box testing should be used. A good example of this would be if you have a function that does a calculation based on specific inputs. A quick C program that calls the function would be much faster and could be written to check all the possible limits of the function. A tool like VU would only be able to access the function through the UI and would not be able to approach the amount of coverage that a C program could do in this situation.
Is there a library of common functions available or will you have to write them yourself? It will save a lot of time if you don't have to develop libraries yourself. No matter how extensive the command set, efficient use of library functions will be essential. Libraries that others have written may be useful; you can modify them to meet your own needs.
What will be the learning curve for a script programmer? The time it takes will depend greatly on the kind of testing you have to do and the experience of the programmer. If you've done your homework on the available test tools, you should know what to expect. Some companies even offer training courses (for a price) in their test software.
Can your automation tool automatically record actions for you? Some tools do this, but don't expect to rely on this too heavily. Tools that I've seen that do this end up creating code that has hard coded strings and tend to be organized in a sequential manner rather than by calling procedures. These recorded scripts are harder to maintain and reuse later. If you plan to use the same script for international testing, modifying the script will mean much more work. If you want to record actions, I recommend that you do it only to create short functions and you should edit the script after recording to remove the unwanted hard coded strings, etc.
Can you run AppleScript scripts from the tool's script language? This is a very useful feature since AppleScript scripts are so easy to write and can add additional functionality to your test tool.
In preparing this article, I encountered several "pearls" worth relating:
"Success in test automation requires careful planning and design work, and it's not a universal solution. ... automated testing should not be considered a replacement for hand testing, but rather as an enhancement." (Software Testing with Visual Test 4.0, forward by James Bach, pg. vii)
"The quality assurance engineers then come on the scene... and begin designing their overall test plan for the features of the product...."
"The goal is to have the test plan and checklists laid out and ready to be manually stepped through by the test engineers when each feature is completed by the programmers. Each item on a checklist is considered a scenario and related scenarios are grouped into test cases." (Software Testing with Visual Test 4.0, pg. 5-6)
Code Complete Phase
What is the Code Complete Phase? At this major milestone the code has been completed. The code has been written, but not necessarily yet debugged. (Development may try to claim they are at code complete even though they may still have major coding still left to do. Go ahead and let them declare the code complete, but don't let them get to Alpha until the code really is completely written.)
Why is the Code Complete Phase important? Sooner or later you'll have to get to a point where new code is no longer being written, and the major effort is in fixing bugs. Development will be relieved to get to this point as now they don't have to be as concerned with the initial coding and can concentrate on refining the existing product. (This is why they will try to claim they are at code complete even when they are not).
Automation activity during the Code Complete Phase Although the UI may still change, QA can begin writing Automatic test cases. The tests that should be written at this point are breadth tests that tell the status of the overall software product. Don't write tests which stress the product until you get close to Alpha. The product will probably break very easily. Some acceptance (or "smoke") tests should also be created to give a quick evaluation of the status of a particular build. Before reaching the Alpha phase there should also be tests written to test the Installer, boundary (or stress tests), compatibility (hardware and OS), performance, and interoperability.
Somewhere just before code complete, you will need to decide which tests should be made into automatic tests and what test tools to use. Use the following checklist to help you determine which tests should be automated:
Automation Checklist
If you answer yes to any of these questions, then your test should be seriously considered for automation.
Can the test sequence of actions be defined?
Is it useful to repeat the sequence of actions many times? Examples of this would be Acceptance tests, Compatibility tests, Performance tests, and regression tests.
Is it necessary to repeat the sequence of actions many times? (See Testing Computer Software, pg. 196 and 282).
Is it possible to automate the sequence of actions? This may determine that automation is not suitable for this sequence of actions.
Is it possible to "semi-automate" a test? Automating portions of a test can speed up test execution time.
Is the behavior of the software under test the same with automation as without? This is an important concern for performance testing.
Are you testing non-UI aspects of the program? Almost all non-UI functions can and should be automated tests.
Do you need to run the same tests on multiple hardware configurations? Run ad hoc tests (Note: Ideally every bug should have an associated test case. Ad hoc tests are best done manually. You should try to imagine yourself in real world situations and use your software as your customer would. As bugs are found during ad hoc testing, new test cases should be created so that they can be reproduced easily and so that regression tests can be performed when you get to the Zero Bug Build phase.) An ad hoc test is a test that is performed manually where the tester attempts to simulate real world use of the software product. It is when running ad hoc testing that the most bugs will be found. It should be stressed that automation cannot ever be a substitute for manual testing.
Alpha Phase
What is the Alpha Phase? Alpha marks the point in time when Development and QA consider the product stable and completed. The Alpha Phase is your last chance to find and fix any remaining problems in the software. The software will go from basically functional to a finely tuned product during this phase.
Why is the Alpha Phase important? Alpha marks a great accomplishment in the development cycle. The code is stable and the most major bugs have been found and fixed.
Automation Activity During The Alpha Phase
At this point you have done the tasks that need to be done in order to reach Alpha. That is, you have all your compatibility, interoperability, and performance tests completed and automated as far as possible. During Alpha you'll be running breadth tests every build. Also you'll run the compatibility, interoperability, and performance tests at least once before reaching the next milestone (beta). After the breadth tests are run each build, you'll want to do ad hoc testing as much as possible. As above, every bug should be associated with a test case to reproduce the problem.
Beta Phase
What is the Beta Phase? The product is considered "mostly" bug free at this point. This means that all major bugs have been found. There should only be a few non essential bugs left to fix. These should be bugs that the user will find annoying or bugs that pose relatively no risk to fix. If any major bugs are found at this point, there will almost definitely be a slip in the shipping schedule.
Automation activity during the Beta Phase
There's no more time left to develop new tests. You'll run all of your acceptance tests as quickly as possible and spend the remaining time on ad hoc testing. You'll also run compatibility, performance, interoperability and installer tests once during the beta phase.
Remember that as you do ad hoc testing every bug should have an associated test case. As bugs are found during ad hoc testing, new test cases should be created so that they can be reproduced easily and so that regression tests can be performed when we get to the Zero Bug Build phase.
Zero Defect Build Phase
What is the Zero Defect Build Phase? This is a period of stability where no new serious defects are discovered. The product is very stable now and nearly ready to ship.
Automation Activity During The Zero Defect Build Phase
Run regression tests. Regression testing means running through your fixed defects again and verify that they are still fixed. Planning for regression testing early will save a lot of time during this phase and the Green Master phase.
Green Master
What is the Green Master Phase? Green Master is sometimes referred to as the Golden Master or the final candidate. The product goes through a final checkout before it is shipped (sent to manufacturing).
Automation activity during the Green Master Phase
After running general acceptance tests, run regression tests. You should run through your fixed defects once again to verify that they are still fixed. Planning for regression testing early will save a lot of time during this phase.
Understanding the PLC will help you select your automation tools
Perhaps this review of the PLC is 'old hat' for you. In my experience, reviewing the process usually helps to focus the project. Since software test has been evolving over the years, and many companies have been struggling with how to implement it, we can always use all the help and advice that we can get. There are some good sources of information that will help. Take a look at the Software Testing Laboratories web site at URL http://www.stlabs.com. There are a few relevant articles in James Bach's archives at http://www.stlabs.com/LABNOTE.HTM. The Software Testing Labs is mostly geared toward MS Visual Test, but the QA principles involved are the same for any platform, Macintosh included.
This review of the PLC represents an ideal situation where Development and QA both buy into this same way of doing the software business. It will also require some commitment on the part of Management to assure that each phase is supported and accepted. Once all of those conditions, everyone involved needs to be focused on making sure that no short cuts are taken around the agreed upon PLC. Hopefully this discussion has been helpful to you in automating your software tests.
Glossary
Word or phrase
Definition.
Ad Hoc Testing
Goal oriented passing through the product. Sometimes to prove or disprove a notion of how the product will behave.
Alpha Test
The part of the Test Phase of the PLC where code is complete and the product has achieved a degree of stability. The product is fully testable (determined by QA). All functionality has been implemented and QA has finished the implementation of the test plans/cases. Ideally, this when development feels the product is ready to be shipped.
Automated Testing
Creation of individual tests created to run without direct tester intervention.
Beta Test
The part of the Test Phase of the PLC where integration testing plans are finished, depth testing coverage goals met; Ideally, QA says product is ready to ship. The product is stable enough for external testing (determined by QA).
Black Box Test
Tests in which the software under test is treated as a black box. You can't "see" into it. The test provides inputs and responds to outputs without considering how the software works.
Boundary Testing
Test which focus on the boundary or limit conditions of the software being tested. (Some of these tests are stress tests).
Breadth Testing
Matrix tests which generally cover all product components and functions on an individual basis. These are usually the first automated tests available after the functional specifications have been completed and test plans have been drafted.
Breath Testing
Generally a good thing to do after eating garlic and before going out into public. Or you may have to take a breath test if you're DUI.
Bug
A phenomenon with an understanding of why it happened.
Code Complete
Phase of the PLC where functionality is coded in entirety; bug fixes are all that are left. All functions found in the Functional Specifications have been implemented.
Code Freeze
When development has finished all new functional code. This is when development is in a "bug fixing" stage.
Coding Phase
Phase of the PLC where development is coding product to meet Functional/Architectural Specifications. QA develops test tools and test cases during this phase.
Compatibility Test
Tests that check for compatibility of other software or hardware with the software being tested.
Concept Phase
Phase of the PLC where an idea for a new product is developed and a preliminary definition of the product is established. Research plans should be put in place and an initial analysis of the competition should be completed. The main goal of this phase is to determine product viability and obtain funding for further research.
Coverage analysis
Shows which functions (i.e., GUI and C code level) have been touched and which have not.
Data Validation
Verification of data to assure that it is still correct.
Debug
To search for and eliminate malfunctioning elements or errors in the software.
Definition Phase
See Design Phase.
Dependency
This is when a component of a product is dependent on an outside group. The delivery of the product or the reaching a certain milestone is affected.
Depth Testing
Encompasses Integration testing, real world testing, combinatorial testing, Interoperability and compatibility testing.
Design Phase
Phase of the PLC where functions of the product are written down. Features and requirements are defined in this phase. Each department develops their departments' plan and resource requirements for the product during this phase.
Dot Release
A major update to a product.
Feature
A bug that no one wants to admit to.
Focus
The center of interest or activity. In software, focus refers to the area of the screen where the insertion point is active.
Functional
Phase of the PLC defining modules.
Specifications
Their implementation requirements and approach, and exposed API. Each function is specified here. This includes the expected results of each function.
GM
See Green Master.
Green Master (GM)
Phase of the PLC where the certification stage begins. All bugs, regressed against the product, must pass. Every build is a release candidate (determined by development).
GUI
Graphical User Interface.
Inline
Phase of the PLC after shipping (STM) where bugs are fixed for interim release. Maintenance of the product involves cleaning up bugs that are found after STM. Inlines are created to address these problems.
Integration Testing
Depth testing which covers groups of functions at the subsystem level.
Interoperability Test
Tests that verify operability between software and hardware.
Load Test
Load tests study the behavior of the program when it is working at its limits. Types of load tests are Volume tests, Stress tests, and Storage tests.
Localization
This term refers to making software specifically designed for a specific locality.
Maintenance Release
See Inline.
Metrics
A standard of measurement. Software metrics are the statistics describing the structure or content of a program. A metric should be a real objective measurement of something such as number of bugs per lines of code.
Milestones
Events in the Product Life Cycle which define particular goals.
Performance Test
Test that measures how long it takes to do a function.
Phenomenon
A flaw without an understanding.
PLC
Product Life Cycle - see Software Product Life Cycle.
Pre-Alpha
Pre-build 1; product definition phase. (Functional Specification may still be in process of being created).
Product Life Cycle
The stages a product goes through.
(PLC)
from conception to completion. Phases of product development includes: Definition Phase, Functional/Architectural Specification Phase, Coding Phase, Code Complete Phase, Alpha, Beta, Zero Bug Build Phase, Green Master Phase, STM, and Maintenance/Inline Phase.
Proposal Phase
Phase of the PLC where the product must be defined with a prioritized feature list and system and compatibility requirements.
QA Plan
A general test plan given at the macro level which defines the activities of the test team through the stages of the Product Life Cycle.
Real World Testing
Integration testing which attempt to create environments which mirror how the product will be used in the "real world".
Regression Testing
Retesting bugs in the system which had been identified as fixed, usually starting from Alpha on.
Resource
People, software, hardware, tools, etc. that have unique qualities and talents that can be utilized for a purpose.
Risk
Something that could potentially contribute to failing to reach a milestone.
STM
See Ship to Manufacturing.
Storage Tests
Test how memory and space is used by the program, either in resident memory or on disk.
Stress Test
Tests the program's response to peak activity conditions.
Syncopated Test
A test that works in harmony with other tests. The timing is such that both tests work together, but yet independently.
Test Case
A breakdown of each functional area into an individual test. These can be automated or done manually.
Test Phase
Phase of the PLC where the entire product is tested, both internally and externally. Alpha and Beta Tests occur during this phase.
Test Plan
A specific plan that breakdown testing approaches on a functional area basis.
Test Suite
A set of test cases.
Usability
The degree to which the intended target users can accomplish their intended goals.
Volume Tests
Test the largest tasks a program can deal with.
White Box Test
It is used to test areas that cannot be reached from a black box level. (Sometimes called Glass Box testing).
Zero Bug Build
Phase of the PLC where the product has stabilized in terms of bugs found and fixed. Development is fixing bugs as fast as they are found, the net resulting in zero bugs on a daily basis. This is usually determined when after a few builds have passed. This is the preliminary stage before Green Master.
Bibliography and References
1. Apple Computer, Inc. Setting Up and Running Virtual User, 1993.
2. Arnold, Thomas R. II. Software Testing with Visual Test 4.0. IDG Books Worldwide, Inc., 1996.
3. Bach, James, "Test-Automation Snake Oil". Windows Tech Journal, October 1996, p. 40.
4. Cem Kaner, Jack Falk, and Nung Quoc Nguyen. Testing Computer Software, Second Edition. New York:Van Nostrand Rienhold, 1993.
5. McCarthy, Jim. Dynamics of Software Development. Redmond, WA:Microsoft Press, 1995.
6. Vineyard, Jeremy. "Software Testing With Virtual User". MacTech Magazine 12:9 (September 1996), pp. 16-20.
7. Software Testing Laboratories web site http://www.stlabs.com/
 
Dave Kelly is Software Quality Assurance Manager for Symantec Corporation responsible for testing several networking products including Norton Administrator for Networks for MacOS. He has worked with the Macintosh since 1984 and is one of the founding editorial board members of MacTutor Magazine (now MacTech Magazine). You can reach him at [email protected].

-------------------- 偶正在努力，偶正在考研

[em071][em071][em071][em071][em071]

谁也帮我翻译一篇文章吧？？？？？？？？~~~~~~~~~~~

就快被折磨死了~~~~~~[em091][em091][em102][em101]

体积数字: 13
议题数字: 10
专栏标签: 质量控制
软件测试自动化和产品生活周期
藉着大卫凯利, Symantec 公司
实现产品生活的软件测试循环
PLC 而且自动化测试
发展的产品阶段被称为产品生活周期。 (PLC) 有在..

哥们儿，您用的是金山全文吧？
有要专业翻译的找我。加QQ或者在这里留言。我在翻译公司。同学的生意给优惠。千万不要用什么词霸全文翻译。翻出来全是DOGSHIT. -------------------- 醒是梦的影子，光是暗的影子，盛开是凋谢的影子，流淌是枯竭的影子，一切都不可避免的会消失。你感到害怕吗？还是说，让我们继续走下去？

哈哈，这翻译出来的是汉语吗？怎么都说的不会话了？

太厉害了！牛人！：） --------------------  ;︶︵︶;，我渴望真实，向往自由，然而世界确是虚伪，封闭，人们都在相互隐瞒，最后我终于明白，在现实中那纯洁的理想原来是那么的脆弱不堪！    

！！ [ 2004-05-31 10:42:11 llj 修改 ]

顶
没话说
好人~ -------------------- 杰哥说的！

[蓝jane]：
你真是个好ＭＭ
不能不帮我翻译一点东西啊
一百字以内的



--------------------
天行健，君子以自强不息；
地势坤，君子以厚德载物。

呵呵！！！！！！！


好呀，你发给我吧 -------------------- 我喜欢蓝色，
爱幻想，
有点忧郁，
有点小孩子气，
我的偶像刘德华！！！

体积数字: 13
议题数字: 10
专栏标签: 质量控制
软件测试自动化和产品生活周期
藉着大卫凯利, Symantec 公司
实现产品生活的软件测试循环
PLC 而且自动化测试
发展的产品阶段被称为产品生活周期。 (PLC) 有在..

[蓝jane]：
你真是个好ＭＭ
不能不帮我翻译一点东西啊
一百字以内的

-------------------- 天行健，君子以自强不息；
地势坤，君子以厚德载物。

体积数字: 13
议题数字: 10
专栏标签: 质量控制
软件测试自动化和产品生活周期
藉着大卫凯利, Symantec 公司
实现产品生活的软件测试循环
PLC 而且自动化测试
发展的产品阶段被称为产品生活周期。 (PLC) 有在使一种产品通过它的 PLC 方面是积极参与的相当工作。 当课已经被学习关于最有效的测试方法学的事时候 , 在许多公司尝试的软件已经成熟。剧照, 有关自动化的软件落实和效力有意见的一种很棒的不同尝试，而且它如何与 PLC 有关。
计算机已经接管我们的社会许多功能是曾经 "手册" 操作。 工厂使用计算机控制制造业的仪器并且已经非常地减低费用。 电子学制造使用计算机从微体电子学到线路卡片集会测试每件事物。 既然自动化已经在这么多区域中是如此成功，它有道理一个软件计画应该用来测试另外的一个软件计画吗? 这为这一个文章的剩余物被称为 "自动化了软件测试" 。
当他们在多样的重复之后很疲累的时候 , 通常使用一个自动的测试计画的尝试软件将会避免人发动的错误。 测试计画将不藉着错误跳越任何的测试。 测试计画也能正确地记录测试的结果。结果可能自动地进入一个可能提供有用的统计学的数据库之内被喂在如何之上软件发展程序正在去的井。 另一方面，用手被测试的软件将会与一的帮助在较不同的情形中找起来疯狂的任意一起测试。 因为一个软件计画通常每时间将不改变它被进行,它不可能找一些疯狂的那一本手册尝试意志。 从不自动化了软件尝试是人工的测试一位完全的替代。
有关自动的软件测试的有用已经有许多辩论。 一些公司以开发者测试被相当满意他的/她自己的工作。 测试你自己的工作被通常想到当危险的自你以后将会可能俯看疯狂的某人不如此的接近到密码 ( 而且不如此的在情绪上附件到它) 将会容易地。 像开发者一样的很快很完成了地他们装船运送它。 另一个极端是有它自己的自动软件测试团体和一个用手测试软件的团体公司。仅仅因为我们有计算机做那低劣的它是花费有效的写测试给测试软件然后度过时间和资源维持他们?答案是的和号码当适当地实现 , 自动化软件测试都当软件接近海运，能节省许多时间，将会被需要的时间。
这是 PLC 进来的地方。 你利用多么的有效 PLC 将会时常依赖你的规画资源和 PLC 的长度。 和软件测试和 PLC 的公司大的和小竞争。有希望地， PLC 的这讨论应该帮助你决定何时该使用自动化而且手册测试何时被。 这应该帮助你问题: "我为什么应该自动化我的软件测试?" "如果自动化对我的产品是正确的，我如何能?" " 最好时间是何时发展我的测试软件?"。
产品生活周期
当我们自动化讨论使用和手册测试,我们在每时期产品生活周期发生的东西。PLC 由六个主要的阶段组成,设计时期,密码完成时期 , 阿尔发时期 , 贝它时期 , 零的缺点建造时期 , 和绿色的主人时期。 你能把 PLC 当做一个时间期限表现软件产品的发展。 这些是捏造产品生活周期的主要里程碑。 为 PLC 的落实跟随这些指导方针的产品将会做到有一好多了机会准时在市场上销售。
PLC 的落实从公司到公司广泛地改变。 你能使用这如针对未来的导引叁考在你的自动化努力中协助你。 你的落实将会从理想在这里被讨论的 PLC 改变，但是你的软件成功可能仰赖你已经实现它的 PLC 多好。 如果你的 PLC 要包括自动化你应该注意的测试在每时期期间自动化工作被运行。
因为每时期我们将会描述它, 定义它的特别重要而且讨论该如何将软件自动化纳入你的计画。 PLC 的最其他的讨论不包括课有学问的大约测试自动化。 这应该是你的关于帮助你的指南 " 多元" 的如何而且当自动化进入 PLC 之内适合的时候。
设计时期
设计时期是什么? 设计时期由一个主意开始。 产品经理，时间表和发展此时聚在一起决定将会被包含在产品中的东西。 计划是设计时期的本质。 由思想的结束开始和由于一件功能的规格。 写下你的全部计划。 你的产品将会做什么? 它解决什么客户问题?
不正确地，一些公司在设计时期中不包括质量保证 (时间表) 。 很非常重要地那一个时间表尽快地是积极参与的。 当开发者正在写密码的时候,时间表将会写测试。 即使时间表将不真的有产品的总照片, 他们将会拿当如可能的在事物上的许多一次跳跃之时。 ,时间表的主要目的要状态。 它很重要甚至在设计时期中初期产品的状态。
设计时期为什么重要的? 如果你认为你准时太短整理你的产品功能的描述, 那么考虑对稍后新的特征是积极参与的额外时间。 增加特征比较迟的 (密码完全的时期一次尤其已经被到达) 是即是 "特征爬". 特征爬可能是一个非常昂贵的偶然方法发展你的产品, 和可能实质上干扰软件的递送。
在设计时期期间的自动化活动。 一旦功能的规格被写, 产生所有的测试情形以便他们用手可能被跑。 是的，那是正确的,用手! 这些人工的测试是按部就班的 "假的" 密码以会允许任何人进行测试。 这方式的利益是:
1. 你的测试情形可能被产生在曾经之前软件的使用者接口.(UI) 它是太很快此时及时自动化测试,但是你能用只有变化的小危险产生人工的测试以将会发生。 这是那些已经试着太早实现自动化测试手写体的人很棒的挫折点。 的很快就像测试手写体是 UI 的书面变化一样多对被介绍是约束的而且在手写体上的所有工作被发现免费。
2. 当密码被修正的时候,你将会总是有人工的程序以可能被适应变化比一个被自动化的测试手写体更更快。 这是保证的一个很棒的方法你将会至少有你能运行的测试，即使自动化变成不是能实行的。 (笔记: 这些可辩解的异议通常起源于自动化被开始太早的事实)。
3. 因为你不须 , 所以测试方法更加完全地可能被想出与自动化工具的规画语言有关系。 最大多数的自动化工具的学问曲线可能到达妨碍写意义深长的测试。
如果你让资源可得的,让他们开始在测试工具上教育以将会被用。 队的一些成员应该开始写作图书馆常式以可能被所有的测试工程师用当开始他们的测试编码。 一些常式将会由数据收集/ 结果报告工具和其他的通常功能所组成。
在测试情形已经被产生的手册和你的测试情形应该是自动化的经理决定之后。 使用自动化检查表在这一个文章中发现起来比较迟的在测试自动化的决定中协助你。 如果你有你可能有一个测试计划队和一个自动化队的充足人力。 测试计划队会用手发展测试，而且自动化队会决定哪一手册测试自动地应该被跑。 (在这一个文章中稍后跟随自动化检查表的指导方针) 自动化队会负责保证的测试可能是成功地和有效地费用自动化。
在设计时期期间某一个时间,像设计一样的很快够坚定的,你将会选择你将会的自动化工具。你不须完全地决定测试需要仍然被自动化,但是你应该有将会被运行的测试和工具的必需能力的类型主意。 当软件变比较靠近到密码完全的时期时候 , 那决心是比较容易的。 你的预算和可得的资源将会开始在这里进入游戏。
只是片刻, 让我们讨论你应该使用的一些考虑在选择测试方面用工具工作你需要。 你也将会在这专栏中变得比较迟。 如果一个测试应该被自动化，它将会帮助你决定。 有一些好测试工具包括苹果计算机的虚拟使用者 (VU)(1996 年九月文章 "和虚拟的使用者的尝试软件",被杰里米葡萄园) 和继续如前的时间表合伙人。 ( 继续如前是发音 "Seg-方法")
有许多使用者接口 (UI) 测试吗? 和许多 UI 的软件为自动化的黑盒子测试被很好地适合。 然而，一些重要的考虑是井然有序在这里。 你以对确定是早的发展变积极参与 UI 可能是自动化工具的 "看到" 。 举例来说: 我已经看到计画在一个虚拟的使用者 '比赛' 工作 (笔记: 一件工作是一个指令所被邀请虚拟的使用者来的)不可以找一个本文编辑领域的本文。因为计画不使用标准的 Macintosh 呼叫, 但是宁可以提供了 UI 特征他们自己的方法习惯图书馆为基础 , 所以在那些情况，这发生。
将会环境招致表现或系统被测试的操作被自动化的测试? 当你正在试着测试最近的系统软件的时候, 你不测试系统变更测试的情况。
速度是测试跑考虑吗? 如果你正在试着测量你将会确定的系统表现情况是尽可能地像真正的世界。 当你正在进行你的测试时候 , 你应该考虑礼物的网络交通的数量。 也，当它参加你的测试跑的时候 , 你的主人处理器的速度能招致。 你应该预定你的测试以便你将干扰在你的网络上的其他人的可能性减到最少。 或隔离从其它来的你网络或他们你将会尝试而且有一种可能性他们的网络活动可能减慢。
测试将会被运行是什么类型的? 降低度的那测试是,更有可能的白色盒子测试应该被用。 一个好例子这会是如果你有一个做基于特定的输入计算的功能。 呼叫功能的计画会是更加快速的而且可能被写检查功能的所有可能的极限一个快的 C。 相似的 VU 只会能够的一个工具经过 UI 存取功能而且不能够接近 a C 计画可以在这一种情形中做的报导数量。
有通常的功能一间图书馆可以用吗，否则你将会必须写他们你自己吗? 如果你不须，它将会节省许多时间发展图书馆你自己。 无论指令放置 , 图书馆功能的有效率的使用将会是必要的多么广泛。 其它已经写的图书馆可能是有用的；你能修正他们符合你自己的需要。
什么将会是一个手写体程序师的学问曲线? 当它拿的时候将会非常仰赖你必须做的尝试类型和程序师的经验。 如果你已经做在可得的测试工具上的你家庭操作,你应该该期待什么。 一些公司更甚至提供教育他们的测试软件的课程 (为价格) 。
你的自动化工具能为你自动地记录行动吗? 一些工具做这, 但是不期待太很重地仰赖这。 我已经看到的工具这一个结束提高创造密码以已经努力地编码线而且容易吗在以继续的方式被组织不愿藉由呼叫程序。 这些记录手写体对稍后维持而且重复使用是比较难的。 如果你计划使用相同的手写体作为国际的测试, 修正手写体更加多将会意谓工作。 如果你记录行动, 我推荐你只做它产生短功能，而且你应该在记录之后编辑手写体除去不必要者努力地编码了线等。
你能跑来自工具的手写体语言的 AppleScript 手写体吗? 这是一个非常有用的特征自 AppleScript 手写体以后对写是如此容易和能把附加的功能性加入你的测试工具。
在准备这一个文章方面，我遇到了一些 "珍珠" 值关联:
"测试自动化的成功需要小心的计划和设计工作，而且它不是全世界的解决。 ... 自动化测试为手测试不应该被考虑一个替换, 但是宁可如提高."( 和视觉的测试 4.0 的尝试软件,被詹姆士巴哈， pg 转寄 vii)。
" 质量保证然后设计来在现场上。。。 而且开始设计他们的全部测试关于的计划产品的特征...."
"目标要有测试计划，而且检查表展开而且当每个特征被程序师完成的时候要被测试用手穿越踏的预备好的状态设计。 在一张检查表上的每个项目被考虑一个情节，而且相关的情节进入测试情形之内被聚集。"(和视觉的测试 4.0 ， pg 的软件测试。 5-6)
密码完全的时期
密码完全的时期是什么? 在密码已经被完成的这个主要的里程碑。 密码已经被写, 但是不必然地仍然除错。 ( 发展可能试着宣称他们在密码完全的即使他们可能仍然让主修科目编码仍然左边做。 前进而且让他们宣布密码完全的, 但是直到密码真的被完全地写不让他们到达阿尔发.)
密码完全的时期为什么重要的? 迟早你将会必须到达新的密码正在不再被写的点，而且主要的努力是在固定方面疯狂的。当现在他们不必是如编码的起始所关心和能专注于精炼已存在的产品时候 , 发展将会被减轻到达这点。 ( 这是他们为什么将会试着宣称他们在密码完全的即使当他们是不).
自动化活动在密码期间完成时期虽然 UI 可能仍然改变，但是时间表能开始写自动机械测试情形。应该在此时被写的测试是全部的软件产品的状态宽度测试。 不要写强调产品直到你到达在阿尔发的附近测试。 产品将会非常容易地或许打破。 一些接受 ( 或 "烟") 测试也应该被产生给一个特别的建造状态的快评估。 在到达阿尔发时期那里之前也应该是被写测试装置，边界 ( 或压迫力测试) ，兼容性 (硬件和操作系统) ，表现和互通性的测试。
某处仅仅在密码之前完全的,你将会决定哪一测试应该被做成自动的测试和使用什么测试工具。 使用下列各项检查表帮助你决定哪一测试应该被自动化:
自动化检查表
如果你是的到这些问题之中的任何一个,那么你的测试应该严重地为自动化被考虑。
行动的测试序列能被定义吗?
许多次重复行动的序列是有用的吗? 例子这会是验收测试，兼容性测试 , 表现测试 , 和复原测试。
许多次重复行动的序列是必需的吗? (测试的计算机软件， pg 。 196 和 282).
自动化行动的序列是可能的吗? 这可能决定自动化对行动的这序列是不适当的。
它是可能的吗到 "半-自动化"一个测试? 自动化一个测试的部分能加速测试实行时间。
在测试下面的软件行为对自动化感到相同吗当做没有? 这是表现测试的重要关心。
你正在尝试计画的非 UI 方面吗? 几乎所有非 UI 动作能和应该被自动化测试。
你进行在多样的硬件结构上的相同测试吗? 进行特别测试 (笔记: 理想地每个虫应该有一个联合的测试情形。 特别测试很最好完成了用手。 你应该试着在真正的世界情形中想像你自己而且如你的客户使用你的软件会。当疯狂的在特别测试期间被发现,新的测试情形应该被产生以便他们容易地可能被再生而且当你到达零的虫建造时期的时候复原测试可能被运行)。一个特别测试是一个用手被运行的测试考试人尝试模拟软件产品的真正世界使用哪里。 它是何时最疯狂的将会被发现的流动特别测试。 它应该紧张自动化不能够曾经是人工的测试替代。
阿尔发时期
阿尔发时期是什么? 阿尔发及时为点作标记当发展而且时间表考虑产品马房而且完成。 阿尔发时期是找并且固定软件的任何剩余的问题你最后机会。 软件将会去从基本上功能到一种精美地调谐的产品在这时期期间。
阿尔发时期为什么重要的? 阿尔发为发展周期的一个很棒的成就作标记。 密码是马房和那最主修科目疯狂的已经被发现和固定的。
自动化活动在阿尔发时期期间
此时你已经做工作要被做的需要为了要到达阿尔发。那是, 你有你所有的兼容性，互通性和表现尽可能远的测试完成而且自动化。 在阿尔发的时候你将会是流动的宽度测试每个建造。 你也将会在到达下一个里程碑之前至少进行兼容性，互通性和表现测试。 (贝它) 在宽度测试之后被跑每个建立,你将会尽可能很多的做特别测试。 如上方，每个虫应该被和一个测试情形联合再生问题。
贝它时期
贝它时期是什么? 产品被考虑 "大概" 虫自由的此时。 这一个方法所有的主要疯狂的已经被发现。 那里应该只是一些非必要的疯狂左边固定。 这些应该是疯狂的使用者将会找令人感到懊恼的或疯狂的那姿势相对地没有危险固定。 如果任何的主要疯狂的在此时被发现,那里将会几乎明确地是海运时间表的滑。
自动化活动在贝它时期期间
剩下没有较多的时间发展新的测试。 你尽快地将会进行你的全部接受测试剩余的时间在特别之上。 你也在贝它时期期间将会进行兼容性，表现，互通性和装置测试一次。
当你做每个虫应该有一个联合的测试情形的特别测试。当疯狂的在特别测试期间被发现,新的测试情形应该被产生以便他们容易地可能被再生而且当我们到达零的虫建造时期的时候复原测试可能被运行。
零缺点建立时期
零的缺点是什么建立时期? 这是没有新的严重缺点被发现的安定一个时期。 产品更预先非常稳定现在而且几乎装船运送。
自动化活动在零的缺点期间建立时期
进行复原测试。 复原尝试方法再一次跑过你的固定缺点和他们仍然被修理的证实。 为尝试的复原计划早在这时期和绿色的主人期间将会节省许多时间时期。
格林主人
绿色的主人时期是什么? 格林主人有时被称为金的主人或最后的候选人。 在它被装船运送之前 , 产品经过一个最后的清单。 (送到制造业)
自动化活动在绿色的主人时期期间
在流动的一般接受之后测试, 奔跑复原测试。 你再一次应该跑过你的固定缺点一次查证他们仍然被修理。 为尝试的复原计划早在这时期期间将会节省许多时间。
了解 PLC 将会帮助你选择你的自动化工具
也许 PLC 的这检讨是你的 ' 落伍的' 。 在我的经验方面, 通常检讨程序帮助集中计画。 既然软件测试已经在数年以来进化，而且许多公司由于该如何实现它已经奋斗，我们总是能使用我们能有的所有帮忙和忠告。 有数据的一些好来源将会帮助。 看一看软件测试位在网址 http:// www.stlabs.com 的实验室网站。 在 http:// www.stlabs.com/LABNOTE.HTM 有詹姆士巴哈文件的一些有关的文章。大概尝试中心的软件向小姐视觉的测试被以齿轮连起, 但是时间表原则积极参与的对任何的月台，被包括在内的 Macintosh 是相同的。
PLC 的这检讨表现理想发展和时间表进入做软件生意的这个相同的方法之内两者都买的情形。它也将会需要在要每时期保证的管理部份上的一些承诺被支援并且一般承认的。一次所有的那些情况, 每个人要把重心集中在确定那没有短切的积极参与的需要在 PLC 之上在同意人的周围被拿。 有希望地这讨论对你在自动化你的软件测试方面已经是有帮助的。
字汇
字或片语
定义。
特别测试
目标导向的通过产品。 有时或证明一个产品将会如何举止的观念是不对的。
阿尔发测试
密码是完全的 PLC 的测试时期的部份而且产品已经达成安定的一个程度。 产品完全可测试。 ( 决定被时间表) 所有的功能性已经被实现，而且时间表已经完成测试计划/情形的落实。 理想地, 这当发展的时候产品准备好被装船运送。
自动化了测试
个别的测试创造产生不需要直接的考试人干涉就能进行。
贝它测试
尝试计划的整合被完成的 PLC 的测试时期的部份,尝试报导目标的深度碰到；理想地，时间表产品准备好装船运送。 产品为外部的测试够稳定的。 ( 决定被时间表)
黑盒子测试
在测试下面的软件被当做一个黑盒子的测试。 你不能检视它。 测试不考虑而提供输入而且回应给输出软件如何工作。
边界测试
把重心集中在边界或界限软件被测试的情况测试。 ( 一些测试是强调测试).
宽度测试
点阵式以个别的方式通常包括所有的产品成份和功能的测试。这些通常是被自动化测试的第一可得的在功能的规格已经被完成而且测试计划已经被起草之后。
呼吸测试
通常一件好事物在做之后吃大蒜和在进入民众之内外出之前。 或如果你是酒醉驾驶 , 你可能必须参加一个呼气检查。
烦扰
和它为什么发生的理解一种现象。
密码完成
功能性在全部中被编码的 PLC 的时期; 虫固定是所有那被留下。 所有的功能在功能的规格中发现已经被实现。
密码结冰
当发展已经完成全新的功能密码的时候。 这是发展是在 "虫固定"阶段中的时候。
编码时期
发展正在编码产品遇见功能的/建筑的规格 PLC 的时期。 时间表在这时期期间发展测试工具而且测试情形。
兼容性测试
为和软件被测试的其他软件或硬件的兼容性检查的测试。
观念时期
一个主意为一种新的产品被发展的 PLC 的时期而且产品的一种初步的定义被建立。 研究计划应该在适当的位置被放，而且竞争的一项起始分析应该被完成。 这时期的主要目标要为进一步的研究决定产品生存能力而且获得资金筹备。
报导分析
动作的表演 (也就是,图形用户接口和 C 密码水平) 已经被接触和已经不。
数据确认
数据的确认保证它仍然是正确的。
除错
寻找而且除去发生故障软件的元素或错误。
定义时期
设计逐步运行。
属国
这是产品的一个成份依赖外面团体的时候。 产品的递送或那个到达一个某里程碑被影响。
深度测试
包含整合测试，测试的真正世界测试， combinatorial 测试，互通性和兼容性。
设计时期
产品的功能被写下的 PLC 的时期。 特征和需求在这时期内被定义。 每个部门在这时期期间为产品发展他们的部门计划和资源需求。
点释放
对一种产品的主要更新。
扮演重要角色
一个没有人承认的虫。
集中
兴趣或活动的中心。 在软件中，焦点提及插入点是活跃的荧屏区域。
功能的
PLC 的时期定义组件。
规格
他们的落实需求和方式 , 和暴露的美国石油协会。 每个功能在这里被叙述。 这包括每个功能的预期结果。
GM
格林主人。
格林主人 (GM)
证明阶段开始的 PLC 的时期。 所有的疯狂,退回对抗产品,一定经过。 每个建造是释放候选人。 ( 决定被发展)
图形用户接口
图解式的使用者接口。
线内
在海运 (STM) 后的时期 PLC 哪里疯狂的为中间的释放被修理。 产品的维护向上包括清洁疯狂的那在 STM 之后被发现。 在线内被产生向这些问题发表演说。
整合测试
在次要系统水平包括群体的功能尝试的深度。
互通性测试
查证在软件和硬件之间的 operability 的测试。
负荷测试
当它正在它的极限工作的时候 , 负荷测试学习计画的行为。 负荷测试的类型是体积测试 , 压迫力测试 , 和储藏测试。
局限
这一个术语提到到明确地发动软件设计来作特定的地方。
维护释放
起来线内。
韵律学
测量的一个标准。 软件韵律学是统计学描述一个计画的结构或内容。 一公制的应该是一个真正的客观测量有点如此的如数字疯狂的每一密码的线。
里程碑
产品生活定义特别的目标周期的事件。
性能测验
测试措施它拿多么的长做一个功能。
现象
没有理解的一个缺点。
PLC
产品生活周期 - 软件产品生活循环。
前阿尔发
前建造 1;产品定义时期。 (功能的规格被产生可能仍然是在程序中).
产品生活周期
阶段一种产品穿过。
(PLC)
从怀孕到完成。 产品发展的时期包括:定义时期 , 功能的/建筑的规格时期,编码时期,密码完成时期，阿尔发,贝它,零虫建造时期,绿色的主人时期， STM 和维护/线内时期。
提议时期
产品一定与一个被优先的特征目录和系统和兼容性需求一起定义的 PLC 的时期。
时间表计划
定义测试的活动同高的一个在句集被给的一般测试计划联成一组过产品生活周期的阶段。
真正的世界测试
尝试产生反映产品将会如何被用于真正的世界环境的尝试整合.
复原测试
再测试疯狂的在是确认为修理的系统中, 通常从阿尔发开始在之上。
资源
人们，软件，硬件，工具等。 那有独特的质量和才能以为一个目的可能被利用。
冒
有点可以可能地有助于到没有到达一个里程碑。
STM
对制造业的船。
储藏测试
测试如何记忆和空间在居留的记忆中或在磁盘片上也被计画用。
压迫力测试
测试计画的回应到达高点活动情况。
省略中间部份了测试
一个与其他的测试和谐地工作的测试。 时间安排是以致于两个测试工作一起, 但是仍然独立地。
测试情形
进入一个个别的测试之内的每个功能的区域崩溃。 这些用手可能被自动化或者完成了。
测试时期
整个的产品被测试的 PLC 的时期, 两者都在内部和外部地。 阿尔发和贝它测试在这时期期间发生。
测试计划
一种特性尝试的崩溃在一种功能的区域基础上接近的计划。
测试随员
一系列测试情形。
可用性
有意的目标使用者能完成他们的有意目标的程度。
体积测试
测试一个计画能处理的最大工作。
白色的盒子测试
它是用测试区域从一个黑盒子水平不能够被到达。 (有时称玻璃盒子为测试).
零虫建立
时期的产品已经以口吻稳定的 PLC 疯狂的发现和固定的。 发展正在固定疯狂的像他们被发现一样的快速, 网造成零疯狂的以每日的方式。 这通常被决定何时在一些建造之后已经经过。 这是初步行动在绿色的主人前的阶段。
叁考书目和叁考
1. 苹果计算机, 公司设定在和上面流动的虚拟使用者,1993.
2. 阿诺，汤姆斯 R.2. 和视觉的测试 4.0 的软件测试. IDG 书全世界的,公司,1996.
3. 巴哈 , 詹姆士,"测试- 自动化蛇油". 窗口技术日记， 1996 年十月， p。 40.
4. Cem Kaner ，杰克 Falk 和 Nung Quoc Nguyen。 测试计算机软件，秒版。 纽约:货车 Nostrand Rienhold,1993.
5. McCarthy ，吉姆。 软件发展的动力学。 Redmond,WA:微软杂志报纸,1995.
6. 葡萄园，杰里米。 "和虚拟的使用者的软件测试". MacTech 杂志 12:9(1996 年九月) ， pp。 16-20.
7. 软件测试实验室网站 http:// www.stlabs.com/

大卫凯利是负责测试一些为 MacOS 的网络网络包括诺顿管理人的产品 Symantec 公司的软件质量保证经理。 他自从 1984 以后已经与 Macintosh 合作并且是建造社论之一董事会 MacTutor 杂志的成员。 ( 现在 MacTech 杂志) 你能在 [email protected] 联络他。 -------------------- 我喜欢蓝色，
爱幻想，
有点忧郁，
有点小孩子气，
我的偶像刘德华！！！

Volume Number: 13
Issue Number: 10
Column Tag: Quality Control
Software Test Automation and the Product Life Cycle
by Dave Kelly, Symantec Corporation
Implementing software test in the product..

是要翻译吗？？？？？？？？我可以帮你！！！！！ -------------------- 我喜欢蓝色，
爱幻想，
有点忧郁，
有点小孩子气，
我的偶像刘德华！！！