Trong chuyên mục này, các bạn chỉ được post các thông tin liên quan đến ISEB
- Posts: 4900
- Joined: Tue 10 Aug, 2010 10:11 am
- Location: HCM
Principles of Testing
In this module you will get an overview of the most fundamental principles of testing.
Firstly, we point out that although you will come across different terminology throughout your career, professional testers are all pretty much agreed on these basic ideas that we describe in this module.
Secondly, we take a look at the need for proper testing look at the cost of getting it wrong and we show why exhaustive testing is neither possible not practical. What are errors and how do they get into the software?
Thirdly, we describe a fundamental test process, base on industry standards, and underline importance of planning) tests and determining expected results in advance of test execution.
We conclude with a look at re-testing and regression testing; why it is so important to go that extra mile and run a suite of tests again, just when you thought it was safe to hit the release date.
After completing this module you will:
- Understand basic testing terminology.
- Understand why testing is necessary.
- Be able to define error, fault and failure.
1.3 TESTING TERMINOLOGY
There is no generally accepted set of testing definitions used by the worldwide testing community. The British Standard for Software Component Testing published in August 1998 provides a new source of testing definitions. When involved in a testing project it is important to ensure everyone understands terminology adopted; you may find different terminology is used in your organization.
The British Standard for Software Component Testing is known as BS___________
A useful glossary of terms used in software testing is called BS___________
Although it is a British Standard published by the BSI (British Standards Institute), the Specialist Interest Group in Software Testing (SIGIST) developed it over a period of several years.
ANSWERS: 7925-PART2, 7925-PART1
1.4 WHY IS TESTING NECESSARY?
This section explains why testing is necessary and closely at the cost and consequences of errors in computer software.
1.4.1 Definitions Errors, Faults and Failures
An error is a human action that produces an incorrect result. A fault is a manifestation of an error in software. Faults are also known colloquially as defaults or bugs. A fault, if encountered, may cause a fault, which is a deviation of the software from its existing delivery or service.
Reliability is the probability that software will not cause the failure of a system for a specified time under specified conditions. Measures of reliability include MTBF (mean time between failure), MTTF (mean time to failure) as well as service level agreements and other mechanisms.
1.4.3 Errors and how they occur
Why do we make errors that cause faults in computer software leading to potential failure of our systems? Well, firstly we are all prone to making simple human errors. This is an unavoidable fact of life. However, this is compounded by the fact that we all operate under real world pressures such as tight deadlines, budget restrictions, conflicting priorities and so on.
1.4.4 Cost of errors
The cost of an error can vary from nothing at all to large amounts of money and even loss of life. The aborted Mercury mission was obviously very costly but surely this is just an isolated example. Or is it? There are hundreds of stories about failures of computer systems that have been attributed to errors in the software. A few examples are shown below:
1.4.5 what happened on October 26th & 27th 1992?
The London Ambulance Service (LAS) covers an area of just over 600 square miles and is the largest ambulance service in the world. It covers a resident population of some 6.8 million, but its daytime population is larger, especially in central London. Since 1974 South West Thames Regional Health Authority has managed it.
1.4.6 London Ambulance Service
In summary the problems were:
1.4.7 Exhaustive testing why not test everything?
It is now widely accepted that you cannot test everything. Exhausted testers you will find, but exhaustive testing you will not. Complete testing is neither theoretically, nor practically possible. Consider a 10-character string that has 280 possible input streams and corresponding outputs. If you executed one test per microsecond it would take approx. 4 times the age of the Universe to test this completely. For a survey of the methodology and limitations of formal proofs of program correctness see [Manna 78]
1.4.8 Testing and risk
How much testing would you be willing to perform if the risk of failure were negligible? Alternatively, how much testing would you be willing to perform if a single defect could cost you your life's savings, or, even more significantly, your life? [Hetzel 88].
1.4.9 Testing and quality
Testing identifies faults whose removal increases the software quality by increasing the software's potential reliability. Testing is the measurement of software quality. We measure how closely we have achieved quality by testing the relevant factors such as correctness, reliability, usability, maintainability, reusability, testability etc.
1.4.10 Testing and legal, contractual, regulatory or mandatory requirements
Other factors that may determine the testing performed may be legal, contractual requirements, normally defined in industry specific standards or based on agreed best practice (or more realistically non-negligent practice).
1.4.11 How much testing is enough?
It is difficult to determine how much testing is enough. Testing is always a matter of judging risks against cost of extra testing effort. Planning test effort thoroughly before you begin, and setting completion criteria will go some way towards ensuring the right amount of testing is attempted. Assigning priorities to tests will ensure that the most important tests have been done should you run out of time.
1.5 FUNDAMENTAL TEST PROCESS
Testing must be planned. This is one of Bill Hetzel's 6 testing principles [Hetzel 88 p25] and he says we are all agreed on this one. However, he points out that the problem is that most of us do not discipline ourselves to act upon it. Good testing requires thinking out an overall approach, designing tests and establishing expected results' for each of the test cases we choose.
1.5.2 Test process stages
See BS7925-2 for diagram of test process. Test planning involves producing a document that describes an overall approach and test objectives noting any assumptions you have made and stating any exceptions to your overall test strategy for your project. Test planning can be applied at all levels. Completion or exit criteria must be specified so that you know when testing (at any stage) is complete. Often a coverage target is set and used as test completion criteria.
1.5.3 Successful tests detect faults
As the objective of a test should be to detect faults, a successful test is one that does detect a fault. This is counter-intuitive, because faults delay progress; a successful test is one that may cause delay. The successful test reveals a fault which, if found later, may be many more times costly to correct so in the long run, is a good thing.
1.5.4 Meaning of completion or exit criteria
Completion or exit criteria are used to determine when testing (at any stage) is complete. These criteria may be defined in terms of cost, time, faults found or coverage criteria.
1.5.5 Coverage criteria
Coverage criteria are defined in terms of items that are exercised by test suites, such as branches, user requirements, and most frequently used transactions etc.
1.6 THE PSYCHOLOGY OF TESTING
The purpose of this section is to explore differences in perspective between tester and developer (buyer & builder) and explain some of the difficulties management and staff face when working together developing and testing computer software.
1.6.2 Different mindsets
We have already discussed that none of the primary purposes of testing is to find faults in software i.e., it can be perceived as a destructive process. The development process on the other hand is a naturally creative one and experience shows that staff that work in development have different mindsets to that of testers.
- Are perceived as very creative - they write code without which there would be no system! .
- Are often highly valued within an organization.
- Are perceived as destructive - only happy when they are finding faults!
- Are often not valued within the organization.
1.6.3 Communication b/w developer and tester
It is vitally important that tester can explain and report fault to developer in professional manner to ensure fault gets fixed. Tester must not antagonize developer. Tact and diplomacy are essential, even if you've been up all night trying to test the wretched software.
1.6.4 How not to approach
Tester: "Hey Fred. Here's a fault report AR123. Look at this code. Who wrote this? Was it you? Why, you couldn't program your way out of a paper bag. We really want this fixed by 5 o'clock or else."
We were unable to print Fred's reply because of the language! Needless to say Fred did not fix the fault as requested.
1.6.6 Why can't we test our own work?
This seems to be a human problem in general not specifically related to software development. We find it difficult to spot errors in our own work products. Some of the reasons for this are:
1. 7 RE-TESTING AND REGRESSION TESTING
We find and report a fault in LOG 3, which is duly fixed by the developer and included in the latest release which we now have available for testing. What should we do now?
1.8 EXPECTED RESULTS
The specification of expected results in advance of test execution is perhaps one of the most fundamental principles of testing computer software. If this step is omitted then human subconscious desire for tests to pass will be overwhelming and tester may perhaps interpret a plausible, yet erroneous result, as correct outcome. .
SOME TESTING TERMINOLOGY
- Faults - a mistake in the code that causes the software to not behave as expected (causes)
- Failures - the act of a product not behaving as expected - the manifestation of a fault (symptoms)
- Validation - establishing the correspondence between the software and its specification - "are we building the right product?"
Why Write Programs?
Concept: If you want your computer to do exactly what you want it to do, you must write a program.
Definition: Code is another name for program.
Review: No single program pleases everyone. When a company sells a program, it must be general enough to please most purchasers. Some people need programs to behave in a specific manner in order to fulfill a specific need. They must resort to writing their own programs. Luckily, Visual Basic takes a lot of the pain out of writing programs.
A Brief History of Textual Programming
Concept: Computers cannot understand just any language. You must learn a language that your computer knows before you can write programs for your computer.
Definition: An application is yet another name for program.
Many people use computers all day long for word processing, database storage, and spreadsheet analysis, without realizing what is going on behind the scenes. You must always keep in mind that computers cannot think. Your computer does not know how to be a word processor. If you want your computer to do word processing, you must supply detailed instructions in the form of a program. Only by following the detailed instructions of a word processor program that you load can your computer perform word processing.
Definition: A bug is a program error.
These are the typical steps that most programmers go through when writing programs. First, you have an idea for a program. Next, you use a program-development system, such as Visual Basic, to write the program. Errors, or bugs, often appear in programs because of the details needed for even the simplest of programs. Therefore, you must test the program thoroughly and fix the errors. Fixing errors is called debugging. Once all the errors are out of the program, you have a finished application.
LOGGING INTO VISUAL BASIC
VBS (adds topics), enter Open, enter
When in the form use - Label 1. Caption = "Hello Class of 2000"
To delete an object from the form Select Edit menu.
ERRORS IN PROGRAMMING
WHY DOES SOFTWARE HAVE BUGS?
1. 9 Prioritization of Tests
Since we have discussed already that there is never enough time to do all of the testing that you would like an obvious approach is to prioritize the tests This will then mean that whenever you stop testing you will have done the best testing possible in the time available.
The test manager should identify the criteria to hi used when prioritizing tests. These will include but are not limited to:
- Visibility of failure.
Keep the test priority with the test at all times. This_ will prevent you from developing and designing lo_ priority tests that never get run. Use a scheme of high, medium, low or a numeric scheme or 1,2, 3, 4 but try not to have more than about 4 categories.
You should now be familiar with some of the fundamental principles of testing and you will recognize much of the terminology. The horror stories that you will have heard are not just sensationalist to motivate the course; they are true stories and there are (unfortunately) many more stories about the costs of not testing systems properly.
In particular you can now:
- Recognize and use basic testing terminology.
- Understand why testing is necessary.
- Define error, fault and failure.
Do thông tin nhiều quá nên mình chỉ để ít thôi, các bạn vui lòng download bản đầy đủ ở file đính kèm nhé