Software testing is a process used to help identify the correctness, completeness and quality of developed computer software. With that in mind, testing can never completely establish the correctness of computer software. Only the process of formal verification can prove that there are no defects. Since the software testing proofs or proof engines themselves are typically complex systems constructed by fallible humans, we aren’t entitled to be entirely confident with formal methods of software testing
There are many approaches to software testing, but effective testing of complex products is essentially a process of investigation, not merely a matter of creating and following rote procedure. One definition of software testing is “the process of questioning a product in order to evaluate it,” where the “questions” are things the software tester tries to do with the product, and the product answers with its behavior in reaction to the probing of the software tester. Hopefully this will help your search into software testing
Overview
Testing can never completely establish the correctness of computer software. Instead, it furnishes a criticism or comparison that compares the state and behavior of the product against oracles principles or mechanisms by which someone might recognize a problem. These oracles may include (but are not limited to) specifications, comparable products, past versions of the same product, inferences about intended or expected purpose, user or customer expectations, relevant standards, applicable laws, or other criteria
Over its existence, computer software has continued to grow in complexity and size. Every software product has a target audience. For example, the audience for video game software is completely different from banking software. Therefore, when an organization develops or otherwise invests in a software product, it presumably must assess whether the software product will be acceptable to its end users, its target audience, its purchasers, and other stakeholders. Software testing is the process of attempting to make this assessment
A study conducted by NIST in 2002 reports that software bugs cost the U.S. economy $59.5 billion annually. More than a third of this cost could be avoided if better software testing was performed
History
The separation of debugging from testing was initially introduced by Glen ford J. Myers in 1979. Although his attention was on breakage testing (“a successful test is one that finds a bug” it illustrated the desire of the software engineering community to separate fundamental development activities, such as debugging, from that of verification. Dr. Dave Gelperin and Dr. William C. Hetzel classified in 1988 the phases and goals in software testing in the following stages:
- Until 1956 – Debugging oriented
- 1957-1978 – Demonstration oriented
- 1979-1982 – Destruction oriented
- 1983-1987 – Evaluation oriented
- 1988-2000 – Prevention oriented
Scope
A primary purpose for testing is to detect software failures so that defects may be uncovered and corrected. This is a non-trivial pursuit. Testing cannot establish that a product functions properly under all conditions but can only establish that it does not function properly under specific conditions. The scope of software testing often includes examination of code as well as execution of that code in various environments and conditions as well as examining the quality aspects of code: does it do what it is supposed to do and do what it needs to do. In the current culture of software development, a testing organization may be separate from the development team. There are various roles for testing team members. Information derived from software testing may be used to correct the process by which software is developed
Defects and Failures
Not all software defects are caused by coding errors. One common source of expensive defects is caused by requirements gaps, e.g., unrecognized requirements that result in errors of omission by the program designer. A common source of requirements gaps is non-functional requirements such as testability, scalability, maintainability, usability, performance, and security.
Software faults occur through the following process. A programmer makes an error (mistake), which results in a defect (fault, bug) in the software source code. If this defect is executed, in certain situations the system will produce wrong results, causing a failure. Not all defects will necessarily result in failures. For example, defects in dead code will never result in failures. A defect can turn into a failure when the environment is changed. Examples of these changes in environment include the software being run on a new hardware platform, alterations in source data or interacting with different software. A single defect may result in a wide range of failure symptoms
Compatibility
A frequent cause of software failure is compatibility with another application or new operating system (or, increasingly web browser version). In the case of lack of backward compatibility this can occur because the programmers have only considered coding the programs for, or testing the software, on the latest operating system they have access to or else, in isolation (no other conflicting applications running at the same time) or under ‘ideal’ conditions (‘unlimited’ memory; ’superfast’ processor; latest operating system incorporating all updates, etc). In effect, everything is running “as intended” but only when executing at the same time on the same machine with the particular combination of software and/or hardware. These are some of the hardest failures to predict, detect and test for and many are therefore discovered only after release into the larger world with its largely unknown mix of applications, software and hardware. It is likely that an experienced programmer will have had exposure to these factors through “co-evolution” with several older systems and be much more aware of potential future compatibility problems and therefore tend to use tried and tested functions or instructions rather than always the latest available which may not be fully compatible with earlier mixtures of software/hardware. This could be considered a prevention oriented strategy that fits well with the latest testing phase suggested by Dr. Dave Gelperin and Dr. William C. Hetzel
Input Combinations and Preconditions
A problem with software testing is that testing under all combinations of inputs and preconditions (initial state) is not feasible, even with a simple product. This means that the number of defects in a software product can be very large and defects that occur infrequently are difficult to find in testing. More significantly, non-functional dimensions of quality (how it is supposed to be versus what it is supposed to do) — for example, usability, scalability, performance, compatibility, reliability — can be highly subjective; something that constitutes sufficient value to one person may be intolerable to another
Static and Dynamic Testing
There are many approaches to software testing. Reviews, walkthroughs or inspections are considered as static testing, whereas actually executing programmed code with a given set of test cases is referred to as dynamic testing. The former can be, and unfortunately in practice often is, omitted, whereas the latter takes place when programs begin to be used for the first time – which is normally considered the beginning of the testing stage. This may actually begin before the program is 100% complete in order to test particular sections of code (modules or discrete functions). For example, Spreadsheet programs are, by their very nature, tested to a large extent “on the fly” during the build process as the result of some calculation or text manipulation is shown interactively immediately after each formula is entered.
Software verification and validation
Software testing is used in association with verification and validation
Verification: Have we built the software right (i.e., does it match the specification?) Its process based
Validation: Have we built the right software (i.e., is this what the customer wants?) Its product based
Software Testing Team
Software testing can be done by software testers. Until the 1950s the term “software tester” was used generally, but later it was also seen as a separate profession. Regarding the periods and the different goals in software testing there have been established different roles: test lead/manager, test designer, tester, test automater/automation developer, and test administrator.
Software Quality Assurance (SQA)
Though controversial, software testing may be viewed as an important part of the software quality assurance (SQA) process. In SQA, software process specialists and auditors take a broader view on software and its development. They examine and change the software engineering process itself to reduce the amount of faults that end up in defect rate. What constitutes an acceptable defect rate depends on the nature of the software. An arcade video game designed to simulate flying an airplane would presumably have a much higher tolerance for defects than mission critical software such as that used to control the functions of an airliner. Although there are close links with SQA, testing departments often exist independently, and there may be no SQA function in some companies.
Software Testing is a task intended to detect defects in software by contrasting a computer program’s expected results with its actual results for a given set of inputs. By contrast, QA is the implementation of policies and procedures intended to prevent defects from occurring in the first place.
Testing System
