Software Testing

Test Metrics

Rajeev Prabhakaran Nair — Wed, 17 Feb 2010 08:53:01 +0000

Test Metrics can be used to measure test coverage prior to software delivery. It provides a measure of the percentage of the software tested at any point during testing.
It is calculated as follows:
Functional Test Coverage = FE/FT
Where,
FE is the number of test requirements that are covered by test cases that were executed against the software
FT is the total number of test requirements

Software Release Metrics
The software is ready for release when:
1. It has been tested with a test suite that provides 100% functional coverage, 80% branch coverage, and 100% procedure coverage.
2. There are no level 1 or 2 severity defects.
3. The defect finding rate is less than 40 new defects per 1000 hours of testing
4. Stress testing, configuration testing, installation testing, Naïve user testing, usability testing, and sanity testing have been completed
Software Maturity Metric
Software Maturity Index is that which can be used to determine the readiness for release of a software system. This index is especially useful for assessing release readiness when changes, additions, or deletions are made to existing software systems. It also provides a historical index of the impact of changes. It is calculated as follows:
SMI = Mt – ( Fa + Fc + Fd)/Mt
Where
SMI is the Software Maturity Index value
Mt is the number of software functions/modules in the current release
Fc is the number of functions/modules that contain changes from the previous release
Fa is the number of functions/modules that contain additions to the previous release
Fd is the number of functions/modules that are deleted from the previous release
Reliability Metrics
Reliability is calculated as follows:
Reliability = 1 – Number of errors (actual or predicted)/Total number of lines of executable code
This reliability value is calculated for the number of errors during a specified time interval.
Three other metrics can be calculated during extended testing or after the system is in production. They are:

MTTFF (Mean Time to First Failure)
MTTFF = The number of time intervals the system is operable until its first failure (functional failure only).

MTBF (Mean Time Between Failures)
MTBF = Sum of the time intervals the system is operable

MTTR (Mean Time To Repair)
MTTR = sum of the time intervals required to repair the system
The number of repairs during the time period

Powerful Web Testing Tools

Rajeev Prabhakaran Nair — Sun, 21 Jun 2009 05:33:47 +0000

Web testing; Nowadays meant to be very crucial, so many Internet browsers and their supporters. Numerous tools available for web testing most of them require higher budget. Few free website testing tools are available online or as add-ones for example firebug,Yslow. These are not only used for web development also for web testing. We can fetch object details ,CSS, JavaScript and more using these kinds of add ones. It is easy to analyze all these information. Open source Testing tools like Selenium, Watir needs web object’s physical properties or it’s description to run successfully, we can use these tools as an object spy.Checkout some powerful web testing tools

1.Firebug
Firebug integrates with Firefox, it is a great invention for developers can find JavaScript errors , page loading time etc. Most of the developers used to fix Html/CSS, JavaScript errors simply in your finger tips while you browse. Firebug is one of the best applications around for debugging issues with front-end code and CSS. If there’s any image or style that’s out of line, checking it out with Firebug is the best response. It’s even possible to change styles within the extension to see how a website will actually appear in the browser. How this is useful in web testing. We can monitor java script errors, web site performance i.e. site loading time/object wise also Html or CSS validation. Inspect elements option in firebug is like GUI Spy.
Visit:- Firebug

2. YSlow
YSlow analyzes web pages and suggests ways to improve their performance based on a set of rules for high performance web pages. YSlow is a Firefox add-on integrated with the Firebug web development tool. YSlow grades web page based on one of three predefined ruleset or a user-defined ruleset. It offers suggestions for improving the page’s performance, summarizes the page’s components, displays statistics about the page, and provides tools for performance analysis, including Smush.it and JSLint.
Visit :- YSlow

3. Load Impact (Online testing)
This is used for Performance Testing (Load/Stress). It simulates users and access your website . It simulates large user load on web servers to determine whether or not they can handle the high traffic load. Free users can only attempt using 50 simulated users But this is very useful . The tool will check (ping) from different locations and results will display as a complete report with graphs
Visit :- LoadImpact

API (Application Programming Interface)

Rajeev Prabhakaran Nair — Mon, 04 May 2009 08:33:01 +0000

An API (Application Programming Interface) is a collection of software functions and procedures, called API calls, that can be executed by other software applications. Application developers code that links to existing APIs to make use of their functionality. This link is seamless and end-users of the application are generally unaware of using a separately developed API.

ANSI / IEEE Standards

Rajeev Prabhakaran Nair — Mon, 27 Apr 2009 09:25:19 +0000

ANSI – ‘American National Standards Institute’
IEEE Standards: Institute of Electrical and electronics Engineers (Founded in 1884) Have an entire set of standards devoted to software. Testers should be familiar with all the standards mentioned in IEEE.

610.12-1990 IEEE Standard Glossary of Software Engineering Terminology
730-1998 IEEE Standard for Software Quality Assurance plans
828-1998 IEEE Standard for Software Configuration Management
829-1998 IEEE Standard for Software Test Documentation
830-1998 IEEE Recommended Practice for Software Requirement Specifications.
1008-1987(R1993) IEEE Standard for Software Unit Testing
1012-1998 IEEE Standard for Software Verification and validation
1012a-1998 IEEE Standard for Software Verification and validation –Supplement to 1012-1998 Content
1016-1998- IEEE Recommended Practice for Software Design description
1028-1997 IEEE Standard for Software Reviews
1044-1993 IEEE Standard Classification for Software Anomalies
1045-1992 IEEE Standard for Software Productivity metrics
1058-1998 IEEE Standard for Software Project Management Plans
1058.1-1987 IEEE Standard for Software Management
1061-1998.1 IEEE Standard for Software Quality Metrics Methodology.

Capability Maturity Model (CMM)

Rajeev Prabhakaran Nair — Sat, 18 Apr 2009 10:08:07 +0000

Developed by the software community in 1986 with the leadership from the SEI.
Has become the factor standard for assessing and improving processes related to software development.
Has evolved into a process maturity framework Provides guidance for measuring software process maturity helps establish process improvement programs.

Maturity levels

Initial
Repeatable
Defined
Manageable
Optimizing

Level 1: Initial
Each maturity level decomposes into several key process areas that indicate the areas an organization should focus on to improve its software process.

Level 2 – Repeatable:Key practice areas

Requirements management
Software project planning
Software project tracking&oversight
Software subcontract management
Software quality assurance
Software configuration management

Level 3 – Defined: Key practice areas

Organization process focus
Organization process definition
Training program
Integrated software management
Software product engineering
Inter group coordination
Peer reviews

Level 4 – Manageable: Key practice areas

Organization process focus
Software Quality Management

Software Test Plan

Rajeev Prabhakaran Nair — Tue, 06 Jan 2009 11:19:36 +0000

A software project test plan is a document that describes the objectives, scope, approach, and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the why and how of product validation.
Test plan is a document through which we can describe objective, approach, scope and focus of software testing efforts. A Test Plan contains following particulars like:
1. Overview
2. Scope
3. Bounds
4. Settings
5. Definitions
6. Quality Risks
7. Schedule
8. Transitions
9. Entry Criteria
10. Stop Criteria
11. Exit Criteria
12. Test Configurations and Environments
13. Test Execution
14. Resources
15. Management of Bugs and Tests
16. Bug isolation and classification
17. Release Management
18. Test Cycles
19. Referenced Documents
20. Change History

Severity of Bug

Rajeev Prabhakaran Nair — Mon, 05 Jan 2009 05:19:07 +0000

Bug Severity indicates how serious the bug is and reflects its impact on the product and customers of the product. It can be critical, major, minor, cosmetic or suggestion.
Critical severity: The bug is of critical severity if it causes system crash, data loss or data corruption.
Major severity: The bug is of major severity if it causes operational errors, wrong results and loss of functionality.
Minor severity: The bug is of minor severity if it causes defect in user interface layout or spelling mistakes.

Priority of Bug

Rajeev Prabhakaran Nair — Mon, 05 Jan 2009 05:12:47 +0000

Priority indicates how important it is to fix the bug and when it should be fixed
Immediate Priority: The bug is of immediate priority if it blocks further testing and is very visible
At the earliest Priority: The bug must be fixed at the earliest before the product is released
Normal Priority: The bug should be fixed if time permits
Later Priority: The bug may be fixed, but can be released as it is

Rules for Displaying Alert or Error Messages in Web Applications

Rajeev Prabhakaran Nair — Sat, 20 Dec 2008 09:43:00 +0000

In all web applications and OS based application, validation is very important for that we are using different types of validation or alert or error messages , is that have any rules ? yes just check it ,

Should start with capital letter(In a standard format)
Error messages should be accessible for as many users as possible regardless of culture, age and impairment.
It should be easy to understand that an error has occurred.
It should be clear what the user has to do to correct the error.
It should be clear for the user where the error was found in the form.
It should be possible to be notified about errors before submitting the form, especially if it is a complex form that takes time to process on the server.
All errors should be displayed at the same time. No one wants to re-submit the form to find a new error.
Do not give any hint about any security event (For example Sign in of an application Incorrect error message -”Password is invalid” , Correct one- “Agent Name or Password is Invalid “)

Where to display these error messages ? Let’s see

Make sure the user can see that an error has occurred. Error messages need to stand out from the rest of the layout. They should be placed at the top of the page if that is the area displayed after the form has been submitted with errors.
Use an error icon if your layout makes it difficult to visually separate errors from the rest of the page
Be consistent when you present errors (the user will expect errors to be displayed at the same location and with the same style on all pages)
Update the page title to indicate that the user is on the same page but with errors (e.g. if the page title is “Registration” you could change it to “Registration – Error occurred”).

Error message text standard

Make sure the error message identifies the related field with the name as it is written in the label for that field
Do not use complicated words
Describe what the user should do to correct the error, especially if it could be difficult to understand
Make it clear if there were more than one error so that the user can correct all errors at once

Error Messages Views

Make sure all user agents can parse error details. Use a heading to identify the error area.
Present each error as an item in a list. This makes it easy for the user to understand what has to be corrected. Also, screen reader users will find it easier to keep errors apart
If you use software specific error id numbers do not hesitate to add them to the id-attribute of the li element. If you use automated functional tests (e.g. Rational Robot) it will be much easier for the test robot to parse and identify errors
Provide an access key for the error message section. This is valuable if your form can generate many errors and enables the user to move back and forth between the error list and the form
Make it possible to navigate from the error message to the related field. This makes it easier for users that navigate with the keyboard

Tips for developers
It is better to use some client side validations in forms (like in .net applications)
Example
<——————————————————————————->
4 Errors were found while registration ,Please Correct these errors and Submit again
#Your user name must be between 6 and 30 characters long.
#Email is invalid
#Password is too short (minimum is 8 characters)
#Terms of Service must be accepted
<——————————————————————————>

Mock Testing

Rajeev Prabhakaran Nair — Wed, 17 Dec 2008 09:04:46 +0000

Set up mock test before you begin your actual test. This is a way to measure the server’s stressed performance. As you progress with your stress testing, you can set up a measurement of metrics to determine the efficiency of the test.
After the initial test, you can determine the breaking point for the server. It may be a processor problem or even a memory problem. You need to be able to check your log to determine the average amount of time that it takes your processor to perform the test. Running graphics or even ASP pages can cause processor problems and a limitation every time you run your stress test
Memory tends to be a problem with the stress test. This may be due to a memory leak or lack of memory. You need to log and monitor the amount of disk capacity during the stress test. As mentioned earlier, the bandwidth can account for the slow down of the processing of the Web site speed. If the test hangs and there is a large waiting period, your processor usage is too low to handle the amount of stress on the system.
Simulate Resources
It is important to be able to run system in a high-stress format so that you can actually simulate the resources and understand how to handle a specific load. For example, a bank transaction processing system may be designed to process up to 150 transactions per second, whereas an operating system may be designed to handle up to 200 separate terminals. The different tests need to be designed to ensure that the system can process the expected load. This type of testing usually involves planning a series of tests where the load is gradually increased to reflect the expected usage pattern. The stress tests can steadily increase the load on the system beyond the maximum design load until the system fails.
This type of testing has a dual function of testing the system for failure and looking for a combination of events that occur when a load is placed on the server. Stress testing can then determine if overloading the system results in loss of data or user service to the customers The use of stress testing is particularly relevant to an e-commerce system with Web database.