Types of Metrics used in Testing

What are different types of metrics used in testing ?

1. User Participation : used to find the involvement of the tester
= Participation test time Vs Total test time

2. Path testing = Number of path tested / total number of paths

3. Acceptance criteria tested = Acceptance criteria verfied Vs total Acceptance criteria
This meets identifies the number of user that were evaluated during the testing process

4. Test cost : used to find resources consumed in the testing
= test cost Vs total system cost
This meets identifies the amount of resources used in testing process

5. Cost to locate defect = test cost / number of defects located in the testing
This metrics shows the costto locate a defect

6. Detected production defect = number of defects detected in production / Application systen size

7. Test automation = Cost of manual test effort / Total test cost

8. Schedule variance = (Actual time taken - Planned time) / Planned time * 100

9. Effort variance = (Actual effort - Planned Effort)/Planned effort * 100

10. Test case efficiency = (Total STRs - STRs not mapped)/Total STRs * 100

11. Test case coverage = (Total Test cases - STRs that cannot be mapped to test cases)/ Total Test Cases * 100

Test Matrices Sample

Test Matrices Sample

___________________________________

Question: I need information about Metrics, which is used to find faults. it is something related with measurement. I need to know how it is used in Quality Assurance. Answer: You can measure the arrival and departure times of developers, if you have them clock in, but that won't tell you much since not all work is done in the office (and it doesn't mean that they're working when they're in the office). This is, however, still a metric. The same holds for a true "quality metric". The most familiar one is defects per thousand lines of (uncommented) code. But this metric assumes that: 1) you count the lines of code 2) the complexity of the code isn't an issue 3) the programmers aren't playing games (like using continuance characters so that what could have been written in one line isn't done in five lines) 4) all defects are uncovered in the code in a single pass 5) each defect discovered is all others 6) defects are uncovered in a linear manner between revisions or builds The fact is that first you need to know what your goal is. Then you need to discover or create a metric that will help you achieve that goal. Then you need to implement it and be prepared to adjust it. You can't use measurements (metrics) to find faults, at least not in software, so that's not a reasonable goal. You can use metrics to help determine if most of the defects have been discovered already. You can use them to tell you how much longer it will take to uncover a reasonable amount of defects. For either of these metrics you will need to know how previous projects of similar size and complexity (using similar languages, etc.) were done in order to get a reasonable comparison. Posted by Walter Gorlitz Test Case: File Open # Test Description Test Cases/ Samples Pass/ Fail No. of Bugs Bug# Comments N/A Setup for [Product Name] setup - - - 1.1 Test that file types supported by the program can be opened 1.1 P/F # # 1.2 Verify all the different ways to open file (mouse, keyboard and accelerated keys) 1.2 P/F # # 1.3 Verify files can be open from the local drives as well as network 1.3 P/F

VSS ( Visual Source Safe)

*->VSS is Visual Source Safe, a popular version control tool developed by Microsoft.

It is used in Configuration Management.

VSS is effective tool if more than one programmer is involved and the project is divided into modules. Even the small modification / change made to the source code
will be available on each version of source code and is saved individually; it can be retrieved any time.

*àVSS is Visual Source Safe, it is a storage place where clients can see the updated documents or files in a project/s. and to enter into that vss user must have a userid and password, with that he/she can view the files and copy into our systems, then update the files into vss, then every one can able to view the files / documents

*à Its basically a version control tool suitable for small & medium size projects
Its lightweight product when compared to others like Clearcase, perforce

*àAccording to my knowledge Visual SourceSafe, simply put, is a source control system. Source control is highly desirable when developing software, web sites, or anywhere you may want to compare two versions of a file or return to a previous version.Microsoft Visual SourceSafe is a file-level version control system that permits many types of organizations to work on several project versions at the same time. This capability is particularly beneficial in a software development environment, where it is used in maintaining parallel code versions. However, the product can also be used to maintain files for any other type of team.

Visual SourceSafe supports cross-platform development by allowing collaborative editing and sharing of data. It is designed to handle the tracking and portability
issues involved in maintaining one source control base, for example, a software code base, across multiple operating systems. For developers, Visual SourceSafe accommodates reusable or object-oriented code. It makes it easier for you to track the applications that use particular code modules.

At a minimum, Visual SourceSafe does the following:
· Helps protect your team from accidental file loss.
· Allows back-tracking to earlier versions of a file.
· Supports branching, sharing, merging, and management of file releases.
· Tracks versions of entire projects.
· Tracks modular code (one file that is reused, or shared, by multiple projects).

Compatibility
The current release of Visual SourceSafe is fully compatible with database versions 6.0 and earlier.
Version Control and File Sharing
Visual SourceSafe allows the quick and efficient sharing of files among projects. The organization of files into projects makes team coordination intuitive. When you
add a file to Visual SourceSafe, the file is stored on the database and made available to other users. Changes that have been made to the file are saved so that any
user can recover an old version at any time. Members of your team can see the latest version of a file, make changes to local file copies, and save new versions in the

database. When a set of files is ready to deliver, Visual SourceSafe makes it easy to share and obtain different versions of the selected set of files.

Extensibility
Using the Visual SourceSafe automation interfaces, you can write extensions based on Visual SourceSafe as needed for your environment. Extensions are usually
provided in the form of stand-alone applications written to the automation interfaces. You can also extend Visual SourceSafe functionality by writing an add-in or

plug-in that is compatible with the integrated development environment (IDE) of the third-party program that will run the software package.

Parallel Development
Visual SourceSafe supports parallel development and cross-platform development techniques. Such support allows individual team members to complete different

parts and versions of a project at the same time, instead of being stalled while waiting for each to other to finish certain tasks. Two- and three-way file merging

operations are supported, and Visual SourceSafe includes a number of mechanisms for resolving merge conflicts. File merge operations enable independent work

without the need to synchronize changes with those made by other individuals.
In support of parallel operations, Visual SourceSafe also includes a label promotion feature to advance files as needed to different versions of a project. It also supports the use of share, pin, and branch operations for parallel development on a project over an extended period of time.

Developer Support
More and more, developers are accessing Visual SourceSafe functions from their development environments within third-party programs. Visual SourceSafe can be
easily integrated with Visual Studio and other development tools, such as Microsoft Access. Visual SourceSafe supports a developer environment in many ways by
fallowing:
· Setting of folder policies to enable group development scenarios.
· Bug fixes
· Easy transition to a new release of an existing project
· Batch/nightly builds
· Automation of source code control events
· Access to automation interfaces
· Source control over slow connections
· Configuration of new projects for isolated Web development
· Addition of a new Web developer to an existing team Web project
· Tracking of programming modules to allow reusable or object-oriented code

Database Maintenance
Visual SourceSafe provides a number of powerful database maintenance tools to keep your databases operating efficiently and securely. It supports archival and
restoration through easy-to-use wizards, as well as several command line-based maintenance utilities.


Managing Your Team Using Visual SourceSafe
Visual SourceSafe was designed to facilitate and promote coordination in teams with members who often need to work on the same files at the same time. This
section describes how you can maximize the team coordination potential of Visual SourceSafe. It contains topics that introduce team concepts and provide

step-by-step instructions for performing team-related tasks, such as setting up a shared database and establishing shadow folders.
Most of the tasks described in this section are intended for the team manager, who should be designated as a database administrator. However, at the manager's
discretion, the tasks can be performed by other team members who have appropriate Windows and Visual SourceSafe access permissions.

Configuration Management Tools

Configuration Management Tools

There are three main tools of Configuration Management:

* Wincvs

* VSS (Visual Safe Source)

* CVS

* SVN (Sub Version)

Check In/Check Out in CM

Understanding how Check In/Check Out works

Dreamweaver knows whether a file has been Checked In or Out by using .LCK files which it writes both locally and on the server. An .LCK file is a simple text file
containing only the user's Check Out Name (and e-mail address, if provided).

Check Out
When a file is checked out from the remote site, an extra file is written to the local site folder. The extra file has the same name as the original file, with an added
.LCK extension. The file is written in the same folder as the checked-out file. For example, checking out index.htm will generate a file by the name of index.htm.LCK
in the same folder.

Dreamweaver will also write an .LCK file in the remote folder for each file that is checked out.
These .LCK files are not visible within the Dreamweaver Site window, but can be seen outside of the Dreamweaver interface or by using a different FTP application.

Check In
When a file is checked in to the remote site, the LCK for that file is deleted on the remote site. Also, the local LCK is deleted from the computer of the user who
checked the file in.When a file is checked in to the remote site, the local version of that same file will be made read-only in the local site

Sample Configuration Management

Sample Configuration Management

Configuration Management is a complex Technical Management model that sits in the realm of Information Management. It focuses on establishing and maintaining
consistency of a product’s performance, functional and physical attributes with its requirements, design and operational information throughout its life.
For information assurance, CM can be defined as the management of security features and assurances through control of changes made to hardware, software,
firmware, documentation, test, test fixtures, and test documentation throughout the life cycle of an information system.

Configuration Management Activities

The view on configuration management in this book is process oriented. Therefore, the definition includes activity areas, which can be described in terms of process
descriptions.

The activity areas described in detail in the following paragraphs are identification, storage, change control, and status reporting.

Configuration management has many interactions with other development and support processes.

Configuration Management.

Configuration Management.

Configuration management (CM) is the detailed recording and updating of information that describes an enterprise's computer systems and networks, including all
hardware and software components. Such information typically includes the versions and updates that have been applied to installed software packages and the locations and network addresses of hardware devices. Special configuration management software is available.
When a system needs a hardware or software upgrade, a computer technician can accesses the configuration management program and database to see what is currently installed. The technician can then make a more informed decision about the upgrade needed.

An advantage of a configuration management application is that the entire collection of systems can be reviewed to make sure any changes made to one system do not adversely affect any of the other systems

Configuration management is also used in software development, where it is called Unified Configuration Management (UCM). Using UCM, developers can keep
track of the source code, documentation, problems, changes requested, and changes made.

Software CM is a discipline for controlling the evolution of software systems.

1. Identification: an identification scheme is needed to reflect the structure of the product. This involves identifying the structure and kinds of components, making

them unique and accessible in some form by giving each component a name, a version identification, and a configuration identification.


2. Control: controlling the release of a product and changes to it throughout the lifecycle by having controls in place that ensure consistent software via the creation

of a baseline product.


3. Status Accounting: recording and reporting the status of components and change requests, and gathering vital statistics about components in the product.


4. Audit and review: validating the completeness of a product and maintaining consistency among the components by ensuring that components are in an
appropriate state throughout the entire project life cycle and that the product is a well-defined collection of components.

The definition includes terminology such as configuration item, baseline, release and version, etc. At a high level, most designers of CM systems incorporate functionality of varying degrees to support these aspects. But at the implementation level, from the user's viewpoint, most CM systems have different functionality.

Among the many reasons for these differences are: disparate hardware platforms, operating system, and programming languages. But an interesting reason is due to
the different kinds of users of a CM system. This stems from the role the user plays in the organization. In particular, a manager, a software engineer developing an
application, an application customer, and an environment builder tend to see CM differently. As a result, they may want differing (albeit complementary) functionality
from their CM system. For example, to a manager the term "CM" conjures up the image of a Configuration Control Board. To a software engineer, the image of baselines arise. To a customer, versions of the application arise. And to the environment builder, mechanisms such as libraries and data compression arise. All these images obviously result in different requirements for a CM system and hence possibly different functionality.

Sample SRS (Software Requirement Specification)

Test Metrics

Test Metrics

What are different types of metrics used in testing ?

1. User Participation : used to find the involvement of the tester
= Participation test time Vs Total test time

2. Path testing = Number of path tested / total number of paths

3. Acceptance criteria tested = Acceptance criteria verified Vs total Acceptance criteria
This meets identifies the number of user that were evaluated during the testing process

4. Test cost : used to find resources consumed in the testing
= test cost Vs total system cost
This meets identifies the amount of resources used in testing process

5. Cost to locate defect = test cost / number of defects located in the testing
This metrics shows the cost to locate a defect

6. Detected production defect = number of defects detected in production / Application system size

7. Test automation = Cost of manual test effort / Total test cost

8. Schedule variance = (Actual time taken - Planned time) / Planned time * 100

9. Effort variance = (Actual effort - Planned Effort)/Planned effort * 100

10. Test case efficiency = (Total STRs - STRs not mapped)/Total STRs * 100

11. Test case coverage = (Total Test cases - STRs that cannot be mapped to test cases)/ Total Test Cases * 100
---------------------------------------------------------------------------------------------------------------------------------------------------------
What are two types of Metrics ?
Metrics are classified into 2 types :-

1. Process metrics : A metric used to measure the characteristic of the methods, techniques & tools employed in developing implementing & maintaining the software system.

2. Product metrics : A metric used to measure the characteristic of the documentation & code characteristic.
----------------------------------------------------------------------------------------------------------------------------------------------------------
What is Product metrics ?
Product metrics is a metric used to measure the characteristic of the documentation & code characteristic.
-------------------------------------------------------------------------------------------------------------------------------------------------------
What is Process metrics ?
Process metrics is a metric used to measure the characteristic of the methods, techniques & tools employed in developing implementing & maintaining the software system.
---------------------------------------------------------------------------------------------------------------------------------------------------------

Sample Traceability Matrix

Sample Traceability Matrix

A traceability matrix is a report from the requirements database or repository. What information the report contains depends on your need. Information requirements
determine the associated information that you store with the requirements. Requirements management tools capture associated information or provide the capability
to add it.

The examples show forward and backward tracing between user and system requirements. User requirement identifiers begin with "U" and system requirements with
"S." Tracing S12 to its source makes it clear this requirement is erroneous: it must be eliminated, rewritten, or the traceability corrected.




For requirements tracing and resulting reports to work, the requirements must be of good quality. Requirements of poor quality transfer work to subsequent phases
of the SDLC, increasing cost and schedule and creating disputes with the customer.

A variety of reports are necessary to manage requirements. Reporting needs should be determined at the start of the effort and documented in the requirements
management plan.

Useful Traceability Matrices

Useful Traceability Matrices

Various traceability matrices may be utilized throughout the system life cycle. Useful ones include:

* Functional specification to requirements document: It shows that each requirement (obtained from a preliminary requirements statement provided by the
customer or produced in the Concept Definition stage) has been covered in an appropriate section of the functional specification.

* Top level configuration item to functional specification: For example, a top level configuration item, Workstation, may be one of the configuration items that
satisfies the function Input Order Information. On the matrix, each configuration item would be written down the left hand column and each function would be written
across the top.

* Low level configuration item to top level configuration item: For example, the top level configuration item, Workstation, may contain the low level configuration
items Monitor, CPU, keyboard, and network interface card.

* Design specification to functional specification verifies that each function has been covered in the design.

* System test plan to functional specification ensures you have identified a test case or test scenario for each process and each requirement in the functional
specification.

Although the construction and maintenance of traceability matrices may be time-consuming, they are a quick reference during verification and validation tasks.

Types Of Traceability Matrix

Types Of Traceability Matrix

1)Baseline Traceability Matrix

Description

A table that documents the requirements of the system for use in subsequent stages to confirm that all requirements have been met.

Size and Format

Document each requirement to be traced. The requirement may be mapped to such things as a hardware component, an application unit, or a section of a design
specification.

2)Building a Traceability Matrix

->Use a Traceability Matrix to:

* verify and validate system specifications

* ensure that all final deliverable documents are included in the system specification, such as process models and data models

* improve the quality of a system by identifying requirements that are not addressed by configuration items during design and code reviews and by identifying extra

configuration items that are not required. Examples of configuration items are software modules and hardware devices

* provide input to change requests and future project plans when missing requirements are identified

* provide a guide for system and acceptance test plans of what needs to be tested.

Need for Relating Requirements to a Deliverable

Taking the time to cross-reference each requirement to a deliverable ensures that a deliverable is consistent with the system requirements. A requirement that cannot
be mapped to a deliverable is an indication that something is missing from the deliverable. Likewise, a deliverable that cannot be traced back to a requirement may mean the system is delivering more than required.

Use a Traceability Matrix to Match Requirements to a Deliverable

There are many ways to relate requirements to the deliverables for each stage of the system life cycle.
One method is to:

* create a two-dimensional table

* allow one row for each requirements specification paragraph (identified by paragraph number from the requirements document)

* allow one column per identified configuration item (such as software module or hardware device)

* put a check mark at the intersection of row and column if the configuration item satisfies the stated requirement

Traceability Matrix

Traceability Matrix

In a software development process, a traceability matrix is a table that correlates any two baselined documents that require a many to many relationship to determine
the completeness of the relationship. It is often used with high-level requirements (sometimes known as marketing requirements) and detailed requirements of the

software product to the matching parts of high-level design, detailed design, test plan, and test cases.

Description

A table that traces the requirements to the system deliverable component for that stage that responds to the requirement.
Size and Format

For each requirement, identify the component in the current stage that responds to the requirement. The requirement may be mapped to such items as a hardware
component, an application unit, or a section of a design specification.

Traceability matrices can be established using a variety of tools including requirements management software, databases, spreadsheets, or even with tables or hyperlinks in a word processor.

A traceability matrix is created by associating requirements with the work products that satisfy them. Tests are associated with the requirements on which they are
based and the product tested to meet the requirement.


Above is a simple traceability matrix structure. There can be more things included in a traceability matrix than shown. In traceability, the relationship of driver to
Satisfier can be one-to-one, one-to-many, many-to-one, or many-to-many.

Traceability requires unique identifiers for each requirement and product. Numbers for products are established in a configuration management (CM) plan.

Traceability ensures completeness, that all lower level requirements come from higher level requirements, and that all higher level requirements are allocated to lower
level requirements. Traceability is also used to manage change and provides the basis for test planning.