Testing Fundamentals

What is a Test Plan?

A test plan can be defined as a document describing the scope, approach, resources, and schedule of intended testing activities. It identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning.

In software testing, a test plan gives detailed testing information regarding an upcoming testing effort, including

1. Scope of testing
2. Schedule
3. Test Deliverables
4. Release Criteria
5. Risks and Contingencies

It is also be described as a detail of how the testing will proceed, who will do the testing, what will be tested, in how much time the test will take place, and to what quality level the test will be performed.

What is Test cases??

A test case is a set of conditions or variables under which a tester will determine whether a system under test satisfies requirements or works correctly.

The process of developing test cases can also help find problems in the requirements or design of an application.

TEST CASE TEMPLATE

A test case can have the following elements. Note, however, that normally a test management tool is used by companies and the format is determined by the tool used.

Test Suite ID	The ID of the test suite to which this test case belongs.
Test Case ID	The ID of the test case.
Test Case Summary	The summary / objective of the test case.
Related Requirement	The ID of the requirement this test case relates/traces to.
Prerequisites	Any prerequisites or preconditions that must be fulfilled prior to executing the test.
Test Procedure	Step-by-step procedure to execute the test.
Test Data	The test data, or links to the test data, that are to be used while conducting the test.
Expected Result	The expected result of the test.
Actual Result	The actual result of the test; to be filled after executing the test.
Status	Pass or Fail. Other statuses can be ‘Not Executed’ if testing is not performed and ‘Blocked’ if testing is blocked.
Remarks	Any comments on the test case or test execution.
Created By	The name of the author of the test case.
Date of Creation	The date of creation of the test case.
Executed By	The name of the person who executed the test.
Date of Execution	The date of execution of the test.
Test Environment	The environment (Hardware/Software/Network) in which the test was executed.

WRITING GOOD TEST CASES

• As far as possible, write test cases in such a way that you test only one thing at a time. Do not overlap or complicate test cases. Attempt to make your test cases ‘atomic’.
• Ensure that all positive scenarios and negative scenarios are covered.
• Language:
  • Write in simple and easy to understand language.
  • Use active voice: Do this, do that.
  • Use exact and consistent names (of forms, fields, etc).
• Characteristics of a good test case:
• Accurate: Exacts the purpose.
• Economical: No unnecessary steps or words.
• Traceable: Capable of being traced to requirements.
• Repeatable: Can be used to perform the test over and over.
• Reusable: Can be reused if necessary.

What is Regression Testing??

Regression testing is a type of software testing that intends to ensure that changes (enhancements or defect fixes) to the software have not adversely affected it.

ELABORATION

The likelihood of any code change impacting functionalities that are not directly associated with the code is always there and it is essential that regression testing is conducted to make sure that fixing one thing has not broken another thing.

During regression testing, new test cases are not created but previously created test cases are re-executed.

LEVELS APPLICABLE TO

Regression testing can be performed during any level of testing (Unit, Integration, System, or Acceptance) but it is mostly relevant during System Testing.

EXTENT

In an ideal case, a full regression test is desirable but oftentimes there are time/resource constraints. In such cases, it is essential to do an impact analysis of the changes to identify areas of the software that have the highest probability of being affected by the change and that have the highest impact to users in case of malfunction and focus testing around those areas.

Difference between "Retesting" and "Regression Testing"?

RETESTING : 
When a bug is found, it needs to be communicated and assigned to developers that can fix it. After the problem is resolved, fixes should be re-tested with the same test data. 

Additionally,determinations should be made regarding requirements, software, hardware, safety impact, etc. to make sure the application works as expected.

Regression Testing 
Testing the application once the bug is fixed with new test data to check the fixes didn't create other problems elsewhere.


What is Test Strategy??

A Test Strategy document is a high level document and normally developed by project manager. This document defines “Testing Approach” to achieve testing objectives. The Test Strategy is normally derived from the Business Requirement Specification document.

The Test Stategy document is a static document meaning that it is not updated too often. It sets the standards for testing processes and activities and other documents such as the Test Plan draws its contents from those standards set in the Test Strategy Document.

Some companies include the “Test Approach” or “Strategy” inside the Test Plan, which is fine and it is usually the case for small projects. However, for larger projects, there is one Test Strategy document and different number of Test Plans for each phase or level of testing.

What are Test Strategy, Test Approach and Test method ?

What Is a Test Strategy?

A test strategy is a planning document that provides the overall direction for the software testing needs of a project. Developing a test strategy is about setting direction and resolving high-level testing questions. The value of the test strategy isn’t in the wording, the writing, or the format of the strategy; the value is in planning an approach for testing.

Test strategies can range from informal to very formal. In its simplest form, the test strategy is exactly that—a strategy. It’s a roadmap for what testing will be done, and details the way in 

which the testing will be accomplished:

How? Answers to this question might identify the types of testing needed, such as manual or automated testing. 

Where? These answers detail the actual test environment, including specific server names, and may include a diagram of all the physical or logical components. 

Who? The test strategy must specify the testing resources and other resources needed to accomplish the testing. 

When? A good test strategy outlines the time of the first internal build for testing and likely includes a rough schedule for the remainder of the project. 

These high-level questions need answers early in a project. The test strategy also might address related testing topics such as purchasing test tools or the defect-reporting process.

Test Approach

The test approach describes the types of tests performed and the sequence of tests as executed in the test lab.

Test Types

The types of tests that were considered for testing the Small IT Solution include:

These tests are limited to specific configurations made within the solution.

Build Verification Tests

The following goals were adopted for build verification testing:

Clarity: The steps should clearly describe the options that must be selected to enable the service. 

Completeness: All configuration options required to configure the service are described. 

Presentation: The process for configuring the service is linearly presented and represents the process that must be followed when initially deploying the solution. 

Consistency: The configuration options presented in the solution should always use the tools that are best suited for configuring the service. Consistency for each service is summarized as follows:

Functional Tests

The following goals were adopted for functional testing:

· Completeness: The functional test cases ensure that the technical goals of the solution are achieved.

Management Tests

The following goals were adopted for security tests:

Technical Writing Tests

The goals of the technical writing tests are as follows:

Security Tests

The following goals were adopted for security tests:

• Principle of least privilege: The configuration recommendations should follow the principle of assigning least privilege. 

• Availability: Verify that configuration recommendations have not been made that reduce the availability of the systems within the solution, due to the activity of some person or system. This includes accidental or malicious activity. 

• Integrity: Verify that recommendations have not been made that may result in a loss of business data integrity. 

• Confidentiality: Ensure that recommendations have not been made that may result in the disclosure of information to unauthorized users. 

• Spoofing: Ensure that recommendations have not been made that may allow a user or system to impersonate an authorized user or system. Ensure that recommendations do not allow for credential theft. 

• Repudiation: Ensure that there is specific and unchangeable proof of critical actions on a server. 

Test Sequence

The test involved the following phases:

1. Build phase: Each test cycle began with this phase. In this phase, the required hardware was installed and connected, network cables were connected, and other hardware configuration was completed. 

2. Build verification testing phase: The test team performed solution configuration using the solution documentation and build verification test cases. This ensured that the systems were built and configured as documented. Build tests quickly exposed human errors made in the guidance as well as errors in the completeness of the deployment guidance that resulted in services not functioning properly. 

3. Functional testing phase: Once build testing was complete, the test team focused on verifying key functions of the products and solution. 

4. Management testing phase: Management testing verified that the requirements of the remote management strategy were met within the solution configuration and design. 

5. Technical writing testing phase: These tests ensured that the communication style and documentation links were correct and consistent. 

6. Security testing phase: The security testing phase was the last phase in each test cycle. This phase ensured that all generated security test cases were executed on the complete end-state environment. 

Test method

A test method is a definitive procedure that produces a test result.

The test result can be qualititive (yes/no), categorical, or quantititive (a measured value). It can be a personal observation or the output of a precision instrument.

Usually the test result is the dependent variable, the measured response based on the particular conditions of the test or the level of the independent variable. Some tests, however, involve changing the independent variable to determine the level at which a certain response occurs: in this case, the test result is the independent variable. 

Components of the Test Strategy document

1. Scope and Objectives
2. Business issues
3. Roles and responsibilities
4. Communication and status reporting
5. Test deliverability
6. Industry standards to follow
7. Test automation and tools
8. Testing measurements and metrices
9. Risks and mitigation
10. Defect reporting and tracking
11. Change and configuration management
12. Training plan.