03 TestCase Development

Introduction to Test Formality

For a newbie, its easy to assume that Testing is executing various section of code ,on a ad-hoc basis and verifying the results. But in real world, Testing is a very formal activity, and is documented in detail.

The degree of formality depends on the type of application under test , standards followed by your organization ,& maturity of development process.

 <div style="position:relative;height:0;padding-bottom:60.94%"><iframe src="https://www.youtube.com/embed/rFMq5bUz7bk?ecver=2" style="position:absolute;width:100%;height:100%;left:0" width="591" height="360" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe></div>

• For all hands - on,  we will be using the Flight Reservation Application ,which comes bundled with Automation Tool ,  QTP.
• Tutorials on this site for QTP & Loadrunner use Flight Reservation .Therefore we have selected flight reservation to reduce your learning curve while studying QTP & Loadrunner. Below, find Link to Introduction Video to Flight Reservation Application.

Resources

1) Refer this guide to download QTP. Flight Reservation comes bundled with thisapplication

2) Refer this tutorial on Flight Reservation for a beginner's introduction to the application.

What is Test Scenario?

What is a Test Scenario?

A Test Scenario is any functionality that can be tested. It is also called Test Condition or Test Possibility. As a tester, you may put yourself in the end user’s shoes and figure out the real-world scenarios and use cases of the Application Under Test.

What is Scenario Testing?

Scenario Testing is a variant of Software Testing where Scenarios are Used for Testing. Scenarios help in an Easier Way of Testing of the more complicated Systems

Let’s study this with the help of the video below -

Why create Test Scenarios?

Test Scenarios are created for following reasons,

• Creating Test Scenarios ensures complete Test Coverage
• Test Scenarios can be approved by various stakeholders like Business Analyst, Developers, Customers to ensure the Application Under Test is thoroughly tested. It ensures that the software is working for the most common use cases.
• They serve as a quick tool to determine the testing work effort and accordingly create a proposal for the client or organize the workforce.
• They help determine the most important end-to-end transactions or the real use of the software applications.
• For studying the end-to-end functioning of the program, Test Scenario is critical.

When not create Test Scenario?

Test Scenarios may not be created when

• The Application Under Test is complicated, unstable and there is a time crunch in the project.
• Projects that follow Agile Methodology like Scrum, Kanban may not create Test Scenarios.
• Test Scenario may not be created for a new bug fix or Regression Testing. In such cases, Test Scenarios must be already heavily documented in the previous test cycles. This is especially true for Maintenance projects.

How to create a Test Scenario

As a tester, you can follow these five steps to create Test Scenarios-

• Step 1: Read the Requirement Documents like BRS, SRS, FRS, of the System Under Test (SUT).  You could also refer uses cases, books, manual, etc. of the application to be tested.
• Step 2: For each requirement, figure out possible users actions and objectives. Determine the technical aspects of the requirement. Ascertain possible scenarios of system abuse and evaluate users with hacker's mindset.
• Step 3: After reading the Requirements Document and doing your due Analysis, list out different test scenarios that verify each feature of the software.
• Step 4: Once you have listed all possible Test Scenarios, a Traceability Matrix is created to verify that each & every requirement has a corresponding Test Scenario
• Step 5: The scenarios created are reviewed by your supervisor. Later, they are also reviewed by other Stakeholders in the project.

Tips to Create Test Scenarios

• Each Test Scenario should be tied to a minimum of one Requirement or User Story as per the Project Methodology.
• Before creating a Test Scenario that verifies multiple Requirements at once, ensure you have a Test Scenario that checks that requirement in isolation.
• Avoid creating overly complicated Test Scenarios spanning multiple Requirements.
• The number of scenarios may be large, and it is expensive to run them all. Based on customer priorities only run selected Test Scenarios

Example 1: Test Scenario for Flight Reservation

For the Flight Reservation Application, a few test scenarios would be

Test Scenario 1: Check the Login Functionality

Test Scenario 2: Check that a New Order can be created 

Test Scenario 3: Check that an existing Order can be opened 

Test Scenario 4: Check that a user, can FAX an order

Test Scenario 5: Check that the information displayed in the HELP section is correct 

Test Scenario 6: Check that the information displayed in About section, like version, programmer name, copy right information is correct

Apart from these six scenarios here is the list of all other scenarios

• Update Order
• Delete Order
• Check Reports
• Check Graphs and so on.

Next, we have already learned exhaustive testing is not possible. Suppose you have time only to execute 4 out of these 6 scenarios which two low priority scenarios of these six will you eliminate. Think, your time starts now

I am sure most of you would have guessed scenarios 5 & 6 since they are not the core functionality of the application. This is nothing but Test Prioritization.

Example 2:  Test Scenarios for a Banking Site

Test Scenario 1: Check the Login and Authentication Functionality

Test Scenario 2: Check Money Transfer can be done

Test Scenario 3: Check Account Statement can be viewed

Test Scenario 4: Check Fixed Deposit/Recurring Deposit can be created

And so on…

How to Write Test Cases: Sample Template with Examples

What is Test Case?

A Test Case is a set of actions executed to verify a particular feature or functionality of your software application.

This tutorial describes test case designing and the importance of its various components.

Now, consider the Test Scenario Check Login Functionality there many possible cases like

Test Case 1: Check results on entering valid User Id & Password

Test Case 2: Check results on entering Invalid User ID & Password

Test Case 3: Check response when User ID is Empty & Login Button is pressed, and many more

This is nothing but Test Case. Test scenarios are rather vague and cover a wide range of possibilities. Testing is all about being very specific.Hence, we need Test Cases

Video Transcript

• Now just Consider the test case, Check the response on entering valid Agent Name and password. It's obvious that this test case needs input values viz.Agent Name & Password
• This is nothing but Test Data. Identifying test data can be time-consuming and may sometimes require creating test data afresh. The reason it needs to be documented
• Before we proceed ahead consider a quote from a witty man who said "To ensure perfect aim, shoot first and call whatever you hit the target" But if you do not live by this philosophy, which I am sure most of you do not then your Test case must have an expected result.
• For our test case, expected result would be, Login should be successful
• If expected results are not documented, we may miss out on small differences in calculations in results that otherwise look OK.
• Consider this example, where you are calculating monthly pay for an employee which involves lots of calculations. The need for documenting expected results becomes obvious.
• Suppose the author of the test case has left the organization or is on a vacation or is sick and off duty or is very busy with other critical tasks and you are recently hired and have been asked to execute the test case.Since you are new, it would certainly help to have test steps documented which, in this case, would be Launch application, Enter Agent Name, Enter Password, Click OK
• You may wonder that for this simple test steps, documentation is not required
• But what is your test steps looked something like this? (pic in video) I think the need will become instantaneously obvious.
• That apart your test case -may have field like, Pre - Condition which specifies things that must in place before the test can run
• For our test case, a pre-condition would be Flight Reservation Application should be installed, which I am sure 50% of the people watching this tutorial do not have
• Also, your test case may also include Post - Conditions which specifies anything that applies after the test case completes.
• For our test case, a post - condition would be time & date of login is stored in the database
• During test case execution, you will document the results observed in the Actual Results column and may even attach some screen shots and based on the results give PASS & FAIL status.
• This entire table may be created in Word, Excel or any other Test management tool.That's all to Test Case Design

Format of Standard Test Cases

Below is format of a standard login Test case

While drafting a test case do include the following information

• The description of what requirement is being tested
• The explanation of how the system will be tested
• The test setup like: version of application under test, software, data files, operating system, hardware, security access, physical or logical date, time of day, prerequisites such as other tests and any other setup information pertinent to the requirements being tested
• Inputs and outputs or actions and expected results
• Any proofs or attachments
• Use active case language
• Test Case should not be more than 15 steps
• Automated test script is commented with inputs, purpose and expected results
• Setup offers alternative to pre-requisite tests
• With other tests, it should be incorrect business scenario order

Best Practice for writing good Test Case Example.

1. Test Cases need to be simple and transparent:

Create test cases that are as simple as possible. They must be clear and concise as the author of test case may not execute them.

Use assertive language like go to home page, enter data, click on this and so on. This makes the understanding the test steps easy and test execution faster.

2. Create Test Case with End User in Mind

Ultimate goal of any software project is to create test cases that meets customer requirements and is easy to use and operate. A tester must create test cases keeping in mind the end user perspective

3. Avoid test case repetition.

Do not repeat test cases. If a test case is needed for executing some other test case, call the test case by its test case id in the pre-condition column

4. Do not Assume

Do not assume functionality and features of your software application while preparing test case. Stick to the Specification Documents.

5. Ensure 100% Coverage

Make sure you write test cases to check all software requirements mentioned in the specification document. UseTraceability Matrix to ensure no functions/conditions is left untested.

6. Test Cases must be identifiable.

Name the test case id such that they are identified easily while tracking defects or identifying a software requirement at a later stage.

7. Implement Testing Techniques

It's not possible to check every possible condition in your software application. Testing techniques help you select a few test cases with the maximum possibility of finding a defect.

Boundary Value Analysis (BVA): As the name suggests it's the technique that defines the testing of boundaries for specified range of values.

Equivalence Partition (EP): This technique partitions the range into equal parts/groups that tend to have the same behavior.

State Transition Technique: This method is used when software behavior changes from one state to another following particular action.

Error Guessing Technique: This is guessing/anticipating the error that may arise while testing.This is not a formal method and takes advantages of a tester's experience with the application

8. Self cleaning

The test case you create must return the Test Environment to the pre-test state and should not render the test environment unusable. This is especially true for configuration testing.

9. Repeatable and self-standing

The test case should generate the same results every time no matter who tests it

10. Peer Review.

After creating test cases, get them reviewed by your colleagues. Your peers can uncover defects in your test case design, which you may easily miss.

Test Case Management Tools

Test management tools are the automation tools that help to manage and maintain the Test Cases. Main Features of a test case management tool are

1. For documenting Test Cases: With tools you can expedite Test Case creation with use of templates
2. Execute the Test Case and Record the results: Test Case can be executed through the tools and results obtained can be easily recorded.
3. Automate the Defect Tracking: Failed tests are automatically linked to the bug tracker , which in turn can be assigned to the developers and can be tracked by email notifications.
4. Traceability: Requirements, Test cases, Execution of Test cases are all interlinked through the tools, and each case can be traced to each other to check test coverage.
5. Protecting Test Cases: Test cases should be reusable and should be protected from being lost or corrupted due to poor version control. Test Case Management Tools offer features like

• Naming and numbering conventions
• Versioning
• Read only storage
• Controlled access
• Off-site backup

Popular Test Management tools are : Quality Center and JIRA

Resources

• Please note that the template used will vary from project to project. Read this Article to Learn Test Case Template with Explanation of Important Fields

Sample Test Case in Excel

What is Test Basis in Software Testing?

Test Basis is defined as the source for creation of test Cases. It can be the Application itself or the requirement documents like SRS (Software Requirement Specification), BRS (Business Requirement Specification), etc.

This tutorial explains "Test-Basis". with the help of a case study

Consider a scenario, where the client sends a request to add a functionality to Flight Reservation to allow sending an order via email.

He also specifies the GUI fields and buttons he wants.

Even though, the application is yet to be developed, try and develop a few test cases for this requirement. 

A few test cases among the many you could have thought of are listed below

• Check the response when valid Email ID is entered, and send is pressed
• Check the response when invalid  Email ID are entered and send is pressed
• Check the response when Email ID is empty and send is pressed

You may have also realized that to create test cases you need to look at something to base your test. This is nothing but Test Basis. This test basis could be the actual Application Under Test (AUT), or maybe even by experience but most of the times, like, in this case, would be based on documents.

In fact, this is what happens during the different phases of V- Model.

.

Where test plans are created using the corresponding documents, and once the code is ready, it is ready for testing.

How to Create Requirements Traceability Matrix (RTM)

What is Traceability Matrix?(TM)

A Traceability Matrix is a document that co-relates any two-baseline documents that require a many-to-many relationship to check the completeness of the relationship.

It is used to track the requirements and to check the current project requirements are met.

What is RTM (Requirement Traceability Matrix)?

Requirement Traceability Matrix or RTM captures all requirements proposed by the client or software development team and their traceability in a single document delivered at the conclusion of the life-cycle.

In other words, it is a document that maps and traces user requirement with test cases. The main purpose of Requirement Traceability Matrix is to see that all test cases are covered so that no functionality should miss while doing Software testing.

Requirement Traceability Matrix – Parameters include

• Requirement ID
• Risks
• Requirement Type and Description
• Trace to design specification
• Unit test cases
• Integration test cases
• System test cases
• User acceptance test cases
• Trace to test script

Types of Traceability Test Matrix

• Forward traceability: This matrix is used to check whether the project progresses in the desired direction and for the right product. It makes sure that each requirement is applied to the product and that each requirement is tested thoroughly. It maps requirements to test cases.


• Backward or reverse traceability: It is used to ensure whether the current product remains on the right track. The purpose behind this type of traceability is to verify that we are not expanding the scope of the project by adding code, design elements, test or other work that is not specified in the requirements. It maps test cases to requirements.


• Bi-directional traceability ( Forward+Backward): This traceability matrix ensures that all requirements are covered by test cases. It analyzes the impact of a change in requirements affected by the Defect in a work product and vice versa.  

How to create Requirement Traceability Matrix

Let's understand the concept of Requirement Traceability Matrix through a Guru99 banking project.

On the basis of Business Requirement Document (BRD) and Technical Requirement Document (TRD), testers start writing test cases.

Let suppose, the following table is our Business Requirement Document or BRD for Guru99 banking project.

Here the scenario is that the customer should be able to login to Guru99 banking website with the correct password and user#id while manager should be able to login to the website through customer login page.

While the below table is our Technical Requirement Document (TRD).

Note: QA teams do not document the BRD and TRD. Also some companies use Function Requirement Documents (FRD) which are similar to Technical Requirement Document but the process of creating Traceability Matrix remains the same.

Let's Go Ahead and create RTM Testing

Step 1: Our Test Case is

"Verify Login, when correct ID and Password is entered, it should login successfully"

Step 2: Identify the Technical Requirement that this test case is verifying. For our test case, the technical requirement is T94 is being verified.

Step 3: Note this Technical Requirement (T94) in the Test Case.

Step 4: Identify the Business Requirement for which this TR (Technical Requirement-T94) is defined

Step 5: Note the BR (Business Requirement) in Test Case

Step 6: Do above for all Test Cases. Later Extract the First 3 Columns from your Test Suite. RTM in testing is Ready!

Advantage of Requirement Traceability Matrix

• It confirms 100% test coverage 
• It highlights any requirements missing or document inconsistencies 
• It shows the overall defects or execution status with a focus on business requirements 
• It helps in analyzing or estimating the impact on the QA team's work with respect to revisiting or re-working on the test cases 

Let's learn RTM with a example in the Video

Tips and Tricks to Generate Test Data

Everybody knows that testing is a process that produces and consumes large amounts of data. Data used in testing describes the initial conditions for a test and represents the medium through which the tester influences the software. It is a crucial part of mostFunctional Testing. But what actually is the test data? Why is it used? Maybe you would wonder ‘Designing Test cases is challenging enough, then why bother about something as trivial as Test Data’ The purpose of this tutorial is to introduce you to Test Data, its importance and give practical tips and tricks to generate test data quickly. So, Let's Begin!

What is Test Data? Why is it Important?

Test data is actually the input given to a software program. It represents data that affects or is affected by the execution of the specific module. Some data may be used for positive testing, typically to verify that a given set of input to a given function produces an expected result. Other data may be used for negative testing to test the ability of the program to handle unusual, extreme, exceptional, or unexpected input. Poorly designed testing data may not test all possible test scenarios which will hamper the quality of the software. 

What is Test Data Generation? Why test data should be created before test execution?

Depending on your testing environment you may need to CREATE Test Data (Most of the times) or atleast identify a suitable test data for your test cases (is the test data is already created).

Typically test data is created in-sync with the test case it is intended to be used for.

Test Data can be Generated -

• Manually
• Mass copy of data from production to testing environment
• Mass copy of test data from legacy client systems
• Automated Test Data Generation Tools

Typically sample data should be generated before you begin test execution because it is difficult to perform test data management. Since in many testing environments creating test data takes many pre-steps or test environment configurations which is very time consuming. Also If test data generation is done while you are in test execution phase you may exceed your testing deadline.

Below are described several testing types together with some suggestions regarding their testing data needs.

Test Data for White Box Testing

In White Box Testing, test data is derived from direct examination of the code to be tested. Test data may be selected by taking into account the following things:

• It is desirable to cover as many branches as possible; testing data can be generated such that all branches in the program source code are tested at least once
• Path testing: all paths in the program source code are tested at least once - test data can be designed to cover as many cases as possible
• Negative Api Testing:
  • Testing data may contain invalid parameter types used to call different methods
  • Testing data may consist in invalid combinations of arguments which are used to call the program's methods

Test Data for Performance Testing

Performance Testing is the type of testing which is performed in order to determine how fast system responds under a particular workload. The goal of this type of testing is not to find bugs, but to eliminate bottlenecks.  An important aspect of Performance Testing is that the set of sample data used must be very close to 'real' or 'live' data which is used on production. The following question arises: ‘Ok, it’s good to test with real data, but how do I obtain this data?’ The answer is pretty straightforward: from the people who know the best – the customers. They may be able to provide some data they already have or, if they don’t have an existing set of data, they may help you by giving feedback regarding how the real-world data might look like. In case you are in a maintenance testing project you could copy data from the production environment into the testing bed. It is a good practice to anonymize (scramble) sensitive customer data like Social Security Number, Credit Card Numbers, Bank Details etc. while the copy is made.

Test Data for Security Testing

Security Testing is the process that determines if an information system protects data from malicious intent. The set of data that need to be designed in order to fully test a software security must cover the following topics:

• Confidentiality: All the information provided by clients is held in the strictest confidence and is not shared with any outside parties. As a short example, if an application uses SSL, you can design a set of test data which verifies that the encryption is done correctly.
• Integrity: Determine that the information provided by the system is correct. To design suitable test data you can start by taking an in depth look at the design, code, databases and file structures.
• Authentication: Represents the process of establishing the identity of a user. Testing data can be designed as different combination of usernames and passwords and its purpose is to check that only the authorized people are able to access the software system.
• Authorization: Tells what are the rights of a specific user. Testing data may contain different combination of users, roles and operations in order to check only users with sufficient privileges are able to perform a particular operation.

Test Data for Black Box Testing

In Black Box Testing the code is not visible to the tester . Your functional test cases can have test data meeting following criteria -

• No data: Check system response when no data is submitted
• Valid data: Check system response when Valid  test data is submitted
• Invalid data: Check system response when InValid  test data is submitted
• Illegal data format: Check system response when test data is in invalid format
• Boundary Condition Data set: Test data meeting boundary value conditions
• Equivalence Partition Data Set: Test data qualifying your equivalence partitions.
• Decision Table Data Set: Test data qualifying your decision table testing strategy
• State Transition Test Data Set: Test data meeting your state transition testing strategy
• Use Case Test Data: Test Data in-sync with your use cases.

Note: Depending on the software application to be tested, you may use some or all of the above test data creation

Automated Test Data Generation

In order to generate various sets of data, you can use a gamut of automated test data generation tools. Below are some examples of such tools:

Test Data Generator by GSApps can be used for creating intelligent data in almost any database or text file. It enables users to:

• Complete application testing by inflating a database with meaningful data
• Create industry-specific data that can be used for a demonstration
• Protect data privacy by creating a clone of the existing data and masking confidential values
• Accelerate the development cycle by simplifying testing and prototyping

Test Data generator by DTM, is a fully customizable utility that generates data, tables (views, procedures etc) for database testing (performance testing, QA testing, load testing or usability testing) purposes.
Datatect  is a SQL data generator by Banner Software, generates a variety of realistic test data in ASCII flat files or directly generates test data for RDBMS including Oracle, Sybase, SQL Server, and Informi.

In conclusion, well-designed testing data allows you to identify and correct serious flaws in functionality. Choice of test data selected must be reevaluated in every phase of a multi-phase product development cycle. So, always keep an eye on it.

Download Sample Test Case Template with Explanation of Important Fields

A good Test Case template maintains test artifact consistency for the test team and makes it easy for all stakeholders to understand the test cases. Writing test case in a standard format lessen the test effort and the error rate. Test cases format are more desirable in case if you are reviewing test case from experts.

The template chosen for your project depends on your test policy. Many organizations create test cases in Microsoft Excel while some in Microsoft Word. Some even use test management tools like HP ALM to document their test cases.

Irrespective of the test case documentation method chosen, any good test case template must have the following fields

Optionally you can have the following fields depending on the project requirements

• Link / Defect ID: Include the link for Defect or determine the defect number if test status is fail
• Keywords / Test Type: To determine tests based on test types this field can be used. Eg: Usability, functional, business rules, etc.
• Requirements: Requirements for which this test case is being written
• References / Attachments: It is useful for complicated test scenarios, give the actual path of the document or diagram
• Automation ( Yes/No): To track automation status when test cases are automated
• Custom Fields: Fields particular your project being tested due to client/project requirements.