• Author:Tom du Pré
  • Comments:0

The lost art of writing tests – Part II

In the first part of this post, I spoke about ISEB’s four E’s method of writing test cases. This is a good start, but I don’t think it’s good enough. Here’s my alternative.

Again, this advice is applicable to both manual and automated testing. Because we’re concerned with test analysis and test design, the method of execution is not important.

The Six S’s

  • Specific
  • Standalone
  • Short
  • Simple
  • Sentences
  • Statements

Specific

The test should target one particular area of the application.

As part of your analysis you will have broken down the application into its smallest units of functionality (functional decomposition). It is then desirable to have one test that covers each of the lowest level functionalality. The result is a clear 1 : 1: 1 mapping between a unit of functionality, a test case, and the bug you raise if/when the test fails.

Tests that are not specific are difficult to prioritise. If the test covers five things, are all five things of equal priority? Unlikely. Tests that are not specific also make it hard to target your testing around particular areas of concern, which is important when time is scarce, as it usually is.

Standalone

The test should be able to be taken in isolation and should still be run-able.

A test must not rely on other tests having been run first, nor must it rely on the tester having seen or read information in other test cases. We need each test to be self sufficient and contain all the information we need to run it so that we are able to pick and choose which tests we want to run.

Imagine a test that says:

When a user leaves the date of birth field blank and submits the form, an error message is shown.

Although the test maybe valid, in isolation it is useless. Taken out of the context of its neighbouring tests it means nothing. It may be that the test lives in a test suite that neatly shows that this it is concerned with the 3rd page in the booking process in a flight reservation system. Even though it may seem repetitive and boring, each test needs to contain all this sort of information. Ensure each test has its own pre-requisites, even if they are identical to the previous tests. You can use and re-use templates to get around this.

Short

All testers know that executing tests can become repetitive and tedious, and to a degree that is unavoidable. This is undesirable because a tester who is bored and feels as though he is going through the motions of executing test after test is unlikely to be observant, inquisitive, aggressive, analytical and all the other things he needs to be if he is going to spot defects. We can help matters by making our tests short. Even the most engaged tester is unlikely to be motivated by a test that has twenty or thirty steps and whose description is a mind bogglingly long sentence.So keep your tests short and punchy.

Also, if your tests are short, they are likely to be specific.

To keep your tests short, consider the following:

Use pre-requisites wisely. In tests where the description is long, the majority of the description is not the test but the pre-requisites. It’s OK to have a long list of pre-requisites if it means the actual test is short. In order to make it clear which are the pre-requisites, and which is the actual meat of the test, separate them. For example,

If a registered and logged on user who has a loyalty card selects an item from the catalogues, adds it to his basket and navigates to the checkout screen does not enter the loyalty card number on the shopping basket page an alert is shown.

This is not a short description. This test is not about the catalogue, nor adding goods to the basket. It’s actually about the alert on the loyalty card number field for customers who have a loyalty card. The length of the test obscures this. The following would be better.

Pre-requisites:
- Test data: Registered user who has signed up to Loyalty card scheme and is logged in.
- User has added good to basket
- User has navigated to the checkout
Description: When a user submits the Checkout page without entering a loyalty card number, an alert is shown.

All tests are concerned with testing cause and effect. Almost every test your write will be the following format.

When <actor1> does <action1>, <actor2> does <action2>.

The “actors” maybe a user or a system, and the actions may be things that people do, or things that systems do. It doesn’t matter. Almost every test case should be able to be simplified down to this structure. And this is a good way of making it short.

Simple

This is possibly the most important of the characteristics. If your test is simple, it is likely that it will also have most of the above characteristics.

This is a good point to re-iterate my earlier statement:

Write your tests assuming that someone else will have to execute them.

Consider who the audience for your tests is. We would hope that the person executing the test is a hugely skilled and knowledgeable tester who has years of experience on the system. If we write our test with this expert in mind, a less experienced person may struggle to understand the test.

Assume your tester is the following person:

  • He has had some basic training in the application.
  • It’s not necessary to assume absolute ignorance of the application. It’s fair to assume that the tester will have had at least some training. (If he hasn’t had any training, he shouldn’t have been asked to run the tests!)
  • He has some experience of software testing.
  • It’s fair to assume that the tester has their ISEB or ISTQB qualification (or local equivalent) and understands basic testing terminology.
  • He has not had the opportunity to extensively review requirements documents, or interview stakeholders.
  • He will need to run the tests without continual support from an expert.
  • It is possible that the test will be executed by an offshore resource while you are asleep. Do you want to be woken up every ten minutes with a question? Or would you rather write your tests in such a way that he doesn’t need to wake your up with lots of questions?
  • English is not his first language.
  • The opposite is also true. If you are writing tests in a language that is not your mother tongue, keeping the test simple will ensure it is understood.

Sentences

Test must be in good English sentences, with good spelling, grammar and punctuation. (Assuming English is the lingua franca in your team.)

Above all, a test is a means of communication between the person who wrote it and the person who must execute it. Good spelling, grammar and punctuation are not just a common courtesy to your audience, they are also essential in ensuring that your tests communicate your intentions accurately.

Tips for writing test in good English:

  • Describe the flow in logical sequential order.
    An entry is written into the data base confirming a payment has been made if the user clicks ‘submit’ and they are logged in and they have added an item to their basket.

    This is unclear because the test is written backwards. It starts with end result then lists the actions in reverse order

    If a logged in user adds an item to their shopping basket and clicks ‘submit’ the confirmation of their payment is written into the database.

    This is exactly the same test as the previous one, but it is written in a logical order.

  • Write in “plain English” http://www.plainenglish.co.uk/howto.pdf
  • Don’t be afraid to use bulleted lists rather than long paragraphs of text. Like this one.They’re ugly, but they’re easy to read.
  • Avoid writing “This test is to ensure that…” or “To verify that…” at the beginning of each test. It doesn’t add any meaning and it doesn’t make the test any more effective.

Statements

One of the interesting things about writing test cases is that you could get 100 testers to each write one test case for the same simple piece of functionality  and they would come up with 100 different test cases. All testers have their own own style of writing and speaking English and this manifests in their test cases. There is nothing wrong with people having a style, so long as it doesn’t interfere with the meaning of the test case.

A good style to use is to write the test as a statement that can be proved or disproved. A statement will be concise, and will include what the expected answer is. You can think of your statement as a hypothesis, which you are setting out to prove or disprove. If you are able to disprove the hypothesis of your test case the test has failed. Look out for future posts that take this idea further, and discuss the scientific method of testing.

Tweet about this on TwitterShare on FacebookDigg thisEmail this to someoneShare on Google+Share on LinkedInPin on PinterestShare on RedditShare on StumbleUponShare on Tumblr

No tags