Test Plans are useless. Test Planning is not.

This article triggered me – in a good way – to describe what I think are the biggest reasons for having a test plan. Obviously, the mileage you get from your test plan will vary, depending on your company, environment, customer, regulatory environment, amongst other things.

As we know, simply having a document doesn’t, in and of itself, add much value. Simply having lots of test cases doesn’t add value. Running them does. Actually, just having documents is waste.

In these agile days where communication is rightly valued over documentation, the poor old test plan gets a bad rep, and is often neglected, or produced on reflex and never read.

There are at least two good reasons for producing a test plan, which I will introduce, naturally, through the medium of appropriate quotes.

Planning is a verb

“Plans are useless. Planning is indispensable”. – Dwight D. Eisenhower.

The value we get from test plans is not from the document, but from the planning process. In producing / reviewing the document, we are effectively testing the project. By asking questions and eliciting responses from the relevant people, we are seeking out where information is missing, or confused, or inconsistent.

Testing is pretty much the first role / stage in a project where questions like “What operating systems do we support?” and “Do we support HTTPS?” have to be answered before the work can be completed. It is therefore vital that we ensure that we ask these questions as early as we can so that a) answers can be found, and b) we can start testing those answers.

Over time you can use the test plan as a repository for organisational lessons learned. In the same way as you might update a test case to test for a specific bug that tends to creep back in, or to ensure that you test a out-of-the-way piece of functionality that you tend to overlook, update your test plan so that you ask the questions that will prevent you making the same mistakes again.

The best way to get value of out of the process around test planning is to involve as many stakeholders and interested parties as possible in the production / review of the plan. Your test plan doesn’t have to be perfect; its primary goal is to serve as a means of reaching agreement between everyone involved, and only subsequently to record the specifics of that agreement.

In longer releases, it’s worth periodically revisiting the plan – really, re-reviewing the plan – in case any of the details have changed that impact any aspect of the planned test activities.

In short, the test plan is actually best used as a vehicle to stimulate communication and record decisions.

Ignorance is not bliss

“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” – Mark Twain.

It is easy in testing to get too comfortable in one’s testing approach. You might be testing yet another release of the same product. Everything is the same as it was last time, so why bother doing test planning?

Well, how do you know everything is the same as last time? Have you checked? Doesn’t your role exist precisely because shit happens? People make mistakes all the time. Things change, and people don’t tell other people. Happens all the time.

As the quote says, it’s not the things you don’t know that cause problems. It’s the things you are both 100% sure you’re right about, and even 1% wrong about, that bite you. As a tester, any feeling approaching total confidence in your knowledge about anything should cause warning lights to blink in your consciousness.

Therefore, force yourselves to go through the planning process again, even if you think everything is the same as last time.

Only difference between science and messing about…

“If you don’t write it down it never happened”. – Somebody

We do test planning because it adds value, to us as a function and to our projects. As a result, it is part of our testing process to ensure that we all do it.  Because it’s part of our process, it’s subject to audit as part of our ISO9001 certification. If we cannot show that any test activity took place, then we are not following our own process. An external ISO auditor could raise non-conformities which, depending on the level, could jeopardise our certification.

Of course, you might not have any process, contractual or regulatory reason or need to document your test planning, in which case it’s up to you to decide whether it adds value or not. I’d personally feel safer writing down agreed test planning points, if only for ass coverage purposes.

In summary…

  • Make test planning a conversation
  • Don’t get complacent about what you think you know
  • Write it down while it adds value

One thought on “Test Plans are useless. Test Planning is not.”

Leave a Reply

Your email address will not be published. Required fields are marked *