Black-Box Test Design Techniques, Applied to One Real Checkout

Introduction

The ISTQB Foundation Level syllabus names four black-box test techniques: equivalence partitioning, boundary value analysis, decision table testing, and state transition testing. Most explanations give each one its own made-up example: an age field for boundaries, an insurance rule for decision tables, a vending machine for states. That makes it hard to see why you would choose one over another, because the examples never overlap.

This guide applies all four to one real thing: the checkout on this site's practice shop, the version without seeded bugs. Every result below comes from a Playwright test that runs against the shop, so if the shop changes, the test fails before this article can go stale.

The short version is that each technique asks a different question about the same screens, and each one found something the others didn't.

Which technique answers which question

A fifth question, "which combinations of independent settings do I have time for?", belongs to combinatorial testing, which the syllabus leaves out. The pairwise test case generator answers it, and pairwise testing on Sauce Demo measures what it catches.

Equivalence partitions and boundary values: the details form

The checkout asks for a first name, a last name, and a postal code. All three are required, and the form states no maximum length. For each field the specification gives two partitions: empty, which should be rejected, and one or more characters, which should be accepted.

Boundary value analysis on a length puts the edge between 0 and 1 characters. 2-value analysis tests exactly those two values. 3-value analysis adds 2 characters, the other neighbour of 1; nothing exists below 0. That is a small set of tests, and it should be: a required field with no upper limit has only one boundary.

Two things came out of it:

• Empty was rejected, and one character was accepted, for each of the three fields.
• A field of spaces counts as empty. Three spaces in the postal code gave "Error: Postal code is required". That isn't a boundary value, because the partition is defined by length and " " has three characters. It comes from error guessing: the form trims what you type before checking it, so the partition the code implements isn't the one the length suggests. Writing partitions down is what makes the difference visible.

There is no upper boundary to test, which is itself worth raising. A form that accepts any length hands the problem to whatever stores the name next. The test cases article found Sauce Demo accepting a 300-character name; here the specification doesn't even say.

Decision tables: which error you see

The same form has a rule that isn't about any one field: what happens when you press Continue. Three conditions (each field filled or empty) give eight combinations. We filled the form in each of the eight ways and pressed Continue.

The shop reported only the first empty field it found, in form order. Two filled fields after an empty first name made no difference: the message was still "Error: First name is required". Entered into the decision table generator, the eight columns merge into four:

A "–" means that condition didn't change the outcome in that column. Four tests cover the rule, and the table explains why four is enough.

It also turns a behaviour into a question. A shopper who leaves all three fields empty fixes one, presses Continue, and is told about the next: one round trip per missing field. That isn't necessarily a bug, but it's a product decision, and the table puts it in front of someone who can make it. Sauce Demo's checkout behaves the same way; the test cases article raised it there too.

State transitions: the checkout as a state machine

Partitions and tables look at one screen. The checkout is a sequence of screens, and the question there is which events are allowed where. Here it is in Mermaid's state diagram notation, the input the state transition tool reads. Mermaid itself wants state names without spaces; the tool accepts them, and its Mermaid export adds the ids Mermaid needs.

[*] --> Login
Login --> Products : log in [right username and password]
Login --> Login : log in [wrong username or password] / show error
Products --> Cart : open cart
Products --> Login : log out
Cart --> Products : continue shopping
Cart --> Your details : checkout [cart not empty]
Your details --> Overview : continue [details complete]
Your details --> Your details : continue [a field empty] / show error
Your details --> Cart : back to cart
Overview --> Cart : cancel
Overview --> Complete : place order / empty the cart
Complete --> Products : back to products

The shop's header has Cart and Log out links on every signed-in page; the model shows them on Products only, to keep the table readable.

The words in brackets are guards: conditions the data has to meet. The syllabus treats an event with a guard as its own column in the state table, so "log in" with the right password and "log in" with the wrong one are two columns. That gives 6 states and 12 columns, and the three coverage criteria count differently:

• All states: 6 coverage items. One test that walks from Login to Complete visits all of them, in 5 steps.
• Valid transitions: 12 coverage items, one per arrow. The state transition test generator covers them in a single test of 17 steps, because every state can reach every other.
• All transitions: every cell of the state table, 72 in all. 12 are the valid transitions; the other 60 are events a state should refuse, such as "place order" on the cart page. The syllabus asks for one invalid event per test, so that one refusal can't hide another, which makes 61 tests.

We ran all of them against the shop. Every valid transition led where the model says, and each of the 52 invalid events we could check was simply not offered: no button, no link. The other 8 are the header links the model simplifies away.

The event the state table doesn't list

A state table only lists the events a page offers, and a browser has one more: the address bar. We typed the checkout's addresses directly.

• Signed out, typing the address of any inner page showed "You need to log in first." That is what the state table predicts.
• Signed in, with something in the cart, typing the overview's address from the product page skipped the details form. The overview showed the order total, Place order was enabled, and pressing it showed "Thank you for your order." No name, no postal code.
• Signed in with an empty cart, the thank-you page opened as well, with nothing ordered.

The practice shop keeps everything in the browser, and there's no real order to protect, so here this is a curiosity. On a real shop it is the test that matters most: the order of pages is a convenience for honest shoppers, not a control, and the server has to refuse an order without the details the form was meant to collect. A state model lists the paths the design intended. Its value as a test design is the question it makes you ask: which other paths exist?

Choosing between them

Start from the question, not the technique.

• One input with ranges or lengths: partitions first, then boundary values at every edge, with the step the field really uses: 1 for a count, 0.01 for money.
• Several conditions deciding one outcome: a decision table. It is exhaustive by design, so it stays useful up to four or five conditions. Beyond that, split it by the condition that matters most.
• A sequence of screens or statuses: a state model. Valid-transition coverage is the usual target. All-transitions coverage is the one that finds missing refusals, and the one to aim for when a wrong step costs money.
• Independent settings that combine: pairwise, which gives up exhaustiveness to fit the time you have.

Most real features need more than one. On this one checkout, boundary values found nothing surprising, the decision table found a product question, and the state model found a path nobody designed.

Exercise

Pick a form you test at work, ideally one with a rule and at least two screens. Write the partitions for one field, the decision table for its submit button, and the state model for the flow it belongs to, using the three tools linked above. For each technique, note one test it produced that the others didn't. If one technique produced nothing new, that is a result too: it tells you where your existing tests already are.

Preparing for the Foundation exam? The test design practice questions generate fresh calculation questions on all four techniques, and explain which mistake leads to each wrong answer.

Conclusion

The four black-box techniques aren't alternatives. They're four questions: which inputs behave alike, where the edges are, which combinations lead where, and which order of events is allowed. Applied to one small checkout, they produced a trimmed field that behaves differently from its length, a form that reports one missing field at a time, and an address that skips a step. Choose the technique by the question you need answered, write down what each one finds, and treat every result you can't explain as something to ask about rather than something to guess.

Sources and further reading

• ISTQB Certified Tester Foundation Level (CTFL) v4.0: section 4.2 defines the four techniques and their coverage items
• Mermaid: state diagrams