Python testing practice — black box, white box and TDD with real programs

In the previous article we covered Python testing theory. Now it’s time to apply it systematically. In this article we build three real test suites — black box testing with equivalence classes, white box testing with flow graph analysis, and a complete TDD cycle building a class from scratch test by test. Each one uses a different strategy with a clear purpose.

Python testing practice — Program 1: Black box testing with equivalence classes

Black box testing designs test cases from the function’s specification — without looking at the code. The key technique is equivalence partitioning: divide the input space into groups (partitions) where every value in the group should produce the same type of result. You only need to test one representative from each partition.

We’ll test a grade validation and classification function.

The specification

classify_grade(grade) — takes a numeric grade
  Valid input:   0.0 to 10.0 (inclusive)
  Invalid input: outside this range, non-numeric

  Classification:
    Outstanding: >= 9.0
    Merit:       >= 7.0 and < 9.0
    Passed:      >= 5.0 and < 7.0
    Failed:      >= 0.0 and < 5.0

  Returns: ('Classification', grade_rounded_to_2dp)
  Raises:  ValueError for out-of-range numeric input
           TypeError for non-numeric input

Step 1: Identify equivalence partitions

VALID partitions (should return a classification):
  P1: grade in [9.0, 10.0]      → Outstanding
  P2: grade in [7.0, 9.0)       → Merit
  P3: grade in [5.0, 7.0)       → Passed
  P4: grade in [0.0, 5.0)       → Failed

INVALID partitions (should raise exceptions):
  P5: grade < 0                  → ValueError
  P6: grade > 10                 → ValueError
  P7: grade is not numeric       → TypeError

BOUNDARY VALUES (always test boundaries between partitions):
  9.0  → Outstanding (lower boundary of P1)
  8.9  → Merit (upper boundary of P2)
  7.0  → Merit (lower boundary of P2)
  6.9  → Passed (upper boundary of P3)
  5.0  → Passed (lower boundary of P3)
  4.9  → Failed (upper boundary of P4)
  0.0  → Failed (lower boundary of P4)
  10.0 → Outstanding (upper boundary of P1)
  -0.1 → ValueError
  10.1 → ValueError

Step 2: The function under test

def classify_grade(grade):
    """
    Classify a grade from 0-10.
    Returns: (classification, rounded_grade)
    Raises: ValueError if out of range, TypeError if non-numeric
    """
    if not isinstance(grade, (int, float)):
        raise TypeError(f'Grade must be numeric, got {type(grade).__name__}')
    if grade < 0 or grade > 10:
        raise ValueError(f'Grade must be between 0 and 10, got {grade}')

    grade = round(grade, 2)

    if grade >= 9.0:
        return ('Outstanding', grade)
    elif grade >= 7.0:
        return ('Merit', grade)
    elif grade >= 5.0:
        return ('Passed', grade)
    else:
        return ('Failed', grade)

Step 3: The test suite — one test per partition + boundaries

import unittest


class TestClassifyGradeBlackBox(unittest.TestCase):
    """
    Black box test suite for classify_grade.
    Tests are designed from the specification — not the implementation.
    One representative per equivalence class + all boundary values.
    """

    # ─── P1: Outstanding [9.0, 10.0] ────────────────
    def test_p1_representative_outstanding(self):
        """P1: typical Outstanding grade"""
        classification, _ = classify_grade(9.5)
        self.assertEqual(classification, 'Outstanding')

    def test_p1_lower_boundary_9_0(self):
        """Boundary: 9.0 → Outstanding (lower bound of P1)"""
        classification, grade = classify_grade(9.0)
        self.assertEqual(classification, 'Outstanding')
        self.assertEqual(grade, 9.0)

    def test_p1_upper_boundary_10_0(self):
        """Boundary: 10.0 → Outstanding (upper bound of valid range)"""
        classification, _ = classify_grade(10)
        self.assertEqual(classification, 'Outstanding')

    # ─── P2: Merit [7.0, 9.0) ───────────────────────
    def test_p2_representative_merit(self):
        """P2: typical Merit grade"""
        classification, _ = classify_grade(8.0)
        self.assertEqual(classification, 'Merit')

    def test_p2_lower_boundary_7_0(self):
        """Boundary: 7.0 → Merit (lower bound of P2)"""
        classification, _ = classify_grade(7.0)
        self.assertEqual(classification, 'Merit')

    def test_p2_upper_boundary_8_9(self):
        """Boundary: 8.9 → Merit (just below P1)"""
        classification, _ = classify_grade(8.9)
        self.assertEqual(classification, 'Merit')

    # ─── P3: Passed [5.0, 7.0) ──────────────────────
    def test_p3_representative_passed(self):
        """P3: typical Passed grade"""
        classification, _ = classify_grade(6.0)
        self.assertEqual(classification, 'Passed')

    def test_p3_lower_boundary_5_0(self):
        """Boundary: 5.0 → Passed (minimum passing grade)"""
        classification, _ = classify_grade(5.0)
        self.assertEqual(classification, 'Passed')

    def test_p3_upper_boundary_6_9(self):
        """Boundary: 6.9 → Passed (just below P2)"""
        classification, _ = classify_grade(6.9)
        self.assertEqual(classification, 'Passed')

    # ─── P4: Failed [0.0, 5.0) ──────────────────────
    def test_p4_representative_failed(self):
        """P4: typical Failed grade"""
        classification, _ = classify_grade(3.0)
        self.assertEqual(classification, 'Failed')

    def test_p4_lower_boundary_0_0(self):
        """Boundary: 0.0 → Failed (lowest valid grade)"""
        classification, grade = classify_grade(0)
        self.assertEqual(classification, 'Failed')
        self.assertEqual(grade, 0)

    def test_p4_upper_boundary_4_9(self):
        """Boundary: 4.9 → Failed (just below pass mark)"""
        classification, _ = classify_grade(4.9)
        self.assertEqual(classification, 'Failed')

    # ─── P5: grade < 0 (invalid) ────────────────────
    def test_p5_negative_raises_value_error(self):
        """P5: negative grade raises ValueError"""
        with self.assertRaises(ValueError):
            classify_grade(-1)

    def test_p5_just_below_zero(self):
        """Boundary: -0.1 → ValueError"""
        with self.assertRaises(ValueError):
            classify_grade(-0.1)

    # ─── P6: grade > 10 (invalid) ───────────────────
    def test_p6_above_max_raises_value_error(self):
        """P6: grade above 10 raises ValueError"""
        with self.assertRaises(ValueError):
            classify_grade(11)

    def test_p6_just_above_ten(self):
        """Boundary: 10.1 → ValueError"""
        with self.assertRaises(ValueError):
            classify_grade(10.1)

    def test_p6_error_message_contains_value(self):
        """ValueError message should contain the invalid value"""
        with self.assertRaises(ValueError) as ctx:
            classify_grade(15)
        self.assertIn('15', str(ctx.exception))

    # ─── P7: non-numeric (invalid) ──────────────────
    def test_p7_string_raises_type_error(self):
        """P7: string raises TypeError"""
        with self.assertRaises(TypeError):
            classify_grade('7.5')

    def test_p7_none_raises_type_error(self):
        """P7: None raises TypeError"""
        with self.assertRaises(TypeError):
            classify_grade(None)

    def test_p7_list_raises_type_error(self):
        """P7: list raises TypeError"""
        with self.assertRaises(TypeError):
            classify_grade([7, 5])

    # ─── Return value format ─────────────────────────
    def test_return_is_tuple(self):
        """Return value must be a 2-tuple"""
        result = classify_grade(7.5)
        self.assertIsInstance(result, tuple)
        self.assertEqual(len(result), 2)

    def test_grade_rounded_to_2_decimal_places(self):
        """Grade in return value must be rounded to 2dp"""
        _, grade = classify_grade(7.555)
        self.assertEqual(grade, 7.56)


if __name__ == '__main__':
    unittest.main(verbosity=2)

Run it:

python -m unittest test_black_box.py -v

Output (all 21 tests passing):

test_p1_lower_boundary_9_0 ... ok
test_p1_representative_outstanding ... ok
test_p1_upper_boundary_10_0 ... ok
...
----------------------------------------------------------------------
Ran 21 tests in 0.003s
OK

Why 21 tests and not just 4? Because the boundaries are where bugs hide. A common mistake is writing if grade > 9.0: instead of if grade >= 9.0: — the test test_p1_lower_boundary_9_0 catches that. The boundary tests are the most valuable tests in the suite.

Python testing practice — Program 2: White box testing with flow graph analysis

White box testing uses knowledge of the implementation to ensure every code path is covered. The formal technique is flow graph analysis — you draw the control flow of the function and verify every edge and node is exercised by at least one test.

We’ll test a calculate_shipping function with complex branching.

The function

def calculate_shipping(weight, destination, express=False):
    """
    Calculate shipping cost.

    weight:      positive float, kg
    destination: 'national', 'europe', 'international'
    express:     bool, adds 50% to base cost

    Returns: (cost, days)
    """
    # Node 1: Input validation
    if weight <= 0:                          # Branch A: weight invalid
        raise ValueError(f'Weight must be positive: {weight}')

    if destination not in ('national', 'europe', 'international'):
        raise ValueError(f'Unknown destination: {destination}')   # Branch B

    # Node 2: Base cost calculation by destination
    if destination == 'national':            # Branch C
        base_cost = 3.99
        days = 3
    elif destination == 'europe':            # Branch D
        base_cost = 8.99
        days = 7
    else:                                    # Branch E: international
        base_cost = 19.99
        days = 14

    # Node 3: Weight surcharge
    if weight > 5:                           # Branch F: heavy package
        surcharge = (weight - 5) * 1.5
        base_cost += surcharge
        days += 2                            # heavy packages take longer

    # Node 4: Express modifier
    if express:                              # Branch G
        base_cost *= 1.5
        days = max(1, days // 2)

    return (round(base_cost, 2), days)

Flow graph — paths to cover

Paths through the function:
  Path 1: weight <= 0                        → ValueError (Branch A)
  Path 2: invalid destination                → ValueError (Branch B)
  Path 3: national + light + no express      → Branches C, not F, not G
  Path 4: europe + light + no express        → Branches D, not F, not G
  Path 5: international + light + no express → Branches E, not F, not G
  Path 6: any + heavy + no express           → Branch F active, not G
  Path 7: any + light + express              → not F, Branch G active
  Path 8: any + heavy + express              → Branches F + G both active

The white box test suite

import unittest


class TestCalculateShippingWhiteBox(unittest.TestCase):
    """
    White box test suite for calculate_shipping.
    Every branch and path through the code is exercised.
    """

    # ─── Branch A: weight <= 0 ──────────────────────
    def test_branch_a_zero_weight_raises(self):
        """Branch A: weight = 0 → ValueError"""
        with self.assertRaises(ValueError):
            calculate_shipping(0, 'national')

    def test_branch_a_negative_weight_raises(self):
        """Branch A: weight < 0 → ValueError"""
        with self.assertRaises(ValueError):
            calculate_shipping(-1, 'national')

    # ─── Branch B: invalid destination ──────────────
    def test_branch_b_unknown_destination_raises(self):
        """Branch B: unknown destination → ValueError"""
        with self.assertRaises(ValueError):
            calculate_shipping(1, 'mars')

    def test_branch_b_case_sensitive(self):
        """Branch B: 'National' (capitalised) is invalid"""
        with self.assertRaises(ValueError):
            calculate_shipping(1, 'National')

    # ─── Path 3: national + light + no express ──────
    def test_path3_national_light_standard(self):
        """Path 3: national, <=5kg, standard"""
        cost, days = calculate_shipping(2, 'national')
        self.assertEqual(cost, 3.99)
        self.assertEqual(days, 3)

    def test_path3_boundary_5kg_national(self):
        """Path 3 boundary: exactly 5kg → no surcharge"""
        cost, days = calculate_shipping(5, 'national')
        self.assertEqual(cost, 3.99)
        self.assertEqual(days, 3)

    # ─── Path 4: europe + light + no express ────────
    def test_path4_europe_light_standard(self):
        """Path 4: europe, <=5kg, standard"""
        cost, days = calculate_shipping(3, 'europe')
        self.assertEqual(cost, 8.99)
        self.assertEqual(days, 7)

    # ─── Path 5: international + light + no express ─
    def test_path5_international_light_standard(self):
        """Path 5: international, <=5kg, standard"""
        cost, days = calculate_shipping(1, 'international')
        self.assertEqual(cost, 19.99)
        self.assertEqual(days, 14)

    # ─── Branch F: weight > 5 (surcharge active) ────
    def test_branch_f_heavy_national_no_express(self):
        """Branch F: 7kg national → surcharge = (7-5)*1.5 = 3.0"""
        cost, days = calculate_shipping(7, 'national')
        self.assertAlmostEqual(cost, 3.99 + 3.0, places=2)   # 6.99
        self.assertEqual(days, 5)   # 3 + 2 for heavy

    def test_branch_f_boundary_just_above_5kg(self):
        """Branch F boundary: 5.1kg → minimal surcharge"""
        cost, days = calculate_shipping(5.1, 'national')
        expected_cost = round(3.99 + 0.1 * 1.5, 2)
        self.assertAlmostEqual(cost, expected_cost, places=2)
        self.assertEqual(days, 5)

    def test_branch_f_heavy_international(self):
        """Branch F: heavy international package"""
        cost, days = calculate_shipping(10, 'international')
        # base 19.99 + (10-5)*1.5 = 19.99 + 7.5 = 27.49
        self.assertAlmostEqual(cost, 27.49, places=2)
        self.assertEqual(days, 16)   # 14 + 2

    # ─── Branch G: express active ────────────────────
    def test_branch_g_national_light_express(self):
        """Branch G: express national, light package"""
        cost, days = calculate_shipping(2, 'national', express=True)
        self.assertAlmostEqual(cost, round(3.99 * 1.5, 2), places=2)
        self.assertEqual(days, 1)   # max(1, 3//2) = max(1,1) = 1

    def test_branch_g_international_light_express(self):
        """Branch G: express international, light package"""
        cost, days = calculate_shipping(1, 'international', express=True)
        self.assertAlmostEqual(cost, round(19.99 * 1.5, 2), places=2)
        self.assertEqual(days, 7)   # max(1, 14//2) = 7

    # ─── Path 8: heavy + express (both F and G) ─────
    def test_path8_heavy_express_national(self):
        """Path 8: both surcharge (F) and express (G) active"""
        # national (3.99) + surcharge (7-5)*1.5=3.0 → 6.99
        # then express: 6.99 * 1.5 = 10.485 → 10.49
        cost, days = calculate_shipping(7, 'national', express=True)
        self.assertAlmostEqual(cost, round((3.99 + 3.0) * 1.5, 2), places=2)
        # heavy adds 2 days → 5 days, express halves → max(1, 5//2) = 2
        self.assertEqual(days, 2)

    # ─── Return value ─────────────────────────────────
    def test_return_is_tuple_of_two(self):
        """Return must be (float, int) tuple"""
        result = calculate_shipping(1, 'national')
        self.assertIsInstance(result, tuple)
        self.assertEqual(len(result), 2)
        cost, days = result
        self.assertIsInstance(cost, float)
        self.assertIsInstance(days, int)


if __name__ == '__main__':
    unittest.main(verbosity=2)

The comment on each test states which branch or path it covers. This is the key benefit of white box testing — you can look at the test suite and know which parts of the code are exercised. In a real project you’d use a coverage tool like coverage.py to measure this automatically.

Python testing practice — Program 3: Complete TDD cycle

This program builds a Queue class from scratch using Test-Driven Development — writing each test before the code that makes it pass.

The specification

A queue is FIFO (First In, First Out) — the first element added is the first removed. Unlike a stack (LIFO), queues model real-world lines: whoever arrived first leaves first.

The TDD cycle — 8 iterations

import unittest


class TestQueue(unittest.TestCase):
    """
    TDD-built Queue test suite.
    Tests were written BEFORE the implementation — one at a time.
    """

    # ─── Iteration 1: Empty queue ───────────────────
    def test_new_queue_is_empty(self):
        """A newly created queue should be empty."""
        q = Queue()
        self.assertTrue(q.is_empty)

    def test_new_queue_size_is_zero(self):
        """A new queue has size 0."""
        q = Queue()
        self.assertEqual(q.size, 0)

    # ─── Iteration 2: Enqueue ───────────────────────
    def test_enqueue_makes_queue_non_empty(self):
        """After enqueue, queue should not be empty."""
        q = Queue()
        q.enqueue(1)
        self.assertFalse(q.is_empty)

    def test_enqueue_increases_size(self):
        """Each enqueue increases size by 1."""
        q = Queue()
        q.enqueue('a')
        self.assertEqual(q.size, 1)
        q.enqueue('b')
        self.assertEqual(q.size, 2)
        q.enqueue('c')
        self.assertEqual(q.size, 3)

    # ─── Iteration 3: Peek (front) ──────────────────
    def test_peek_returns_first_element(self):
        """Peek returns the front element without removing it."""
        q = Queue()
        q.enqueue(10)
        q.enqueue(20)
        self.assertEqual(q.peek(), 10)

    def test_peek_does_not_remove_element(self):
        """Peek does not change the size."""
        q = Queue()
        q.enqueue(1)
        q.peek()
        self.assertEqual(q.size, 1)

    def test_peek_on_empty_raises(self):
        """Peeking on an empty queue raises an error."""
        q = Queue()
        with self.assertRaises(IndexError):
            q.peek()

    # ─── Iteration 4: Dequeue ───────────────────────
    def test_dequeue_returns_first_element(self):
        """Dequeue returns the front element (FIFO)."""
        q = Queue()
        q.enqueue('first')
        q.enqueue('second')
        self.assertEqual(q.dequeue(), 'first')

    def test_dequeue_removes_element(self):
        """Dequeue reduces size by 1."""
        q = Queue()
        q.enqueue(1)
        q.enqueue(2)
        q.dequeue()
        self.assertEqual(q.size, 1)

    def test_dequeue_fifo_order(self):
        """Elements come out in FIFO order."""
        q = Queue()
        for i in [1, 2, 3, 4, 5]:
            q.enqueue(i)
        result = [q.dequeue() for _ in range(5)]
        self.assertEqual(result, [1, 2, 3, 4, 5])

    def test_dequeue_on_empty_raises(self):
        """Dequeue on empty queue raises IndexError."""
        q = Queue()
        with self.assertRaises(IndexError):
            q.dequeue()

    def test_dequeue_error_message(self):
        """IndexError message should be descriptive."""
        q = Queue()
        with self.assertRaises(IndexError) as ctx:
            q.dequeue()
        self.assertIn('empty', str(ctx.exception).lower())

    # ─── Iteration 5: Alternating enqueue/dequeue ───
    def test_enqueue_after_dequeue(self):
        """Can enqueue after dequeuing."""
        q = Queue()
        q.enqueue(1)
        q.dequeue()
        q.enqueue(2)
        self.assertEqual(q.size, 1)
        self.assertEqual(q.peek(), 2)

    def test_interleaved_enqueue_dequeue(self):
        """Alternating enqueue/dequeue maintains FIFO."""
        q = Queue()
        q.enqueue('a')
        q.enqueue('b')
        self.assertEqual(q.dequeue(), 'a')
        q.enqueue('c')
        self.assertEqual(q.dequeue(), 'b')
        self.assertEqual(q.dequeue(), 'c')

    # ─── Iteration 6: Clear ─────────────────────────
    def test_clear_empties_queue(self):
        """Clear removes all elements."""
        q = Queue()
        for i in range(5):
            q.enqueue(i)
        q.clear()
        self.assertTrue(q.is_empty)
        self.assertEqual(q.size, 0)

    def test_clear_on_empty_queue(self):
        """Clear on empty queue does nothing."""
        q = Queue()
        q.clear()    # should not raise
        self.assertTrue(q.is_empty)

    # ─── Iteration 7: Contains ──────────────────────
    def test_contains_present_element(self):
        """'in' operator works for elements in queue."""
        q = Queue()
        q.enqueue(42)
        q.enqueue('hello')
        self.assertIn(42, q)
        self.assertIn('hello', q)

    def test_contains_absent_element(self):
        """'in' operator returns False for missing elements."""
        q = Queue()
        q.enqueue(1)
        self.assertNotIn(99, q)

    def test_contains_on_empty_queue(self):
        """'in' operator returns False on empty queue."""
        q = Queue()
        self.assertNotIn(1, q)

    # ─── Iteration 8: Magic methods ─────────────────
    def test_bool_empty_queue_is_false(self):
        """Empty queue is falsy."""
        q = Queue()
        self.assertFalse(bool(q))

    def test_bool_non_empty_queue_is_true(self):
        """Non-empty queue is truthy."""
        q = Queue()
        q.enqueue(1)
        self.assertTrue(bool(q))

    def test_len_equals_size(self):
        """len(q) equals q.size."""
        q = Queue()
        q.enqueue(1)
        q.enqueue(2)
        self.assertEqual(len(q), q.size)
        self.assertEqual(len(q), 2)

    def test_str_shows_contents(self):
        """str(q) shows queue contents."""
        q = Queue()
        q.enqueue(1)
        q.enqueue(2)
        q.enqueue(3)
        result = str(q)
        self.assertIn('Queue', result)
        self.assertIn('1', result)

    def test_iteration(self):
        """Can iterate over queue (front to back)."""
        q = Queue()
        for i in [10, 20, 30]:
            q.enqueue(i)
        items = list(q)
        self.assertEqual(items, [10, 20, 30])


# ─── Queue implementation (written AFTER each test) ─────────────────
class Queue:
    """FIFO Queue implementation — built TDD-style."""

    def __init__(self):
        self._items = []

    @property
    def is_empty(self):
        return len(self._items) == 0

    @property
    def size(self):
        return len(self._items)

    def enqueue(self, item):
        self._items.append(item)

    def dequeue(self):
        if self.is_empty:
            raise IndexError('Cannot dequeue from empty queue')
        return self._items.pop(0)

    def peek(self):
        if self.is_empty:
            raise IndexError('Cannot peek empty queue')
        return self._items[0]

    def clear(self):
        self._items.clear()

    def __contains__(self, item):
        return item in self._items

    def __bool__(self):
        return not self.is_empty

    def __len__(self):
        return self.size

    def __iter__(self):
        return iter(self._items)

    def __str__(self):
        return f'Queue({self._items})'

    def __repr__(self):
        return f'Queue({self._items})'


if __name__ == '__main__':
    unittest.main(verbosity=2)

Run all tests:

python -m unittest test_tdd_queue.py -v

test_bool_empty_queue_is_false ... ok
test_bool_non_empty_queue_is_true ... ok
test_clear_empties_queue ... ok
test_clear_on_empty_queue ... ok
test_contains_absent_element ... ok
...
----------------------------------------------------------------------
Ran 26 tests in 0.004s

OK

The TDD discipline that matters most: each test was written to fail first. Only then was the minimum code written to make it pass. The implementation above looks simple because the tests guided it to exactly what was needed — nothing more. TDD prevents over-engineering.

Visualise with Python Tutor

Copy this code into pythontutor.com and step through it:

import unittest

class Queue:
    def __init__(self):
        self._items = []

    def enqueue(self, item):
        self._items.append(item)

    def dequeue(self):
        if not self._items:
            raise IndexError('Empty queue')
        return self._items.pop(0)

    @property
    def size(self):
        return len(self._items)

class TestQueue(unittest.TestCase):
    def setUp(self):
        self.q = Queue()

    def test_fifo_order(self):
        self.q.enqueue(1)
        self.q.enqueue(2)
        self.q.enqueue(3)
        self.assertEqual(self.q.dequeue(), 1)
        self.assertEqual(self.q.dequeue(), 2)

suite = unittest.TestLoader().loadTestsFromTestCase(TestQueue)
unittest.TextTestRunner(verbosity=2).run(suite)

Step through setUp and test_fifo_order. Notice that setUp is called before the test method — self.q is a fresh Queue every time. Inside test_fifo_order, three enqueue calls add 1, 2, 3 to _items. The first dequeue removes and returns _items[0] which is 1 — FIFO. The assertEqual compares the returned value with 1 — they match, assertion passes. If you change the expected value to 2, watch AssertionError get raised inside assertEqual and caught by unittest internally — the test is marked as FAILED and the next test still runs.

Summary and next step

In this article you practised three testing strategies. Black box testing showed equivalence partitioning and boundary value analysis — designing 21 tests from the specification without reading the implementation. White box testing showed flow graph analysis — mapping every branch and ensuring full path coverage with 14 targeted tests. TDD showed the red-green-refactor cycle — 26 tests written before the code, guiding the implementation to exactly what was needed.

In the next article you’ll find exercises to solve on your own — designing test suites for the BankAccount and Stack classes from previous articles.