Test Types

Test levels tell you how much of the system you are testing. Test types tell you what kind of quality you are checking: does the feature give the right answer, is it fast enough, does an old feature still work after a change? One test level can include many test types.

In this lesson you will learn functional and non-functional testing, black-box and white-box testing, and the change-related types every tester uses daily: retesting, regression, smoke and sanity testing.

Functional and non-functional testing

Functional testing checks what the system does: given these inputs, does it produce the right outputs and behaviour? It is based on requirements such as "a user can reset the password with an OTP".

Non-functional testing checks how well the system does it. The main kinds are:

The same feature usually needs both. This example checks a product search: one functional check (correct results) and one non-functional check (response time):

import time

products = [f"Product {n}" for n in range(100_000)] + ["Cricket bat (Kashmir willow)"]

def search(term):
    return [p for p in products if term.lower() in p.lower()]

# Functional: does it find the right product?
results = search("cricket")
print("Functional:", "PASS" if results == ["Cricket bat (Kashmir willow)"] else "FAIL")

# Non-functional: does it answer within half a second?
start = time.perf_counter()
search("willow")
seconds = time.perf_counter() - start
print("Performance:", "PASS" if seconds < 0.5 else "FAIL")

Functional: PASS
Performance: PASS

The performance check uses a time limit rather than an exact value, because timings change from run to run and machine to machine.

Black-box, white-box and grey-box testing

• Black-box testing designs tests from the requirements and behaviour only, without looking at the code. Most manual testing is black-box. Techniques include equivalence partitioning and boundary values.
• White-box testing designs tests from the code's internal structure, for example making sure every if branch runs at least once. Developers do most of it, often measured with code coverage tools.
• Grey-box testing mixes the two: you test through the user interface or API but use some knowledge of the inside, such as the database tables, to design better tests.

Retesting and regression testing

When a developer fixes a bug, two different checks follow:

• Confirmation testing (retesting) runs the failed test again to confirm the bug is really fixed.
• Regression testing reruns tests of features that used to work to make sure the change did not break them. A regression is a feature that worked before and is broken now.

Regression suites grow over time and are run after every change, so they are the first tests teams automate. This JavaScript example runs the same regression suite against two versions of a shipping rule:

// Version 1, released last month
function shippingV1(total) {
  return total >= 499 ? 0 : 40;
}

// Version 2: a developer added express delivery
function shippingV2(total, express = false) {
  if (express) return 99;
  return total > 499 ? 0 : 40; // ">=" was changed by accident
}

const regressionSuite = [
  { name: "small order pays 40", input: 200, expected: 40 },
  { name: "499 is free", input: 499, expected: 0 },
  { name: "big order is free", input: 2500, expected: 0 },
];

for (const [label, fn] of [["v1", shippingV1], ["v2", shippingV2]]) {
  const failures = regressionSuite.filter((t) => fn(t.input) !== t.expected);
  const names = failures.map((t) => t.name).join(", ") || "none";
  console.log(`${label}: ${regressionSuite.length - failures.length}/${regressionSuite.length} passed, failed: ${names}`);
}
console.log("New feature check, express:", shippingV2(2500, true) === 99 ? "PASS" : "FAIL");

v1: 3/3 passed, failed: none
v2: 2/3 passed, failed: 499 is free
New feature check, express: PASS

The new express feature works, so a tester checking only the new feature would approve version 2. The regression suite catches the old rule that broke.

Smoke and sanity testing

Two more names appear in almost every job description:

The name "smoke test" comes from hardware: switch the device on and see if smoke comes out. If login or the home page fails, there is no point running 500 detailed tests. Companies use these two words a little differently, so always check how your team defines them.

Teams usually tag tests so they can run a small smoke set or the full regression set from the same code. pytest uses markers for this; here is the idea in plain Python:

def home_title():
    return "Welcome to Jaipur Books"

def login(user, password):
    return user == "priya" and password == "Secret@123"

def apply_coupon(total, percent):
    return round(total * (100 - percent) / 100, 2)

def test_home_page_opens():
    assert "Welcome" in home_title()

def test_login_works():
    assert login("priya", "Secret@123")

def test_coupon_rounding():
    assert apply_coupon(1299, 15) == 1104.15

def test_wrong_password_rejected():
    assert not login("priya", "secret@123")

SUITE = [
    (test_home_page_opens, {"smoke", "regression"}),
    (test_login_works, {"smoke", "regression"}),
    (test_coupon_rounding, {"regression"}),
    (test_wrong_password_rejected, {"regression"}),
]

def run(tag):
    selected = [test for test, tags in SUITE if tag in tags]
    print(f"Running {len(selected)} {tag} test(s)")
    for test in selected:
        try:
            test()
            print("  PASS", test.__name__)
        except AssertionError:
            print("  FAIL", test.__name__)

run("smoke")
run("regression")

Running 2 smoke test(s)
  PASS test_home_page_opens
  PASS test_login_works
Running 4 regression test(s)
  PASS test_home_page_opens
  PASS test_login_works
  PASS test_coupon_rounding
  PASS test_wrong_password_rejected

Common mistakes

• Confusing retesting with regression testing. Retesting checks the fixed bug; regression testing checks everything else that might have broken.
• Skipping non-functional testing. A correct page that takes 20 seconds to load is still a failure for users.
• Running the full suite on a broken build. Run the smoke tests first and reject unstable builds quickly.

Key points

• Functional testing checks what the system does; non-functional testing checks how well it does it.
• Black-box tests come from requirements, white-box tests from the code structure, and grey-box tests use some inside knowledge.
• Retesting confirms a fix; regression testing confirms that old features still work after a change.
• A smoke test is a broad, shallow check that a build is stable enough to test.
• A sanity test is a narrow check that a specific change works sensibly.
• Tagging tests lets one suite serve smoke, sanity and regression runs.

def apply_coupon(total, percent):
    return round(total * (100 - percent) / 100, 2)

def test_coupon_zero_percent():
    assert apply_coupon(1299, 0) == 1299

SUITE = [(test_coupon_zero_percent, {"sanity", "regression"})]

def run(tag):
    selected = [test for test, tags in SUITE if tag in tags]
    print(f"Running {len(selected)} {tag} test(s)")
    for test in selected:
        try:
            test()
            print("  PASS", test.__name__)
        except AssertionError:
            print("  FAIL", test.__name__)

run("sanity")

Running 1 sanity test(s)
  PASS test_coupon_zero_percent