Session 1.6 - Effective vs Exhaustive Testing

Module 1: Introduction to Software Testing | Duration: 1 hour

Learning Objectives
  • Contrast practical, risk-based testing with infeasible exhaustive approaches.
  • Explain combinatorial growth and apply sampling to control scope.
  • Define coverage/adequacy targets and exit criteria aligned to risk.

Why Exhaustive is Impractical

Even modest systems have astronomically large input spaces. Hardware, environment, data states, and user behavior multiply combinations beyond what time and budgets allow. The aim is not to test everything, but to test the right things.

Combinatorial Explosion

3 Params 5 Values Each 5^3 = 125 Combos 10 Params 5 Values Each 5^10 ≈ 9.7M

This growth demands disciplined sampling and prioritization.

Principles of Effective Testing

Risk-based: Test where failure hurts most (business impact, safety, security).
Representative: Use realistic data, environments, and user journeys.
Diversity: Vary inputs, states, and paths to expose different bug classes.
Feedback Loops: Learn from defects and production signals to focus testing.

Sampling Techniques

  • Equivalence Class Partitioning (EC): Partition input domains; test representatives.
  • Boundary Value Analysis (BVA): Focus on edges where faults cluster.
  • Pairwise/Combinatorial: Cover all 2-way (or t-way) interactions with minimal sets.
  • Error Guessing & Heuristics: Leverage experience-driven scenarios.
Full Space EC + BVA Pairwise

Coverage vs Adequacy

Coverage metrics indicate how much we touched (branches, paths, requirements). Adequacy asks whether we tested enough in the right places. Use coverage trends to detect gaps, not as a sole target.

Stopping & Exit Criteria

  • Critical risks mitigated; Sev-1/Sev-2 defects addressed or accepted by stakeholders.
  • Agreed adequacy thresholds met (e.g., 2-way combos, boundary sets, branch coverage).
  • Stability achieved: low flake rate, test pass rate plateau, no new high-severity defects in N runs.

Summary & Assignment

Effective testing is strategic. By sampling wisely and aligning to risk, we deliver confidence without chasing impossible exhaustive coverage.

Assignment: For a feature with 6 boolean flags and 3 input ranges, propose a test plan using pairwise + BVA. Estimate test count and justify adequacy.
Course Home