Session 4.7 - Regression Types & Techniques

Overview of Types & Techniques

Regression testing encompasses both what we test (types) and how we manage the test suite (techniques). This session covers the taxonomy used in ISTQB and the Chowdhury textbook:

Category	Sub-category	Key Question
Regression Types (What we test)	Bug-Fix Regression	Did the fix work and did it cause related defects?
Regression Types (What we test)	Stability / Side-Effect Regression	Did the change break anything that was previously working?
Regression Techniques (How we manage the suite)	Retest-All	Can we afford to run every test?
	Test Case Selection	Which subset is sufficient?
	Test Case Prioritization	In what order should we run them?
	Test Suite Reduction	Which tests are redundant and can be removed?

Important: Types and techniques are orthogonal. You choose a type based on the change being made. You choose a technique based on the time and resources available. In practice, a single regression cycle often combines a type (e.g., stability regression) with a technique (e.g., prioritized selection).

Regression Testing Types

The Chowdhury textbook (Chapter 8) identifies two primary types of regression testing, each addressing a distinct risk:

Type 1: Bug-Fix Regression

Performed after a defect has been reported and fixed. The goal is twofold:

Repeat the test cases that originally exposed the defect to confirm the fix is effective.
Execute related test cases in the same module to check for any new defects introduced by the fix.

Type 2: Stability / Side-Effect Regression

Involves retesting a substantial part of the product. The goal is to prove that a change — whether a bug fix, new feature, or refactoring — has no detrimental effect on something that was earlier in order (previously working functionality).

This is the "insurance" type of regression: we are not looking for a specific known defect but for any side effects of the change across the wider system.

Relationship between the two types:

Dimension	Bug-Fix Regression	Stability Regression
Trigger	Specific defect was fixed	Any change to the codebase
Scope	Narrow: fixed module + related tests	Broad: substantial part of product
Primary question	"Is the bug fixed?"	"Did we break anything else?"
Test cases used	Original defect tests + nearby tests	Existing passing regression suite
Frequency	Every time a bug is fixed	Every significant change; every release

Bug-Fix Regression — In Depth

Bug-fix regression is the most targeted form of regression. Its three-step structure directly addresses all three regression objectives from Session 4.5:

Step 1: Confirmation testing — Re-execute the exact test case(s) that originally exposed the defect. If they now pass, the fix addresses the reported problem (Objective 1: verify the bug is fixed).

Step 2: Related defect discovery — Execute test cases covering other scenarios in the same module or function. Defects often cluster around shared root causes (Objective 2: find other related bugs).

Step 3: Side-effect check — Execute test cases for modules that depend on the fixed component. This bridges into stability regression (Objective 3: check the effect on other parts of the program).

Example: Bug-fix regression test plan

Bug reported: "Tax calculation returns negative value when discount equals item price."

Fix: Added a guard clause: if (price - discount) < 0: taxable_amount = 0

Step	Test Cases	Purpose
Confirmation	TC-TAX-042: price=50, discount=50 → expect tax=0	Confirms the exact reported scenario now passes
Related defects	TC-TAX-040 to TC-TAX-055: all discount + tax combinations	Checks for similar edge cases (price<discount, price=0, discount=0)
Side effects	TC-INV-010 to TC-INV-020: Invoice total calculation tests	Invoice module uses tax; guard clause may affect invoice total

Stability / Side-Effect Regression — In Depth

Stability regression is broader in scope. It is also called full regression when the entire suite is involved, or partial regression when a selected subset is used.

When is full stability regression justified?

Before a major product release.
After a large-scale refactoring of core infrastructure.
After a platform migration (new OS, DB version, cloud provider).
After a framework or library upgrade that affects many modules.
After multiple simultaneous bug fixes in a hotfix release.

Partial stability regression: A risk-stratified subset of the full regression suite, selected to cover:

All critical business process paths.
All modules that depend on the changed component (impact analysis output).
Historically failure-prone areas (high historical defect density).
Recently changed areas (higher regression risk than stable, untouched code).

Regression Techniques Overview

The Chowdhury textbook (Chapter 8) identifies four regression testing techniques that address the regression testing problem (P, P′, T). Each technique attacks the problem from a different angle:

Technique	Core Approach	Test Suite T Effect	Best Used When
Retest-All	Run the entire existing test suite T	Unchanged — full T run	Small test suites; critical releases; time is not a constraint
Test Case Selection	Run a subset T′ ⊂ T relevant to the change	Reduced per cycle; full T preserved	Large test suites; targeted changes with clear impact boundaries
Test Case Prioritization	Re-order T to run highest-value tests first	Same size; execution order changes	When early defect detection is required; time-boxed regression windows
Test Suite Reduction	Permanently remove redundant tests from T	Permanently smaller T	Bloated suites; redundant tests causing execution overhead

Retest-All Technique

Retest-All

Definition: Execute the entire existing test suite T against the modified program P′. No selection or prioritization is applied.

Advantages

Highest confidence — no regression can hide.
Simple to manage: no selection decisions needed.
Appropriate as the final pre-release gate.
Detects unexpected regressions in untouched areas.

Disadvantages

Most expensive in time and resources.
Impractical for large suites and frequent changes.
Does not scale with system growth.
Cannot be used in CI/CD with short build cycles.

When is Retest-All justified?

The test suite is small (e.g., <200 tests that run in under 10 minutes).
The change was so large or cross-cutting that impact analysis cannot meaningfully limit scope.
A major release is imminent and maximum confidence is required.
Automated execution makes full-suite runs fast enough to be practical.

Regression Test Selection Techniques

Regression Test Selection (RTS) attempts to reduce the time required to retest a modified program by selecting some subset T′ of the existing test suite T that is sufficient to detect regressions introduced by the change.

Test Case Selection

Goal: Find the smallest T′ ⊂ T that provides equivalent regression confidence to running all of T for the specific change P → P′.

Three main sub-techniques for regression test selection:

1. Minimization Techniques
Use formal analysis (e.g., code coverage, control flow analysis) to identify the minimum set of tests that covers all changed code paths. Selects T′ = only those tests that exercise the modified code. Provides theoretical minimum while maintaining change coverage.
Formal / algorithmic Highest precision Tool-supported

2. Ad-hoc / Random Selection
Randomly samples a subset of the existing test suite. No formal analysis; relies on statistical coverage. Simple and fast to apply but provides no guarantee of covering the changed code paths. Most useful as a quick sanity check when formal analysis is unavailable.
Simple No tooling required No coverage guarantee

3. Selective Retest Technique
The most widely used approach in practice. Select tests based on: (a) impact analysis results, (b) the components directly and indirectly affected by the change, and (c) historical defect data for high-risk areas. Combines human judgment with systematic analysis.
Practical Risk-based Combines analysis + judgment

Selection sub-techniques compared:

Sub-technique	Selection Basis	Precision	Effort	Tool Required
Minimization	Code coverage analysis of changed paths	Highest	High	Yes (coverage tools)
Ad-hoc / Random	Random sampling	Lowest	Very low	No
Selective Retest	Impact analysis + risk + history	Medium-High	Medium	Optionally (RTM, dependency graph)

Safe vs Unsafe Selection:

Safe selection: The selected subset T′ is guaranteed to detect any regression that the full suite T would detect (no regression can escape through gaps in T′). Minimization techniques aim for safe selection.
Unsafe selection: T′ may miss some regressions that T would catch. Ad-hoc and some selective methods may be unsafe. The risk is accepted in exchange for reduced execution time.

Test Case Prioritization Technique

Test case prioritization reorders the regression test suite so that higher-priority tests execute earlier in the test run. It does not reduce the number of tests run — it changes the order to maximise early defect detection.

Test Case Prioritization

Goal: If the regression window is time-boxed (e.g., 2-hour CI pipeline), prioritized ordering ensures the most valuable tests run first. If time runs out, the remaining (lower-priority) tests have not yet run — a known and managed risk.

Two types of prioritization are defined in Chapter 8:

General Test Case Prioritization

Tests are ordered to be useful over multiple future regression cycles, not just the current change. Priority is based on general importance independent of any specific change.

Prioritization criteria:

Business criticality of the feature tested
Historical defect density (tests that found defects before are more likely to find them again)
Code complexity (complex code is more regression-prone)
Frequency of use (heavily-used features must always work)
Customer-facing vs internal functionality

Version-Specific Test Case Prioritization

Tests are ordered based on their relevance to the specific change in the current version. Priority reflects how likely a test is to detect a regression caused by this particular modification.

Prioritization criteria:

Tests that directly exercise changed code paths (highest priority)
Tests for direct dependents of the changed component
Tests for indirect dependents (transitive impact)
Tests for high-risk integration points
Tests that have not been recently executed (stale)

Prioritization in practice — priority ordering example:

P1 (Highest)

Tests that directly execute modified code; tests for previously failing scenarios; smoke tests for critical user journeys.

P2 (High)

Tests for direct dependents of changed module; tests in historically high-defect areas; tests for business-critical workflows.

P3 (Medium)

Tests for indirect dependents; tests for related but lower-risk features; tests not recently executed (coverage refresh).

P4 (Low)

Tests for components with no dependency on the change; stable, well-tested areas with low historical defect rate.

Test Suite Reduction Technique

Over time, regression test suites grow large and acquire redundant tests: tests that cover the same code paths, test cases made obsolete by requirement changes, and tests duplicated across test levels. Test suite reduction permanently eliminates these tests to reduce future execution overhead.

Test Suite Reduction

Goal: Reduce testing costs by permanently eliminating redundant test cases from the regression suite in terms of code paths exercised or functionalities covered. Unlike selection (which temporarily excludes tests), reduction permanently removes tests.

Criteria for identifying redundant tests:

Code coverage overlap: Two tests exercise identical code paths and trigger the same branches. One can be removed without loss of coverage.
Requirement obsolescence: The requirement the test was designed for has been removed or replaced. The test no longer represents system behaviour.
Superseded by new tests: A new, more comprehensive test covers all scenarios of an older, narrower test.
Test duplication across levels: The same scenario is tested as both a unit test and an integration test with no incremental value from one of them.
Always-passing tests: Tests that have never failed in 20+ regression cycles may be testing trivially stable code that does not need repeated verification.

Benefits of suite reduction

Faster regression cycles as suite size shrinks.
Reduced maintenance effort (fewer tests to update on code change).
Cleaner, higher signal-to-noise test suite.
Less ambiguity when failures occur (fewer tests means clearer failure attribution).

Risks of suite reduction

Incorrectly identifying a non-redundant test as redundant removes regression coverage.
Historical tests that rarely fail may catch infrequent but high-impact regressions.
Reduction requires careful analysis — do not delete without coverage verification.
Always archive deleted tests rather than permanently discarding them.

Safe reduction process:

Use coverage tools to identify tests with overlapping code path coverage.
Verify the test to be removed is genuinely redundant (the remaining test covers all its paths).
Verify the requirement being tested has not been silently changed.
Archive the test (do not delete from version control) before removing from the active suite.
Run the reduced suite and verify coverage metrics are maintained.

Technique Comparison & Selection Guide

Technique	Suite Size Effect	Confidence Level	Effort	Best Scenario
Retest-All	No change (full T)	Highest	Highest	Small suites; pre-release gates; infrequent releases
Minimization (Selection)	Reduced per cycle	High (safe)	High (tooling required)	Large suites with good code coverage tooling
Selective Retest (Selection)	Reduced per cycle	Medium-High	Medium	Most practical day-to-day regression in Agile
Ad-hoc (Selection)	Reduced per cycle	Low	Very low	Quick sanity check; no formal tooling available
General Prioritization	No change (full T, reordered)	Same as Retest-All (if all run)	Low-Medium	Time-boxed regression; CI pipeline ordering
Version-Specific Prioritization	No change (full T, reordered)	Highest early-detection rate	Medium	When early defect detection for current change is priority
Suite Reduction	Permanently reduced	Maintained (if done correctly)	High (one-time; careful analysis)	Bloated suites with confirmed redundancy

Combining techniques: In practice, all four techniques are used together:

Reduction: Periodically prune the suite to remove confirmed redundancy.
Selection: From the pruned suite, select tests relevant to the current change.
Prioritization: Order the selected tests to run highest-priority first.
Retest-All: Before release, run all remaining tests (after selection and prioritization have already found most defects early).

Worked Example: Flight Booking System

System: A flight booking platform with modules: Search, Booking, Payment, Seat Selection, Check-in, Notifications, and Loyalty Points.

Current test suite T: 480 test cases across all modules.

Change: Bug fix — Loyalty Points were not being credited for international flights booked through the mobile app.

Step 1: Determine regression type
This is a Bug-Fix Regression (fix of a reported defect) combined with limited Stability Regression (check side effects on Booking and Notifications, which interact with Loyalty Points).

Step 2: Apply Selection technique (Selective Retest)

Module	Tests Selected	Reason
Loyalty Points	TC-LP-001 to TC-LP-040 (40 tests)	Directly changed module; confirmation + related defects
Booking (int’l)	TC-BK-050 to TC-BK-070 (21 tests)	Bug was specific to international bookings; booking module triggers loyalty credit
Notifications	TC-NOT-010, TC-NOT-015, TC-NOT-020 (3 tests)	Loyalty credit triggers a notification; check notification content
Payment	TC-PAY-001 to TC-PAY-005 (5 tests)	Smoke test; payment flow was not changed but interacts with booking
Search, Seat, Check-in	0 tests	No dependency on loyalty points; impact analysis shows no risk

Selected T′: 69 tests (14% of full suite) instead of 480.

Step 3: Apply Version-Specific Prioritization to T′

Priority	Tests	Rationale
P1 (run first)	TC-LP-018: "International booking via mobile app — loyalty credit" (original defect test)	Directly confirms the reported fix
P2	TC-LP-001 to TC-LP-040 minus TC-LP-018 (39 tests)	Related loyalty scenarios
P3	TC-BK-050 to TC-BK-070 (21 tests)	International booking flows
P4	TC-NOT-010, TC-NOT-015, TC-NOT-020; TC-PAY-001 to TC-PAY-005 (8 tests)	Side-effect smoke checks

Step 4: Suite Reduction opportunity identified
During the selection analysis, TC-LP-022 and TC-LP-023 are found to exercise identical code paths with the same test data. TC-LP-023 is archived and removed from the active suite (suite: 480 → 479). This will save marginal time in future regression cycles.

Selection Method Details — Minimization Techniques

Minimization-based selection uses formal program analysis to identify which tests are relevant to a change. The key methods are:

Control Flow Graph (CFG) Based Selection
Construct a CFG for the modified function/module. Identify all nodes and edges that changed. Select tests whose execution paths pass through at least one changed node or edge. Guarantees that every changed code path is exercised.

Data Flow Analysis Based Selection
Track definitions and uses of variables across the CFG. Select tests that cover definition-use pairs (du-pairs) involving the changed variables. Particularly effective for catching data-related regressions (wrong value propagated).

Firewall (Modification-Traversing) Selection
Identify the "firewall" — the set of code entities (functions, modules) modified or affected by the change. Select tests that exercise any entity within the firewall. Simpler than full CFG analysis; practical for larger systems.

Code example: CFG-based selection

# Modified function P' (changed condition: >= instead of >)
def apply_loyalty_credit(booking, user):
    if booking.flight_type == 'international':   # unchanged
        points = booking.fare * 0.05              # unchanged
        if user.app_version >= '3.0':            # CHANGED: was > '3.0'
            user.add_points(points)
            return True
    return False

# CFG selection: tests that traverse the changed condition (app_version >= '3.0')
# are selected. Tests covering only the outer 'if' or the return False path
# are not changed and can be de-prioritized.

Common Mistakes

Applying only bug-fix regression
Teams confirm the fix but skip stability regression. The most dangerous regressions are side effects in other modules — these are exactly what stability regression catches.

Treating reduction and selection as the same thing
Selection temporarily excludes tests from a given cycle; reduction permanently removes them. Treating them as equivalent can lead to permanent deletion of tests that were meant to be temporarily excluded.

Prioritizing by execution speed rather than risk
Ordering tests from fastest to slowest runs fast tests first — but may run low-risk tests before high-risk tests. Prioritization should be risk-driven, not speed-driven.

No baseline for minimization
Minimization techniques require an execution baseline (which tests previously passed against P). Without this, it is impossible to determine whether a failure is a regression or a pre-existing issue.

Deleting tests during reduction without archiving
Permanently deleting tests from version control removes the ability to restore them if the redundancy analysis turns out to be wrong. Archive first; delete only after verification.

Using ad-hoc selection for high-risk changes
Random sampling may completely miss tests for the changed module. Ad-hoc selection is only appropriate for very low-risk changes or as a quick sanity check, not as the primary regression strategy.

Class Activity

Scenario: You are the test lead for an e-learning platform. The regression test suite has grown to 600 test cases. The following changes are shipped in the next release:

Bug fix: Quiz timer was not pausing when a student navigated away from the quiz page and returned.
New feature: Students can now download course completion certificates as a QR-code-enabled PDF.
Refactoring: The database query layer was rewritten using an ORM. No functional change intended.

Tasks (in groups of 3–4):

Classify each change by regression type (bug-fix / stability / both). Justify each classification.
For change 1 (quiz timer), apply selective retest selection to identify which module tests to include. Assume 8 modules exist: Login, Dashboard, Course Content, Quiz Engine, Timer, Results, Certificates, and Payments.
For change 3 (ORM refactoring), justify why this change warrants a retest-all approach rather than selective retest.
Design a version-specific prioritization order for change 1's selected test set. List your P1, P2, and P3 categories with reasoning.
From the 600-test suite, you identify 35 tests as potentially redundant. Describe the safe process you would follow to decide which (if any) to permanently reduce.

Evaluation Rubric (10 marks)

2 marks: Correct type classification for all 3 changes with justification.
2 marks: Selective retest scope for change 1 is impact-driven and justified.
2 marks: Retest-all justification for change 3 is sound (ORM touches all DB queries).
2 marks: Prioritization order is risk-driven and logically consistent.
2 marks: Reduction process is safe (coverage verification, archive before delete).

Exit Ticket

Name the two regression testing types and state in one sentence what each is trying to achieve.
What is the difference between general prioritization and version-specific prioritization?
A team has 1,000 test cases. After a bug fix, they run only the 80 tests for the changed module. Is this a retest-all, selection, prioritization, or reduction approach? Justify your answer.
Why is test suite reduction a permanent operation and what safeguard should always accompany it?

Summary & Assignment

Regression testing has two types (bug-fix and stability) and four techniques (retest-all, selection, prioritization, and reduction). In practice, they are combined: selection narrows the scope, prioritization orders the selected tests, and reduction keeps the suite lean over time. Retest-all is reserved for when maximum confidence is needed and time permits.

Key takeaways:

Bug-fix regression: targeted; confirms fix and checks related defects.
Stability regression: broad; confirms nothing was broken in the wider system.
Retest-all: maximum confidence, maximum cost — use for release gates.
Selection (minimization, ad-hoc, selective retest): reduces execution cost per cycle.
Prioritization (general, version-specific): maximizes early defect detection rate.
Reduction: permanently shrinks the suite by removing confirmed redundant tests.
In CI/CD: selection + prioritization runs on every commit; retest-all gates releases.

Assignment (Module 4 Capstone): Using your mini-project:

Identify the most recent bug fix and classify the regression testing type required. Design a 3-step bug-fix regression test plan (confirmation, related defects, side effects).
Apply selective retest selection to identify the minimum test set for this regression cycle. Show the impact analysis that drives your selection.
Prioritize your selected test set into P1/P2/P3/P4 categories using version-specific criteria. Justify the priority of each category.
Identify 5 tests in your existing suite that may be candidates for reduction. For each, state the specific redundancy criterion and describe the verification you would perform before removing it.
Draw a CI/CD regression pipeline diagram for your project showing: trigger, unit regression gate, integration regression gate, E2E regression gate, and release gate. Label each with the technique(s) applied.

Regression Types & Techniques

Learning Objectives