Module 6 Session 6.4 Automation Guidelines & Tool Demos

Automation Guidelines & Tool Demos

Apply automation guidelines and explore tool capabilities through worked examples — Pressman Ch. 22 | Fewster & Graham (1999)

Learning Objectives

Apply the core guidelines for writing maintainable, reliable automated tests.
Explain and apply the Page Object Model pattern to separate test logic from UI implementation details.
Read and interpret representative test scripts in JUnit 5 (unit), Playwright (UI), and k6 (performance).
Describe how automated tests are integrated into a CI/CD pipeline and what artifacts they produce.
Identify common anti-patterns in automation scripts and explain how to refactor them.

Why Guidelines Matter

Automated tests are code. They must be written, reviewed, maintained, and debugged just like production code. Teams that treat test scripts as throwaway artifacts accumulate a test maintenance burden that eventually makes the suite unreliable and expensive to operate. Guidelines prevent this by establishing shared standards for quality.

Key insight (Fewster & Graham, 1999): The biggest risk in test automation is not technical failure but unmaintainability. Tests that break with every minor UI change, tests that nobody understands, and test suites that take hours to run because of poor design are the primary reasons automation programs are abandoned.

Core Automation Guidelines

1. One Concern Per Test

Each test should verify exactly one behavior. Tests that verify multiple concerns fail for multiple reasons, making diagnosis difficult. Apply the Single Responsibility Principle to test design: a failing test should immediately indicate what is broken.

Rule: If a test has more than one logical assertion about different behaviors, split it into multiple tests.

2. Tests Must Be Independent

No test should depend on the execution order of other tests, or on state created by a previous test. Each test must set up its own preconditions (Arrange) and clean up after itself (teardown). Interdependent tests break in non-obvious ways when the suite is run in a different order or in parallel.

Rule: A test must pass when run in isolation and when run as part of the full suite, in any order.

3. Assert What Matters, Not What Is Convenient

Avoid asserting implementation details (e.g., internal variable values) or irrelevant outputs (e.g., timestamp fields). Assert the observable behavior that the business cares about. Over-specified assertions create brittleness; under-specified assertions miss defects.

Rule: Assertions should reflect the acceptance criteria of the feature under test, not the structure of the implementation.

4. Eliminate Arbitrary Waits

Hard-coded sleep() calls are a pervasive anti-pattern. They make tests slow on fast machines and flaky on slow ones. Replace arbitrary waits with explicit waits (wait for a specific element state, HTTP response code, or database record) that poll until a condition is met or a timeout is reached.

Rule: No Thread.sleep() or time.sleep() in test scripts. Use framework-provided wait mechanisms.

5. Control Test Data

Tests must own their data. Do not rely on data created by other tests, manual operations, or shared environment state. Use factories, fixtures, or setup scripts to create required data before each test and delete it in teardown. This ensures repeatability across environments.

Rule: Every test must set up all data it requires and leave the environment in the same state it found it.

6. Meaningful Names and Documentation

Test names are the first failure message a developer sees. A good test name describes the scenario and expected outcome: should_return_404_when_user_not_found is far more diagnostic than test3. Organize tests by feature and document non-obvious setup steps.

Rule: Test names must describe the behavior under test and the expected outcome. No numbered or generic names.

Naming & Organization

A consistent directory structure and naming convention allows the team to quickly locate tests, understand coverage, and integrate with CI tooling.

Poor Organization

tests/
  test1.py
  test2.py
  misc_tests.py
  LoginTest.java
  NewTest.java

No structure, no naming convention, no relationship to features. Impossible to gauge coverage or find tests for a given feature.

Good Organization

tests/
  unit/
    auth/
      test_login.py
      test_logout.py
    billing/
      test_invoice.py
  integration/
    api/
      test_user_api.py
  e2e/
    checkout_flow_test.py

Mirrors source structure, grouped by feature and layer. Any developer can find or add tests for a given module.

Writing for Maintainability

Fragile Test (Hard-coded, Duplicate Logic)

// Duplicated setup in every test
void testCheckoutSuccess() {
  driver.get("https://shop.example.com");
  driver.findElement(By.id("email"))
        .sendKeys("user@test.com");
  driver.findElement(By.id("pass"))
        .sendKeys("password123");
  driver.findElement(By.id("login-btn")).click();
  Thread.sleep(3000); // arbitrary wait
  // ... 40 more lines ...
}

Maintainable Test (Fixture, Page Object, Explicit Wait)

// Shared fixture handles login
@BeforeEach
void setUp() { loginPage.loginAs(TEST_USER); }

@Test
void checkout_succeeds_with_valid_card() {
  cartPage.addItem(PRODUCT_SKU);
  checkoutPage.enterCard(VALID_CARD);
  OrderConfirmation confirm = checkoutPage.submit();
  assertThat(confirm.getStatus())
        .isEqualTo("CONFIRMED");
}

Page Object Model (POM)

The Page Object Model is the most widely adopted design pattern for UI test automation. It separates test logic from UI interaction details by encapsulating each page or component in a class.

Problem Without POM

Test scripts contain raw locators (By.id("submit-btn")) scattered throughout. When the UI changes, every test that references the changed element must be updated. A single ID change can break dozens of tests.

Solution: Page Objects

Locators and interaction methods live in a Page Object class. Tests call methods like loginPage.submitForm(). If the submit button ID changes, only the Page Object changes — no test script changes needed.

LoginPage.java — Page Object

public class LoginPage {
  private final WebDriver driver;
  private final By emailField = By.id("email");
  private final By passwordField = By.id("password");
  private final By submitButton = By.id("login-submit");

  public LoginPage(WebDriver driver) { this.driver = driver; }

  public DashboardPage loginAs(String email, String password) {
    driver.findElement(emailField).sendKeys(email);
    driver.findElement(passwordField).sendKeys(password);
    driver.findElement(submitButton).click();
    return new DashboardPage(driver); // returns next page object
  }
}

POM benefits: Single point of change for locators; tests read as user stories; page interactions are reusable across multiple tests; page state validations are centralized.

Demo: JUnit 5 + Mockito Unit Test

A unit test for a PaymentService that calculates the order total with discount logic. The payment gateway is mocked to isolate the unit.

PaymentServiceTest.java

@ExtendWith(MockitoExtension.class)
class PaymentServiceTest {

  @Mock PaymentGateway mockGateway;
  @InjectMocks PaymentService service;

  @Test
  void applyDiscount_reduces_total_by_percentage() {
    Order order = new Order(100.00, "SAVE10");
    when(mockGateway.validateCoupon("SAVE10"))
        .thenReturn(new Coupon(10));

    double total = service.calculateTotal(order);

    assertThat(total).isEqualTo(90.00);
    verify(mockGateway).validateCoupon("SAVE10");
  }

  @Test
  void calculateTotal_throws_for_expired_coupon() {
    Order order = new Order(100.00, "EXPIRED");
    when(mockGateway.validateCoupon("EXPIRED"))
        .thenThrow(new CouponExpiredException());

    assertThrows(CouponExpiredException.class,
        () -> service.calculateTotal(order));
  }
}

Key observations:

@Mock creates a Mockito mock; @InjectMocks injects it into the service under test.
Each test is fully independent — no shared state between them.
Test names describe the behavior and outcome: applyDiscount_reduces_total_by_percentage.
Two tests cover two distinct behaviors; they are not combined into one.
verify() asserts interaction with the mock — confirms the gateway was called with the right argument.

Demo: Playwright UI Test (TypeScript)

An end-to-end test verifying that a user can log in and see their dashboard. Uses Playwright's auto-waiting and page object pattern.

import { test, expect } from '@playwright/test';
import { LoginPage } from './pages/LoginPage';

test.describe('Authentication', () => {
  test('valid credentials show user dashboard', async ({ page }) => {
    const loginPage = new LoginPage(page);
    await loginPage.goto();
    await loginPage.login('user@example.com', 'ValidPass1!');

    await expect(page.getByRole('heading', { name: 'Dashboard' }))
        .toBeVisible();
    await expect(page.getByTestId('user-greeting'))
        .toContainText('Welcome, User');
  });

  test('invalid password shows error message', async ({ page }) => {
    const loginPage = new LoginPage(page);
    await loginPage.goto();
    await loginPage.login('user@example.com', 'wrongpass');

    await expect(page.getByRole('alert'))
        .toContainText('Invalid email or password');
  });
});

Key observations:

No sleep() calls — Playwright auto-waits for elements to be visible and interactive.
Uses semantic locators (getByRole, getByTestId) rather than fragile CSS selectors.
Page Object (LoginPage) encapsulates navigation and form interaction.
Positive and negative scenarios are separate tests with independent setup.

Demo: k6 Load Test Script

A k6 script that ramps from 10 to 200 virtual users over 2 minutes and verifies the API responds within SLA under load.

load_test.js

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '1m', target: 100 },   // ramp up
    { duration: '1m', target: 200 },   // peak load
    { duration: '30s', target: 0 },    // ramp down
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'],  // 95th percentile < 500ms
    http_req_failed:   ['rate<0.01'],  // error rate < 1%
  },
};

export default function () {
  const res = http.get('https://api.example.com/products');
  check(res, {
    'status is 200': (r) => r.status === 200,
    'response time OK': (r) => r.timings.duration < 500,
  });
  sleep(1); // simulate user think time
}

Key observations:

thresholds codify the SLA. If p95 latency exceeds 500ms, the test exits with a non-zero code, failing the CI pipeline gate.
stages model realistic load patterns: ramp-up, sustained peak, ramp-down.
sleep(1) here is intentional — it simulates user think time between requests, which is correct for load modeling (unlike arbitrary waits in functional tests).
k6 scripts are pure JavaScript/ES6; no GUI, no XML configuration.

CI Pipeline Integration

Automated tests deliver value only when they run automatically on every code change. The standard integration pattern is: commit → build → test → report → gate.

GitHub Actions: .github/workflows/test.yml (excerpt)

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with: { java-version: '21' }
      - name: Run unit tests
        run: mvn test
      - name: Publish test report
        uses: dorny/test-reporter@v1
        with:
          name: JUnit Tests
          path: target/surefire-reports/*.xml
          reporter: java-junit
      - name: Upload coverage to SonarCloud
        run: mvn sonar:sonar

Fail Fast
Unit tests run first. If they fail, the build stops before running slower integration and E2E tests. This provides fast feedback on the most common failure type.

JUnit XML Reports
Most CI platforms (GitHub Actions, Jenkins, GitLab CI) consume JUnit XML report format natively. Produce this format from every test layer to enable unified reporting.

Quality Gates
Configure CI to fail the build if: test pass rate drops below threshold, coverage falls below minimum, or SonarQube quality gate is not met.

Common Mistakes

Hard-coded credentials in test scripts: Storing passwords, API keys, or tokens directly in test code creates a security vulnerability if the code is pushed to a shared repository. Use environment variables or secrets management.

Record-and-playback as a permanent strategy: Recorded scripts are useful for initial script generation but produce extremely fragile tests. Always refactor recorded scripts to use explicit waits, page objects, and meaningful assertions before committing them.

Testing implementation, not behavior: Asserting internal state (e.g., checking a private field value) rather than observable outputs makes tests tightly coupled to implementation. When the implementation changes (even correctly), the test breaks.

Ignoring flaky tests: Flaky tests (non-deterministically pass or fail) erode trust in the suite. Track flakiness rate and fix or quarantine flaky tests immediately. A suite that sometimes fails for no reason will be ignored by the team.

Class Activity

Code Review: Spot the Anti-Pattern (20 minutes)

Review the following test script. Identify every guideline violation, explain why it is a problem, and write a corrected version.

void bigTest() throws Exception {
  // Test login
  driver.get("http://app.test.com/login");
  driver.findElement(By.cssSelector("input[name='email']"))
        .sendKeys("admin@test.com");
  driver.findElement(By.cssSelector("input[name='pass']"))
        .sendKeys("SuperSecret123"); // hardcoded
  driver.findElement(By.cssSelector("button.login")).click();
  Thread.sleep(5000); // wait for login

  // Test profile update
  driver.findElement(By.id("profile-link")).click();
  Thread.sleep(2000);
  driver.findElement(By.id("first-name")).clear();
  driver.findElement(By.id("first-name")).sendKeys("Alice");
  driver.findElement(By.id("save-btn")).click();
  Thread.sleep(2000);
  assertEquals("Profile saved", driver.findElement(By.id("msg")).getText());
  assertEquals("Alice", driver.findElement(By.id("first-name")).getAttribute("value"));
  // also check logout works
  driver.findElement(By.id("logout")).click();
  Thread.sleep(1000);
  assertEquals("Login", driver.getTitle());
}

Exit Ticket

State two core automation guidelines and explain why each is important for long-term suite maintainability.
What problem does the Page Object Model solve? Give a concrete example of a change that breaks scripts without POM but requires only a single change with POM.
In a k6 script, what is the difference between sleep(1) used to simulate user think time and Thread.sleep(3000) used as an arbitrary wait in a Selenium test? Why is one acceptable and the other not?
How does a CI pipeline quality gate work? What should trigger a build failure in a properly configured pipeline?

Summary & Preview

Key takeaways from Session 6.4:

Six core guidelines: one concern per test; independence; assert behavior not implementation; eliminate arbitrary waits; control test data; meaningful names.
The Page Object Model centralizes locators and interactions, dramatically reducing maintenance cost for UI tests.
Automation scripts are production-quality code: they must be reviewed, organized, and maintained to the same standard.
CI integration is mandatory: tests run automatically on every commit, produce JUnit XML reports, and gate deployments via quality thresholds.

Coming up — Session 6.5: OO Testing Basics
Sessions 6.5 and 6.6 shift focus to testing object-oriented systems. Session 6.5 reviews the OO concepts that most affect testability: encapsulation, inheritance, polymorphism, and dynamic binding — and establishes why OO systems require testing strategies beyond traditional procedural approaches.

Previous Session Course Home Next Session