Skip to content
Code Automation Lab

Pragmatic Programming Principles in Test Automation

“The Pragmatic Programmer” by Andy Hunt and Dave Thomas is a classic programming book published in 1999. The authors share practical advice for writing better code and building software that works.

These principles work great for test automation too. When you apply them to your test code, you get tests that are easier to maintain, debug, and understand.

Let’s look at key principles and how to use them in your test automation projects.

Principle 1: DRY (Don’t Repeat Yourself)

Section titled “Principle 1: DRY (Don’t Repeat Yourself)”

Each piece of logic should live in one clear place.

If you’re copying and pasting code, you’re breaking DRY — and setting yourself up for extra bugs and maintenance headaches.

DRY in Automation

Section titled “DRY in Automation”

Don’t duplicate locators, API calls, or test data across files. Centralize them in page objects, utilities, or fixtures. This keeps your tests maintainable and reduces breakage when the app changes.

Adhering to the DRY principle is crucial for creating a robust and scalable test suite. It directly impacts your tests’ reliability and maintainability, saving significant time and effort in the long run.

For instance, instead of copying login logic into every single test, you create a single helper function. When the login page changes, you only have to update that one function, and all your tests will instantly be up to date. This prevents the classic “find-and-replace” nightmare and ensures consistency.

Example:

❌ Before (Violating DRY):

// Multiple test files with duplicate login logic
test('user can view dashboard', async ({ page }) => {
  await page.goto('/login');
  await page.fill('[name="email"]', 'user@test.com');
  await page.fill('[name="password"]', 'password123');
  await page.click('button[type="submit"]');
  // test continues...
});


test('user can edit profile', async ({ page }) => {
  await page.goto('/login');
  await page.fill('[name="email"]', 'user@test.com');
  await page.fill('[name="password"]', 'password123');
  await page.click('button[type="submit"]');
  // test continues...
});

✅ After (Following DRY):

// Reusable login helper
async function loginAsUser(page) {
  await page.goto('/login');
  await page.fill('[name="email"]', 'user@test.com');
  await page.fill('[name="password"]', 'password123');
  await page.click('button[type="submit"]');
}


test('user can view dashboard', async ({ page }) => {
  await loginAsUser(page);
  // test continues...
});


test('user can edit profile', async ({ page }) => {
  await loginAsUser(page);
  // test continues...
});

Impact: Reduces code duplication by 70%, makes updates require only one change, improves test reliability.

DRY Principle Diagram

Principle 2: ETC (Easy To Change)

Section titled “Principle 2: ETC (Easy To Change)”

Good code is easy to change. Every design decision should make future changes easier, not harder.

In test automation, requirements change constantly. New features appear, UI gets redesigned, APIs evolve. Your tests should adapt quickly without massive rewrites.

ETC in Automation

Section titled “ETC in Automation”

Write tests that isolate change. When the application changes, you should update configuration or a single module—not hundreds of test files. Use abstractions, configuration files, and clear boundaries between test logic and implementation details.

The ETC principle is about anticipating change and designing your test framework to handle it gracefully. It’s not about predicting the future—it’s about making your code flexible enough to adapt when the inevitable changes arrive.

For example, hardcoding URLs in every test makes changes painful. But storing them in a config file means one update fixes all tests. Similarly, using Page Objects means UI changes only affect the page object, not every test that uses it.

Example:

❌ Before (Hard to Change):

// Tests are tightly coupled to implementation details
test('user can checkout', async ({ page }) => {
  await page.goto('https://staging.myapp.com/checkout');
  await page.fill('#credit-card-number', '4111111111111111');
  await page.fill('#cvv', '123');
  await page.click('button.submit-payment');


  // Direct DOM assertions
  expect(await page.locator('div.success-message').textContent())
    .toContain('Payment successful');
});


test('user can view orders', async ({ page }) => {
  await page.goto('https://staging.myapp.com/orders');
  await page.fill('#search-box', 'ORDER-123');
  // More hardcoded selectors...
});

Problems:

  • URL change requires updating every test
  • Selector changes break multiple tests
  • Environment switching needs find-and-replace
  • No separation between “what” and “how”

✅ After (Easy to Change):

config/environments.js
// Configuration-driven approach
export const config = {
  baseURL: process.env.BASE_URL || 'https://staging.myapp.com',
  timeout: 30000
};


// pages/CheckoutPage.js
export class CheckoutPage {
  constructor(page) {
    this.page = page;
    this.cardNumberInput = page.getByLabel('Card Number');
    this.cvvInput = page.getByLabel('CVV');
    this.submitButton = page.getByRole('button', { name: 'Submit Payment' });
    this.successMessage = page.getByTestId('success-message');
  }


  async navigate() {
    await this.page.goto('/checkout');
  }


  async submitPayment(cardNumber, cvv) {
    await this.cardNumberInput.fill(cardNumber);
    await this.cvvInput.fill(cvv);
    await this.submitButton.click();
  }


  async getSuccessMessage() {
    return await this.successMessage.textContent();
  }
}


// tests/checkout.spec.js
test('user can checkout', async ({ page }) => {
  const checkout = new CheckoutPage(page);


  await checkout.navigate();
  await checkout.submitPayment('4111111111111111', '123');


  expect(await checkout.getSuccessMessage())
    .toContain('Payment successful');
});

What Changed:

  • URL centralized → Environment changes affect one config file
  • Selectors in Page Object → UI changes update one class
  • Clear methods → Test intent is obvious, implementation hidden
  • Semantic locators → Less brittle, survives UI refactoring

Impact: Environment changes take 1 line instead of 50+ tests. UI redesigns update 1 page object instead of dozens of tests. New team members can change tests without breaking them.

Practical ETC Strategies

Section titled “Practical ETC Strategies”

1. Configuration Over Hardcoding

// ❌ Hard to change
await page.goto('https://api-staging.example.com/users');


// ✅ Easy to change
await page.goto(`${config.apiBaseURL}/users`);

2. Use Test Data Files

// ❌ Hard to change
test('login with admin', async ({ page }) => {
  await login(page, 'admin@test.com', 'Admin123!');
});


// ✅ Easy to change
import { testUsers } from './test-data/users.js';


test('login with admin', async ({ page }) => {
  await login(page, testUsers.admin.email, testUsers.admin.password);
});

3. Feature Flags for Conditional Logic

// ✅ Easy to adapt to feature rollouts
test('user sees new dashboard', async ({ page }) => {
  if (config.features.newDashboard) {
    await expect(page.getByTestId('dashboard-v2')).toBeVisible();
  } else {
    await expect(page.getByTestId('dashboard-v1')).toBeVisible();
  }
});

4. Abstraction Layers

// ✅ API client hides implementation
// If API changes from REST to GraphQL, tests don't need to know
const user = await apiClient.getUser(userId);
expect(user.name).toBe('John Doe');

Questions to Ask

Section titled “Questions to Ask”

When writing test code, ask yourself:

  • “If this URL changes, how many files do I update?” Should be 1 (config file)
  • “If this selector changes, how many tests break?” Should be 1 (page object)
  • “Can I switch environments without code changes?” Should be yes (environment variables)
  • “If the API version changes, what breaks?” Should be isolated (API client layer)

If the answer is “many files” or “everywhere,” your code isn’t ETC-compliant.

Principle 3: Orthogonality

Section titled “Principle 3: Orthogonality”

In mathematics, orthogonal means independent—changing one thing doesn’t affect another. In code, orthogonal components are self-contained and decoupled.

When test components are orthogonal, you can change, test, and reuse them independently. A bug in one area doesn’t cascade into unrelated failures. Tests become more reliable and easier to maintain.

Orthogonality in Automation

Section titled “Orthogonality in Automation”

Keep tests independent from each other. One test’s execution should never depend on another test running first or leaving behind specific data. Each test should set up its own state, perform its actions, and clean up after itself.

Orthogonal test design means you can run tests in any order, in parallel, or individually—and they’ll always work. This is critical for fast CI/CD pipelines and reliable test results.

The opposite of orthogonality is coupling. When tests are coupled, they become fragile. Test A fails, which causes Test B to fail, which cascades into Test C, D, and E failing. You waste hours debugging false failures instead of real bugs.

Example:

❌ Before (Coupled Tests):

// Test 1 creates data that Test 2 depends on
test('admin creates new user', async ({ page }) => {
  await page.goto('/admin/users');
  await page.fill('#username', 'testuser123');
  await page.fill('#email', 'test@example.com');
  await page.click('button[type="submit"]');


  await expect(page.getByText('User created')).toBeVisible();
});


// Test 2 DEPENDS on Test 1 having run first
test('admin can edit user', async ({ page }) => {
  await page.goto('/admin/users');


  // Assumes 'testuser123' exists from previous test
  await page.click('tr:has-text("testuser123") button:has-text("Edit")');
  await page.fill('#email', 'newemail@example.com');
  await page.click('button:has-text("Save")');


  await expect(page.getByText('User updated')).toBeVisible();
});


// Test 3 also depends on the same data
test('admin can delete user', async ({ page }) => {
  await page.goto('/admin/users');


  // Still assuming 'testuser123' exists
  await page.click('tr:has-text("testuser123") button:has-text("Delete")');
  await page.click('button:has-text("Confirm")');


  await expect(page.getByText('User deleted')).toBeVisible();
});

Problems:

  • Tests must run in specific order
  • If Test 1 fails, all others fail
  • Can’t run tests in parallel
  • Can’t run individual tests for debugging
  • Shared state creates race conditions

✅ After (Orthogonal Tests):

// Each test is self-contained and independent
test('admin creates new user', async ({ page }) => {
  await page.goto('/admin/users');


  // Generate unique data for this test
  const username = `user_${Date.now()}`;
  const email = `${username}@example.com`;


  await page.fill('#username', username);
  await page.fill('#email', email);
  await page.click('button[type="submit"]');


  await expect(page.getByText('User created')).toBeVisible();


  // Clean up: Delete the user we just created
  await page.click(`tr:has-text("${username}") button:has-text("Delete")`);
  await page.click('button:has-text("Confirm")');
});


test('admin can edit user', async ({ page }) => {
  // Set up: Create the user we need for THIS test
  const username = `user_${Date.now()}`;
  await createUserViaAPI(username, 'test@example.com');


  // Execute: Test the edit functionality
  await page.goto('/admin/users');
  await page.click(`tr:has-text("${username}") button:has-text("Edit")`);
  await page.fill('#email', 'newemail@example.com');
  await page.click('button:has-text("Save")');


  await expect(page.getByText('User updated')).toBeVisible();


  // Clean up: Remove test data
  await deleteUserViaAPI(username);
});


test('admin can delete user', async ({ page }) => {
  // Set up: Create the user we need for THIS test
  const username = `user_${Date.now()}`;
  await createUserViaAPI(username, 'test@example.com');


  // Execute: Test the delete functionality
  await page.goto('/admin/users');
  await page.click(`tr:has-text("${username}") button:has-text("Delete")`);
  await page.click('button:has-text("Confirm")');


  await expect(page.getByText('User deleted')).toBeVisible();


  // No cleanup needed - user is already deleted
});

What Changed:

  • Each test creates its own test data
  • Unique identifiers prevent conflicts
  • Tests can run in any order
  • Tests can run in parallel
  • Each test cleans up after itself
  • Setup via API is faster than UI

Impact: Tests become 100% reliable regardless of execution order. Parallel execution reduces test suite time by 70%. Debugging becomes trivial—run one test in isolation.

Practical Orthogonality Strategies

Section titled “Practical Orthogonality Strategies”

1. Independent Test Data

// ❌ Coupled - shared test data
const TEST_USER = { email: 'test@example.com', password: 'pass123' };


test('login test', async () => {
  await login(TEST_USER.email, TEST_USER.password);
});


// ✅ Orthogonal - unique data per test
test('login test', async () => {
  const user = await createTestUser(); // Generates unique user
  await login(user.email, user.password);
  await deleteTestUser(user.id);
});

2. API Setup for Speed

// ✅ Use API to set up state quickly
test('user can update profile', async ({ page }) => {
  // Fast: Create user via API (50ms)
  const user = await createUserViaAPI({
    username: `user_${Date.now()}`,
    email: 'test@example.com'
  });


  // Slow: Test the UI functionality
  await page.goto('/profile');
  await loginAsUser(page, user);
  await page.fill('#bio', 'My new bio');
  await page.click('button:has-text("Save")');


  await expect(page.getByText('Profile updated')).toBeVisible();


  // Fast: Clean up via API
  await deleteUserViaAPI(user.id);
});

3. Fixtures for Isolation

// Playwright fixtures provide automatic cleanup
import { test as base } from '@playwright/test';


const test = base.extend({
  authenticatedUser: async ({ page }, use) => {
    // Setup: Create and authenticate user
    const user = await createTestUser();
    await loginAsUser(page, user);


    // Provide to test
    await use(user);


    // Automatic cleanup after test completes
    await deleteTestUser(user.id);
  }
});


test('authenticated user can view dashboard', async ({ page, authenticatedUser }) => {
  // User is already authenticated and will be cleaned up automatically
  await page.goto('/dashboard');
  await expect(page.getByText(`Welcome ${authenticatedUser.username}`)).toBeVisible();
});

4. Database Transactions (where applicable)

// ✅ Wrap tests in transactions that rollback
test('user registration creates database entry', async ({ database }) => {
  await database.beginTransaction();


  try {
    // Test creates data
    await registerUser('testuser', 'test@example.com');


    // Verify it exists
    const user = await database.query('SELECT * FROM users WHERE username = ?', ['testuser']);
    expect(user).toBeDefined();


  } finally {
    // Always rollback - no cleanup needed
    await database.rollback();
  }
});

5. Avoid Global State

// ❌ Coupled - global state
let currentUser = null;


test('login', async () => {
  currentUser = await login('user@test.com', 'pass');
});


test('view profile', async () => {
  // Depends on currentUser from previous test
  await viewProfile(currentUser.id);
});


// ✅ Orthogonal - local state
test('login', async () => {
  const user = await login('user@test.com', 'pass');
  // Use user only within this test
  expect(user.isAuthenticated).toBe(true);
});


test('view profile', async () => {
  const user = await login('user@test.com', 'pass');
  await viewProfile(user.id);
});

Signs of Non-Orthogonal Tests

Section titled “Signs of Non-Orthogonal Tests”

Watch out for these warning signs:

  • Tests fail when run in different order - Clear sign of coupling
  • Tests fail when run in parallel - Shared state conflicts
  • Cascade failures - One failure causes many others to fail
  • Setup files with shared data - beforeAll that creates test data used by multiple tests
  • Tests that can’t run individually - Must run entire suite
  • Flaky tests that pass/fail randomly - Often due to race conditions with shared state

Benefits of Orthogonality

Section titled “Benefits of Orthogonality”

Reliability: Each test is isolated, reducing false failures

Speed: Parallel execution without conflicts

Debugging: Run one test at a time to isolate issues

Maintainability: Changes to one test don’t break others

Confidence: Test results are consistent and trustworthy

Principle 4: Tracer Bullets

Section titled “Principle 4: Tracer Bullets”

Tracer bullets are rounds that light up when fired, showing you where your shots are going. In programming, tracer bullets mean building a minimal end-to-end implementation first, then iterating.

Instead of building everything before testing anything, create a simple working version that connects all the pieces. You get immediate feedback and can adjust your approach early.

Tracer Bullets in Automation

Section titled “Tracer Bullets in Automation”

Start with one complete test flow before building your entire framework. Get something working end-to-end quickly, then expand and refine.

This approach reveals problems early. You discover authentication issues, environment problems, or architectural flaws when it’s still easy to fix them—not after weeks of building on a broken foundation.

For example, don’t spend two weeks building a perfect page object framework before writing a single test. Instead, write one simple test that exercises the full stack, then extract patterns and build your framework based on real needs.

Example:

❌ Before (Big Design Up Front):

// Week 1-2: Build elaborate framework before any tests work


// Abstract base page class
class BasePage {
  constructor(page) { this.page = page; }
  async navigate(path) { /* implementation */ }
  async waitForLoad() { /* implementation */ }
  async takeScreenshot() { /* implementation */ }
  // ... 20 more methods nobody asked for
}


// Complex page object hierarchy
class LoginPage extends BasePage {
  // ... elaborate setup
}


class DashboardPage extends BasePage {
  // ... elaborate setup
}


// Sophisticated test data management
class TestDataFactory {
  // ... complex builder pattern
}


// Custom reporting system
class TestReporter {
  // ... over-engineered solution
}


// After 2 weeks: Still no tests running
// Problem: You built all this without knowing if it works
// or if you even need it

Problems:

  • Weeks of work before first test runs
  • Building features you might not need
  • No feedback on whether approach works
  • Hard to change direction once committed
  • Team can’t contribute until framework is “done”

✅ After (Tracer Bullet Approach):

// Day 1: Get ONE test working end-to-end


// Iteration 1: Simplest possible test
test('user can login and see dashboard', async ({ page }) => {
  // Hardcode everything - just make it work
  await page.goto('https://staging.myapp.com/login');
  await page.fill('[name="email"]', 'test@example.com');
  await page.fill('[name="password"]', 'password123');
  await page.click('button[type="submit"]');


  await expect(page.locator('h1')).toHaveText('Dashboard');
});
// ✅ Test passes - we have a tracer bullet!


// Iteration 2: Extract configuration (based on real need)
const config = {
  baseURL: process.env.BASE_URL || 'https://staging.myapp.com',
  testUser: { email: 'test@example.com', password: 'password123' }
};


test('user can login and see dashboard', async ({ page }) => {
  await page.goto(`${config.baseURL}/login`);
  await page.fill('[name="email"]', config.testUser.email);
  await page.fill('[name="password"]', config.testUser.password);
  await page.click('button[type="submit"]');


  await expect(page.locator('h1')).toHaveText('Dashboard');
});
// ✅ Test still passes - configuration works


// Iteration 3: Extract reusable login (when you need it in test #2)
async function loginAsUser(page, user) {
  await page.goto(`${config.baseURL}/login`);
  await page.fill('[name="email"]', user.email);
  await page.fill('[name="password"]', user.password);
  await page.click('button[type="submit"]');
}


test('user can login and see dashboard', async ({ page }) => {
  await loginAsUser(page, config.testUser);
  await expect(page.locator('h1')).toHaveText('Dashboard');
});
// ✅ Test still passes - helper function works


// Iteration 4: Add page object (when selectors get complex)
class LoginPage {
  constructor(page) {
    this.page = page;
    this.emailInput = page.locator('[name="email"]');
    this.passwordInput = page.locator('[name="password"]');
    this.submitButton = page.locator('button[type="submit"]');
  }


  async login(email, password) {
    await this.emailInput.fill(email);
    await this.passwordInput.fill(password);
    await this.submitButton.click();
  }
}


test('user can login and see dashboard', async ({ page }) => {
  await page.goto(`${config.baseURL}/login`);
  const loginPage = new LoginPage(page);
  await loginPage.login(config.testUser.email, config.testUser.password);
  await expect(page.locator('h1')).toHaveText('Dashboard');
});
// ✅ Test still passes - page object works

What Changed:

  • Working test on Day 1 instead of Week 3
  • Each iteration adds value based on real needs
  • Framework emerges from actual requirements
  • Team can contribute immediately
  • Easy to pivot if something doesn’t work

Impact: Framework development guided by real needs. Team gets value immediately. Risk reduced through continuous validation.

Practical Tracer Bullet Strategies

Section titled “Practical Tracer Bullet Strategies”

1. Start with Manual Steps

// First: Just get it working manually
test('checkout flow', async ({ page }) => {
  // Hardcode everything, inline all steps
  await page.goto('https://myapp.com/products');
  await page.click('text=Blue Shirt');
  await page.click('button:has-text("Add to Cart")');
  // ... continue until it works
});

2. Extract Patterns When You See Repetition

// Second test reveals pattern - NOW extract helper
test('checkout flow for shirt', async ({ page }) => {
  await addToCart(page, 'Blue Shirt');
  await checkout(page, paymentInfo);
});


test('checkout flow for shoes', async ({ page }) => {
  await addToCart(page, 'Red Shoes');
  await checkout(page, paymentInfo);
});

3. Build Infrastructure When You Hit Limits

// After 5 tests, you realize you need better data management
// NOW build test data system - because you understand the need
const testProducts = {
  shirt: { name: 'Blue Shirt', price: 29.99 },
  shoes: { name: 'Red Shoes', price: 79.99 }
};

4. Evolve Your Framework Organically

// Your framework grows naturally:
// Test 1-3: Inline code
// Test 4-6: Extract helpers
// Test 7-10: Add page objects
// Test 11+: Build data management
// Each step validated by working tests

Tracer Bullets vs Prototyping

Section titled “Tracer Bullets vs Prototyping”

Tracer Bullets:

  • Production code, kept and refined
  • End-to-end skeleton you build upon
  • Tests working from day one

Prototype:

  • Throwaway code, discarded after learning
  • Exploring unknowns before committing
  • Proof of concept, not production

Use tracer bullets for test automation. The code you write on Day 1 should evolve into your final framework, not be thrown away.

Principle 5: Design by Contract

Section titled “Principle 5: Design by Contract”

A contract defines responsibilities. Each function promises to do something specific if you give it the right inputs. The caller promises to provide valid inputs. Both sides honor the agreement.

In code, this means clear preconditions (what must be true before), postconditions (what will be true after), and invariants (what stays true throughout).

Design by Contract in Automation

Section titled “Design by Contract in Automation”

Write tests and utilities with clear expectations. Document what inputs are valid, what the function promises to do, and what it returns. Fail fast with clear error messages when contracts are violated.

This prevents confusion and debugging nightmares. When a test fails, you immediately know whether it’s a test bug, a framework bug, or an application bug.

Example:

❌ Before (No Contract):

// Vague function - what does it expect? What does it return?
async function createOrder(page, data) {
  await page.click('button');
  await page.fill('input', data);
  await page.click('.submit');
  return await page.textContent('.result');
}


// Usage is unclear and error-prone
test('create order', async ({ page }) => {
  // What should data be? A string? Object? Array?
  const result = await createOrder(page, someData);
  // What is result? Text? Object? Might be null?
  expect(result).toBe('???');
});


// When this fails, you have no idea why:
// - Invalid input?
// - Wrong page state?
// - Selector changed?
// - Function returned unexpected format?

Problems:

  • No clear expectations
  • Silent failures possible
  • Hard to debug
  • Easy to misuse
  • No guidance for users

✅ After (Clear Contract):

/**
 * Creates an order through the checkout flow
 *
 * Preconditions:
 * - User must be authenticated
 * - User must be on the checkout page
 * - Cart must contain at least one item
 *
 * @param {Page} page - Playwright page object
 * @param {Object} orderData - Order information
 * @param {string} orderData.paymentMethod - 'card' or 'paypal'
 * @param {string} orderData.shippingAddress - Full shipping address
 * @param {string} [orderData.promoCode] - Optional promo code
 *
 * @returns {Promise<Object>} Order confirmation
 * @returns {string} return.orderId - Unique order identifier
 * @returns {number} return.total - Order total in cents
 * @returns {string} return.status - Order status ('confirmed' or 'pending')
 *
 * @throws {Error} If user is not authenticated
 * @throws {Error} If cart is empty
 * @throws {Error} If payment method is invalid
 */
async function createOrder(page, orderData) {
  // Validate preconditions - fail fast with clear messages
  if (!orderData || typeof orderData !== 'object') {
    throw new Error('orderData must be an object');
  }


  if (!['card', 'paypal'].includes(orderData.paymentMethod)) {
    throw new Error(`Invalid payment method: ${orderData.paymentMethod}. Must be 'card' or 'paypal'`);
  }


  if (!orderData.shippingAddress || orderData.shippingAddress.length < 10) {
    throw new Error('shippingAddress must be at least 10 characters');
  }


  // Verify page state
  const currentUrl = page.url();
  if (!currentUrl.includes('/checkout')) {
    throw new Error(`Must be on checkout page. Current URL: ${currentUrl}`);
  }


  // Execute the action
  await page.selectOption('[name="payment"]', orderData.paymentMethod);
  await page.fill('[name="address"]', orderData.shippingAddress);


  if (orderData.promoCode) {
    await page.fill('[name="promo"]', orderData.promoCode);
  }


  await page.click('button:has-text("Place Order")');
  await page.waitForSelector('.order-confirmation');


  // Ensure postconditions - return value matches contract
  const orderId = await page.textContent('[data-testid="order-id"]');
  const totalText = await page.textContent('[data-testid="order-total"]');
  const status = await page.textContent('[data-testid="order-status"]');


  if (!orderId) {
    throw new Error('Order ID not found - order may have failed');
  }


  return {
    orderId,
    total: parseFloat(totalText.replace(/[^0-9.]/g, '')) * 100, // Convert to cents
    status: status.toLowerCase()
  };
}


// Usage is now clear and safe
test('create order with credit card', async ({ page }) => {
  // Contract tells us exactly what we need
  await loginAsUser(page, testUser);
  await addItemToCart(page, 'Blue Shirt');
  await page.goto('/checkout');


  const order = await createOrder(page, {
    paymentMethod: 'card',
    shippingAddress: '123 Main St, City, State 12345'
  });


  // Contract tells us exactly what we get back
  expect(order.orderId).toMatch(/^ORD-\d+$/);
  expect(order.total).toBeGreaterThan(0);
  expect(order.status).toBe('confirmed');
});


// Invalid usage fails immediately with clear message
test('create order fails with invalid payment', async ({ page }) => {
  await loginAsUser(page, testUser);
  await page.goto('/checkout');


  await expect(async () => {
    await createOrder(page, {
      paymentMethod: 'bitcoin', // Invalid!
      shippingAddress: '123 Main St, City, State 12345'
    });
  }).rejects.toThrow("Invalid payment method: bitcoin. Must be 'card' or 'paypal'");
});

What Changed:

  • Clear documentation of inputs and outputs
  • Explicit validation of preconditions
  • Descriptive error messages
  • Type information and examples
  • Guaranteed return format

Impact: Misuse caught immediately. Debugging time reduced by 80%. New team members understand APIs instantly.

Practical Contract Strategies

Section titled “Practical Contract Strategies”

1. Validate Inputs Early

function searchProducts(query, options = {}) {
  // Fail fast with specific error messages
  if (!query || typeof query !== 'string') {
    throw new Error('query must be a non-empty string');
  }


  if (options.maxResults && options.maxResults < 1) {
    throw new Error('maxResults must be at least 1');
  }


  // Now we can trust the inputs
  // ... implementation
}

2. Use TypeScript for Compile-Time Contracts

interface OrderData {
  paymentMethod: 'card' | 'paypal';
  shippingAddress: string;
  promoCode?: string;
}


interface OrderResult {
  orderId: string;
  total: number;
  status: 'confirmed' | 'pending';
}


async function createOrder(
  page: Page,
  orderData: OrderData
): Promise<OrderResult> {
  // TypeScript enforces the contract at compile time
  // ... implementation
}

3. Assert Postconditions Before Returning

async function getUserBalance(apiClient, userId) {
  const response = await apiClient.get(`/users/${userId}/balance`);


  // Verify the contract before returning
  if (response.status !== 200) {
    throw new Error(`Failed to get balance: HTTP ${response.status}`);
  }


  const balance = response.data.balance;


  if (typeof balance !== 'number') {
    throw new Error(`Invalid balance type: ${typeof balance}`);
  }


  if (balance < 0) {
    throw new Error(`Invalid balance: ${balance}. Balance cannot be negative`);
  }


  return balance; // Guaranteed to be valid number >= 0
}

4. Document Expected State

/**
 * Verifies user can access admin dashboard
 *
 * REQUIRES:
 * - User must have 'admin' role
 * - User must be authenticated
 * - Admin features must be enabled in environment
 *
 * ENSURES:
 * - Admin dashboard is visible
 * - Admin navigation menu is present
 * - User sees admin-only content
 */
test('admin can access dashboard', async ({ page }) => {
  // Test implementation with clear contract
});

Principle 6: Crash Early

Section titled “Principle 6: Crash Early”

When something goes wrong, fail immediately with a clear error message. Don’t limp along hoping things will work out. They won’t.

Silent failures are debugging nightmares. The error happens in one place, but symptoms appear somewhere else. You waste hours tracking down the root cause.

Crash Early in Automation

Section titled “Crash Early in Automation”

Tests should fail loudly when preconditions aren’t met, when unexpected errors occur, or when assumptions are violated. A test that fails with “Error: User must be authenticated” is infinitely better than one that fails with “Timeout: element not found.”

This principle transforms debugging from archaeology into instant diagnosis. You know exactly what went wrong and where.

Example:

❌ Before (Silent Failures):

async function updateUserProfile(page, profileData) {
  // No validation - just hope everything works
  await page.fill('#username', profileData.username);
  await page.fill('#email', profileData.email);
  await page.click('button[type="submit"]');


  // Assume success - don't verify anything
  return true;
}


test('update profile', async ({ page }) => {
  await page.goto('/profile');


  // profileData might be undefined - test will fail cryptically later
  const result = await updateUserProfile(page, undefined);


  // This will timeout with "element not found"
  // instead of failing with "profileData is undefined"
  await expect(page.locator('.success-message')).toBeVisible();
});


// Error message: "Timeout 30000ms exceeded waiting for selector .success-message"
// Real problem: profileData was undefined
// Time wasted: 30+ seconds waiting + debugging time

Problems:

  • Fails 30 seconds later at wrong location
  • Error message doesn’t explain root cause
  • Wastes time with timeouts
  • Hard to debug what actually went wrong
  • Symptoms appear far from cause

✅ After (Crash Early):

async function updateUserProfile(page, profileData) {
  // Validate immediately - fail fast with clear messages
  if (!profileData) {
    throw new Error('profileData is required but was undefined');
  }


  if (!profileData.username || profileData.username.length < 3) {
    throw new Error(`Invalid username: "${profileData.username}". Must be at least 3 characters`);
  }


  if (!profileData.email || !profileData.email.includes('@')) {
    throw new Error(`Invalid email: "${profileData.email}". Must be a valid email address`);
  }


  // Verify we're on the right page
  const currentUrl = page.url();
  if (!currentUrl.includes('/profile')) {
    throw new Error(`Cannot update profile - not on profile page. Current URL: ${currentUrl}`);
  }


  // Verify elements exist before interacting
  const usernameField = page.locator('#username');
  if (!(await usernameField.isVisible())) {
    throw new Error('Username field not found - page may have loaded incorrectly');
  }


  await usernameField.fill(profileData.username);
  await page.fill('#email', profileData.email);
  await page.click('button[type="submit"]');


  // Verify success explicitly
  const successMessage = page.locator('.success-message');
  try {
    await successMessage.waitFor({ timeout: 5000 });
  } catch (error) {
    // Check for error messages instead
    const errorMsg = await page.locator('.error-message').textContent().catch(() => 'none');
    throw new Error(`Profile update failed. Error message: ${errorMsg}`);
  }


  return true;
}


test('update profile', async ({ page }) => {
  await page.goto('/profile');


  // Now fails immediately with clear message
  await updateUserProfile(page, undefined);
  // Error: "profileData is required but was undefined"
  // Time to failure: <100ms
  // Debugging time: seconds instead of minutes
});


test('update profile with valid data', async ({ page }) => {
  await page.goto('/profile');


  const result = await updateUserProfile(page, {
    username: 'newuser',
    email: 'new@example.com'
  });


  expect(result).toBe(true);
});

What Changed:

  • Fails in <100ms instead of after 30s timeout
  • Error message explains exact problem
  • Validation at entry points
  • Explicit verification of assumptions
  • No silent failures

Impact: Debugging time reduced by 90%. Test failures pinpoint exact problems. CI/CD feedback loops become instant.

Practical Crash Early Strategies

Section titled “Practical Crash Early Strategies”

1. Validate Environment Setup

// At the start of test suite
if (!process.env.API_KEY) {
  throw new Error('API_KEY environment variable is required but not set');
}


if (!process.env.BASE_URL) {
  throw new Error('BASE_URL environment variable is required but not set');
}

2. Check Prerequisites Before Tests

test.beforeEach(async ({ page }) => {
  await page.goto('/');


  // Verify app loaded correctly
  const title = await page.title();
  if (!title) {
    throw new Error('Page title is empty - application may have failed to load');
  }


  // Verify required cookies/auth
  const cookies = await page.context().cookies();
  if (cookies.length === 0) {
    throw new Error('No cookies found - user may not be authenticated');
  }
});

3. Wrap API Calls with Clear Error Handling

async function apiRequest(endpoint, options) {
  let response;


  try {
    response = await fetch(endpoint, options);
  } catch (error) {
    throw new Error(`Network request failed for ${endpoint}: ${error.message}`);
  }


  if (!response.ok) {
    const body = await response.text();
    throw new Error(
      `API request failed: ${response.status} ${response.statusText}\n` +
      `Endpoint: ${endpoint}\n` +
      `Response: ${body}`
    );
  }


  return response.json();
}

4. Assert Intermediate States

test('multi-step checkout', async ({ page }) => {
  // Step 1
  await page.click('text=Add to Cart');
  const cartCount = await page.locator('.cart-count').textContent();
  if (cartCount === '0') {
    throw new Error('Cart count is still 0 after adding item');
  }


  // Step 2
  await page.goto('/checkout');
  const checkoutTitle = await page.locator('h1').textContent();
  if (!checkoutTitle.includes('Checkout')) {
    throw new Error(`Expected checkout page but got: "${checkoutTitle}"`);
  }


  // Continue with confidence that each step worked
});

5. Use Assertions Liberally

// Don't just hope - verify!
const user = await createTestUser();
expect(user).toBeDefined(); // Crash if undefined
expect(user.id).toBeTruthy(); // Crash if missing ID


await loginAsUser(page, user);
const isLoggedIn = await page.locator('[data-testid="user-menu"]').isVisible();
expect(isLoggedIn).toBe(true); // Crash if login failed


// Now proceed knowing everything is valid