Skip to main content

Integration Tests

Our end-to-end (E2E) integration tests, powered by Playwright, are designed to validate the "happy path" of our core user-facing features. These tests simulate real user interactions in a browser, providing confidence that our applications work as expected from the frontend to the database. They focus on complete user flows rather than exhaustive edge cases, which are covered by our API tests.

The Foundation: Consistent Test Data

Reliable E2E tests require a consistent and predictable database state. We achieve this through a seeding process that populates a test database before the suite runs.

Seed Generation (seeds.rb)

The db/seeds.rb script is the source of truth for all test data. It uses our Rails factories to create a comprehensive set of records, including users, brands, packages, and orders in various states. This script serves a dual purpose:

  1. Populating the restaurant demo environment with synthetic data that is also realistic and predictable.
  2. Generating a seeds.json file for our Playwright tests to consume.

At the end of the seeding process, a database dump is created, which acts as a fast restore point when iterating on tests.

# ...

result = Seeds.new.tap(&:run)

File.write('e2e/state/seeds.json', JSON.pretty_generate(result.to_h))
%x[mysqldump -u #{ENV['DB_USERNAME']} ... > ./tmp/2-seeds.sql] unless ENV['IS_RUNNING_DOCKER']

Decoupling Tests with seeds.json

To avoid writing brittle tests that rely on hardcoded database IDs, the seeds.rb script exports key objects into a seeds.json file. The Seeds::OBJECTS constant in the script declares which variables are included in this export.

class Seeds
  OBJECTS = %i[
    admin brand main_brand_menu main_brand_menu_review menu_item menu_review_item fmo_brand
    # ... more objects
  ].freeze
  # ...
end

This JSON file is then imported directly into our Playwright tests, allowing us to reference data contextually, such as SEED_STATE.brand.id.

{
  "brand": {
    "id": 1,
    "name": "True Foods",
    "url": "novacaters.com",
    // ... more brand data
  }
  // ... more seeded data
}

The "Setup DB" Test

When developing features, seed data may need to be updated. After running the seeds.rb script and generating a new dump, our test suite can be quickly reset to this known-good state. This is handled by a dedicated "setup db" test defined in our Playwright configuration, which restores the database from the dump before the main test suite runs.

Writing a Playwright Test

Our Playwright tests are written in TypeScript and follow a clear, readable structure that describes a user's actions and expectations. The deals.spec.ts file provides a great example of testing the full lifecycle of a feature.

The test below demonstrates how a caterer can create a new deal. It uses the brand data from seeds.json to navigate to the correct URL and then programmatically fills out the form, submits it, and asserts that the correct notification appears.

import { expect, test } from "@playwright/test";
import SEED_STATE from "@state/seeds.json";
// ...

// Serial so we can hide the same deal that we create
test.describe.serial("Caterer - Deals", () => {
  // ...
  const couponCode = "WELCOME2024";

  test("create a new deal", async ({ page }) => {
    const { brand } = SEED_STATE;

    await page.goto(`/caterer/${brand.id}/deals`);

    await expect(page.getByRole("heading", { name: "Deals" })).toBeVisible();

    await page.getByTestId("add-deal").click();

    await page.getByTestId("discount-percent").fill("10");
    await page.getByTestId("deal-code").fill(couponCode);
    await page.getByTestId("free-delivery").click();
    await page.getByTestId("limited_to_first_time_customers").click();

    await page.getByTestId("require_order_minimum").click();
    await page.getByTestId("minimum-order-size").fill("100");
    await page.getByTestId("create-deal").click();

    await expectNotification(page, `Deal ${couponCode} successfully created.`);

    // Now verify the deal shows up on their profile
    await page.goto(`/catering/${brand.id}`);

    await gotoAndAwait(page, `/caterer/${brand.id}/deals`, "/caterer/coupons");
    await expect(page.getByRole('heading', { name: 'WELCOME2024' })).toBeVisible()
  });
});

Because the tests in this file are marked as test.describe.serial, they run sequentially, allowing the update a deal and delete a deal tests to operate on the exact same deal created in the first test.

Running in CI and Debugging Failures

Our E2E tests are an integral part of our Continuous Integration (CI) pipeline.

Test Environment

To closely mirror our production environment, we use Docker Compose to orchestrate all the necessary services, including our Rails API and Nuxt frontend applications. We also utilize Caddy as a reverse proxy to replicate production routing schemes.

CI Triggers

Tests are automatically executed in our CI environment on every push to the develop branch or to any feature branch that includes e2e in its name.

Debugging with Traces and Videos

When a test fails in CI, debugging is made easy with two key artifacts:

  • Video Recordings: On the first retry of a failed test, Playwright records a full video of the test execution.
  • Playwright Traces: For every failure, a detailed trace file is generated. This trace captures a complete DOM snapshot, network requests, console logs, and actions for every step of the test. It's an invaluable tool for diagnosing issues without needing to re-run the test locally. Traces can be easily viewed by uploading the generated .zip file to the Playwright Trace Viewer.

Below you can find an example .webm video recording and a trace.zip file from a failed test run.

trace.zip

Contributors: Nate Mortensen