← Back to Blog

Should You Mock the Mailer in Email OTP E2E Tests?

Mock sendEmail for unit speed; use a real catcher or disposable inbox for E2E. When page.route helps, when it lies, and how OTPBox fits CI.

testingotpe2eci-cdq-and-a
Should You Mock the Mailer in Email OTP E2E Tests?

Mock the mailer in unit and integration tests. Do not call that suite "E2E" for email OTP. End-to-end coverage needs a real catcher — Mailpit when you own SMTP, or a disposable inbox like OTPBox when CI must receive a public delivery.

This Q&A separates spy-based mail tests from delivery-path tests. Claims below match Playwright's network mocking docs and the live OTPBox API. No invented competitor benchmarks.

What does "mocking the mailer" mean?

Teams usually mean one of three things:

  1. Stub sendEmail() in the app process — the function records arguments (to, subject, template id) and returns success without talking to SES, SendGrid, Postmark, or SMTP.
  2. Inject a fake mail module in unit tests — same idea, often with Jest/Vitest spies.
  3. page.route in Playwright — fulfill or rewrite browser HTTP(S) calls from the page so the UI never hits a real API.

All three are valid for call-site coverage. None of them prove that a verification email was rendered, accepted by a provider, or readable as a code in an inbox.

When is mocking the right move?

Mock (or spy) when you want fast, deterministic checks:

  • Controllers call the mailer with the right recipient and template after signup
  • Rate-limit or "already verified" branches skip the send
  • Payload builders put a six-digit code or magic-link token in the message body string before handoff

Those tests belong next to the mail module. They should not open browsers, wait on SMTP, or burn CI create quotas. Keep them green on every PR.

What mocking is not for: claiming the production email path works because a spy returned undefined.

Why does mocking fail as E2E for email OTP?

Email OTP E2E has extra failure modes a mock never sees:

  • Wrong From / Reply-To or broken provider API keys
  • Template render bugs (missing {{code}}, HTML-only body your extractor misses)
  • Provider deferrals and spam filtering
  • Inbox isolation under parallel workers
  • Polling and timeouts when delivery is eventual

A green mock only proves intent to send. The flake catalog in 7 Mistakes That Make Email OTP Tests Flake in CI is almost entirely about real inboxes, shared addresses, and fixed sleeps — not about spies.

Hard-coded OTP bypasses in production ("if email ends with @test.com, accept 000000") are a different smell: they skip delivery and weaken auth if the bypass ships. Prefer a disposable inbox in test env over a permanent back door.

Does Playwright page.route mock email delivery?

No — not for server-side mail. Playwright's page.route intercepts requests the page makes (XHR, fetch, navigations). It can mock your frontend's /api/signup response so the UI shows "check your email" without a backend. It cannot see SMTP, provider REST calls from the API process, or messages landing in an inbox.

Use page.route when you are testing UI behavior given a known API response. Use a catcher or disposable inbox when you are testing the path from "submit signup" to "I hold a real code." Component tests that never start a mail stack do not need OTPBox either; reserve hosted inboxes for full E2E.

What should E2E use instead?

Pick the layer you are proving:

Goal Tool
Fast unit/integration of "we called send" Mock / spy sendEmail
Local or Compose with redirectable SMTP Mailpit (SMTP 1025, UI/API 8025)
Cloud CI that must receive a real code without an SMTP sidecar OTPBox (*@box.otpbox.app)

Mailpit wins when the app under test can point SMTP_HOST at a sidecar. OTPBox wins when the app always sends through a real provider to a normal address. The honest split is written up in Mailpit vs OTPBox for CI Verification Codes. Wiring for Playwright lives in the Ultimate Guide to Email OTP Testing with Playwright and How Playwright expect.poll Beats Fixed Sleeps for Email OTP.

Mock vs Mailpit vs OTPBox

Dimension Mock / spy Mailpit OTPBox
Proves send was invoked Yes Yes (captured) Yes (delivered)
Proves public deliverability No No Yes
Needs SMTP config change No Yes No
Parallel isolation N/A Filters / wipe / per-job service One inbox per test
OTP extraction You invent the code DIY from body otp / link on /latest
Best for Unit / integration Laptop + Compose GitHub Actions E2E

Qualitative only — no fake latency numbers.

How do I wire OTPBox without inventing bypass codes?

From the docs: Free tier is 10 inbox creates/hour per IP with about 1h TTL; Pro is $9/mo with an API key for 120 creates/hour and 6h TTL. Polling /latest is unrestricted on both. Product addresses are *@box.otpbox.app.

Pattern: create once, trigger the real send, poll until a code appears.

import { test, expect } from "@playwright/test";

async function createInbox(apiKey?: string) {
  const headers: Record<string, string> = {};
  if (apiKey) headers.Authorization = `Bearer ${apiKey}`;
  const res = await fetch("https://otpbox.app/api/inbox", {
    method: "POST",
    headers,
  });
  if (!res.ok) throw new Error(`OTPBox create failed: ${res.status}`);
  return (await res.json()) as { email: string; token: string };
}

test("signup verifies a real email OTP", async ({ page }) => {
  test.setTimeout(90_000);

  const { email, token } = await createInbox(process.env.OTPBOX_API_KEY);
  console.log(`OTPBox inbox: ${email}`);

  await page.goto("/signup");
  await page.getByLabel("Email").fill(email);
  await page.getByLabel("Password").fill("correct-horse-battery");
  await page.getByRole("button", { name: "Create account" }).click();
  await expect(page.getByText(/check your email|enter the code/i)).toBeVisible();

  const otp = await expect
    .poll(
      async () => {
        const res = await fetch(`https://otpbox.app/api/inbox/${token}/latest`);
        if (!res.ok) return null;
        const data = (await res.json()) as { otp?: string | null };
        return data.otp ?? null;
      },
      {
        message: "waiting for OTPBox /latest otp",
        timeout: 60_000,
        intervals: [2_000, 3_000],
      }
    )
    .not.toBeNull();

  await page.getByLabel("Verification code").fill(otp as string);
  await page.getByRole("button", { name: "Verify" }).click();
  await expect(page).toHaveURL(/\/(dashboard|home|app)/);
});

Create with optional OTPBOX_API_KEY (Bearer). Poll without the key. Do not create a new inbox inside the poll loop — that burns the create quota. Shell and Actions examples: How to Test Email OTP Verification in CI and /docs.

Checklist

  • Unit/integration: mock or spy the mailer; assert recipients and templates
  • E2E: catch real mail (Mailpit or OTPBox); no production OTP bypass
  • Do not treat page.route as SMTP coverage
  • One inbox (or filtered Mailpit message) per parallel worker
  • Create OTPBox once; poll /latest with a deadline (expect.poll)
  • Log the disposable address on failure for provider traces

Related reading

Resources

Mock the mailer when you are testing whether you called send. Catch real mail when you are testing whether a human could enter the code. Keep both layers — and keep their names honest.