E2E tests¶

End-to-end (E2E) tests check that your workspace works the way your users use it. A test creates the records that it needs, opens a real browser, signs in as one of those records, and goes through a flow: it opens pages, fills in forms, and checks what appears on screen. Then it can check the records that the flow changed.

Tests never change your live data. Each run gets its own test branch: a copy of your workspace's configuration with no records except your own account and the records of your static resources. The tests act on that branch.

Tests are part of your workspace. They are stored with it, copied into branches, and included in workspace exports. A package can include test suites, so a workspace that installs the package can run the tests that the package author wrote.

System Administrators find E2E tests under Settings → Tests.

Concepts¶

A suite is a named group of tests. You run a whole suite, or one test of it.
A test has three parts: an optional setup script, the body of a Playwright test, and an optional assert script. Tests in a suite run one after another, in the order you set.
A run is one execution of a suite or of one test. It records the result, the error, and the console output of each test, a video of each test, and a screenshot and trace of each failed test.
A test branch is the branch that a run acts on. Diagonal creates it when the run starts and discards it when the run passes.
A runner is the service that executes runs in a browser. Diagonal operates it. You do not install or register a runner.

Create a suite¶

Open Settings → Tests and select New suite….
Enter a name and a description.
Select Add test… and enter a name.
Add an optional description to explain what the test checks and why it matters.
Write the test code on the Test tab. Add a setup script on the Setup tab and an assert script on the Assert tab when the test needs them.

Test names, descriptions, code, scripts, and settings save automatically.

Work with the assistant¶

Select Ask AI beside a test to prepare a message with that test attached. You can ask the assistant to write a new test, explain the code, or improve its coverage. Edit the message and send it when it is ready. Pending test changes save before the message is sent.

On a failed result, select Diagnose to ask about that specific run and test. The assistant can read the code that the run executed, even if the test has changed since then. Review its diagnosis before asking it to apply a fix or start another run.

Write a test¶

A test runs in three steps:

The setup script creates the records that the test needs and returns them.
The test code opens a browser, signs in as one of those records, and goes through the flow.
The assert script reads the records that the flow changed and fails the test when a value is wrong.

The setup and assert scripts are optional.

Setup script¶

The setup script is a workflow script, written as in a script workflow. Diagonal runs it in the run's branch before the test code. It acts as the person who started the run. Return the records that the test needs in an object:

import { defineWorkflow, data } from '@diagonal/workflow';
import { resources } from '@diagonal/workflow/workspace';

export default defineWorkflow(async () => {
  const member = await data.create({
    resource: resources.Member,
    data: { name: 'Ada Lovelace', email: 'ada@example.com' }
  });
  const event = await data.create({
    resource: resources.Event,
    data: { name: 'Annual dinner' }
  });
  return { member, event };
});

The run's branch has no records except your account and the records of your static resources. A static resource holds fixed records that your pages and workflows expect, such as statuses or categories. To mark a resource as static, turn on Static resource in the resource settings. The setup script must create the other records that the test needs.

Calling workflow.run from a setup or assert script requires manage_workflows or System Administrator, in addition to the permission to run tests.

A setup or assert script can use the data functions, such as data.create, data.find, data.search, and data.update, the date, time, timezone, money, number, and phone helpers, and workflow.run. Pass a workflow slug as workflow when possible; it is portable across branches. You can pass a workflow ID instead. The value can also be { id, params? }. You can pass input, runAsResource, runAsRecord, and executionMode. The @diagonal/workflow/workspace module exposes resource IDs, not workflow IDs, so prefer the slug. A production workflow's slug or ID runs its copy in the test-run branch. The child runs its published version in the run's target workspace as the acting account, unless the child or the call sets Run As. A workflow.run call that sets Run As follows the record sign-in rule. An account that holds impersonate_identities may run as any record that can sign in. Any other account may run as itself or as a record its setup script created whose permissions it holds. Its steps run as normal workflow steps and can use the capabilities that workflow supports. Direct calls from the test script remain limited to data.*, workflow.run, and pure helpers. Writes go through the normal record rules, so validation and access policies apply.

A synchronous child run returns its output. The setup script's synchronous child records count as created by the setup script for Sdk.loginAs. An asynchronous child run is queued and returns its pending execution event. Its records do not count as setup-created records.

Test code¶

The test code is the body of an async function. These values are in scope:

Name	What it is
`page`	A Playwright page. Relative paths, such as `page.goto('/events')`, open on this workspace.
`expect`	Playwright assertions.
`context`, `browser`, `request`	The Playwright browser context, browser, and API request context.
`Sdk`	Diagonal's test helpers. The rows below list its members.
`Sdk.setup`	The object that the setup script returned. It is `{}` when the test has no setup script.
`Sdk.resources`	The resources of the run's workspace by script name, such as `Sdk.resources.SalesOrders` for Sales Orders.
`Sdk.loginAs(record, { redirectTo })`	Signs the page in as a record and opens `redirectTo` (default `/`). The record must be of a resource whose records can sign in.
`Sdk.newPageAs(record, { redirectTo })`	Opens a second browser, signed in as another record, and returns its page.
`Sdk.url(path)`	An absolute URL on the run's workspace.
`Sdk.workspace`	The `id`, `name`, and `baseUrl` of the run's workspace.
`Sdk.date`, `Sdk.time`, `Sdk.timezone`	The date, date-time, and timezone helpers of the workflow script SDK, such as `await Sdk.date.add({ date, value: 7, unit: 'days' })`.
`Sdk.money`, `Sdk.number`, `Sdk.phone`	Format a money value, a number, or a phone number as the platform does.

Sign in as a record that the setup script returned:

await Sdk.loginAs(Sdk.setup.member, { redirectTo: '/events' });
await page.getByRole('link', { name: 'Annual dinner' }).click();
await page.getByRole('button', { name: 'Register' }).click();
await expect(page.getByText('You are registered')).toBeVisible();

Sdk.loginAs and Sdk.newPageAs take a record that the setup script returned. For another record, pass { resource: Sdk.resources.Member, id }. A text key such as 'admin' fails the test. The acting account must hold manage_tests unless it is a System Administrator. An account that holds impersonate_identities can sign in as any record that can sign in. Any other account, including one with only System Administrator access, can sign in only as itself or as a record the setup script created whose permissions it holds.

A test that needs two users opens a second page. The setup script returns both records:

await Sdk.loginAs(Sdk.setup.member, { redirectTo: '/requests/new' });
await page.getByLabel('Subject').fill('Parking permit');
await page.getByRole('button', { name: 'Submit' }).click();

const staffPage = await Sdk.newPageAs(Sdk.setup.staff, {
  redirectTo: '/admin'
});
await expect(staffPage.getByText('Parking permit')).toBeVisible();

The test code cannot import modules, and it cannot create or read records. Each test has a default time limit of 60 seconds. The setup and assert scripts count toward it. Set Timeout on the test to change it, up to 600 seconds.

Prefer locators that users see, such as roles, labels, and text. The tests of one run share one branch and run in order. Make each test create the data that it needs.

Assert script¶

The assert script runs after the test code passes. Its input is the object that the setup script returned. Read the records and call fail() when a value is wrong. The test then fails with your message:

import { defineWorkflow, data, fail } from '@diagonal/workflow';
import { resources } from '@diagonal/workflow/workspace';

export default defineWorkflow<{ member: { id: string } }>(async ({ member }) => {
  const saved = await data.find({
    resource: resources.Member,
    mode: 'record',
    record: member.id
  });
  if (saved.status !== 'registered') fail(`Expected registered, found ${saved.status}.`);
});

When the test code fails, the assert script does not run. Lines that a script writes with console.log appear in the test's console output with a [setup] or [assert] prefix.

Run a suite¶

Select Run full suite on the suite page. To run one test, select Run test on that test. A suite has one active run at a time.

A run goes through these steps:

Preparing branch: Diagonal creates the run's test branch. The branch gets the workspace configuration, your account with its roles, and every record of your static resources. It gets no other records and no files. Because the branch copies no file binaries, media-library images on your pages may not load during a test.
Queued: the run waits until the runner accepts it.
Running: the run page shows each test's result as it finishes.

The tests act as the person who started the run, in the run's branch. The account must hold manage_tests to start a run. System Administrators also pass.

A run that you start inside a branch acts on that branch. It creates no test branch, and it does not discard the branch at the end.

The run page links to the run's branch with Open branch. When a run passes, Diagonal discards its branch. When a run ends before a runner starts it, Diagonal also discards its branch, because the tests created no data. When a run that started fails, errors, or is cancelled, the branch stays, so you can open it and inspect the data that the tests created. Discard the branch from its branch page when you are done. Test branches do not show in Settings → Branches.

A run of one test does not count for the package publish check. See Tests in packages.

A run ends with one of these statuses:

Status	Meaning
Passed	Every test passed or was skipped.
Failed	At least one test failed, timed out, or did not finish.
Errored	Diagonal could not prepare the branch, the run's account cannot run tests, the runner could not run the suite or stopped responding, or the runner did not accept the run within 30 minutes.
Cancelled	An administrator cancelled the run.

Open a test to watch its video. The video shows the main browser page of the test from start to end. A page from Sdk.newPageAs is not recorded. Open a failed test to see the error, the line in your test code, the console output, and a screenshot. A setup or assert script that fails shows The setup script failed: or The assert script failed: and its message. Download the trace and open it at trace.playwright.dev to step through the test.

Test execution¶

Diagonal prepares a test branch and submits each run to its browser execution service. The service receives the workspace URL, tests, and short-lived API credentials for that run. It reports results and uploads videos, screenshots, and traces back to Diagonal.

Diagonal runs the setup and assert scripts of each test when the runner asks for them. Sdk.loginAs() and the other helpers call Diagonal's API while the test runs. Test data stays in the run's branch.

If test execution is unavailable, contact your Diagonal administrator.

Tests in packages¶

A package release can include test suites. The release carries the suites and their tests. Tests create their own data, so an installed suite needs no setup. After you install a package with tests, open the suite under Settings → Tests and run it.

A package author can publish a release that includes a suite only when the suite's latest run of the whole suite passed and ran the tests that the release contains. A run of one test does not count. Run the suite, fix any failures, build the release, and then publish. When you change a test after you build the release, run the suite and build the release again. A release that contains a suite that you deleted cannot be published.

The package author's policy decides what you can change. You can always rename a test, change its description, and switch it off. Package updates change the tests the same way they change other package content: when you and the author both changed the code or a script of a test, the update review offers to merge the two versions.

Branches and exports¶

A branch copies the workspace's suites and tests. A run that you start in a branch acts on the branch's own data. You can edit a test in a branch, run it there, and merge the change. A workspace export includes the same content. Run history is not copied or exported.