logo
Guides

Browser-based tests, with representative stacks

Nik Zherebtsov
Nik Zherebtsov

Why is it still so hard to set up an environment for end-to-end (e2e) Web tests? It is easy if you have only a single server with no dependencies, but it is rarely the case. How do you ensure the testing environment is isolated, consistent with production, and deterministic?

Enter Namespace: the cloud-native application platform that can orchestrate a stack of servers consistently across development, testing, and production in Kubernetes. In the last blog post, we talked about reproducible environments, and now we'll demonstrate how to use them for Web testing.

Browser-based testing today

Different teams use different strategies today when writing browser-based tests.

  • Some teams use a set of pre-configured environments on dedicated machines and start sweating if the configuration changes or when you add a new machine.

    Lack of isolation and repeatability makes such an approach challenging to maintain.

  • Others create separate configurations for tests in addition to the ones for development and production.

    Supporting these configurations is expensive in terms of computing resources and human time costs. A common mitigation is to mock some of the backend and database dependencies. However, changes to the mocked-out dependencies may go unnoticed.

Something that works best is to automate the creation of an ephemeral testing environment. The server stack, including its dependencies, is brought to life for the test duration and then shut down after it is over.

Ideally, the server's dependencies under test should match the ones in production (potentially with a test-specific configuration like database seeding).

Let's explore how we can achieve that goal with Namespace.

Browser-based testing with Cypress

Cypress is one of the most popular web test automation frameworks. We’ll use it to drive our test-cases, while Namespace drives the application stack under test.

A similar setup would work for Playwright or other frameworks, too.

Example Stack

For demonstration purposes, we create a simple server stack:

  • An API backend that takes a name from the request and responds with Hello, ${name}.
  • A Web frontend that calls the backend and displays the response as HTML:

You can find the full example here.

Install Cypress

Let's create a separate package to host Cypress tests and install Cypress there as we normally would:

cd tests/browser
npm install cypress

We’ll keep our tests in a separate directory, away from other configurations. This gives us some confidence that their configuration is isolated. Namespace makes it trivial to work with different servers and tests across different directories, whether in the same repository or different ones (we call them packages).

Maintaining your test server dependencies under a single top-level package.json is thus not necessary.

Write a test

Following the Cypress guide, you create a new directory cypress/e2e under tests/browser, where we’ll host the test code below. The only difference from the official guide is that the URL of the server to test comes from an environment variable; see the next section.

describe("My First Test", () => {
	it("Smoke", () => {
		// Visit the home page.
		// The environment variable is injected by Namespace.
		cy.visit("http://" + Cypress.env("ROOT_HOST"));
 
		// Verify that the page contains the expected text.
		cy.contains("Hello, world!");
	});
});

Configure and run tests

Now we need to configure Namespace to run this code via ns test. We use a simple Dockerfile to instruct Namespace on how to build the test. In a later blog post, we'll discuss improving the build performance and reducing boilerplate by replacing the manually managed Dockerfile with Namespace-built integrations.

Our test.cue looks like this:

tests: {
  mycypresstest: {
    integration: "dockerfile"
    env: {
	  // Injecting the address of the server under test as an environment variable.
	  // CYPRESS_XYZ can be accessed with Cypress.env("XYZ") from the test.
      CYPRESS_ROOT_HOST: fromServiceEndpoint: "namespacelabs.dev/example-cypress/web:webservice"
    }
    serversUnderTest: [
      // Starting our server before the test is run.
      "namespacelabs.dev/example-cypress/web",
    ]
  }
}

We confirmed with ns test that our test passed, but what happened under the hood?

  • Validated test configuration: Namespace confirmed that it could run the test. In this case, it ensured that WebFrontend is a direct dependency of the test and that it exposes the required endpoint so that Namespace can inject it.

  • Resolved dependencies: The test depends on WebFrontend, which Namespace resolved by adding the server and its dependencies (in this case, another server: ApiBackend) to the deployment plan.

  • Produced a DAG that represents the deployment plan: From the resulting dependency graph, Namespace instantiated an execution plan, which includes a compiled list of Kubernetes resources required to support the servers above.

    • An ephemeral Namespace to host the test resources, managed by our scheduler.
    • A Pod for the ApiBackend server and a Service that points at its endpoint.
    • A Pod for the WebFrontend server and a Service that points at its endpoint.
    • A Pod for the Cypress test.

    Note: As you can see, Namespace maps each server to a Pod. Since we are running a test, these Pods have a restart policy never to not hide flakiness/crash bugs. If we were to deploy the stack to a non-test environment, Namespace would allow the Pods to restart and manage them with Deployments.

  • Deployed the plan to Kubernetes: Namespace's scheduler executed this plan respecting the dependency order:

    • First, it created the new Kubernetes namespace.
    • Then, it deployed the ApiBackend server to that namespace.
    • After ApiBackend was ready, the scheduler deployed the WebFrontend server.
    • After the WebFrontend was ready, the scheduler deployed the Cypress test. The deployment order guarantees that the test driver can use the injected endpoint immediately.
    • Finally, the scheduler deletes the Kubernetes namespace.

Iterating on tests

ns test is great to verify that everything is working, as a part of a CI pipeline or locally. But for development, we want to run the test runner locally against a Namespace-managed stack; to utilize the GUI (if available), live reload and debugging capabilities of the testing framework.

All we have to do is to start the server stack with ns dev, and then run Cypress in a separate terminal from the test directory with CYPRESS_ROOT_HOST set to the generated URL of the server that ns dev prints. This URL is stable and doesn't change over time, unless the server is renamed or moved to a different package.

Example:

ns dev web
 
…
 
   Development mode, services forwarded to 127.0.0.1:
 
    [✓] WebFrontend/webservice
        http://web-frontend-sc0bslb9heckkakk.dev.nslocal.host:40080/
# In a separate terminal, pass the server address as the environment variable.
CYPRESS_ROOT_HOST=web-frontend-sc0bslb9heckkakk.dev.nslocal.host:40080 npx cypress open

For iterations, the test runner watches the source code of the tests and re-runs them when needed. ns dev watches the code of our server stack under test and keeps it up-to-date.

What to do next

Share this story on