Data Privacy in Test Automation: Keep Sensitive Data On-Premise

What an end-to-end test sees

Data privacy in test automation is the practice of knowing, and limiting, where the data your tests touch ends up.It matters most for end-to-end tests, because they work through your application's screens.

An end-to-end test drives your application the way a user does.So it sees everything a user sees: customer names, account numbers, patient records, prices, addresses and the internal tools your staff use every day.

And it keeps evidence: a screenshot of each step, a video of the run, logs, and more and more often the prompts sent to an AI model.That evidence makes a failure quick to understand.It is also a copy of your data, made every time the tests run.

Where test data goes: screenshots, videos and AI prompts

Every testing tool makes copies of what it sees.The question is where they are kept, for how long, and who can open them.

Open source frameworks keep their evidence on the machine that ran the test, unless you upload it somewhere.A Playwright trace records a screencast, DOM snapshots of every action, the console and every network request, and its videos are written to the test output folder.A trace of a login form holds whatever that form showed.

Selenium Grid runs on machines you set up yourself, so its files stay where you put them.The cost is the time to build, run and maintain that grid.

Cloud grids such as BrowserStack, Sauce Labs and TestMu AI (formerly LambdaTest) run your tests on their own browsers and devices.BrowserStack's documentation lists session logs, video, screenshots and Selenium logs as places where sensitive data may appear, and documents settings to mask values and turn recordings off.Sauce Labs records a video of every test by default, and documents settings to turn the video, the step screenshots and the logs off.

Visual testing services compare screenshots on a server.Applitools Eyes, for example, sends each screenshot to its Eyes server, which can be its public cloud, a dedicated cloud or an installation in your own network.

AI features add one more path.Self-healing, test generation and visual review often send page content, screenshots or both to a cloud model.

Data privacy in test automation: GDPR, HIPAA, PCI DSS and SOC 2

This is general information, not legal advice.Ask your data protection officer or your counsel about your own case.

GDPR

Under the GDPR, any information about an identifiable person is personal data, and PII in screenshots is no exception.The rules apply to test environments as much as to production.A testing service that stores your screenshots processes personal data on your behalf, which calls for a data processing agreement.If it keeps them outside the European Economic Area, the transfer rules apply too.Data minimization means keeping only what you need, for as long as you need it.

HIPAA

Under HIPAA, a service that creates, receives, maintains or transmits protected health information for a covered entity is a business associate, and needs a business associate agreement.The US Department of Health and Human Services says this holds for a cloud service even when it stores the data encrypted without the key.A testing service that keeps recordings of patient records can fall under that rule.

PCI DSS

PCI DSS v4.0 says live card numbers are not used in pre-production environments, unless those environments are protected as part of the cardholder data environment.Test with your payment provider's test card numbers, and a screenshot of a payment form never shows a real one.

SOC 2

SOC 2 is an auditor's report on a service organization's controls, not a certificate.When a testing tool stores your screenshots, it becomes one more vendor your own audit has to account for.Fewer places that hold copies means fewer vendors to review.

Test data management: synthetic data and masking

The safest screenshot is one that shows nothing real.

A data privacy checklist for test automation tools

Before you choose a tool, or before you turn on a new feature of the one you have, ask five questions.

1. 1Where are screenshots, videos and logs stored, and in which country?
2. 2How long are they kept, and can we delete them ourselves?
3. 3What is sent to an AI model, to which provider, and under whose account?
4. 4Can tests, runs and reports stay entirely on our own computers?
5. 5What does the tool still need from the internet, and what does that connection carry?

For a side-by-side look at how today's end-to-end testing tools work, see the comparison study.

How on-premise test automation keeps your screens in-house

On-premise test automation means the testing tool and everything it produces run on computers you control.GUI Test Studio works that way.

You record an end-to-end test in GUI Test Runner-Recorder, or describe it in a prompt, and you edit it in GUI Test Studio on a visual timeline.A test is one file, saved in a folder of its own, wherever you choose.

Every run is kept in the results folder inside the test's own folder: its video, its Confidence Report and its history entry.No test, recording, screenshot or result is ever sent to us.

The GTS Engine + AI finds and checks every element on your computer.It asks the application where an element is, through its accessibility information, and finds it by sight when the application says nothing.Text checks read the words on the screen with text recognition that is installed with the suite, so no outside service reads your screens.How the engine finds every element.

In a CI/CD pipeline, gtr-cli runs the same tests on your own build agents and writes every artifact into a folder you choose.See CI/CD.

When a test fails, a bug report is a PDF or a workbook written to your disk.If you connect Jira, the issue is created on your own Jira site, straight from your computer: nothing passes through our servers.

Local AI test automation: AI checks that stay on your computer

AI Smart Verification is the optional AI side of the GTS Engine + AI.When a check fails, it takes a second look, says in plain words what changed, and gives the final verdict of that check.It is off until you turn it on, and you choose where it runs.

With a cloud provider, a check that passes is never sent.A failed visual check sends two pictures: the element as it was expected, and as it was found.A failed text check sends the part of the screen it read and the words it expected.Any AI Instructions you wrote for that check travel with them.Your API key is stored encrypted, for your Windows account only.

A local AI model, a local LLM, is downloaded once, when you ask for it, from its publisher.Before it is ever used, it is checked against its published checksum, and a file that does not match is deleted.After that, the model needs no internet connection.Set up for me recommends the best model your graphics card and memory can run well, and downloads it.

Rather not run a model yourself?On the Enterprise plan, we can run AI Smart Verification for your team on our own service, as an option priced separately.Like any cloud option, it means failed checks leave your computers, so choose it when your rules allow it.See pricing.

Tests from a prompt, without sending your screens

GUI Test Studio can also create a test from a prompt.You describe the workflow, an AI assistant performs it on your computer, and it saves an ordinary test.

With Claude or ChatGPT, the assistant works from what it sees: screenshots of your screen and the texts it reads there.Those go to the assistant's provider, under your account with it.

For a screen with real data, use a local AI model instead.The This computer card creates the test with a model in GUI Test Studio's built-in engine, or with one you bring, and your request and your screen never leave the computer.See Create a test from a prompt.

Enlarge

What still connects to the internet

Here is what the apps connect to, and what each connection carries.

You turn usage statistics off in either app's Preferences, or in the installer.Turning them off also deletes what your computer sent before.A silent install turns them off with /TELEMETRY=off.

Questions

Playwright, Selenium, BrowserStack, Sauce Labs, TestMu AI, LambdaTest, Applitools, Jira, LM Studio, Claude, ChatGPT, OpenAI, Anthropic and Windows are trademarks of their respective owners.