Test Evidence for Audits: What an Auditor Wants From an Automated Test
Test evidence for audits: what an auditor asks of an automated test, and how every run keeps its own report, video, pictures and history as proof.
Test evidence is the proof that a test really ran: what was tested, when, on which version, with what result, and what the screen showed. An auditor wants it for every run, readable months later.
In a bank, at a supplier of health software, in a public service or in any company that is audited, "the tests passed" is not an answer. The auditor asks to see it. Then a tester spends the days before the audit pasting screenshots into a document, for tests that ran weeks ago.
What is test evidence?
Definition
Test evidence is everything that lets someone who was not there check that a test ran and what it found.
- It names the test and what it checks.
- It gives the date, the version and the result.
- It shows what the screen looked like, for the steps that passed as well as for the one that failed.
Good evidence can be opened a year later by a person who has never seen the app.
An audit trail for testing is the same idea over time: the evidence of every run, in order, so that anyone can follow what was tested from one release to the next.
Why a green tick in a build log is not evidence
A pipeline that ends in green proves one thing: a command ran and returned without an error. It does not say which steps ran, what each check compared, or what the screen showed at that moment.
A build log
A claim, with nothing to open.
The evidence of the same run
Proof that anyone can open later.
A result is a claim. Evidence is what lets someone else check the claim.
The gap matters most on the day something goes wrong. When a defect reaches customers, the question is no longer whether the tests passed. It is what exactly they looked at, and nobody can answer that from a tick.
What an auditor asks, and the evidence that answers
Whether the audit is about financial controls, a quality system for medical software or information security, the questions about testing have the same shape. Each one has an answer that should already exist when the question comes.
| What the auditor asks | The evidence that answers |
|---|---|
| What was tested? | The test itself: a file that holds every step and every check, readable in plain words. |
| When did it run? | The date and time of the run, shown on its report and kept with its results. |
| On which version? | The build of your pipeline that ran the test, with the evidence kept as that build's artifact. |
| What was the result? | The verdict of the run and of every check: passed, or failed and why. |
| What did the screen show? | The video of the run, and the expected picture beside what the run found. |
| Did it pass before? | The history of the earlier runs of the same test. |
| What happened to the failure? | A bug report with the steps to reproduce, or the issue it became in your tracker. |
| Who decided that it passed? | The check's own comparison, the same at every run, or an AI verdict that is saved with its explanation. |
| Who changed what the test expects? | The history of the test file in your version control: an author and a date for every change. |
Why collecting test evidence by hand does not last
Many teams answer these questions with a document: a tester runs the scenario, takes a screenshot of each screen, pastes it under the step and signs the page.
It works for one release, then the costs show.
- It takes days that return before every audit and every release.
- It shows only the screens someone thought to capture, never the ones in between.
- It can be done after the fact, which is exactly what an auditor is paid to doubt.
- It depends on one careful person, and it stops when that person is away.
What GUI Test Studio keeps for every run, by itself
GUI Test Studio automates end-to-end tests of any app without code: you record a workflow once, or describe it in a prompt, and the test replays it. Every run, on your desk or in a pipeline, leaves the same evidence behind without anyone asking for it.
One run
Confidence Report
Every step, what passed, what failed and why
Video of the run
Every step, the ones that passed included
Pictures
What was expected, beside what the run found
Results files
JUnit and JSON, for your pipeline
History
The earlier runs of the same test
Bug report
PDF or Excel, with the steps to reproduce
| Evidence | What it holds |
|---|---|
| The Confidence Report | The verdict, then every step in order: what passed, what failed and why. It is saved with the run and reopens exactly as it was. |
| The video of the run | The whole run from the first step to the last, so the steps that passed can be seen too. |
| Expected beside actual | For a visual check, the picture the test expects beside what the run found, with what changed lit up. |
| The results files | A JUnit file for your CI tool and a JSON file for your own scripts, written by gtr-cli. |
| The history | Every earlier run of the same test, check by check. |
| The bug report | For a failure: a PDF or an Excel workbook with the steps to reproduce and the pictures. |
The Confidence Report guide explains everything the report shows, and how to read a test report walks through a failure from the verdict to the next step.
An audit trail for testing: the history of every run
One run answers what happened today.An audit asks what happened over a year.History, in the report's toolbar, follows every check across all the runs of a test: which run first failed, how each check did over time, and what changed between two runs.Any past run's report opens from there.
The test itself has a trail too.
- A test is one file, so it fits the version control or the backed-up storage your team already uses.
- Each change of what a test expects then has an author and a date.
- A run from the command line never changes the test file, so what ran is what was reviewed.
Who decided? Checks and AI verdicts an auditor can inspect
As soon as AI enters testing, an auditor asks a new question: who decided that this passed? The answer has to be something a person can open and read.
During the run: the recorded steps replay by themselves, and no AI drives the app.
AI off
Each check decides on its own comparison
The same screen gives the same verdict at every run.
AI Smart Verification on
A failed check gets a second look
The AI's verdict and its explanation are saved with the run.
| Your setting | Who decides a check | What is kept |
|---|---|---|
| AI off, as installed | The check's own comparison. It passes or fails the same way at every run. | The verdict and the pictures. |
| AI Smart Verification on | For a failed check, the AI takes a second look and gives the final verdict of that check. | The verdict, the explanation in plain words, every difference found and the instructions the AI was given. |
AI is optional, and the choice is yours: a cloud AI with your own key, a local AI model, ours or your own, or none.The answers are kept with the run, so reopening a report asks nothing again, and History shows what the AI said in each past run.See AI Smart Verification.
Evidence that stays on your own computers
An auditor's next question is where the evidence lives and who can reach it. With GUI Test Studio, tests, recordings, runs and reports are files on your own computers. Every run is kept in the results folder inside the test's own folder.
That makes three decisions yours.
- How long the evidence is kept follows your own retention rules, not a provider's.
- Who can open it follows the permissions of your own folders and build server.
- Where it is archived is wherever you already keep other records.
A video shows whatever the screen showed, so run your tests with test accounts and made-up data.Data privacy in test automation explains how to keep real records out of every screenshot, and AI sovereignty in software testing covers who decides which AI sees your screens.
How to keep the evidence of every run in your pipeline
gtr-cli, the command-line runner, runs the same tests in any CI/CD tool and writes the evidence where you tell it to.Run tests in your pipeline lists every option.
- 1
Keep the tests where your pipeline can read them
A test is a file. Keep it with your project, in version control or on a share the build machine reads, so every change to what a test expects has an author and a date.
- 2
Run the tests with gtr-cli and ask for the results files
Add the command to your pipeline. The JUnit file feeds the test results page of your CI tool, and the JSON file holds the same results for your own scripts and dashboards.
Run every test of a folder and keep its evidencegtr-cli run tests --junit results\tests.xml --json results\tests.json --artifacts evidence --bug-report pdf - 3
Gather the evidence in one folder
The artifacts option gathers the video, the Confidence Report and the failure pictures of every test in the folder you name.
- 4
Ask for a bug report for every failed test
The bug report option writes a filled-in report as a PDF for each test that failed, with the steps to reproduce.
- 5
Keep the folder as an artifact of the build
Tell your CI tool to keep the results and evidence folders as artifacts. The build number then ties the evidence to the version that was tested, and your CI tool's retention applies.
- 6
Open a run again when someone asks
Pipeline runs also land in each test's results folder. Open the test in GUI Test Runner-Recorder, and History lists them beside your own. Any past run's report reopens exactly as it was.
| Option | What it keeps |
|---|---|
| --junit | The results file your CI tool reads for its test results page. |
| --json | The same results for your own dashboards and scripts. |
| --artifacts | The video, the Confidence Report and the failure pictures of every test, in one folder. |
| --bug-report pdf | A filled-in bug report for every failed test. |
When a run fails: the bug report is evidence too
An audit does not expect every test to pass. It expects every failure to be followed up. A failed run already holds what that takes: the steps that led there, what was expected, what appeared instead, and how often it failed before.
Bug Report, in the report's banner, writes it as a PDF or an Excel workbook, with a zip of the pictures beside it, or creates an issue on your own Jira site.See Create a bug report, and Connect Jira to the Confidence Report for the issue a failed run becomes.
Build 418 fails on the order summary. The evidence folder of that build holds the report, the video and a bug report as a PDF. The developer fixes the total, build 419 passes, and its report is kept the same way. A year later, the whole story can be opened: the failure, the bug, the fix and the proof of the fix.
What to know before you start
| Limit | What it means |
|---|---|
| It does not make you compliant | GUI Test Studio gives you the evidence to show. Whether it is the right evidence, kept for the right time, is decided by your own rules. |
| It does not know your app's version | Your pipeline does. Keep the evidence as an artifact of the build that was tested, and the build number links the two. |
| A test checks what it was told to check | The evidence proves that those checks ran and what they found, not that nothing else is wrong. |
| Keeping the files is your job | The evidence is yours: back it up, and decide who may delete a run or clear a test's history. |
Key takeaways
- 1Test evidence is proof someone else can open: what was tested, when, on which version, with what result, and what the screen showed.
- 2A green tick is a claim, and evidence gathered by hand costs days and can always be doubted.
- 3GUI Test Studio keeps the evidence of every run by itself: a saved report, a video, pictures, results files and a history.
- 4An auditor can see who decided: each check's own comparison, or an AI verdict saved with its explanation, and AI can stay off.
- 5The evidence is files on your own computers, so retention and access follow your rules.It does not make you compliant: it gives you what to show.
Questions
What is test evidence?
Test evidence is the proof that a test really ran and what it found. It says what was tested, when, on which version, with what result, and it shows what the screen looked like. A line that says passed is a claim, and the evidence is what backs it.
What does an auditor want to see from automated tests?
An auditor wants to see what each test checks, when it ran, on which version of the software, what the result was, and proof that can be opened later. For a failure, the auditor also wants to see what was done about it.
Is a passed build enough as proof of testing?
No. A passed build proves that something ran and ended without an error. It does not show which steps ran, what each check compared, or what the screen showed. Keep the report, the video and the results files of the run with the build.
How do I create an audit trail for automated tests?
Keep three things for every run: the test as it was, the results of the run, and the build they belong to. With GUI Test Studio, gtr-cli writes the results files and gathers the report, the video and the pictures in one folder, and your pipeline keeps that folder as an artifact of the build.
How long should test evidence be kept?
As long as your own rules say. The framework you are audited against, your contracts and your quality system decide, not the testing tool. GUI Test Studio writes the evidence as files on your computers, so you apply the retention you already use for other records.
Can AI be used in tests that are audited?
Yes, when its part is clear and its answers are kept. In GUI Test Studio no AI drives a run. AI Smart Verification is optional, looks only at failed checks, and its verdict and explanation are saved with the run. With it off, every check passes or fails on its own comparison.
Does GUI Test Studio make my team compliant?
No. Compliance is decided by your auditor against your own processes, and no tool can grant it. GUI Test Studio gives you the evidence to show: a saved report, a video, pictures, results files and a history for every run.
Where is the test evidence stored?
On your own computers, as files. Every run is kept in the results folder inside the test's own folder, and a pipeline run can also gather everything in a folder you name. Who can open those files and how long they are kept is your team's decision.
Jira and Windows are trademarks of their respective owners.


