How to Read a Test Report: What Failed, Why, and What to Do Next
How to read a test report: five questions that tell you what failed, why, and whether to file a bug, accept the change or edit the check.
To read a test report, ask five questions: did it pass, where did it stop, what did the screen show, is it a bug or an intended change, and has it happened before. A good report answers all five.
Most test results do not. They are a wall of red and green, a stack trace, and a line such as element not found at line 214. After reading them, a tester still does not know whether the app broke or the test did.
Why are most test results hard to read?
Definition
A test report is what a test run leaves behind for a person to act on.
- Many are written for the program that ran the test, not for that person.
They say where the code stopped, when the reader needs to know what the user would have seen.
A wall of results
FAILED: checkout test 42
Element not found at line 214
Assertion error: expected true, got false
Timeout exceeded
Did the app break, or the test?
A report that answers
You know what to do next.
| What you get | What you still do not know |
|---|---|
| A red line with the test's name | Which step of the workflow failed. |
| A stack trace and a line number | What was on the screen at that moment. |
| Element not found | Whether the button is gone, moved, renamed, or was slow that day. |
| A screenshot of the end of the run | What the screen should have looked like. |
So someone runs the test again by hand, watches the screen, and works out what happened. The report was meant to save that time.
The five questions a test report must answer
Reading test results is easier with a fixed order. These five questions go from the quickest to the deepest, and you can stop as soon as you know what to do.
| Question | Where the Confidence Report answers it |
|---|---|
| 1. Did it pass? | The verdict at the top: passed, failed or aborted, with how many checks and actions passed. |
| 2. Where did it stop? | The banner names the step, and every row of the list carries the name you gave that step. |
| 3. What did the screen show? | The failed check's panel: expected beside actual, the difference lit up, and the run's video at that moment. |
| 4. Is it a bug or an intended change? | Two answers per check, found and same, and an explanation in plain words when AI is on. The decision stays yours. |
| 5. Has it happened before? | History: every run of the test, and the run where the check first failed. |
The Confidence Report is the report GUI Test Studio opens after every run.GUI Test Studio automates end-to-end tests of any app without code, and this report is where a run is read.The guide Read the Confidence Report describes every button.
Did it pass, and where did it stop?
The banner at the top says how the run ended, how many checks and actions passed out of how many the test holds, and how long it took. When a run failed, it says where: the step, the check and the part of the test it belongs to.
| Verdict | What happened |
|---|---|
| Passed | Every step ran and every check passed. |
| Failed: not found | A check waited its whole time and never found what it expected. |
| Failed: different | The element was found, but it looks different from what the test expects. |
| Aborted | Someone stopped the run, or a time limit ended it. |
Under the banner, the list shows every action and every check in order, with a green or red mark.Each row carries the name you gave its step in GUI Test Studio, so a failure reads Confirm order and not a line number.F8 jumps to the next failure, and a filter shows the failed rows alone.
Test failure analysis: was it found, does it look the same?
A check that looks at an element answers two questions, one after the other. First, was the element found on the screen. Then, does it look the same as expected.
One failed check, read from top to bottom
Was the element found?
FoundIt is on the screen, so the app reached the right place.
Does it look the same?
DifferentSomething about it changed: a word, a colour or an icon.
What changed, in plain words?
Optional AIThe Confirm order button changed from active to disabled.
| Which answer failed | What it means | Where to look |
|---|---|---|
| Not found | The element never showed up, or it changed so much that it is no longer recognised. | Is the app on the right screen? Is something covering the element? Was it removed or redesigned? |
| Found, but different | The element is there, and something about it changed: a word, a colour, an icon. | The element itself. Either the app has a bug, or it changed on purpose. |
First you spot your friend in the crowd, and a new coat does not stop you. Then you look closer and notice the new haircut. The first look is forgiving, the second is careful, and the report keeps the two apart.
The comparison works like a human eye.It notices what a person would notice, such as a wrong digit or a missing icon, and ignores what nobody sees.Text checks are simpler: the report shows the words expected beside the words read.A check that ran out of time on a slow day is another story, told in fixed sleeps in tests.
What did the screen show? Expected beside actual
A sentence about a difference is never as quick as seeing it. Click a failed check and its panel opens: what was expected on the left, what the run found on the right.
The report gives you several ways to look.
- Side by side, or one picture laid over the other.
- A view that lights up only what changed, so your eye goes straight to it.
- The run's timeline, with the frames of its video and every check marked at its moment.
A report should show what the user would have seen, not where the code stopped.
Is it a bug or an intended change? An explanation in plain words
With AI Smart Verification on, a failed check gets a second look.The AI gives the final verdict on that check and writes the sentence a colleague would write: the Confirm order button has changed from active to disabled.
The explanation is optional, and which AI writes it is your team's choice.
- A cloud AI, with your own key.
- A local AI model, one we provide or one you bring.
- No AI at all: the pictures still show what changed.
The reports, the pictures and the recordings are files on your computers, kept with the test.That also makes them evidence: see test evidence for audits.
Has it happened before? The history of earlier runs
One failure means little without its past. A check that failed today for the first time, after weeks of passing, points at yesterday's change. A check that fails one run in five points at timing.
| What History shows | What it tells you |
|---|---|
| The run where a check first failed | Which build or which day to look at. |
| A check that passes and fails in turn | A timing problem, or something that differs from one run to the next. |
| What changed between two runs | Whether the test was edited, or only the app moved. |
Every run's report is saved with the test and reopens exactly as it was, so any past run can be read again.Tests that fail without a reason are the subject of flaky tests and false positives.
What to do next: three choices
Reading a report ends in a decision, and there are only three.
A failed check, and what you saw on the screen
The app is wrong
File a bug report
A PDF, an Excel file or a Jira issue, filled in from the run.
The change was intended
Accept the new look
One click in the report, and it can be undone.
The check asks the wrong thing
Edit the check
The test opens in GUI Test Studio on that very check.
| You found | What you do | How |
|---|---|---|
| A bug in the app | File a bug report. | The Bug Report button opens a form filled in from the run: the title, what failed, the steps to reproduce, the environment. Save it as a PDF or an Excel file, or create a Jira issue. |
| An intended change | Accept the new look. | One click makes what the run found the new expected look. It can be undone, and Save writes it into the test. |
| A check that asks the wrong thing | Edit the check. | Change a text check's expected words in the report, or open the test in GUI Test Studio on that check. |
Create a bug report and When the app changes on purpose walk through each of them.Accept only what you have looked at: a test that follows every change by itself hides real bugs, as self-healing tests explains.
How to read a failed run in five steps
- 1
Read the verdict at the top
The banner of the Confidence Report says how the run ended, how many checks and actions passed, and the step where it failed.
- 2
Go to the first failure
Press F8 or click Next failure.The row carries the name you gave that step, so you know which part of the workflow it is.
- 3
Read the two answers of the check
Was the element found, and does it look the same. Not found points at the screen the app was on, different points at the element itself.
- 4
Look at what the screen showed
Compare expected and actual side by side, or let the report light up what changed. Read the explanation when AI Smart Verification is on.
- 5
Decide what to do next
Open History to see whether it failed before.Then file a bug report, accept the new look when the change was intended, or edit the check.
Test results from a CI/CD pipeline
A run on a build machine is read the same way. gtr-cli, the command-line runner, gives the CI tool its results as JUnit, JSON or Markdown, and keeps the run's video, its pictures and its saved report as artifacts. Nothing about a run is lost because nobody was watching it.
A failed build can also file its own Jira issue, with the steps and the pictures: see every failed test becomes a Jira bug.Run tests in your pipeline lists the options.
What to know before you start
| Good to know | What it means |
|---|---|
| A report shows what the test checked | A problem outside every check is not in it. A test checks what it was told to check. |
| A stopped run is a partial run | After a failure the rest depends on, the later steps did not run. Fix the first failure, then run again. |
| AI answers after the run | With AI on, the report says that it is verifying while the failed checks get their second look, and the verdict settles once every answer is in. |
| The decision needs a person | The report shows and explains the difference. Whether it is a bug is for someone who knows what the app should do. |
Key takeaways
- 1A test report is for a person who has to act, so it must say what the user would have seen, not where the code stopped.
- 2Ask five questions in order: did it pass, where did it stop, what did the screen show, is it a bug or an intended change, has it happened before.
- 3Each check gives two answers: not found points at the screen the app was on, different points at the element itself.
- 4The Confidence Report shows expected beside actual, with your own step names, the run's video and the history of earlier runs.
- 5Every reading ends in one of three actions: a bug report filled in for you, a new look accepted in one click, or an edited check.
Questions
How do you read a test report?
Start at the verdict, then go to the first failure. For that step, ask what the screen showed, whether the difference is a bug or an intended change, and whether it has happened before. Then file a bug, accept the change or edit the check.
What should a good test report contain?
The answer to five questions: did the run pass, where did it stop, what did the screen show, is it a bug or an intended change, and has it happened before. A list of red and green lines answers only the first.
How do I know whether the app broke or the test did?
Look at what the screen showed at the failed step, beside what was expected. If the screen is wrong, the app broke. If the screen is right and only looks new, the app changed on purpose and the test should follow.
What is the difference between not found and different in a failed check?
Not found means the element never showed up while the check waited. Different means the element was there, but something about it changed, such as a word, a colour or an icon. The first sends you to the screen the app was on, the second to the element itself.
Do I need AI to understand a failed test?
No. The Confidence Report shows expected beside actual with the difference lit up, without any AI. AI Smart Verification is optional: it adds an explanation in plain words, with a cloud AI and your own key, or with a local AI model that we provide or that you bring.
What do I do when a test fails because the app changed on purpose?
Accept the new look. In the Confidence Report, one click makes what the run found the new expected look of that check, it can be undone, and Save writes it into the test.
Can a failed test become a bug report by itself?
Yes. The Confidence Report's Bug Report button opens a form already filled in from the run: the title, what failed, the steps to reproduce and the environment. You save it as a PDF or an Excel file, or create a Jira issue from it.
Where is the report of an old run?
With the test. Every run's Confidence Report is saved beside the test file and reopens exactly as it was, with its pictures. The test's History lists every run and opens any of them.
Jira and Excel are trademarks of their respective owners.
Keep reading
All articles
How it works
How GUI Test Studio Finds Elements: Accessibility First, Vision When the App Says Nothing
6 min read

Test automation
What Is End-to-End Testing? A Plain Guide With One Real Example
10 min read

Test automation
Smoke Testing vs Sanity Testing vs Regression Testing: Which One to Run, and When
9 min read