Read the Confidence Report
What passed, what failed, why, and how close each check came.
Open a report
The Confidence Report opens by itself after every run.To see it again later, open the test in GUI Test Runner-Recorder and click Confidence Report (Ctrl+Shift+R).Every run's report is saved with the test, and reopens exactly as it was.
The verdict at the top
The banner says how the run ended, how many checks and actions passed out of how many the test holds, and how long it took.When a run failed, it says where: the step, the check and its Scenario.
Its weakest links point at the checks that passed by the smallest margin.They are the ones most likely to fail next, worth a look before they do.
When a key of the test was recorded in a keyboard layout this computer does not have, the report says so on its banner.The run still goes on, and that alone never fails it.
The banner's buttons
| Button | What it does |
|---|---|
| Bug Report | Opens the bug report form, filled in from the run.It offers two destinations: a file, PDF or Excel, or a Jira issue.See Create a bug report. |
| Connect Jira | Shown while no Jira site is connected on this computer: connect once, and Bug Report offers Jira from then on.Once this run was reported, the issue's key stands here instead, and a click opens the issue in Jira or creates another. |
| Folder | Opens the folder where the run's files are kept. |
| Clear History | Removes the earlier runs of the test from its History; the run you are reading stays. |
| Verdict | What happened |
|---|---|
| Passed | Every step ran and every check passed. |
| Failed: not found | A check waited its whole timeout and never found what it expected.For a visual check, this is the Search Phase. |
| Failed: different | The element was found but looks different from the recording.This is the Verification Phase. |
| Aborted | Someone pressed the abort key, or a time limit ended the run. |
The two phases of a visual check
Every visual check is decided in two phases, and the report shows both.Knowing which phase failed tells you what kind of problem you have.
| Phase | The question it answers | When it fails | What to look at |
|---|---|---|---|
| Search Phase | Is the element on the screen?While the test runs, the element is looked for, up to the check's timeout. | Not found: the element never showed up, or it changed so much that it is no longer recognised. | Is the app on the right screen?Is something covering the element?Was it removed or redesigned? |
| Verification Phase | Does it look as expected?Once found, the element is compared closely with what the test expects. | Different: the element is there, but something about it changed, such as a word, a colour or an icon. | Either the app has a bug, or it changed on purpose and you accept the new look. |
First you spot your friend in the crowd, and a new coat does not stop you: that is the Search Phase.Then you look closer and notice the new haircut: that is the Verification Phase.
The Search Phase is forgiving on purpose: the element only has to be recognisable, so a small change never hides it.The Verification Phase is the strict one: it leaves out what the eye cannot see and reports what really changed.
In a check's panel the phases read from top to bottom: found or not, then matches or differs.Two settings stand behind them.How forgiving the search is, is the test's Search Sensitivity, in Project Settings.How strict the comparison is, is each check's Verification Strictness.The defaults suit most tests: see Settings worth knowing.
Text and layout checks are simpler.They are decided in one step while the test runs: the text is read, or the elements are located, and the check passes or fails.
Every step, in order
The list shows every action and every check of the run, with a green or red mark.Filter it by kind (visual, text, layout, events), by result (failed only), by Scenario, or by a part of the test: drag across the run's timeline to look at that stretch only.
Each row carries the name you gave its block in GUI Test Studio.When the test has Scenarios, their steps are listed under a header in the Scenario's colour.Fold a header to hide the steps you are not looking at.
| To | Do this |
|---|---|
| Jump to the next failure | F8 |
| Find a step | Type in the search box: a name, a word, a Scenario |
| See the steps as a table | Switch the view at the top of the list |
The middle of the toolbar counts the failures, between Prev failure and Next failure.History, at its right end, switches between this run's steps and the test's History.
The run's timeline
Under the list, the run's timeline shows every step at its moment, with the frames of the video.Every check is marked at its moment, a failed one in red: click a mark to open that check.Click any other moment to open its step, and walk from step to step with the arrow keys.
It stands at the bottom of the window.Drag it by its title bar to any other edge, or out of the window to float on its own.
Look at one check
Click a check: its panel opens under it.On the left, the picture.In the middle, three rows give the verdict of each phase in plain words: the Search Phase, the Verification Phase and AI Smart Verification.On the right, what you can do about it.
The picture opens in Pulse: what changed glows on the screen as it is now, so your eye goes straight to it.
Compare the two pictures
The buttons above the picture switch the view: side by side, a split you drag, one laid over the other, a difference view, and Pulse.Any picture opens larger with its Enlarge button.
"Visual check failed.A significant visual difference was detected in Save button, concentrated in one area."The difference view shows the area: the label now reads "Save draft".Either the app has a bug, or it changed on purpose and you accept the new look.
Text and layout checks
A text check shows the words it expected and the words it read, side by side.A layout check shows the frame with each element's expected place in green, where it stood in red, and the name of every element that moved.
When a desktop app tells Windows what its text says, a text check asks the app first, and the report shows what the app said beside what was read on the screen.
Found outside its search area
A check that found its element only after looking beyond its search area says so, and offers Widen the Search Scope in GUI Test Studio....It opens the test on that check, in its editor, so you can widen the area in one move.
History: when did it start failing?
History, in the report's toolbar, follows every check across all the runs of the test: which run first failed, how close each check came over time, and what changed between two runs.Open any past run's report from there.
Another browser or device
For a web test made for several browsers and devices, each one has its own report and its own history.The Browsers & Devices button in the report's toolbar lists them all with their last verdict, and opens the report of the one you pick.See Every browser and device from one recording.
Next: When the app changes on purpose, or Create a bug report.
Still stuck?Tell us what you were trying to do, and we answer within one working day.Ask our team