Accessibility first, vision when the app says nothing: how GUI Test Studio finds every element
Most end-to-end tests break because they cannot find an element any more.The GTS Engine + AI finds each element two ways: it asks the application through its accessibility information, and finds the element by sight when the application does not say.Why two ways to one element make a test robust, with no locator to write or maintain.
6 min readBy the GUI Test Studio team
Why end-to-end tests break
Ask a team why its end-to-end tests are red, and the answer is rarely a bug.More often, a test could not find a button any more: a developer renamed a class, a framework update changed the page's structure, a dialog opened a little lower, a theme changed a colour.The application was fine; the test was not.
Code-based tests find elements with locators: an id, a CSS selector, an XPath, a position.Each one is a guess about how the application is built, and every redesign breaks some of them.Image-based tools have the opposite weakness: they see the screen, but they know nothing of what the application says about it.
Two ways to the same element
The GTS Engine + AI, the engine behind every GUI Test Studio check, does not choose between the two.It finds each element two ways, and uses the one that works on the screen in front of it.
Ask
The application says where the element is
See
Or the engine finds it by sight
Compare
What is there, against what you expect
Report
The verdict, in plain words
First: ask the application
Most applications describe their own interface to Windows: every button, field, list and label with its name, its role and its place on the screen.It is the accessibility information that screen readers use, and it is precise and immediate.A web page does the same through its own elements.
When a test is recorded, or described in a prompt, GUI Test Studio keeps what the application said about each element a check was made from.When the test runs, the engine asks the live application for that element first: where it stands now, whether it is visible, whether something covers it.A text check asks the application what the element says before it reads the words on the screen.
| What the application says | What it gives the test |
|---|---|
| The element's name and role | The same button is found again, even among others that look alike |
| Its place on the screen | The element is found where it moved, at once |
| Whether it is visible or covered | A dialog or a menu in front of it is noticed, never clicked through |
| What it says | A text check gets the exact words, never a misreading |
Then: find it by sight
Some applications say nothing.A game draws its menus itself.A canvas, a chart or a map is one picture to Windows.A remote desktop, a virtual machine or a phone shown on your screen is a single window with nothing inside.Older programs drawn by their own toolkit look like one blank pane.
There, the GTS Engine finds the element by sight, on the live screen, the way a person does: it looks for the element's picture in its search area, tolerates the small differences every screen has, and tells look-alikes apart by where they stand.It also takes over when the application's answer does not hold: an element it reports but that is hidden, covered or nowhere near where it should be.
Why two ways make a test robust
Each way covers the other's weak spot.The accessibility information does not care how an element looks, so a new theme, a font or a different screen scaling does not lose it.Vision does not care how the application was built, so a renamed class, a new framework or an app with no accessibility information at all does not lose it either.
| What changed | What still finds the element |
|---|---|
| The code behind the button was rewritten | Both: the application still names it, and it still looks the same |
| The window moved, or the layout shifted | The application says where it is now, and vision finds it there |
| A new colour or theme | The application's accessibility information |
| A game, a canvas, a remote desktop | Vision |
| Several identical buttons | The application's names and the element's place tell them apart |
The result is a test that keeps finding what it is looking for while your application changes around it, and that fails for one reason only: what the user would see is wrong.
Finding is half the check
Once the element is found, the GTS Engine compares what is on the screen with what you expect: the picture, the words, the arrangement.It ignores the noise every screen has, anti-aliasing and compression, and catches the real regressions a person would call a change: a wrong colour, a missing icon, a word that changed case.
When a check fails, AI Smart Verification, the optional AI side of the engine, can take a second look and explain the difference in plain words, with a cloud model and your own key or a local model on your own computer, one we provide or one you bring.See AI Smart Verification.
Nothing to set up
There is no locator to write, no accessibility setting to turn on, no mode to choose.Record the test yourself, or describe it in a prompt to Claude, ChatGPT or an AI model on your own computer: GUI Test Studio keeps what it needs about every element, and the engine decides at every run which way finds it.Everything runs on your own computer.
Questions
Does GUI Test Studio use the accessibility tree?
Yes.The GTS Engine + AI first asks the application where an element is, through its accessibility information on Windows, or the web page's own elements in a browser.It finds the element by sight on the screen only when the application does not say.
What if my application has no accessibility information?
The test still works.Games, canvases, remote desktops and programs drawn by their own toolkit expose nothing, so the GTS Engine finds their elements by sight, on the screen, the way a person does.
Do I have to write locators or selectors?
No.GUI Test Studio keeps what it needs about each element by itself when the test is recorded or described in a prompt.There is no locator, selector or element id to write or maintain.
Is the check still visual when the element is found through accessibility?
Yes.Finding the element is the first half of a check.The second half always compares what is on the screen with what you expect, so a test catches what your users would see, such as a wrong colour or a missing icon.
Ready to record your first test?
Every feature, for 15 days.
Rather see it on the kind of app you test?Book a free 30-minute call with our team.