Visual Bugs Your Automated Tests Miss: How to Test What Users See
Visual bugs your automated tests miss: why toBeVisible() and isDisplayed() pass covered or unreadable elements, and how to test what users really see.
10 min readBy the GUI Test Studio team
What are visual bugs?
Your end-to-end tests are green.Then a customer sends a screenshot of the checkout page: the order total is hidden behind a promotion banner.Every automated test passed, because every automated test asked the page's code, not the screen.Visual bugs like this one are what visual testing is for: it checks what your users actually see.
What the test asked
await expect(
page.getByTestId('order-total')
).toBeVisible();What the customer saw
Definition
A visual bug is a defect people see on the screen while the code behind it looks fine.
- The element exists, it has a size, it holds the right text.
- A check of the code passes.
The person in front of the screen cannot read it, find it or press it.
This article shows where the gap comes from, which bugs fall into it, and how to build end-to-end tests that see the screen the way your users do.
Your user asks
- Can I see it?
- Can I read it?
- Is it where it should be?
- Does it look the way it did yesterday?
A check of the code asks
- Is the element in the page?
- Does it have a size?
- Is it styled as hidden?
- Does it hold the right text?
What "visible" means to Playwright, Cypress and Selenium
A coded end-to-end test finds an element through a selector and asserts something about it.The assertion closest to what a user experiences is "is it visible?".Each tool answers it its own way, and none of them looks at the screen.
Playwright
Playwright's toBeVisible() passes when the element has a non-empty bounding box and no visibility: hidden style.Its documentation adds that an element with opacity: 0 counts as visible.It does not ask whether another element covers the element, or what color its text is.
Actions are stricter than assertions.Before a click, Playwright checks that the element is the hit target at the click point, so a covered button makes the click wait until it times out.Playwright also offers toBeInViewport() for an element that must be inside the viewport, and toHaveText() reads the element's textContent by default.
Cypress
Since Cypress 16, the default visibility check delegates to the browser's own Element.checkVisibility().An element counts as hidden when it has no width or height, display: none, visibility: hidden or, for an assertion, opacity: 0.Before a click, Cypress checks that the center of the element is not covered.
An assertion is another matter.Cypress's documentation says it plainly: "The modern algorithm does not hit test, so it reports the covered element as visible."
Selenium
Selenium's isDisplayed() is not defined by the W3C WebDriver specification, because of "the impossibility of covering all potential conditions", as Selenium's documentation puts it.Selenium runs a large JavaScript function instead, which weighs the element's style, its size and the ancestors that clip it.It does not check whether another element paints over it, or whether its text can be read against its background.
A click on a covered element fails with an "element click intercepted" error.An element the test only reads is still reported as displayed.
| The check | Is in the page, with a size | Not covered by another element | Text readable on its background |
|---|---|---|---|
Playwright toBeVisible() | Yes | No | No |
Cypress be.visible | Yes | No | No |
Selenium isDisplayed() | Yes | No | No |
| A check of the screen | Yes | Yes | Yes |
9 visual bugs that pass a coded check
Each of these bugs leaves the element in the page's code, with a size and the right text.A coded test that asserts the element exists, is visible or holds its text passes.Your user sees something else.
1. An element covered by another
A cookie banner, a chat widget, a sticky header or a dialog that never closed sits on top of the element, often after a z-index change.The element is still visible by every definition above.A click on it is refused, but a test that only reads it, such as a price, an error message or a confirmation, passes while nobody can see it.
2. Text nobody can read
White text on a white background, gray on gray after a theme change, or a label left transparent by a broken animation.No visibility check reads a color.Playwright's toBeVisible() also passes for opacity: 0.
3. Text cut off
A label too long for its box ends in an ellipsis, or a fixed height hides the second line.The page still holds every word, so an assertion that reads the text passes.Your user reads "Confirm your or...".
4. An element pushed out of view
A wider table or a longer heading pushes the Buy button off the right edge of a narrow screen.The button keeps its size, so toBeVisible() still passes.The test catches it only if it also asks toBeInViewport() for that button.
5. The wrong picture or icon
A product shows another product's photo, an icon font fails and leaves an empty square, or a sprite update swaps two icons.The image element and the icon's class are unchanged, so the code looks right.
6. A layout broken by a CSS change
One rule changes a margin, a flex direction or a grid column.Cards overlap, a column drops under another, a form loses its alignment.Every element is still in the code, with its text and its size, so every assertion passes.
7. A translated label that does not fit
German or French words often run longer than English ones.A label overflows its button, wraps onto a second line, or pushes the next button away.The translation is correct, so a text assertion passes in every language.
8. A font that failed to load
When a web font fails to load, the browser falls back to another font: lines wrap differently, and icons drawn with a font turn into empty boxes.The text in the code is the same, so nothing fails.
9. A spinner that never stops
The data arrives and the rows are added to the page, but the loading overlay stays on top.The test finds its rows in the code and passes.Your user waits in front of a spinner.
Cypress's own documentation sums it up in one line: "an application can pass every functional test and still ship visibly broken."
Can visual regression testing with screenshots catch them?
Yes, and it is the usual answer.Visual regression testing compares a picture of the screen with a reference picture taken earlier.
Playwright has it built in with toHaveScreenshot().Its documentation warns that browser rendering can vary with the host operating system, its version and settings, the hardware and headless mode.So each browser and platform keeps its own reference picture.
Cypress takes screenshots, and its documentation says it "does not perform image comparison itself".Plugins or visual testing services add the comparison.
Services such as Percy and Applitools Eyes add picture checkpoints to your coded tests.Percy captures and compares screenshots during your test runs.Applitools Eyes adds a check call inside a test.
For these tools side by side, see the comparison study.
Why manual testing does not close the gap
A person sees every one of these bugs at a glance.That is why many teams still click through their application before a release.Manual testing has its limits, though.
| Limit | What happens |
|---|---|
| Time | A full regression pass takes hours, so it runs before a release, not on every build. |
| Attention | By the twentieth screen of the same pass, the eye skims.A small change of color or a missing icon goes unnoticed. |
| Repeatability | Two testers look at different things, and the same tester looks differently on a Friday evening.A bug found once may never be looked for again. |
| Coverage | Every browser, device and language multiplies the work. |
How do you build end-to-end tests that see what users see?
Whatever tool you use, six principles make an end-to-end test experience your application like a user.
- 1
Drive the real application
Click and type the way a person does, on the real screen.A test that cannot press a covered button fails where your user would.
- 2
Check what is shown
Check the screen three ways: the picture, so an element looks right; the words, so it says the right thing; the arrangement, so everything stands where it belongs.
- 3
Find the element first, then compare it
A window that moved or a page that scrolled is not a bug.A test should find the element where it is now, then judge how it looks.
- 4
Ignore noise, never a real change
Screens differ slightly between machines, so the comparison should ignore rendering noise and still catch every change a person would notice.
- 5
Show the evidence
A failure should show the expected screen beside the one found, with the change marked, so anyone on the team can read it.
- 6
Keep the cost of change low
When the application changes on purpose, accepting the new look should take a click, not a rewrite.
How GUI Test Studio tests what users see
GUI Test Studio builds end-to-end tests on these principles, without code.It tests web apps, Windows desktop apps, Android phones and games with one kind of test.Read more about visual testing and GUI testing with it.
Record the workflow once
In GUI Test Runner-Recorder, click Record and do the workflow as you always do: it records your mouse, your keyboard and the screen.You can also describe the workflow in a prompt to Claude, ChatGPT or a local AI model we provide or you bring, which records it for you.
When you stop, it asks what the test should verify.With the default answers, every element you click or type into gets a Visual Assert just before your action.What appears on its own, such as a menu, a dialog or a message, gets a Group Assert.
Checks of what the screen shows
| Check | What it makes sure of |
|---|---|
| Visual Assert | An element is on the screen and looks right: its picture, its colors, its icon. |
| Visual Absent Assert | Something is gone: a spinner, a banner, a dialog that must close. |
| Text Assert | The words are right, whatever the font or the color. |
| Layout Assert | Elements stand where they belong: nothing moved, overlapped or left the screen. |
| Group Assert | Several checks of one screen at the same moment, and optionally their arrangement. |
Each one has its guide: visual checks, text checks, and layout and group checks.
Find the element, then compare it
Every visual check is decided in two phases.The Search Phase asks whether the element is on the screen, and waits for it up to the check's timeout.The GTS Engine + AI finds the element by asking the application first, through its accessibility information or the page's own elements, and by sight when the application does not say.See how GUI Test Studio finds every element.
When the element is found somewhere else than in the recording, the clicks after it are aimed at where it is now.And a test never clicks blindly through a banner: when the element is covered, its check fails before the click.
Read what changed
After a run, the Confidence Report shows each check with the picture expected beside the picture found, the change lit up, and the verdict in one plain sentence.
With AI Smart Verification on, a failed check also gets a second look from an AI, which says in plain words what changed.It runs with a cloud model and your own key, or with a local AI model on your own computer: one we provide, or one you bring.
When the change is on purpose, accept the new look from the report, and the test expects it from then on.The command line, gtr-cli, runs the same tests from any CI/CD pipeline, on every build.
Which check catches which visual bug
The nine bugs again, with the GUI Test Studio check that catches each one.
| The bug | The check | What the report says |
|---|---|---|
| An element covered by another | Visual Assert | Not found, or different: the screen no longer shows the element's picture. |
| Text nobody can read | Visual Assert | Different: the colors changed. |
| Text cut off | Visual Assert | Different: the end of the label is missing. |
| An element pushed out of view | Layout Assert | The element is named, with how far it moved. |
| The wrong picture or icon | Visual Assert | Different: the picture changed. |
| A layout broken by a CSS change | Layout Assert, or a Group Assert with its layout check | Each element that moved, with how far. |
| A translated label that does not fit | Visual Assert, Layout Assert | Different, or moved. |
| A font that failed to load | Visual Assert | Different: the letters changed. |
| A spinner that never stops | Visual Absent Assert | Still on the screen when its time ran out. |
Where coded checks still belong
Questions
Why does Playwright's toBeVisible() pass when an element is covered?
Playwright's visibility check asks only whether the element has a non-empty bounding box and no visibility: hidden style.It does not ask whether another element sits on top of it.A click on that element does check it, so the bug slips through when the test reads the covered element instead of clicking it.
Can Selenium detect an element hidden behind another element?
Not with isDisplayed(), which looks at an element's style, size and clipping, never at what paints over it.A click on a covered element fails with an element click intercepted error.An element the test only reads is still reported as displayed.
What is the difference between visual testing and functional testing?
Functional testing checks that a feature behaves correctly: a click leads to the right page, a form saves its data.Visual testing checks what the screen shows: the right elements, readable and in their place.An end-to-end test that does both catches what your users would notice.
Is visual testing the same as screenshot comparison?
Not quite.A raw screenshot comparison flags every changed pixel, including rendering noise that differs between machines.Good visual testing finds each element first, ignores harmless rendering noise and still catches real regressions, then shows what changed.
Do I need to write code to add visual checks in GUI Test Studio?
No.You record the workflow once, and GUI Test Studio adds a visual check before every element you click or type into, and checks what appears on the screen.You adjust the checks on a visual timeline.
Can automated visual tests replace manual testing?
They replace the repetitive part: walking through the same screens on every build.A person is still best at judging a new design or exploring a feature for the first time.
Selenium, Playwright, Cypress, Percy and Applitools Eyes are trademarks of their respective owners.
Ready to record your first test?
Every feature, for 15 days.
Rather see it on the kind of app you test?Book a free 30-minute call with our team.