Connect Jira to the Confidence Report: every failed end-to-end test becomes a Jira bug, automatically

The bug report nobody has to write

When an end-to-end test fails, someone has to tell the developers: what broke, where, how to get there, and what it should look like.Writing that well takes time, and the pictures and steps are often missing or vague.

A failed GUI Test Studio run already knows all of it.It recorded every step, the picture or text it expected, what the screen showed instead, and how often the test failed before.The bug report turns that into a ticket, and with Jira connected to the Confidence Report it lands in your backlog directly: a functional regression, a broken workflow, a wrong screen, each as a ready-to-fix issue.

In this article Jean is the developer, Pierre wrote the test, James looks after the backlog in Jira, and John runs TeamCity.The shop and its test are a small demo, but every picture is from a real run.

What you need

Step 1: Connect Jira to the Confidence Report

1. 1

   Create an API token

   In your Atlassian account, open Security, API tokens and create one.Copy it: it is shown only once.On Jira Data Center, create a personal access token in your Jira profile instead.

2. 2

   Connect GUI Test Runner-Recorder

   In a run's Confidence Report, click Connect Jira beside Bug Report (the bug report form offers it too, under Create a Jira issue).Paste the address of any page of your Jira site, your email and the token.The window checks them at once and says who you are connected as.From then on every Confidence Report can file its bug report in Jira.

   

   Enlarge

Step 2: Send a failed run to Jira from the app

Open the failed run's Confidence Report and click Bug Report (or press Ctrl+B).Choose Create a Jira issue.Everything is filled in from the run: you only check it.

Enlarge

Click Create Jira Issue.The issue's key and a link to it appear at once, and the report remembers it: its Jira button opens that issue from then on.The form itself is described in Create a bug report.

Step 3: Let TeamCity do it on every failed build

In a pipeline nobody opens a report, so gtr-cli creates the issue itself.Start from the command of Run your end-to-end tests on every build and add the Jira options:

gtr-cli run tests --countdown 0 --format teamcity --artifacts ci-out\artifacts ^
  --jira on-failure --jira-project SCRUM --jira-assignee "James" ^
  --jira-labels gui-test-studio,ui-regression

The connection comes from three environment variables, so the token never appears in the command or in the build log:

Enlarge

When a test fails, the build log says which issue it became, with its link, and the issue links back to the build.

Enlarge

What the Jira issue holds

Here is the issue the failed build created, as James finds it in the backlog.

Enlarge

Steps to reproduce

The run's own steps, numbered, up to the one that failed.A developer can follow them by hand without guessing.

Enlarge

What the test saw

For every check that failed: the picture the test expects, the picture it found, and the found picture again with each change outlined in red.That last one is the Confidence Report's Pulse view: the eye goes straight to what changed.

Enlarge

The PDF bug report and a zip of every picture are attached, for anyone who needs the full story or wants to keep it.

One defect, one issue

A failure seen again must not flood the backlog.Each failure carries a label made from the test and from what failed.When the same defect fails again, on the next build, on another browser or device, or in two runs at the same time, gtr-cli finds its open issue and adds a comment with the new run instead of a duplicate.

Enlarge

Once the issue is done, a new failure files a new issue: a defect that comes back is news.

Not using Jira or TeamCity?

Jira and TeamCity are the example, not a requirement.

1. 1

   Any CI tool

   gtr-cli runs the same way in Jenkins, GitHub Actions, GitLab, Azure DevOps or a script of your own, and --jira on-failure works in all of them.See Run tests in your pipeline.

2. 2

   Any tracker

   Without Jira, --bug-report pdf (or excel, or both) writes the same filled-in bug report and its evidence as files, ready to attach to a ticket in any tracker.The --json result file has everything a script needs to file it for you.

3. 3

   No pipeline at all

   From the app, Bug Report saves the report as a PDF or an Excel workbook, or sends it to Jira, for any run you choose.

Where to go next