Regression runs took hours, backlogs were full of unprioritized tests, and teams had little visibility into automation value. Confidence in releases was low, despite large test suites. To address this, I conducted a test automation assessment and introduced a decision matrix to prioritize tests by business value and automation complexity. This helped teams focus on meaningful tests, reduce overhead, and apply a risk-based mindset. In this session, I'll share how this approach turned automation into a reliable quality enabler.

Target Audience: Testers, QA, Team Leads, Test Managers & Product Owners
Prerequisites: Project Experience and Agile Development Methods
Level: Basic

Extended Abstract:

Regression runs take several hours, backlogs are filled with unprioritized test cases, and teams have little visibility into what has actually been automated. Releases are viewed as high-risk, confidence is low, and although the automation suite has many test cases, it offers little real value. This was the situation I faced when I started my current project about a year ago with one of the project teams, where I conducted a test automation assessment.

Like any detective story, the first step was to investigate the details. I looked at how tests were structured, how pipelines were set up, how results were reported, and how automation engineers worked together. I noticed a trend: teams concentrated on coverage statistics, added more test cases, and created flaky suites that slowed down feedback instead of speeding it up. Visibility was low, and priorities were unclear, making it challenging to understand which tests were important.

The key step was introducing a decision matrix that compared business value to automation complexity, which I had used in my previous project. Instead of asking, 'How many tests do we have?' teams began to ask, 'Which tests give us confidence to move forward?'. This change reduced the overhead, enabled backlog prioritization, and turned automation into a source of confidence rather than just quantity. We applied this method in one project team and expanded it to a second team. Soon, in the third team, we are implementing it from the start to avoid repeating past mistakes. 

In this presentation, I will guide you through the investigation, show the decision matrix in practice and explain how the teams and the overall test automation approach were adjusted. You will see how a risk-based method makes test automation manageable, reliable, and a valuable quality enabler.