How a large UK ‘orange’ retailer transformed their Test Automation with AutomatePro

Outcome Highlights:

• 75% increase in automated test coverage
• 4 weeks upgrade timeline reduction
• 84 days reduction in upgrade effort days

The Hunt

Initially, Selenium was decided upon, known for its versatility and popularity in the industry However, Selenium proved to be less than adequate to cover critical workflows specific to the retailer’s operations. Over the course of three years, only 1% of the retailer’s critical workflows were covered, falling significantly short of their goal. Moreover, the test coverage achieved was unreliable and difficult to maintain. Ultimately, Selenium’s limitations prevented them from accurately replicating complex scenarios and edge cases.

In an effort to overcome the limitations experienced with Selenium, the retailer switched to Gauge, which presented its own set of challenges. Firstly, building tests were difficult due to their internal knowledge gaps, the significant maintenance efforts required demand considerable time and resources from the retailer, and the unreliability of the test results weakened the retailer’s confidence.

The Solution

The search for a suitable automation tool led the retailer to AutomatePro—a solution that proved to be a perfect fit for their needs. Unlike their previous experiences with generic tools, AutomatePro was specifically designed for ServiceNow, aligning seamlessly with the retailer’s existing technology infrastructure.

AutomatePro’s AutoTest module solves the three main test automation frustrations in the marketplace – how to create, maintain and execute tests at speed and scale. By automating various testing procedures, the tool significantly reduced the time, stress, and costs associated with ServiceNow updates. A key element is AutomatePro’s upgrade comparison reporting, enabling the retailer to effectively baseline what changes have been introduced, fixed, or not fixed between family and patch upgrades and direct developer efforts to immediately resolve the issue.

The retailer also found value in implementing automated documentation. Unlike previous tools, they were able to generate documentation written in simple and accessible language, making it user-friendly for everyone on their team. A key advantage was the ability to include screenshots in each document,  which provide visual cues and make it easier for users to understand complex workflows and processes.

Also, AutomatePro’s AutoDoc module automatically updates documents with the running of the regression pack, eliminating the need for manual updates and saving significant time and effort. Additionally, each piece of documentation could be tailored to the client’s specific requirements, whether that be a User Guide, Training Guide or Knowledge Base Article.

The Results

Prior to adopting AutomatePro’s testing solutions, the retailer struggled to surpass a mere 5% test coverage for their critical workflows. Now, they boast an impressive 70-80% test coverage for their critical workflows. Achieved with the implementation of the collaborative approach and the QuickStart AI technology.

Additionally, as part of their partnership with AutomatePro, the retailer leveraged automated testing for the first time during their ServiceNow upgrades. The results were outstanding. Their upgraded timeline was reduced from an arduous 6 weeks to a mere 2 weeks.

Furthermore, with the current test coverage, the retailer has realised substantial time savings. On average, a remarkable reduction of 84 days of user effort is required for each ServiceNow upgrade. This impressive efficiency gain not only minimises the impact on the workforce but also allows the retailer to allocate valuable resources to other strategic initiatives, fostering productivity and driving overall business growth.