AutoPlan. Capture requirements right.
4 min read
The Test Pyramid has been around since 2009. But technology has moved on since then. Is it time for a re-think?
Most people involved in testing will be familiar with the Test Pyramid depicted below.
The Test Pyramid was first described by Mike Cohen in 2009 in his book ‘Succeeding with Agile’. The arguments he makes for the shape of the triangle were certainly logical and relevant at the time, especially for large business applications built in that era built from the ground up by large teams of software developers.
Over a decade later, a number of things have changed that perhaps should cause us to re-think the pyramid – perhaps even turned it on its head!
The shape of the pyramid was derived from the desirable quantity of automated tests that should be created for each type of test. More automated Unit Tests should be created than Integration Tests, and very few automated UI tests should be created.
The reasons given for why lots of Automated Unit Tests should be created are as follows:
The reasons given for why only a few Automated UI tests should be created were as follows:
But wait! A lot has changed in technology since 2009.
Firstly, most organisations do not build business applications from the ground up. Most organisations use at least one Platform as a Service solution like ServiceNow, Salesforce.com etc.
PaaS solutions no longer require thousands of lines of code, instead they are:
Some code is written sometimes, but this is normally kept to a minimum to avoid customising the out of the box configuration too much.
In this world, where the focus is on configuration rather than development, unit testing developed code units is not as relevant. The ‘units’ are provided by the platform vendor and therefore expected to work.
What is of paramount importance when testing these types of systems, is testing of user interface and the end-to-end business processes i.e. the business workflow.
Logic follows that the majority of tests should therefore be of the UI and the testing of the business processes. If the business processes work, then how this is done in the code at the unit level is of little or no relevance.
Unit testing is largely a relic of large software application development projects where many coded algorithms or new code classes needed to be developed and unit tested.
In the PaaS world, these ‘units’ are provided by the vendor and have been thoroughly tested, if not thoroughly by the vendor pre-release, then very thoroughly by virtue of a very large user community using them everyday in live operation.
So is it time the Test Pyramid was turned on its head for the testing of PaaS solutions like ServiceNow?
It would seem so.
But wait. What about the list of UI automation issues that meant it was not desirable to do very many UI automated tests?
Well, technology has moved on since 2009 and there are now automated testing platforms, that remove all of these UI test limitations. One such platform is AutomatePro designed to overcome these traditional automated testing obstacles.
In addition, platforms like AutomatePro provide additional benefits of building many UI based process tests. For example, test results can be viewed by UAT testers, dramatically reducing the amount of time and resource needed for UAT. User guides and process documentation can automatically be created from the UI test results, reducing the overall cost of the effort to create the UI tests.
So, perhaps it is time to turn the test Pyramid on its head?