The Testing (Automation) Pyramid

The Testing Automation Pyramid (sometimes just called the “Testing Pyramid”) is an Agile methodology that involves a heavy reliance on Unit Tests as opposed to UI-based testing. It’s referred to as a “pyramid” because the width of the layers in the pyramid refers to the ideal number of tests per layer. The bottom, widest layer is Unit Testing, the middle layer consists of service-related tests, and the tip of the pyramid represents UI Tests. A picture of the structure is shown below, sourced from Martin Fowler’s blog.

Moving up the pyramid, the type of testing done costs more resources to execute and more time to assess.

Part of the reason why UI testing takes significantly more resources is because it involves interacting with and logging the operations of the interface in order to see where something may go wrong. Running through the operations of the UI to see where it breaks takes a very long time, as there are so many levels of abstraction between the UI and the underlying infrastructure, and so much functionality within User Interfaces. On top of this, as the product is updated and refactored, tests derived from the original UI interface becomes obsolete, making the lifespan of higher-level tests shorter than those aimed towards the low-level structure of the system.

This is why Unit Tests are so good. They’re a cheaper (in terms of both resources and time) alternative, and they provide a means of testing the actual root of the issue. Unit Tests are preventative and they ensure that a bug presenting itself in the UI level stays fixed, rather than allowing the bug to manifest itself in a variety of different ways.

There is quite a bit of criticism of this model, as if it goes unchecked it ends up as the so called “Ice Cream Cone” anti-pattern, where a small layer of exploratory manual testing above the UI testing layer ends up becoming mandatory manual testing, which becomes both a resource and a time sink. This is far better explained by viewing Alister Scott’s blog post on it over at his blog, WatirMelon.

Martin Fowler, whose blog I found this concept on and referenced earlier, summed it all up in a pretty wonderful way: “If you get a failure in a high level test, not just do you have a bug in your functional code, you also have a missing or incorrect unit test.” Martin’s blog post elaborates on the idea much better than I’m able to, and his blog is extremely readable and has really great content. I’ll definitely be using it as a resource in the future.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s