The role of the Automated Tester, like many things, was created with the best of intentions. In an attempt to acknowledge the importance and effectiveness of the automated test suites which now forms the backbones of many large-scale (purportedly) agile projects which utilize scrum and XP practices (notably Iterative Development, TDD, and Continuous Integration) the industry has created a role which supports the idea of automated testing while attempting to marry it with the continued existence of a separate QA department.

Despite these good intentions the introduction of this role harkens back to the bad old days of waterfall development, in which everything everything fell neatly into one of a number of familiar phases: Ideation, Requirements Gathering, Design, Development, Testing, Deployment, Maintenance, etc. In this case we are once again separating Development and Testing into distinct activities. Developers are no longer required to wear as many hats, as a new member of the team would be responsible for wearing The Testing Hat. Now, instead of having armies of off-team QA Testers who follow scripts to repetitively, manually (read: prone to human error) test functionality organizations now have armies of on or (worse) off-team Automated Testers who write reams of often poorly written, unmaintainable, and flaky functional/e2e tests.

Introducing this role means that the organization has implicitly said to developers “Don’t worry about writing the right tests, that is the Automated Tester’s job”. Whether this was their intent or not is irrelevant. By adopting this strategy the organization is making one of two statements. They are either saying “Developers can’t be trusted to write the right tests for their own software”, or they’re saying “Developers have more important things to do than to write their own tests”. Either of these statements is an indication that something is rotten in state of Denmark.

Give a tester a hammer and everything looks like an end-to-end test.

Automated testers frequently relegate themselves to the functional/e2e test layers, and therefore make it their business to write more of these tests. Understand, this is through no fault of their own, as they are simply performing the job assigned to them. The organization has requested that they write more tests, so they write more tests in “their” section of the codebase, which is itself antithetical to the concept of Collective Code Ownership.

Unfortunately this is often counterproductive and does nothing more than to create an inverted test pyramid. This causes the build to run longer, which increases the length of the feedback cycle, which in turn makes introducing new functionality take longer. As we all know time is money, and the longer it takes to develop features the longer the project takes, and the longer the project takes the more budget it burns.

Furthermore, functional/e2e tests can often be slow and flaky due to the need to wait on UI interactions, outside dependencies that often have a degree of latency, and other issues that arise in a production-like environment. The long run times and flakiness can drastically reduces the effectiveness, reliability, and therefore value of huge suites of functional/e2e tests.

Automated Testers and TDD are like water and oil.

No matter how you look at it the introduction of the Automated Tester to a project works against Test-Driven Development.

Let’s examine the situation. Let’s say that an organization is attempting to be forward-thinking and tries to deliver working software frequently. They practice Iterative Development and have a concept of a sprint. In this scenario perhaps the Automated Tester works with the Product Owner and developers in a Three Amigos meeting. They agree to the requirements, and the Automated Tester writes one or more functional/e2e tests before (or while) the developers begin their own coding. Sounds good right? Heck, it even sounds like it lines up with TDD. After all, there will be failing tests that will (hopefully) pass when the developers are done.

Here’s what happens in actuality. The Automated Tester goes off to write their tests with some kind of design in mind. The UI will behave this way, or the service will have this contract. The developers go off to begin their own Test-Driven approach to meet the requirements. They don’t get too far before they realize that the contract they came up with doesn’t work. Different parameters need passed to the service. A different approach to the UI is required. The hooks the Automated Tester is relying on need to be changed to meet the technical requirements. The tests that the Automated Tester created don’t pass, and need to be re-written to account for the actual behavior and technical requirements.

Because we’ve separated the functional/e2e tests from the rest of the development we have effectively knee-capped the “Driven” part of Test-Driven. We have already come up with a design that probably doesn’t match reality, because the developers who are responsible for implementing the functionality were not the ones who wrote the first failing tests. They have also made assumptions about what the design should be before allowing a single test to drive that design in the first place.

For those developers who are not practicing TDD, now that the role of the Automated Tester has been introduced some developers may interpret this as a license to once again “throw it over the wall” so to speak. They will no longer feel the pressure to properly cover their own code with tests at the right level. Instead, they can rely on the Automated Tester to cover the behavior of the functionality in the higher-level tests, and nevermind that any number of corner cases may not be covered at this level (or if they are they are represented as additional slow, expensive tests which shouldn’t be in the e2e layer to begin with). Again this harkens back to the bad old days when developers would churn out some production code and simply pass it on to QA, who would then either pass it or throw it back over the wall to the developer again.

Let’s also consider the point at which the tester’s code is integrated in the codebase. Should this before before the developer begins? How far ahead of development do they need to be? Should their code compile for functionality that doesn’t exist yet? Should it be commented out? Ignored? At what point do we consider this code rot? How about after the developer’s work? Are these post-development tests valuable? Are they duplicating tests which have been written in lower layers of the system as faster, more reliable unit or integration tests? Should the tester add their tests to a separate codebase altogether, divorced from the regular build? Experience has shown me that this drastically increases the feedback loop time. How about on another branch? How often does this branch get merged? Who’s responsibility is it to do the merging? The developer’s? The tester’s? As you can see even the nuts and bolts of how the Automated Tester conducts their work comes into question.

We don’t need Automated Testers, we need developers who can write tests.

Some of you are probably thinking that I’m being uncharitable to Automated Testers, that I’m not accounting for skilled developers who happen to be in that role, or who have an affinity for writing tests. I acknowledge that these individuals exist, but my experience has been that they are by and large in the minority. I would also encourage these individuals to consider moving into development roles, as there is no shortage of a need for developers who know how to write tests and testable code.

Projects would be better served by moving tests from the slower functional/e2e test layer down to the integration and unit test layers, and putting them back in the hands of the developers. Developers are the ones positioned to know what tests should be written, and in what layers. They’ll be able to determine if a test is best written as a unit test, an integration test, or if required an e2e test.

Developers are ultimately responsible for the functionality of the software, and as such they should be the ones to insure that their own code works the way they say it does.