Functional Testing Pipeline for web applications

I do not know what the rise of dynamic languages have bought, but one good thing is that it has thrown a very bright light on the importance of testing and has created a community of people who are practicing testing as a part of software development. While this is happening on one side, there are still many others who keep debating about do we really need unit test, does integration tests matter more than unit tests, does TDD pay off, what to test and what not to test, is TDD dead, etc...

Me and my team are currently engaged in building an ASP.NET website. This website on its back-end has to talk with multiple services to provide what is needed for the user. Let me try to explain what we have done and if it makes sense, it might be useful for some poor soul who is in the same situation as us.

A standard web application, or any user facing application, typically does these things

1. Get user input
2. Do validation
3. Convert the input to a format that is easy to process.
4. Process the input, which is generally called the business logic.
5. Convert it to a format that can be persisted.
6. Persist the data.
7. Take the result, and convert it to a format that can be sent back.
8. Do some processing on that result.
9. Transform it to a view able format.
10. Display it to the user. 

What should be tested? All of it. 

Will one test suffice all? No!!!

How do we design it? Do we write all these as one module? No. We split it across multiple layers and give each layer a cohesive functionality. That exactly applies to tests too. We need different types of tests based on the what area is being tested.

Unit Tests

Unit test should be testing some sort of logic implemented in the program. If there is no logic, we do not need to write unit tests. I have had push from management to get a 100%  unit test coverage, but that can be done only for code that has logic. So, in our case we have close to 100% coverage on business layer, 20% on the Presentation Layer and 20% on Data Access Layer. The 20% coverage in Data Access Layer and Presentation Layer attributes to test cases that cover transforming data between layers and error handling. Otherwise, there is nothing much to unit test in these layers.

The unit tests start with a mocked input and applies it on the unit(method or class) with mocked dependencies and checks the result against expected output. These tests validate the code that we have written and does not validate 

1. Object/Data Format.(We hardcode them, who knows if the actual data from from the other layer might be completely different)
2. Dependency on configuration files.
3. Dependency on databases.
4. Dependency on external systems.
5. User interface
6. Input/Output channels

Integration Tests

Our integration tests exactly covers the items that the unit tests have missed. We start our integration tests with fabricated , but valid inputs at the Presentation Layer level.In ASP.NET MVC jargon, we hit the Controllers directly with a pre-populated HttpContext and any other data and look for the response. This test exercises all the external systems and also dependencies.There are no mocks, the application talks with the real systems and gives real outputs. Since we are not mocking the external systems, we will end up setting up some data  though external means on those systems or during the "Test Initialize" phase of the test.This makes it harder and time consuming(to both code and to run) than the unit tests and hence we do not do an extensive tests. With these Integration tests, we try to cover most of the user scenarios and functionality and achieve around 80% coverage of the code.

What is not tested as a part of the Integration tests?

• User Interface.
• Usability.
• Client side code that runs in the browser.
• Any interceptor code like Web Filters, that are hit before the flow reaches the Controllers. 

End to End Tests

These tests are the closest to user behavior,We do not mock data, we do not mock systems. The tests start right from the browser and the results are also verified at the browser. We use Selenium web driver to perform these tests and have lately moved from Chrome to Phantomjs driver. Seems good till now and we're able to run more test cases concurrently with phantomjs. 

The unit and the integration tests are run as a part of CI and hence we never have an unstable build hosted, even in test environments. The end to end tests are scheduled to run everyday. With these 3 tests in place we cover almost 80% of the testing and 100% of the functionality. 

The 20% is left out for manual testing. It is a web app and it is a human who is going to use the application. The usability and user interface can only be experienced and there is no replacement for manual testing in this area. With some sort of an automation we can figure out that button is in the right place or the text is in the prescribed size. however, only a human can tell whether the right place for that button is convenient and the prescribed size for the text is comfortable.

We have a feeling of completeness and are confident that our product is well tested. The coming years will be the real test for our process and will reveal the pros and cons, which I sure will share with you.

References

Just Say No to More End-to-End Tests - by Mike Wacker
Test Pyramid - by Martin Fowler