Wouldn’t you love to work on a Greenfield project? A fresh start where you can make sure that you don’t repeat any of your prior mistakes! A great TDD application with a code coverage that will make everyone jealous. Oh, and off course you will finally have the opportunity to play with all the newest technologies!
We had this opportunity three years ago. A brand new project to build a Social Intranet solution that would rock the world. We started with great ideas but not everything went as planned.
In this blog I would like to show some of the things that went wrong, what we learned from them and how we got our project back on track.
A great start (?)
The project started about three years ago. I can still remember how the empty Visual Studio Solution Explorer looked like when I created the basic skeleton structure.
We started with the following building blocks:
- C# 4.0
- ASP.NET 4.0 WebForms
- Entity Framework 4
- WCF
We used the basic layered architecture approach:
We used the Model View Presenter pattern to make the WebForms part better testable. Because we where certain we would get a massive amount of users we made sure that our back-end was scalable by introducing a WCF service layer.We used abstract factories for creating our services so we could easily switch from in memory to the WCF model.
At the time of starting this project the Entity Framework team had just released Self Tracking Entities. This was the way to go when using EF with WCF. Instead of coding all kinds of DTO’s by hand we could just serialize our STEs and send them over the wire.
In our data layer we choose for the Unit Of Work and Repository pattern. We made sure the T4 template for generating our entities was in a separate project where they could be completely POCO.
The first Unit Tests where created and our project started to grow. We used a setup with Team Foundation Server Gated Builds that would run all our unit tests before allowing anyone to check in some code. The code was then automatically deployed to an internal test server.
At this point, I moved to another project. I saw how the project made good progress (we’re using it internally). Things looked good! But when talking to the colleagues working on this project I understood there where some serious issues.
Testing Hell
When we started this project we had little experience with unit testing. We used it in some small projects and everything looked good. But this project was our first serious try at doing some real unit testing.
And we made some mistakes. As it turned out, all of our unit test where actually integration tests.What’s the difference? A Unit Test makes sure that a piece of code is tested in complete isolation. It doesn’t depend on other code and it definitely doesn’t depend on external things like a database or a WCF service being configured and running. They can be executed really fast and can be controlled to test each path trough your code.
An integration test, tests your all components of your application to make sure that everything works together. Because of this they are slower but they can be really useful in testing the most important functionality of your application (see this article: Unit Testing, hell or heaven? for a more complete explanation).
At the start of running our unit test, a setup function would execute that would make sure the database was setup correctly. Each time we added functionality, we expanded our setup function to facilitate all our tests. To make sure the database was in a consistent state, we wrapped all of our test code in a transaction. We started creating base classes of our tests that would provide all kinds of helper functions.
A typical Unit Test would look like this:
[TestMethod]
public void TestMyViewIsCorrectlyInitialized()
{
using (TransactionScope scope = new TransactionScope())
{
IMyView view = MockRepository.GenerateStub<IMYView>();
MyPresenter presenter = new MyPresenter(view);
presenter.PrepareView();
Assert.AreEqual(3, view.NumberOfItems);
}
}
When reading this test, ask yourself what does this test actually do? Why is the answer three? What’s happening and what am I testing?
Well, the answer is three because this test goes all the way to the database to get some records (you know, from the setup function!), return those to the presenter over a WCF connection and then initialize the view. Oh, and in the mean time it will also update some files on disk and write some auditing information. For the production environment this was no problem, but for a developer trying to get into his ’Red Green Refactor’ mode this was a big problem.
These tests would also break for all kinds of reasons. Maybe because someone changed the database initialization code (I need 4 items for my scenario!) or maybe because somewhere in the code there was a hard dependency (like throwing an exception if a certain file didn’t exist on disk).
So slowly on more and more tests where turned off and eventually the gated build was turned off because it would just take way to long to do a check in. The project was still making progress. Nice functionality was added but the underlying code started to become a big ball of mud.
All the typical problems from a ‘not unit testing world’ where showing up. Bugs where returning after they where already fixed and code would keep braking when changes where made.
And some other problems
Next to testing problems, there where other issues. The Self Tracking Entities where not working out as we hoped. They where causing all kinds of bugs when serializing multiple graphs and trying to submit changes to them. They where also becoming bigger and bigger. To remove STE , we also need to get rid of WCF.
We came to the conclusion that we where using a Anemic Domain Model. Our service layer did all the actual work. It contained all the business logic and our entities only consisted of getters and setters that where mapped to DTO’s to be send to the presentation layer. Eventually, there was a repository for each entity. The repositories itself where also exploding. All kinds of methods where added for getting data in all kinds of ways.
Because TDD was abandoned more and more code ended up in the code behind of our aspx files and functionality started to leak from the layer where it should have been. More and more coupling, untestable code, duplicated code and all the typical architecture problems started to show.
What would you have done
A couple of weeks ago I was asked to have a look at the project and see if we could turn things around. What’s your take on this? What would you do?
We have chosen for a quite drastic approach. Tomorrow I will share some of the steps we took and the key points we learned from it.