Continuous Integration Matters
Recently I’ve been spending quite a bit of time trying to work out the best way to enable my team at work to be able to rapidly meet several impending deadlines for new projects. In this I have once again started looking at some agile practices that have been largely tossed to the sidelines. Some of the more important practices that I am currently trying to preach again are the old of code standards and unit testing. While there are many people who see these as barriers to progress, viewing them as yet more things they need to do, my hope is to perhaps give them the tools to be able to see the truth; that these practices exist to empower developers.
One of the core issues that I see day in a day out is that testing almost always takes 2-3 times longer than originally expected. This is by no means the tester’s fault though. It has been shown over and over again that following a waterfall style approach to development is in fact the most inefficient way of producing software. The reason for this is because when you put testing at the end of the process, you force everything to grind to a halt as you backtrack to fix bugs.
In fact, unit-tests are designed to help prevent this. Now while I won’t go into the specifics on what unit-tests are or why they are good, as that has been covered by many wiser men than myself, I will say that holding developers accountable for writing them is a key component to making this ecosystem work.
Enter Continuous Integration
This begs the question of, what exactly is continuous integration. To quote the venerable Martin Fowler:
Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.
More specifically it means automating an incremental build system, that encourages developers to find and fix bugs as they create them. This is after all the moment that they are in the zone, working with that code and most likely to be able to react quickly, already emerged in the problem. Including some standards support to warn the developer when they are about to commit code the is needlessly complex, can provide warnings that they may want to re-factor a small bit before continuing.
The important part of this whole process is that the developer be made aware of the issue as close to immediately as possible. If you’re in a month long project and major testing doesn’t happen until the fourth week, then the likelihood bugs uncovered in that fourth week pushing out the deadline is increased. This is simply because it is harder to go back to code that you previously wrote, than it is to fix it immediately after writing it.
Not to say that you will find every problem up front, but I can say from experience that when ever I spend the time to write unit tests and verify those tests before I push code, testing time is almost always miniscule in comparison to when I do not.
A recent example comes to mind. I was writing a small rules engine that would based on some simple conditions allow a CMS to control, based on dynamic input, whether-or-not a small bit of code would show up in the page. During this process, I did not do my due diligence and write my unit tests as needed. Because of this, some simple rules went back and forth with the testers for 3 days.
Finally, realizing my error, I stopped, wrote out a set of 20 test cases for how I expected the rules to behave, ran them, found five failures. Spent 20-minutes working them out and finally got my code to stable. At this point, not only was I confident that the code would work, but it went through the QA process with flying colors. Moral of the story, had I done the right thing up front and spent that 20-minutes working out the test cases, I would have saved 3-days of time while bugs bounced back and forth from me to the testers and back.
Continuous Integration is about keeping developers accountable for doing these things and verifying that the code they write doesn’t break someone else’s code before it makes it to the testing phase.
When does it take to start?
In generally you need three things to start using continuous integration in your development cycle.
The first is the simplest, version control. This can be anything from GIT, to Subversion, all the way to CVS. The latter having fallen out of fashion for various reasons, though still in use. No matter what, every project and team should be using some sort of version control. It is your contingency plan if errors occur. It is how you find out what changed and who changed it.
Second we have the CI server itself. This can be either easy or difficult, depending on your platform. .Net developers have Team Foundation Server available to them, which can be configured to provide this type of service automatically. The specific server is called Team Build, which will keep track of build successes and failures and help keep the developer informed of problems.
A more general solution for the rest of us developers is Cruise Control. This server has various plugins that allow it to work for Java, Ruby, Python, PHP and just about any other language out there. It contains reporting tools and very tight integration with Subversion. If you are in need of a solid, quick start package, this is where I would start looking.
Lastly, you need someone to keep the developers motivated and encouraged to keep following these practices. It is very easy to “forget” to create a build process for the new application that you’re developing. It can definitely seem like more of an obstacle than an option if you are only looking at the immediate gains (though I argue confidence is a huge immediate gain). So, having someone (or a group of someones) to keep people excited and motivated to use these tools.
Actively show the successes that they have provided. Gather some metrics that show how many bugs were avoided in the testing phase, because of the incremental builds. How testing time has decrease. How production bugs have gone down. All of these things are the benefits of failing early.
When should I start?
Right now!
Whether you are starting a new project, or deep in the trenches of an old one, if nothing else, then start writing unit tests. However, taking a few hours to set up Cruise Control or some other build server will be paid back in full very quickly. If you have the server set up, then make sure when you create every new project that along with setting up your repository, that creating an automated build is on your list of first things done.
If you can have this set up and ready to go before you write your first line of code, then you are in a rock-solid position. Just remember, it is an active process. Continuous integration is a way to reinforce the good habits, it does not replace them. It is up to every developer to do his part.
Google Wave: Why I care.
The Internet has been buzzing lately about Google’s new application, Wave. Though it is still under a limited preview, everyone seems to be talking about it. I have heard everything from the evangelicals touting this as a sign of the coming singularity, to people just not getting it or thinking it too complicated to understand.
For most people, Google Wave indeed will not make much sense. At least not yet. However, for the developers and engineers out there though and the people who work remotely or just on teams, there is something more to be seen in its future. When I look at Google Wave, I see the ability to email out a requirement document for a project to a whole team. The ability to change and modify it as the project grows and shrinks (as it always does). The ability to keep track of the changes, much in the way one would do with a wiki. The ability to add someone to an existing conversation and for them to be able to easily use that history and the comments along with it to understand why changes are being made.
In short, I see a complete change in how a project team works.
There is nothing truly revolutionary here. With the use of IM clients, a project management tool like Base Camp, a wiki and an E-mail client, you could do this today. Why would you want to though? Keeping track of these disparate systems is possible, but if there is a better way, I would take it. Wave gives you a way to do all of these things succinctly.
For me, Google Wave is more than some preview software. It is a preview of the next phase in the web’s evolution. It may not be the final draft of what is to come, but it shows us that there is a better way to communicate than E-mail really provides us. We do not need to be slaves to one off messages. Chained by the droves of responses and replies.
What Wave has to offer may not seem immediately obvious to you. That is okay. Its benefits are not short term ones. Until it can act as an e-mail client as well, so that people can more easily transition into it, adoption will be slow. However, it is an open protocol, just as E-mail itself is. Which means we will hopefully see more servers out there in the coming months and years.
Google may have started this thing we now know as Wave, but I by no means think it is anywhere near finished.
