In the natural progression of software development to faster deployments and shorter release cycles continuous deployment pushes the code-release time delta as close to 0 as we humans can get.

This does not come for free. We need highly skilled, highly disciplined developers, versatile enough to assume responsibility for the entire code base and rely on infrastructure as intricate and intelligent as our production systems.

With the drive for continuous deployment systems the development environment increasingly adopts features of productions systems with caching, load balancing etc. Large scale development environments go even further adopting advanced code analysis techniques in order to minimize test execution times and shorten that time delta (check Ashish Kumar’s InfoQ talk about Google’s development practices).

How do you manage the code that is your development environment? Is it subjected to the same rigorous testing and vetting procedures you use for your product?

I consider our development environment a production system. It is in production much much earlier than the actual product we are building and usually has much stricter stability and performance requirements. And it’s users are more demanding and a lot more impatient.

Honestly, I am more stressed in the beginning of a project ramping up automation and stabilizing the development environment than on the eve of a release.

Failures in our build system are not tolerated. Case in point: In my current team an hour’s downtime due to a build script error translates to three work days lost not considering frustration and disappointment factors. If the system is down a day I might as well scratch a month’s worth of budget off.

I’ll take a step back now and constrain (or rather expand) the term ‘development environment’. It is not just your build scripts, your build server and your test infrastructure. It also includes your source control infrastructure, the issue management system, the knowledge servers (wikis, blogs, CMS, information radiators) and all the little tools/scripts you wrote to facilitate menial tasks in the project.

And while we use many third party servers/tools,nothing just works off the shelf. Each project has unique requirements and so a bit of custom “glue” code.

And I will ask again, do you treat your development environment as a production system? Are there tests, versioning, releases? How about continuous integration/deployment?

I fall into a case of infinite recursion thinking about the build system that builds the build system that builds the build system…ad infinitum. But instead of the mental picture of falling into constantly smaller build systems I like to think about the world snake, the snake that eats it’s own tail. Perfect symbol for dogfooding and infinite recursion in one.

So we need a development environment that can support it’s own evolution and the development of our product. Which highlights the need for code reuse, scalability…need I go on? Aren’t these the same qualities we want for our “production” systems?

The answer to how you effectively manage your development infrastructure is typically the same answer you’ll get if you ask how to manage a software project because as it so happens, your development infrastructure is a software project.

And while there might not be a silver bullet or a universal HOWTO on how to do it right, it is important that you do not underestimate the complexity or importance of said environment and plan accordingly.