Fast feedback is important. Knowing you broke something a month ago or just a few minutes ago can make a huge difference.
More and more teams use a build server to compile the code and run a variety of checks on each check in. This is often referred to as the commit stage. One nice feature of TFS to make sure the quality of your code base is guarded, is a Gated Check-in. A Gated Check-means the whole commit stage is executed on some code a developer wants to check in. Only when the commit stage succeeds, the check in is allowed.
This way, you can be sure that the code on your server is always in a good state. An important characteristic of successful gated check-ins is that they are fast. If developers have to wait a long time, they will start checking in less regularly and find ways around your build server to share code with other developers.
How can you optimize the build time of your Gated check-in?
What’s acceptable
An acceptable build time depends on your team. However, from experience you should aim for a Gated Check-in build of less then five minutes. Somewhere around one to two minutes would be perfect but that’s often hard to realize. It’s important to regularly check your build time and make sure you constantly optimize whenever necessary.
1. Check your hardware
The Team Foundation Server build architecture consists of two important parts:
- Build Controllers
- Build Agents
A Build Controller connect to Team Foundation Server. They monitor your build queue and contain the logic of executing the build workflow. The actual build runs on a Build Agent. This means that Build Controllers and Build Agents have different resource needs.
A Controller is memory intensive. An Agent is both CPU and Disk intensive. When you do a default installation of a TFS Build Server, both the Agent and the Controller are installed on the same machine. This is an easy setup and requires less servers but if build speed becomes an issue, an easy solution is just to scale out your build hardware. Moving your agents to a different server then your controller is a first and easy step.
Instead of scaling out you can also scale up by using a machine with more CPU cores, more memory and faster disks. I was at a customer once where their Build took 90 minutes to run. Because they where constrained on their on-premises hardware, we moved the build server to Azure. By using a larger VM size on Azure the build time dropped to 15 minutes. That’s a very quick win.
2. What are you building?
When you create a new standard Build Definition in Visual Studio, you get a build that monitors all the code in your team project. This is easy to get started since you probably only have a single source tree with one solution file in it that contains your code.
This is defined in your Source Settings in your Build Definition file as you can see in the following image:
Here you see there are two work folders defined. One points to your source code and is marked as Active. The Cloaked working folder makes sure that your Drops folder (which is created by a build that copies the resulting binaries to the TFS server) is not monitored.
When the Build server starts a new Build it will get a fresh copy (in the following section you will see if this is incremental or from scratch) of all the source files beneath your work folder mapping. If you start getting multiple branches or other folders that contain documentation or other non-essential stuff for your build, the build server will download all those files every time a build runs.
Modifying your Source Settings to only include the files you really need can speed up your build time quite substantially.
3. How are you getting your sources?
If you create a new Build Definition you will see the following setting in the Process tab:
What’s important to note is that the Clean workspace property is set to true. This means that every time a build runs, all the source files are first removed from the Build Server and then the whole source tree is downloaded.
By setting this value to false, you will do an incremental get, just as you would in Visual Studio. The Build Server will only download the files that have changed since the last build.
Now of course, this is a nice default. Having a clean slate every time you run a build makes sure you won’t have any artifacts left from previous builds.
But you know your code and your team best. You can experiment with setting this value to false and checking what it does to your build time.
4. Where are you getting your sources from
Normally you connect your Build Server directly to Team Foundation Server. This means that when the Build Server needs to download a newer version of a file, it will go directly to the TFS server and download it from their.
By using a Team Foundation Server Proxy you can use an in-between cache that sits between your Build Server and the TFS Server. The proxy will cache source files and optimizes getting files from TFS. Especially when your Build Server and TFS Server are not co-located, a proxy can save you huge amounts of time by installing it nearby your Build Server (on a controller machine or on a separate server in the same location as your Build infrastructure).
See the following MSDN article for configuration details: Configure Team Foundation Build Service to Use Team Foundation Server Proxy
5. How are you building
There is a huge change your Build Agent is running on a multi-core machine. MSBuild, which is used in the Build Template to do the actual compiling of your code, has an option to start using multiple cores and parallelizing your build process.
If you look at MSDN you will see the following documentation:
/maxcpucount[:number]
/m[:number]
Specifies the maximum number of concurrent processes to use when building. If you don’t include this switch, the default value is 1. If you include this switch without specifying a value, MSBuild will use up to the number of processors in the computer. For more information, see Building Multiple Projects in Parallel with MSBuild. The following example instructs MSBuild to build using three MSBuild processes, which allows three projects to build at the same time: msbuild myproject.proj /maxcpucount:3
By adding the /m option to your TFS Build you will start utilizing multiple cores to execute your build. You can add this option in your Process tab for the MSBuild arguments property. This is also a setting you have to test and make sure it works for you. Sometimes you will get Access Denied errors because multiple processes are trying to write to the same folder.
6. Building less
When you are dealing with a huge application that maybe exists of several sub applications, you can start splitting your application in distinct parts that can be build separately. By using NuGet and your own NuGet repository you can then have those subparts publish a precompiled and ready to go NuGet package. This package can then be used by other applications that depend on it.
This means that you won’t have to build all your source code every time you run a build. Instead you only build those parts that have changed and reuse the resulting NuGet packages in other parts of your build.
If you have a look at the NuGet documentation you will find some easy steps that allow you to create a package and setup your own NuGet server.
These are my favorite steps for optimizing a TFS build. What are your favorite steps? Anything that I missed? Please leave a comment!