No Free Time

July 17, 2008

Howto: Only include part of a project workspace in a TFS 2008 Team Build

Filed under: .net, CI, tfs — Tags: , , — andrewmyhre @ 12:56 am

I’m working on a website with a large design aspect, so we have a lot of .psd files (about 200mb) kicking about the place. We want them source controlled, so they live in the TFS 2008 team project folder in source control. But then our Team Build includes them when downloading the source files, so that quickly fills up our build server’s drive and makes a single build take up to 11 minutes, which is far too long for a basic eCommerce website.

So I trimmed these files from our build process. Here’s what I did:

  1. Copy the .psd files and any other files that aren’t required for a build into a seperate folder in the root of the team project. (All our website/source files are in another folder called ‘Current’, as in the ‘current version’, as opposed to ‘Phase 1′, ‘Phase 2′)
  2. In Team Explorer, right click the build you want to edit and selected Edit Build Definition…
  3. In the Workspace tab, enter the path within source control where you want to start downloading files from. In our case I chose the $/[project]/Current folder, so the /PSDs folder will be ignored and not downloaded.

Now, thanks to the above steps my build history folders are 1/10th the size they were to begin with, and the build process takes less than half the time!

May 13, 2008

Where to manage the build location in TFS 2008

Filed under: .net, CI, tfs — Tags: , , — andrewmyhre @ 11:11 pm

Until now I’d been confused between the agent working directory, the build service accounts’s temp folder, and the workspace created by Team Build. Do I have to manage all three settings? Do they have to be different? Today I discovered there’s just one location I need to manage for each build agent.

In my efforts to clean up and standardise where projects are being downloaded and built on our build server I discovered that the workspaces automatically created by Team Build don’t actually determine where the build happens. This is because every time TFS performs a build it deletes and recreates a workspace for that build. The source directory path it uses comes from the Build Agent definition:

Build Agent Working Directory

May 9, 2008

Making Team Build and Web Deployment Projects play nice: Part 2

Filed under: .net, CI, tfs — Tags: , , — andrewmyhre @ 10:02 pm

Right, so yesterday I figured out how to make ASP.Net 2.0/3.5 Website projects work in a Team Build environment using Web Deployment Projects. Today I figured out how to get Web Application Projects to work.

The problem is explained in this thread - take a look at maharik’s post and BradleyB’s reply. maharik was having the same problem I was having, where the Web Deployment project was trying to build the wrong path. At this stage I still don’t fully understand what’s going on (why does the web deployment project need to be pointed to the compiled website?) but I know that it works.

So as background, I’m talking about a .Net 2.0 solution containing a Web Application Project, a Web Deployment Project and zero or more class libraries. The solution is part of a Team Project and I want to set up a Team Build to build the solution.

The Team Build must be set up using the Mixed Platforms as the platform (contrary to what the article I linked to yesterday said - but that article is still correct as far as Website projects are concerned). Choose a configuration (Release/Debug) and make sure your Web Deployment project is configured to build under the same configuration.

Try running the Team Build now. Everything but the WDP should build fine, and you should get errors like this:

error ASPPARSE: Could not load type 'masterpages_default'.

Now you need to follow the directions posted by BradleyB. Find the <SourceWebPhysicalPath>…</SourceWebPhysicalPath> element in your Web Deployment Project file. Comment it out and replace it with:

<SourceWebPhysicalPath Condition=”‘$(OutDir)’ != ‘$(OutputPath)’”>$(OutDir)_PublishedWebsites\WebApplication1</SourceWebPhysicalPath>
<SourceWebPhysicalPath Condition=”‘$(OutDir)’ == ‘$(OutputPath)’”>..\WebApplication1</SourceWebPhysicalPath>

Where ‘WebApplication1′ is the name of your website. Now check in the WDP file.

Now when you run the Team Build again you should find that everything runs successfully, and in your release folder there’ll be a deployment folder with the same name as your Web Deployment Project. Great! Now you can follow the last step from yesterday’s post to add the AfterCompile step to copy the website to your deployment folder.

Nice!

Make Team Build and Web Deployment Projects play nice

Filed under: CI, tfs — Tags: , , — andrewmyhre @ 4:30 am

Update: Part 2, Web Application Projects.

The thing I like about Web Deployment Projects is the easy configuration of things like compilation (I like to compile to a single assembly), configuration section replacement (so, so much easier than setting this up in NAnt) and deployment. I also really like Team Builds, because they’re so easy to set up (compared to NAnt and CruiseControl.Net). But they don’t have the customisation features of Web Development Projects that I mentioned above. Plus the deployment option in a Team Build is absolutely useless to me, because it deploys the web project to a different folder every time. So I need a way to combine the two configuration features plus make sure the website is deployed to a specific folder on our development web server.

Initially I thought this would be really easy, in fact I expected that the Visual Studio team had specifically designed Team Build and WDPs to compliment each other. Ummmmm not exactly, it seems. I had loads of trouble getting the two to work together, with loads of builds failing due to stupid problems… usually due to the WDP losing its context and building the wrong stuff.

The best guide I found on the subject is this one. It explains that ASP.Net Websites and Web Application Projects are treated differently by Team Build (not why, just that they are) and that you need to use specific, different platform configurations.

To get my ASP.Net Website project (I haven’t tried Web Application Projects yet) working I created a WDP for it. I specified that it should build under the Release configuration, then checked in my solution. I then created a Team Build which targeted the Release|Mixed Platforms configuration. At that point I ran the team build and happily everything came up green, including the WDP.

To my dismay though, although the Web Deployment Project ran, it wouldn’t copy the build to the folder I specified in the WDP configuration. It just wouldn’t do it, and I have a feeling it’s something like the Team Build overriding the drop folder property that the WDP picks up.

The bottom line is that you need to manually override a build task to perform the copy yourself. This isn’t that difficult, and I’m happy to say it’s a step that I can expect other the other developers I work with to perform without needing to know too much about what’s going on.

So the final step was to modify the Team Build project file slightly. First I added the following property group, just below the closing </ProjectExtensions> element:

<PropertyGroup>
    <!-- change this to the name of the Web Deployment Project -->
    <BuildFileName>Engelbart2</BuildFileName>
    <!-- change this to location where you want the website deployed to -->
    <DeployLocation>\\engelbart\wwwroot\projects\tequila\digital playground\buildtest\staged</DeployLocation>

    <!-- don't worry about this -->
    <WebBinariesLocation>$(SolutionRoot)\..\Binaries\Mixed Platforms\Release\_PublishedWebSites\$(BuildFileName)</WebBinariesLocation>
  </PropertyGroup>

Then I added the following below the last </ItemGroup> element (I don’t think it actually matters where you put this):

<!-- deploy the compiled site to where we want it to go -->
  <Target Name="AfterCompile">
    <MakeDir Directories="$(DeployLocation)" />
    <!--<Exec Command="xcopy /y /e '$(WebBinariesLocation)' '$(DeployLocation)'"/>-->
    <ItemGroup>
      <ProjectSourceFiles Include="$(WebBinariesLocation)\**\*.*"/>
    </ItemGroup>
    <Copy
        SourceFiles="@(ProjectSourceFiles)"
        DestinationFiles="@(ProjectSourceFiles->'$(DeployLocation)\%(RecursiveDir)%(Filename)%(Extension)')">
      <Output
          TaskParameter="CopiedFiles"
          ItemName="SuccessfullyCopiedFiles"/>
    </Copy>
  </Target>

This works because your web deployment project is build as part of the team build into it’s own special folder. So for instance I could have team builds called ‘Development’, ‘Staging’ and ‘Production’, each with special web.config settings which the WDP would handle for me. The output from Team Build will be three websites, each in a distinct folder with the correct configuration. It wouldn’t be difficult to modify the above script to copy three seperate releases.

Hope this clears up some mysteries…..

March 13, 2008

Change TFS Build Agent Build Location

Filed under: CI, tfs — Tags: , , — andrewmyhre @ 2:49 am

When you create a TFS 2008 build, there are three build locations you need to be aware of.

First is the location that the build agent ‘gets’ the source to for compilation. You can specify this when creating the Build Agent - just set the Working directory to the location you want the source to be downloaded to. Normally this will be set to the windows temp folder for the account the build service is running under.

The second location is where the build is compiled. This is defined within the Build Definition as the Workspace for the build. TFS copies the source files from the ‘working directory’ to the ‘workspace’ before compilation. It is compiled into a new sub-folder with the same name as the build definition.

Thirdly, you have the option of specifying a build drop location. Your compiled build will be copied here into a new sub-folder, datestamped and versioned by the build number. I haven’t worked out yet how to instruct the build script to deploy a compiled website to a static location. Would love to know if anyone can shed any light!

February 26, 2008

Build Helpers

Filed under: CI, nant — Tags: , — andrewmyhre @ 8:08 am

I’ve made a couple of build helper files which may come in handy. They cover compilation, testing and deployment, and hopefully can be reused between projects with minimal need to be rewritten. I’ll share the compilation one today.

Compilation Build Scripts

As background my folder structure looks like this:

/Solution/ClassLibrary1
/Solution/ClassLibrary2
/Solution/Website1
/Solution/Resources
/Solution/Build
/Solution/Build/bin
/Solution/Build/src
/Solution/Build/testlib
/Solution/Build/deploy

Resources is where everything like NAnt, MBUnit, log4net, NCover etc all reside. Various things happen within the Build folder: src is where all projects are copied into before being compiled, bin is where each compiled assembly is placed for easy reference for other assemblies (e.g: where ClassLibrary2 depends on ClassLibrary1), testlib is where all test assemblies are compiled and run, and deploy is where our final compilation will end up.

The reason each assembly is copied into the src folder prior to compilation is so that we can do tricky things like substituting *.config and AssemblyInfo.cs files, as you’ll see below.

The build file starts this way:

    1 <?xml version="1.0"?>
    2 <!--EXTERNAL_PROPERTIES: output.dir;AssemblyName;debug;assembly.label-->
    3 <project name="general compilation">
    4   <target name="clean" >
    5     <delete dir="${output.dir}"/>
    6   </target>
    7
    8   <target name="init" depends="clean" >
    9     <mkdir dir="${output.dir}" />
   10     <mkdir dir="${output.dir}/bin" />
   11   </target>

Straight forward. Note that the ${output.dir}, ${AssemblyName}, ${debug} and ${assembly.label} properties are expected to be set - these will be set from your calling build file. ${output.dir} can simply be “c:pathSolutionBuild”. ${AssemblyName} is just the name of the assembly you want to compile at that time.

Assembly compilation:

   13 <target name="general.compile.assembly" description="compiles a satellite assembly">
   14     <echo message="*******************************" />
   15     <echo message="***COMPILING ${AssemblyName}" />
   16     <echo message="***TO ${AssemblyOutputFolder}" />
   17     <echo message="*******************************" />
   18
   19     <delete dir="${output.dir}src${AssemblyName}" />
   20
   21     <copy todir="${output.dir}src" flatten="false">
   22       <fileset>
   23         <include name="${AssemblyName}***.*" />
   24         <exclude name="${AssemblyName}bin" />
   25         <exclude name="${AssemblyName}config" />
   26         <exclude name="${AssemblyName}**AssemblyInfo.cs" />
   27       </fileset>
   28     </copy>
   29
   30     <call target="general.labelassembly" />
   31
   32     <csc target="library" output="${output.dir}bin${AssemblyName}.dll" debug="${debug}">
   33     <sources>
   34       <include name="${output.dir}src${AssemblyName}/**/*.cs" />
   35     </sources>
   36     <references refid="assembly.resources" />
   37     </csc>
   38     <copy file="${output.dir}bin${AssemblyName}.dll" tofile="${AssemblyOutputFolder}${AssemblyName}.dll" />
   39   </target>

As the description says, this target compiles a satellite assembly, or class library. It first deletes the src folder (/Solution/Build/src/AssemblyName) before copying the latest source into it. Then it calls the “general.labelassembly” target, which we’ll see shortly. Finally it compiles all .cs files into a dll.

Note line 36, where the references tag uses assembly.resources. This is because you should call the target like this:

    5     <!-- Class Library 1 -->
    6     <property name="AssemblyName" value="ClassLibrary1" />
    7     <assemblyfileset id="assembly.resources">
    8       <include name="${resources.dir}/log4net.dll" />
    9     </assemblyfileset>
   10     <call target="general.compile.assembly" />

This way you only specify exactly what you need to in order to build the assembly.

Now, websites. First we need to prepare a few things:

   41 <target name="prepare.web" >
   42     <mkdir dir="${output.dir}src${website.name}" />
   43     <echo message="*******************************" />
   44     <echo message="***COMPILING ${website.name}" />
   45     <echo message="*******************************" />
   46
   47     <copy todir="${output.dir}src" flatten="false">
   48       <fileset>
   49         <include name="${website.name}***.*" />
   50         <exclude name="${website.name}config" /> <!-- this are where we put environment-specific config files -->
   51         <exclude name="${website.name}bin*.*" />
   52         <exclude name="${website.name}web.config" />
   53       </fileset>
   54     </copy>
   55     <copy todir="${output.dir}src${website.name}bin" flatten="true">
   56       <fileset>
   57         <include name="${output.dir}bin*.*" />
   58       </fileset>
   59     </copy>
   60     <copy todir="${output.dir}src${website.name}bin" flatten="true">
   61       <fileset refid="website.resources" />
   62     </copy>
   63   </target>

Then we can compile the site itself. Note that we don’t copy the web.config over with the website. This is because I have a seperate target to copy a config file, replacing environment specific variables as it copies. I’ll come to that in a bit.

The next target is straightforward - compile the website to the /Build/Deploy/WebsiteName folder.

   65 <target name="general.compile.website" description="compiles a website" depends="prepare.web">
   66     <mkdir dir="${output.dir}Deploy${website.name}" />
   67     <exec program="aspnet_compiler.exe"
   68         basedir="C:WINDOWSMicrosoft.NETFrameworkv2.0.50727"
   69         workingdir="${output.dir}"
   70         commandline="-u -v /${website.name} -p ${output.dir}src${website.name} ${output.dir}Deploy${website.name}"  />
   71   </target>

Here’s a target used to copy a config file.

   73 <target name="general.compile.website.copyconfig" description="Copies an XML file to the website's web.config, replacing tokens as specific in fileset">
   74     <copy file="${template.sourcefilename}" tofile="${output.dir}Deploy${website.name}web.config">
   75       <filterchain refid="config.settings" />
   76     </copy>
   77   </target>

And finally the target to label the assembly.

   79   <target name="general.labelassembly">
   80     <asminfo output="${output.dir}src${AssemblyName}AssemblyInfo.cs" language="CSharp">
   81       <imports>
   82         <import namespace="System.EnterpriseServices" />
   83         <import namespace="System.Reflection" />
   84         <import namespace="System.Runtime.CompilerServices" />
   85       </imports>
   86       <attributes>
   87         <attribute type="AssemblyVersionAttribute" value=""${assembly.label}"" asis="true" />
   88         <attribute type="AssemblyTitleAttribute" value=""Canon Create"" asis="true" />
   89         <attribute type="AssemblyDescriptionAttribute" value=""assembly for Canon Create"" asis="true" />
   90         <attribute type="AssemblyCopyrightAttribute" value=""Copyright (c) 2008"" asis="true" />
   91         <attribute type="ApplicationNameAttribute" value=""${AssemblyName}"" asis="true" />
   92       </attributes>
   93       <references>
   94         <include name="log4net.dll" />
   95       </references>
   96     </asminfo>
   97   </target>

Using the build scripts

You can invoke the website compiler like so:

   13 <!-- build the frontendwebsite website -->
   14     <property name="website.name" value="FrontEndWebsite" />
   15
   16     <assemblyfileset id="website.resources">
   17       <include name="${resources.dir}/DynamicPDF.Generator.Server.dll" />
   18       <include name="${resources.dir}/DynamicPDF.Merger.Server.dll" />
   19       <include name="${resources.dir}/NetSpell.SpellChecker.dll" />
   20       <include name="${resources.dir}/FreeTextBox.dll" />
   21       <include name="${resources.dir}/log4net.dll" />
   22     </assemblyfileset>
   23     <!-- compile the site -->
   24     <call target="general.compile.website" />

And to copy config settings:

   26     <!-- copy the frontendwebsite web.config with replacing tokens -->
   27     <filterchain id="config.settings">
   28       <replacetokens>
   29         <token key="token.defaultConnectionString" value="server=engelbart;database=CANON_CREATE_V6;uid=sa;pwd="/>
   30         <token key="token.smtpserver" value="10.111.1.55" />
   31         <token key="token.baseSiteUrl" value="http://localhost:4870" />
   32         <token key="token.XHTMLTemplateUploadPath" value="C:Inetpubvirtual2005ProjectsCanonCanonCreate2005FrontEndWebsitexhtmltemplateuploads" />
   33         <token key="token.ConfigurationFilesDirectory" value="C:Inetpubvirtual2005ProjectsCanonCanonCreate2005ConfigurationFiles" />
   34       </replacetokens>
   35     </filterchain>
   36
   37     <!-- copy the frontendwebsite.config.template.xml file to /frontendwebsite/web.config with tokens replaced -->
   38     <property name="template.sourcefilename" value="template.config.FrontEndWebsite.xml" />
   39     <call target="general.compile.website.copyconfig" />

Here’s how I’m labelling my build. Generally I just want to use the CCNetLabel variable passed through from CCNet, but for added flexibility The target expects the ${assembly.label} variable. I simply check for CCNetLabel and copy it to the new variable.

   68 <!-- label the assembly -->
   69     <ifnot propertyexists="CCNetLabel">
   70       <fail message="CCNetLabel property not set, so can't create labelled distribution files" />
   71     </ifnot>
   72     <trycatch>
   73       <try>
   74         <echo message="build number ${CCNetLabel}" />
   75         <property name="assembly.label" value="${CCNetLabel}" />
   76       </try>
   77       <catch>
   78         <property name="assembly.label" value="1.0.0.1" />
   79       </catch>
   80     </trycatch>

February 7, 2008

Small NAnt Gotcha

Filed under: CI, nant — Tags: , — andrewmyhre @ 12:34 pm

Say I have two XML build files. The first is intended to call a target in the second.

Example1.xml:

.cf { font-family: Courier New; font-size: 8pt; color: black; background: white; } .cl { margin: 0px; } .cln { color: #2b91af; } .cb1 { color: blue; } .cb2 { color: #a31515; } .cb3 { color: red; }

    1 <?xml version="1.0"?>
    2 <project name="Master build file" default="build" basedir=".">
    3
    4   <echo message="first" />
    5
    6   <include buildfile="example2.xml" />
    7
    8   <target name="build" description="Main target">
    9     <echo message="second" />
   10     <call target="secondary.target" />
   11   </target>
   12 </project>

Example2.xml:

.cf { font-family: Courier New; font-size: 8pt; color: black; background: white; } .cl { margin: 0px; } .cln { color: #2b91af; } .cb1 { color: blue; } .cb2 { color: #a31515; } .cb3 { color: red; }

    1 <?xml version="1.0"?>
    2
    3 <target name="secondary.target">
    4   <echo message="third" />
    5 </target>

I had expect that the secondary.target target would not be called until after the build target was called. Instead:

.cf { font-family: Courier New; font-size: 8pt; color: black; background: white; } .cl { margin: 0px; }

NAnt 0.85 (Build 0.85.2478.0; release; 14/10/2006)
Copyright (C) 2001-2006 Gerry Shaw
http://nant.sourceforge.net

Buildfile: file:///C:/Inetpub/virtual2005/Projects/Canon/CanonCreate2005/example1.xml
Target framework: Microsoft .NET Framework 2.0
Target(s) specified: build

[echo] first
[echo] third

build:

[echo] second

BUILD FAILED

Target 'secondary.target' does not exist in this project.

Total time: 0 seconds.

Huh? Why did the output go “first”, “third”, then “second”? And why does it say secondary.target doesn’t exist?

The documentation for the <include> task states:

Any global (project level) tasks in the included build file are executed when this task is executed. Tasks in target elements are only executed if that target is executed.

Which means my build file should be executing correctly. It’s in a target, isn’t it?

So I tried wrapping the secondary target in a project tag, and lo and behold:

.cf { font-family: Courier New; font-size: 8pt; color: black; background: white; } .cl { margin: 0px; }

NAnt 0.85 (Build 0.85.2478.0; release; 14/10/2006)
Copyright (C) 2001-2006 Gerry Shaw
http://nant.sourceforge.net

Buildfile: file:///C:/Inetpub/virtual2005/Projects/Canon/CanonCreate2005/example1.xml
Target framework: Microsoft .NET Framework 2.0
Target(s) specified: build

[echo] first

build:

[echo] second

secondary.target:

[echo] third

BUILD SUCCEEDED

Total time: 0 seconds.

So the NAnt docs aren’t entirely complete, are they. So I suppose the tag <project> is really a way of saying “this is build script which I want to control the execution for, instead of just processing the whole file as NAnt reads it”.

Older Posts »

Blog at WordPress.com.