Tuesday, April 24, 2012

Transform Web.config for debugging using MsBuild 4.0 and VS.NET 2010

This is the old topic and there are quite a few post about this. I found following articles are quite helpfull and actually, I based on their help to make a solution that fit my need:

Well, in my project, I have a 3 different configuration files which are web.config, appSettings.config and Autofac.config that need to be transformed when I debug. There are many settings need to be changed between those configurations so doing it manually is not an interesting job.

I want a simple solution that can be reused for other project, and on top of it, I don't want to unload my project, put some chunk of Msbuild tasks to it to do the job. Therefore, an external msbuild script seems to be a nice option. Then what I actually need to do is modify the build event of the project to call MsBuild command to transform these things. However, I don't want my hack to affect the build process on server so I use a environment variable which is only available on my machine to enable the transformation.

So using this way, I have to aware 3 following pointss:

  • Work only for MSbulid 4.0 since I'm using a feature of this version
  • The Web.config for instance will be overwrite everytime I build the project, and I should not checkin these changes.
  • I should have a Web.Debug.config to switch everthing back to normal in Debug configuration.

Here is the Msbuild Script:
<!--  To use:
- Put this file at project folder
- Set DeveloperMachine to enable transforming on current machine, otherwise it will be ignored
- Set following command as the Post build event:
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\MSBuild $(ProjectDir)TransformXml.msbuild /p:ProjectPath=$(ProjectDir);Configuration=$(ConfigurationName);ConfigFile=Web.config
<Project ToolsVersion="4.0"

    <UsingTask TaskName="TransformXml"

        <ProjectPath Condition=" '$(ProjectPath)' == '' ">.\</ProjectPath>
        <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
        <ConfigFile Condition=" '$(ConfigFile)' == '' ">Web.config</ConfigFile>
        <ConfigFileName>$(ConfigFile.Substring(0, $(ConfigFile.IndexOf('.'))))</ConfigFileName>

    <Target Name="Transform" Condition="'$(DeveloperMachine)' == 'True'" >
        <TransformXml Source="$(TransformInputConfigFile)"
                      StackTrace="$(StackTraceEnabled)" />                    
    <Target Name="Init" Condition="'$(DeveloperMachine)' != 'True'">
        <Message Text="Set system environment [DeveloperMachine] to True to transform $(TransformOutputConfigFile) while debugging !!!" />                
    <Target Name="Execute" DependsOnTargets="Init;Transform" />        

As you see in the comment, to use this script:
  1. Put this file at project folder
  2. Set DeveloperMachine to enable transforming on current machine, otherwise it will be ignored
  3. Set following command as the Post build event:

  4. C:\Windows\Microsoft.NET\Framework64\v4.0.30319\MSBuild $(ProjectDir)TransformXml.msbuild /p:ProjectPath=$(ProjectDir);Configuration=$(ConfigurationName);ConfigFile=Web.config

Because I have 3 files to transform, I'll have to add 3 lines for these 3 files in my build event.


Thursday, April 19, 2012

MVC Enum and Nullable object JSON ModelBinder

Warning: This post is quite long with so many unusable code except the last one which is my solution. Stop reading or get your own a cup of tea first :)

- When posting Json data to an MVC action method, you could face a problem if you're using numeric values for enum field instead of string representations of the enum names. There are some posts on stackoverflow.com to solve this problem but I don't really like it. The problem with that approach is that it can not bind nullable enum, array of enum, etc. Indeed, when trying to solve a problem with Model Binders we could have following options:
  • Override the DefaultModelBinder, if the ModelType to bind is the type we want, do it, otherwise delegate the job the the base class.
  • Make a specific ModelBinder for your type.
  • Change the way JsonValueProviderFactory works.
  • Hack TypeDescripter.
  • ... potential other options

- The first approach is the one used in the stackoverflow post. That seems to be the good one because we could have many enums. The "One size fits all" approach supposes to be the good choice. However, let check out other options first.

- The second option is really good if you have a specific type that MVC does not know how to bind such as Mongo ObjectID. However, for enum types, this approach is not very convenient because you have to add following line for every single enum type you have in the project:
ModelBinders.Binders.Add(typeof(YourEnumType), new EnumModelBinder());

- I found the third option when I tried to find different ways to solve my problem without using the code in stackoverflow post. Well, by reading the MVC source code and it's quite interesting to see how MVC team make something virtual but it's not that easy to extend the behavior you need. In short, when you post something to server, the MVC engine will use all registered ValueProviderFactory to get an IValueProvider.

- Obviously, when you post Json, the content type is "application/json" and nothing but the built-in JsonValueProviderFactory will return a IValueProvider which is an object of type DictionaryValueProvider. There is an interesting point here, if you post an array in JSON, something like {selectedFields:[1,2,3]}, JsonValueProviderFactory will put following keys/values to the dictionary:
selectedFields[0] = 1
selectedFields[2] = 2
selectedFields[3] = 3

- That potentialy is an issue if lately you want to retrieve the value of that field because the model name here is selectedFields, not selectedFields[0] nor selectedFields[1]. Anyways, that's the way MVC works and I believe if MVC serialize array that way, it can get the values back properly.

- Continue digging into class DictionaryValueProvider, you will see that it has an "private readonly" Dictionay. ValueProviderResult is the one I said that has method virtual but there is no really simple way to extend behavior just by overriding methods. I realized that if I override method ConverTo, I can fix the Enum binding problem here. Here is the code:
/// <summary>
/// This class is created to fix the Enum parser 
/// </summary>
public class JsonEnumValueProviderResult : ValueProviderResult
    public JsonEnumValueProviderResult(object rawValue, string attemptedValue, CultureInfo culture)
        : base(rawValue, attemptedValue, culture)
    public override object ConvertTo(Type type, CultureInfo culture)
        int rawValue;
        var underlyingType = Nullable.GetUnderlyingType(type);

        if ((type.IsEnum || (underlyingType != null && underlyingType.IsEnum)) && int.TryParse(RawValue == null ? "0" : RawValue.ToString(), out rawValue))
            return Enum.ToObject(underlyingType ?? type, rawValue);
        return base.ConvertTo(type, culture);

As I said, thing is not that simple. I have to somehow make DictionaryValueProvider use my JsonEnumValueProviderResult as well. And I would have to create this class:
/// <summary>
/// Inherit DictionaryValueProvider, just to return JsonEnumValueProviderResult for method GetValue
/// </summary>
/// <typeparam name="TValue"></typeparam>
public class CustomDictionaryValueProvider<TValue> : DictionaryValueProvider<TValue>
    private readonly Dictionary<string, JsonEnumValueProviderResult> _values = new Dictionary<string, JsonEnumValueProviderResult>(StringComparer.OrdinalIgnoreCase);

    public CustomDictionaryValueProvider(IDictionary<string, TValue> dictionary, CultureInfo culture)
        : base(dictionary, culture)
        foreach (var entry in dictionary)
            object rawValue = entry.Value;
            string attemptedValue = Convert.ToString(rawValue, culture);
            _values[entry.Key] = new JsonEnumValueProviderResult(rawValue, attemptedValue, culture);

    public override ValueProviderResult GetValue(string key)
        if (key == null)
            throw new ArgumentNullException("key");

        JsonEnumValueProviderResult vpResult;
        _values.TryGetValue(key, out vpResult);
        return vpResult;


Well, that's still not enough. I have to make JsonValueProviderFactory to return my CustomDictionaryValueProvider, either overriding it which is impossible because MVC team sealed this class :( or completly replace it. So if I keen on this approach I have to copy its source code and change only 1 line:
/// <summary>
/// JsonValueProviderFactory is sealed. This class is copied from JsonValueProviderFactory and will try to use CustomDictionaryValueProvider
/// </summary>
public class CustomJsonValueProviderFactory : ValueProviderFactory
    private static void AddToBackingStore(Dictionary<string, object> backingStore, string prefix, object value)
        IDictionary<string, object> d = value as IDictionary<string, object>;
        if (d != null)
            foreach (KeyValuePair<string, object> entry in d)
                AddToBackingStore(backingStore, MakePropertyKey(prefix, entry.Key), entry.Value);

        IList l = value as IList;
        if (l != null)
            for (int i = 0; i < l.Count; i++)
                AddToBackingStore(backingStore, MakeArrayKey(prefix, i), l[i]);

        // primitive
        backingStore[prefix] = value;

    private static object GetDeserializedObject(ControllerContext controllerContext)
        if (!controllerContext.HttpContext.Request.ContentType.StartsWith("application/json", StringComparison.OrdinalIgnoreCase))
            // not JSON request
            return null;

        StreamReader reader = new StreamReader(controllerContext.HttpContext.Request.InputStream);
        string bodyText = reader.ReadToEnd();
        if (String.IsNullOrEmpty(bodyText))
            // no JSON data
            return null;

        JavaScriptSerializer serializer = new JavaScriptSerializer();
        object jsonData = serializer.DeserializeObject(bodyText);
        return jsonData;

    public override IValueProvider GetValueProvider(ControllerContext controllerContext)
        if (controllerContext == null)
            throw new ArgumentNullException("controllerContext");

        object jsonData = GetDeserializedObject(controllerContext);
        if (jsonData == null)
            return null;

        Dictionary<string, object> backingStore = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);
        AddToBackingStore(backingStore, String.Empty, jsonData);
        return new CustomDictionaryValueProvider<object>(backingStore, CultureInfo.CurrentCulture);

    private static string MakeArrayKey(string prefix, int index)
        return prefix + "[" + index.ToString(CultureInfo.InvariantCulture) + "]";

    private static string MakePropertyKey(string prefix, string propertyName)
        return (String.IsNullOrEmpty(prefix)) ? propertyName : prefix + "." + propertyName;

- So the final step is replace the default JsonValueProviderFactory:
var oldFactory = ValueProviderFactories.Factories.FirstOrDefault(x => x is JsonValueProviderFactory);
if (oldFactory != null)
ValueProviderFactories.Factories.Add(new CustomJsonValueProviderFactory());

- It's not fun anymore, I hate to copy source code and change 1 line because I cannot extend it. So let's talk about last option. Again if you look in MVC source code class ValueProviderResult, you'll see that in method ConvertSimpleType where it convert simple types like int, enum, etc, It use TypeDescriptor to get the converter. So I thought If I change that static class, I'll have what I want. And here's the code:
internal class JsonEnumTypeConverter<T> : TypeConverter
    public override bool CanConvertFrom(ITypeDescriptorContext context, Type sourceType)
        return (sourceType == typeof(int) || sourceType.IsEnum || sourceType == typeof(string)) && CanConvertTo(context, typeof(T));

    public override bool CanConvertTo(ITypeDescriptorContext context, Type destinationType)
        if (destinationType.IsEnum)
            return true;

        var underlyingType = Nullable.GetUnderlyingType(destinationType);
        return underlyingType != null && underlyingType.IsEnum;

    public override object ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, object value)
        if (value == null)
            return null;
            int numericValue;
            return int.TryParse(value.ToString(), out numericValue)
                ? Enum.ToObject(typeof(T), numericValue)
                : Enum.Parse(typeof(T), value.ToString());
        catch (Exception)
            return base.ConvertFrom(context, culture, value);

- But again, I would have to register every single enums in the project which I don't like. So I'll have to get back the the first option which suppose to be the best choice now. Indeed, it works just fine for most scenarios but list of enum, nullable of enum. Futhur more, MVC is not good in handling nullable type like float?, int?. You post {value: 1} or even {value:1.0}, you will get null in the float? field. Unless you post {value: '1.0'} :D. So I'm thinking of getting the underlying type of nullable type then get the binder for that type and bind the value to the underlying type is likely the solution. Here is the code:
public class FixedDefaultModelBinder : DefaultModelBinder
    public override object BindModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
        if (bindingContext.ModelType.IsEnum)
            var valueProviderResult = bindingContext.ValueProvider.GetValue(bindingContext.ModelName);
            if (valueProviderResult == null)
                return base.BindModel(controllerContext, bindingContext);

            int numericValue;
            return int.TryParse(valueProviderResult.AttemptedValue, out numericValue)
                ? Enum.ToObject(bindingContext.ModelType, numericValue)
                : Enum.Parse(bindingContext.ModelType, valueProviderResult.AttemptedValue);

        var underlineType = Nullable.GetUnderlyingType(bindingContext.ModelType);
        if (underlineType != null)
            var newBindingContext = new ModelBindingContext(bindingContext)
                ModelMetadata = ModelMetadataProviders.Current.GetMetadataForType(() => bindingContext.Model, underlineType),
                ModelName = bindingContext.ModelName
            return ModelBinders.Binders.GetBinder(underlineType).BindModel(controllerContext, newBindingContext);

        return base.BindModel(controllerContext, bindingContext);

This is the work solution.

It works :). What you have to do last is setting it as the default model binder:
ModelBinders.Binders.DefaultBinder = new FixedDefaultModelBinder();

It's clean and simple, isn't it? Appologise for letting you read through such a long post with so many unusable code except the last one because I was so excited to find out as many ways to solve my project's problem as possible.


Wednesday, April 4, 2012

Deploy window service artifact to a remote server after a successful build on TeamCity

There is a new version of this approach: http://thoai-nguyen.blogspot.com.au/2012/10/deploy-dot-NET-window-service-TeamCity-Updated.html

Deploy window services from TeamCity is such a pain since you don't have something like WebDeploy as deploying a web application. Optopus is a product on the market to do it but I don't want to pay for this simple task. So here is the way I did.

Tool: powershell, window commands, wget, 7zip
Prerequisite: TeamCity server and the remote server have to be configured to use Remote Commands in Windows PowerShell

1/ Config both servers follow the above link. TeamCity server has to be a trusted client to the remote server and vice versa

2/ Download wget and put somewhere on the remote server such as C:\DeploymentScripts\wget.exe

3/ Download and install 7zip on remote server where you installed your windows service.

4/ Write deployment script and put somewhere on the remote server such as C:\DeploymentScripts\MyApp\deploy.bat

It can be a powershell script but I decided to use a batch file for simplicity.
Basically, the script when triggered, will
  • Use wget to download the latest artifact from TeamCity.
  • Use 7zip to extract the artifact, copy it to a temporary folder.
  • Save the artifact to Artifacts folder
  • Backup the whole current service folder
  • Stop target window service
  • Copy over new files and old configuration files
  • Start target window service
  • Clean up

Here is the script, replace some value in the script to match your requirement.
@echo off:: Deployment scripts for my window service

:: 0/ --------- Set some local variables
SET Environment.ExecutingFolder="C:\DeploymentScripts\MyApp"
SET Environment.7zip="C:\Program Files (x86)\7-Zip\7z.exe"
SET Environment.Wget="C:\DeploymentScripts\wget.exe"
SET TeamCity.User=van
SET TeamCity.Password=vanpassword
SET TeamCity.BuildTypeId=bt19
SET TeamCity.Artifact=MyAppService.{build.number}.zip
SET TeamCity.Branch=Default Branch
SET MyApp.TargetFolderName=MyApp.Service
SET MyApp.TargetFolderPath=C:\WhereMyAppInstalled\%MyApp.TargetFolderName%
SET MyApp.ServiceName=MyAppServiceName
SET MyApp.ImageName=MyApp.Service.exe
CD /D %Environment.ExecutingFolder%

ECHO 1/ --------- Get latest artifact from TeamCity, MyApp %TeamCity.Branch% ----------------- bt19: build type ID for MyApp on %TeamCity.Branch%
%Environment.Wget% -q --http-user=%TeamCity.User% --http-password=%TeamCity.Password% --auth-no-challenge http://my.teamcity.url.com/repository/download/%TeamCity.BuildTypeId%/.lastSuccessful/%TeamCity.Artifact%
REN *.zip* *.zip
ECHO Found following artifact
DIR /B *zip

ECHO 2/ --------- Extract the artifact to folder __Temp ---------------
%Environment.7zip% e -y -o__Temp *.zip

ECHO 3/ --------- Store the artifact ------------------ 
MOVE /Y *.zip Artifacts\

ECHO 4/ --------- Backup current service folder --------------- 
for %%a in (%MyApp.TargetFolderPath%) do set Temp.LastDate=%%~ta
SET Temp.LastDate=%Temp.LastDate:~6,4%-%Temp.LastDate:~0,2%-%Temp.LastDate:~3,2% %Temp.LastDate:~11,2%%Temp.LastDate:~14,2%%Temp.LastDate:~17,2%
ECHO Last deployment: %Temp.LastDate%
ECHO Now backup files to folder %MyApp.TargetFolderName%.%Temp.LastDate%
XCOPY /E /I /H /R /Y %MyApp.TargetFolderPath% "%MyApp.TargetFolderName%.%Temp.LastDate%"

ECHO 5/ --------- Stop %MyApp.ServiceName% service ---------------
net stop %MyApp.ServiceName%
ECHO Wait 15 seconds
ping -n 1 -w 15000 > nul
ECHO Try to kill the process %MyApp.ImageName% if it's still running
TASKKILL /F /FI "IMAGENAME eq %MyApp.ImageName%" /T > nul

ECHO 6/ --------- Deploy new files & copy over old configs ----------------------
XCOPY /E /H /R /Y __Temp %MyApp.TargetFolderPath%
COPY /Y "%MyApp.TargetFolderName%.%Temp.LastDate%\*.config" %MyApp.TargetFolderPath%

ECHO 7/ --------- Start %MyApp.ServiceName% service ---------------
net start %MyApp.ServiceName%

ECHO 8/ --------- Cleanup --------------------------------- 
RD /S /Q __Temp

5/ Open powershell as Administrator on TeamCity server and execute following command:
read-host -assecurestring | convertfrom-securestring | out-file C:\Users\Administrator\MyAppSecurePassword.txt

Then enter the password for the remote server. The secure password will be stored in the above file. It will then be used by TeamCity for future powershell commands against the remote server.

6/ Go to TeamCity management website, add a new project with 1 step that will execute the following powershell command:
$username = "Administrator"
$password = cat C:\Users\Administrator\MyAppSecurePassword.txt | convertto-securestring
$cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $username, $password
$serverNameOrIp = "xxx.xxx.xxx.xxx"
Invoke-command -computername $serverNameOrIp -ScriptBlock {C:\DeploymentScripts\MyApp\deploy.bat} -Credential $cred

  • Administrator is the Admin username on the remote server
  • C:\Users\Administrator\MyAppSecurePassword.txt is the secure file that store the secure password we created at step 5.
  • serverNameOrIp value has to be changed to the remote server's ip address where you installed the windows service

So after the code is checked in and successfully built, we can trigger the deployment by clicking "Run" from the above created project. The deploy.bat script above can be triggered anytime from the remote server for testing purpose. It's simple enough to read and modify when more steps need to be involved.