Microsoft.Diagnostics.Runtime

Not that long ago .NET Runtime team announced the beta release of Microsoft.Diagnostics.Runtime component (aka ClrMD).
The package delivers managed API for .NET process and crash dump inspection which is similar to SOS Debugging Extensions. If you are a seasoned Windows programmer you probably have been using WinDbg a couple of times, for example to track and identify logical memory leaks in your applications.
ClrMD simply brings some of that capabilities as an API. That is a big deal for a couple of reasons:

• Makes memory profiling and process analysis much easier for regular C#/.NET developers,
• Allows people and businesses to write custom diagnostic tools tailored for their needs.

Remember that ClrMD is still in beta phase and also that attaching and debugging is an invasive process (don't run it on production servers).

For more in depth introduction to the topic please read the original blog post

ScriptCS and REPL

ScriptCS is a cool project started by @gblock and inspired by @filip_woj that uses Roslyn and NuGet to make C# scripting easy (no .csprojs required). It is getting a lot of traction in the community recently - it is definitely one of my favourite initiatives in the C# world, putting Roslyn to a great use.

Just recently Glenn added initial REPL (Read-eval-print loop) support to the project meaning that now you can use C# in an interactive shell, kind of like a JavaScript console.

ClrMD + ScriptCS = ScriptCS.ClrDiagnostics

Based on the above I've created a script pack that brings ClrMD into ScriptCS and is aimed to provide interactive CLR diagnostics environment under ScriptCS REPL.
It is available on GitHub and NuGet as ScriptCS.ClrDiagnostics

Here is how you can use it:

• Install ScriptCS using nightly builds (this is needed due to a bug in current version that prevents from installing prerelease packages) cinst scriptcs -pre -source https://www.myget.org/F/scriptcsnightly/ Alternatively you can build from source
• Install ScriptCS.ClrDiagnostics: scriptcs -install ScriptCs.ClrDiagnostics -pre
• Launch ScriptCS in REPL mode (you may need to get latest version for that): scriptcs.exe
• You should be able to load the script pack and attach to a process like this:

// Load ClrDiag object
> var c = Require<ClrDiag>();

// Attach to process 
> c.Attach(6152)
Attaching to process PID=6152
Using CLR Version=v4.0.30319.18033 DACFileName=mscordacwks_amd64_Amd64_4.0.30319.18033.dll
Succesfully attached to process PID=6152 Name=WpfApplication2

You can also use the process name:

c.Attach("MyApplication")

Check current state

> c.IsAttached 
True         

> c.Process.WorkingSet64
51265536

ClrMD API is available to use, eg. ClrRuntime

> c.Clr.Threads.Count
2

Script pack also provides some helpers that should make analysis easier:

> c.PrintTypes("System.");
Total size               Count   Name
167.82KB                 398     System.Object[]
143.29KB                 2404    System.String
126.98KB                 2385    System.Delegate[]
126.98KB                 2385    System.Delegate[]

PrintTypes() will simply output all the types on managed heaps sorted by total memory consumed, it lets you optionally filter by type name and limit returned result set.

You can also display each thread's call stack using PrintStackTrace():

> c.PrintStackTrace()                                                                                                                                     
Stacktrace for ThreadId=3200                                                                                                                              
           0       A9E9B8 InlinedCallFrame                                                                                                                
           0       A9E9B8 InlinedCallFrame                                                                                                                
 7F92AF92093       A9E990 DomainBoundILStubClass.IL_STUB_PInvoke(System.Windows.Interop.MSG ByRef, System.Runtime.InteropServices.HandleRef, Int32, Int32)
 7F92AF87640       A9EA60 System.Windows.Threading.Dispatcher.GetMessage(System.Windows.Interop.MSG ByRef, IntPtr, Int32, Int32)                          
 7F92AF85E9E       A9EB20 System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame)                                     
 7F9272172DA       A9EBC0 System.Windows.Application.RunInternal(System.Windows.Window)                                                                   
 7F927216BD7       A9EC60 System.Windows.Application.Run()                                                                                                
 7F8E3700107       A9ECA0 WpfApplication2.App.Main()

Or for one thread using index:

> c.PrintStackTrace(0)                                                                                                                                     
Stacktrace for ThreadId=3200                                                                                                                              
           0       A9E9B8 InlinedCallFrame                                                                                                                
           0       A9E9B8 InlinedCallFrame                                                                                                                
 7F92AF92093       A9E990 DomainBoundILStubClass.IL_STUB_PInvoke(System.Windows.Interop.MSG ByRef, System.Runtime.InteropServices.HandleRef, Int32, Int32)

After you are done, detach from the process like this:

> c.Detach()                                                    
Successfully detached from process PID=6152 Name=WpfApplication2

Troubleshooting

ScriptCs.ClrDiagnostics lets you also specify DAC file (mscordacwks.dll) location while attaching:

> c.Attach(6152, @"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\mscordacwks.dll");

If you encounter an error with StackOverflow being thrown in REPL console - it is caused by jsv serialization recently added to ScriptCS. You may need to wait for the fix or manually apply a simple workaround (use simple .ToString() in place of object serialization to jsv).

Summary

The project has been created as an experiment, but it shows how great capabilities ScriptCS REPL has.

Oh, and you can also do that:

> c.Play().ImperialMarch();

What do I mean by 'reactive app'?

MVVM and data-binding is by its very nature reactive. Any changes made to the view model should automatically propagate to the data-bound view. Thus being said, in many scenarios the true source of such change comes from the model that resides on remote server (eg. because a new entity has been added). Very often we want to be able to push this change across the wire so that it triggers relevant view updates. This is where SignalR can be leveraged - it can help us extend 'reactivity' beyond the network boundary.

In traditional scenarios apps that consume HTTP services will usually pull data from the server. This may happen on demand - e.g. when opening a new screen that presents a list of customers, or in regular intervals to check for any updates. There is nothing inherently bad with this approach and it suits a large number of cases. Sometimes, however constant polling for changes is undesired and can be sub optimal from performance perspective. Consider for example a financial app that should provide real-time like updates to the user (eg. commodity quotes) - in this case it feels natural to have data pushed to us from the service as new updates arrive.

From technical point of view SignalR may still use polling, but from logical point of view it creates an abstraction over the actual mechanism (be it long polling, web sockets or anything else) which makes our apps independent from it. This is a huge benefit.

Let's see how we can use SignalR to write a simple 'financial' app.

Push service with ASP.NET SignalR

We will use SingalR hubs feature to create a push service. Create an Empty ASP.NET Web Application in Visual Studio and NuGet install or reference SignalR.

Install-Package Microsoft.AspNet.SignalR.SystemWeb -Version 1.0.1 

protected void Application_Start(object sender, EventArgs e)
{
    RouteTable.Routes.MapHubs();
}

The example will operate on a simple model - a Quote class representing individual.. 'quote' (e.g. a currency exchange rate).

public class Quote
{
    public int Id { get; set; }
    public string Name { get; set; }
    public decimal Price { get; set; }
    public decimal PriceChange { get; set; }
}

The hub will simply push quote updates to all interested parties.

[HubName("Quote")]
public class QuoteHub : Hub
{
    public void UpdateQuote(Quote quote)
    {
        Clients.All.updateQuote(quote);
    }
}

For the sake of example we will generate some random data and push quote updates in regular intervals. This code is not that important, but may help if you want an example on how to generate sample data in SignalR projects.

internal class DataStreamThread
{
    private const int UpdateIntervalInMilliseconds = 4000;

    private object randomLock = new object();

    private Dictionary<string, decimal> quotes 
                        = new Dictionary<string, decimal>()
                              {
                                {"EUR/USD", 1.320m},
                                {"GBP/USD", 1.550m},
                                {"AUD/USD", 1.032m},
                                {"PLN/USD", 0.3168m},
                                {"USD/JPY", 99.045m},
                                {"Gold", 1467.2m},
                                {"Silver ", 24.02m},
                              };

    public void Start()
    {
        Random random = new Random();

        int counter = 0;
        foreach (KeyValuePair<string, decimal> quote in quotes)
        {
            counter++;
            int id = counter;
            ThreadPool.QueueUserWorkItem(_ =>
            {
                var hubContext = GlobalHost
                        .ConnectionManager.GetHubContext<QuoteHub>();

                Quote q = new Quote()
                {
                    Id = id,
                    Name = quote.Key,
                    Price = quote.Value, 
                    PriceChange = 0,
                };

                while (true) //sic
                {
                    double randomChange;
                    lock(randomLock)
                    {
                        randomChange = (random.NextDouble() - 0.42) / 100;
                    }
                    decimal change = Math
                    .Round((decimal)randomChange * q.Price, 6);
                    q.Price += change;
                    q.PriceChange = change;
                    try
                    {
                        hubContext.Clients.All.updateQuote(q);
                    }
                    catch (Exception ex)
                    {
                        System.Diagnostics
                        .Trace.TraceError("Error thrown while updating clients: {0}", ex);
                    }
                    var intervalAdj = random.Next(-700, 700);
                    Thread.Sleep(UpdateIntervalInMilliseconds + intervalAdj);
                }
            });
        }
    }
}

That's all we need on the server side. The most important line - hubContext.Clients.All.updateQuote(q); will push sample data to all hub clients.

Windows Phone 8 XAML application

As a next step let's write a XAML client that will present this data. I will use a Windows Phone 8 app as an example, but Windows 8 (or WPF and Silverlight) will be conceptually similar.

Here is how we want it to look like. The quotes should update automatically, ideally with some kind of subtle animation.

Start by adding a new Windows Phone 8 XAML application and installing MVVM Light via NuGet. Instead of reusing the Quote class (by adding as link to WP8) we will create a separate model on the client side in order to implement INotifyPropertyChanged.

public class Quote : INotifyPropertyChanged
{
    private string _name;
    private decimal _price;
    private decimal _priceChange;

    public int Id { get; set; }

    public string Name
    {
        get { return _name; }
        set
        {
            if (value == _name) return;
            _name = value;
            OnPropertyChanged();
        }
    }

    public decimal Price
    {
        get { return _price; }
        set
        {
            if (value == _price) return;
            _price = value;
            OnPropertyChanged();
        }
    }

    public decimal PriceChange
    {
        get { return _priceChange; }
        set
        {
            if (value == _priceChange) return;
            _priceChange = value;
            OnPropertyChanged();
            OnPropertyChanged("ChangePercentage");
        }
    }

    public double ChangePercentage
    {
        get 
          { 
            return (double) (_priceChange / (_price - _priceChange))
                                                             * 100; 
          }
    }

    public event PropertyChangedEventHandler PropertyChanged;

    protected virtual void OnPropertyChanged([CallerMemberName] string propertyName = null)
    {
        PropertyChangedEventHandler handler = PropertyChanged;
        if (handler != null) handler(this, 
            new PropertyChangedEventArgs(propertyName));
    }
}

Now let's think about how our view model(s) should receive update notifications from SignalR hub. We could open connection directly in the view model, get hub proxy instance and subscribe to updateQuote event. Something like that:

public MainViewModel()
{
    _connection = new HubConnection(EndpointAddress);
    _hubProxy = _connection.CreateHubProxy(StockHubName);
    _hubProxy.On<Quote>("updateQuote", q => UpdateQuoteHandler);

    _connection.Start();
}

This would work and maybe is not that bad, but... it doesn't seem that doing all this is something our view model should be concerned about. What it really cares about is data updates and connection state changes. How it happens and what is the underlying technology should be owned by another class.
Also if we had more than one view model using the same hub (e.g. list->detail scenario) we would need to duplicate the code and connections.

Let's use MVVM Light's messaging capabilities to propagate updates in a decoupled way (we could have also used an interface that exposes events).
This also means we will introduce more abstractions and complexity, so bear that in mind especially when working on small applications.

public class ConnectionStateChangedMessage
{
    public ConnectionState OldState { get; set; }
    public ConnectionState NewState { get; set; }
}

public class QuoteUpdatedMessage
{
    public Quote Quote { get; set; }
}

Now we need a class that will connect to SignalR hub and push the updates.

public interface IDispatcher
{
    void Dispatch(Action action);
}

public interface IConnectedDataProvider
{
    Task StartAsync();
    void Stop();
}

public class SignalRDataProvider : IConnectedDataProvider
{
    private HubConnection _connection;
    private IHubProxy _hubProxy;

    private const string EndpointAddress 
            = "http://192.168.1.18/Piotr.XamlSignalR.Service/";
    private const string StockHubName = "Quote";
    private const string QuoteUpdateName = "updateQuote";

    private readonly IMessenger _messenger;
    private readonly IDispatcher _dispatcher;

    public SignalRDataProvider(IMessenger messenger, IDispatcher dispatcher)
    {
        _messenger = messenger;
        _dispatcher = dispatcher;
        _connection = new HubConnection(EndpointAddress);
        _hubProxy = _connection.CreateHubProxy(StockHubName);
        _hubProxy.On<Quote>(QuoteUpdateName, p => _dispatcher
                .Dispatch(() => UpdateQuote(p)));

        _connection.StateChanged += _connection_StateChanged;
    }

    void _connection_StateChanged(StateChange stateChange)
    {
        ConnectionState oldState = ConnectionStateConverter
            .ToConnectionState(stateChange.OldState);
        ConnectionState newState = ConnectionStateConverter
            .ToConnectionState(stateChange.NewState);

        var msg = new ConnectionStateChangedMessage()
        {
            NewState = newState,
            OldState = oldState,
        };

        _dispatcher.Dispatch(() => _messenger
            .Send<ConnectionStateChangedMessage>(msg));
    }

    public Task StartAsync()
    {
        return _connection.Start();
    }

    private void UpdateQuote(Quote quote)
    {
        var msg = new QuoteUpdatedMessage()
        {
            Quote = quote
        };
        _messenger.Send<QuoteUpdatedMessage>(msg);
    }

    public void Stop()
    {
        _connection.Stop();
    }
}

Please forgive me interface/class name ;)

The rationale behind providing IDispatcher interface is to be able to provide platform specific implementations (WP8, Windows 8, WPF, etc.). SignalR action handler can start from a non-UI thread and we need to be able to marshal execution to the UI thread.
Oh and remember - the WP8 emulator runs as a virtual machine, so make sure you don't use localhost as the endpoint address.

Now we can instantiate this class (e.g. in Application object) and start the connection.

private readonly IConnectedDataProvider _dataProvider 
    = new SignalRDataProvider(Messenger.Default, new PhoneDispatcher());

///...

private async void Application_Launching(object sender, LaunchingEventArgs e)
{
    await _dataProvider.StartAsync();
}

private void Application_Closing(object sender, ClosingEventArgs e)
{
    _dataProvider.Stop();
}

///...

This will ensure that update events are being sent and can be consumed by the view model... which leads us to the view model itself.

public class MainViewModel : ViewModelBase
{
    private ConnectionState _connectionState;
    public ConnectionState ConnectionState
    {
        get { return _connectionState; }
        set
        {
            if (_connectionState == value) return;
            _connectionState = value;
            RaisePropertyChanged("ConnectionState");
            RaisePropertyChanged("IsConnected");
        }
    }

    public bool IsConnected
    {
        get { return ConnectionState == ConnectionState.Connected; }
    }

    public MainViewModel()
    {
        Items = new ObservableCollection<Quote>();

        MessengerInstance
            .Register<QuoteUpdatedMessage>(this, UpdateQuoteHandler);
        MessengerInstance
            .Register<ConnectionStateChangedMessage>(this,
                            ConnectionStateChangedHandler);
    }

    private void ConnectionStateChangedHandler(ConnectionStateChangedMessage msg)
    {
        ConnectionState = msg.NewState;
    }

    private void UpdateQuoteHandler(QuoteUpdatedMessage msg)
    {
        var quote = msg.Quote;
        var match = Items.FirstOrDefault(q => q.Name == quote.Name);
        if (match != null)
        {
            match.Price = quote.Price;
            match.PriceChange = quote.PriceChange;
        }
        else
        {
            Items.Add(quote);
        }
    }

    public ObservableCollection<Quote> Items { get; private set; }

    public override void Cleanup()
    {
        MessengerInstance.Unregister(this);
        base.Cleanup();
    }
}

It is quite simple and hopefully readable. MessengerInstance is a property provided by ViewModelBase and can be mocked for the sake of unit tests. Now, we need to write XAML view that will present data to the customer. For brevity I will only include most important parts here, you can have a look at full markup in the source code provided.

        <ListBox x:Name="QuoteList" Opacity="0.5" Margin="0,0,-12,0"
            ItemsSource="{Binding Source={StaticResource ItemsViewSource}}">

            <ListBox.ItemTemplate>
                <DataTemplate>

                    <Grid x:Name="TemplateContainer">
                        <!-- (...) -->
                        <i:Interaction.Triggers>
                            <ec:DataTrigger Binding="{Binding PriceChange, Converter={StaticResource DecimalToDoubleConverter}}" Comparison="GreaterThan" Value="0.0">
                                <ec:GoToStateAction StateName="Up" TargetObject="{Binding ElementName=TemplateContainer}"/>
                            </ec:DataTrigger>
                            <ec:DataTrigger Binding="{Binding PriceChange, Converter={StaticResource DecimalToDoubleConverter}}" Comparison="LessThan" Value="0.0">
                                <ec:GoToStateAction StateName="Down" TargetObject="{Binding ElementName=TemplateContainer}"/>
                            </ec:DataTrigger>
                            <ec:DataTrigger Binding="{Binding PriceChange, Converter={StaticResource DecimalToDoubleConverter}}" Comparison="Equal" Value="0.0">
                                <ec:GoToStateAction StateName="NoChange" TargetObject="{Binding ElementName=TemplateContainer}"/>
                            </ec:DataTrigger>
                        </i:Interaction.Triggers>
                        <Grid>
                            <!-- Item layout goes here -->
                        </Grid>
                    </Grid>
                </DataTemplate>
            </ListBox.ItemTemplate>
        </ListBox>

Because we want to have a strong order of elements on the list we use CollectionViewSource to do the sorting.

<phone:PhoneApplicationPage.Resources>
    <CollectionViewSource x:Key="ItemsViewSource" Source="{Binding Items}">
        <CollectionViewSource.SortDescriptions>
            <scm:SortDescription PropertyName="Id"/>
        </CollectionViewSource.SortDescriptions>
    </CollectionViewSource>
</phone:PhoneApplicationPage.Resources>

As you can see we didn't have to write a single line of C# in the view's code-behind. Sorting, updates and visual state transitions will all happen thanks to XAML.

Unfortunately I ran out of time to include a Windows 8 sample, but the code has been built in such a way that creating a Win8 application should be a simple task as we can reuse most parts (including view model, SignalR data provider).

ASP.NET SignalR bridges the gap between reactive world of MVVM and HTTP services that reside on a remote server.

The complete project is available on bitbucket and below you can see the end result. Of course the data is completely unrealistic :)

http://vimeo.com/65718738

Client libraries

At the time of writing there are two popular and actively developed C# client libraries for Redis available:

• ServiceStack.Redis - created by Demis Bellot of ServiceStack fame and based on Miguel de Icaza's redis-sharp project,
• BookSleeve - mantained by Mark Gravell and as I understand used by Stack Exchange.

Before making a choice I would suggest trying both of them and deciding which API and capabilities better suit your project.
BookSleeve has non-blocking (asynchronous) API, provides thread-safe connection object, while ServiceStack implementation provides JSON serialization, connection pool like client factory and uses convention to simplify POCO object persistence.

In this article I will use ServiceStack.Redis, but remember that BookSleeve has been proved in a big real-world web application and is also very capable.

Redis in a nutshell

If you are reading this article then very likely you already know what Redis is. If you are an experienced Redis user interested in ASP.NET Web API integration you can safely jump to the next part.

In order to use Redis efficiently and avoid potential pitfalls one needs to understand a little bit about how it works and how different it is from relational databases.I strongly recommend reading one of books or online materials available on the topic.

Simply put Redis is an in-memory key-value data store that supports durability.
In-memory and key-value sounds much like a memory cache - and indeed you can think of Redis as of a specialized and more advanced memory cache. Unlike other caches (such as memcached) Redis delivers richer feature set including things like sorted sets and even Lua scripting capabilities.

It's main advantage over 'traditional' databases comes from the fact that it stores and retrieves data directly to / from operating memory - which means it is really fast.

Redis is simple and specialized - unlike relational databases it does not provide any table-like abstractions nor relational capabilities. Instead, it provides five fundamental data types along with specialized operations that can manipulate those types (stored values). This is why it is sometimes refered as a data structure server:

• strings - the most basic and atomic type that can be used to store any data (integers, serialized POCO objects, etc.),
• lists - lists of strings that are sored by insertion order,
• sets - logical sets of strings,
• hashes - maps between string-only keys and string values,
• sorted sets - similar to sets, but each element is associated with a score that is being used to sort.

Examples of specialized commands:

• strings - SET, INCR, APPEND, INCRBY, STRLEN, SETBIT,
• lists - LPUSH, LPOP, LTRIM, LINSERT,
• sets - SADD, SDIFF, SINTER, SUNION, etc.

Hopefully this should give you a basic feel for what Redis is about :)

Why would I use it?

Whether and how easily your application could benefit from Redis depends on its architecture, data volume, data complexity and experienced loads. When used correctly Redis can be bring major performance improvements and may help scale application out.

Here are some use cases I can think of:

• as a main data store,
• as one of multiple data stores, for example storing small, but frequently accessed information,
• as a highly performant read-only view over your domain model,
• as a cache.

Bearing in mind that Redis operates in memory, the first option is quite extreme and viable only if your data sets are small (or you can afford to have lots of RAM).
Because in this article I want to focus on ASP.NET Web API integration not architectural aspects I will choose this option.

Using Redis in a ASP.NET Web API application

I will use an empty ASP.NET Web API application as my starting point along with two third party libraries:

• ServiceStack.Redis - C# Redis client,
• Autofac - dependency injection container with Web API integration.

Obviously we will also need a working Redis server instance. If you don't have one running already you can download Windows port provided by MS Tech. Please note that the port is not considered production ready yet (you need to use one of the official packages for that), but is good for development scenarios.

Model

For the sake of this example lets consider the following requirements:

• the API should provide capability to store Clients, retrieve Client details and retrieve list of all Clients in the system,
• Clients may place orders that consist of multiple items,
• API should expose a list of N best selling items.

Here is how we could design the model:

public class Customer
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public IList<Guid> Orders { get; set; }
    public Address Address { get; set; }
}

Properly defining your data model will help you use Redis in an efficient way. Redis stores values as byte blobs internally and *ServiceStack.Redis* will serialize the whole object graph for us. Thus it is important that we define aggregate boundaries. As you can see Address is a *value object* and will be persisted and retrieved as a part of Customer *aggregate*, while *Orders* property is a list of ids.

public class Order
{
    public Guid Id { get; set; }
    public Guid UserId { get; set; }
    public IList<OrderLine> Lines { get; set; }
}

public class OrderLine
{
    public string Item { get; set; }
    public int Quantity { get; set; }
    public decimal TotalAmount { get; set; }
}

public class Address
{
    public string Line1 { get; set; }
    public string Line2 { get; set; }
    public string City { get; set; }
}

Now let's define repository contracts:

public interface ICustomerRepository
{
    IList<Customer> GetAll();
    Customer Get(Guid id);
    Customer Store(Customer customer);
}

public interface IOrderRepository
{
    IList<Order> GetCustomerOrders(Guid customerId);
    IList<Order> StoreAll(Customer customer, IList<Order> orders);
    Order Store(Customer customer, Order order);
    IDictionary<string, double> GetBestSellingItems(int count);
}

The implementation can look like this:

public class CustomerRepository : ICustomerRepository
{
    private readonly IRedisClient _redisClient;

    public CustomerRepository(IRedisClient redisClient)
    {
        _redisClient = redisClient;
    }

    public IList<Customer> GetAll()
    {
        using (var typedClient = _redisClient.GetTypedClient<Customer>())
        {
            return typedClient.GetAll();
        }
    }

    public Customer Get(Guid id)
    {
        using (var typedClient = _redisClient.GetTypedClient<Customer>())
        {
            return typedClient.GetById(id);
        }
    }

    public Customer Store(Customer customer)
    {
        using (var typedClient = _redisClient.GetTypedClient<Customer>())
        {
            if (customer.Id == default(Guid))
            {
                customer.Id = Guid.NewGuid();
            }
            return typedClient.Store(customer);
        }
    }
}

public class OrderRepository : IOrderRepository
{
    private readonly IRedisClient _redisClient;

    public OrderRepository(IRedisClient redisClient)
    {
        _redisClient = redisClient;
    }

    public IList<Order> GetCustomerOrders(Guid customerId)
    {
        using (var orderClient = _redisClient.GetTypedClient<Order>())
        {
            var orderIds = _redisClient.GetAllItemsFromSet(RedisKeys
                        .GetCustomerOrdersReferenceKey(customerId));
            IList<Order> orders = orderClient.GetByIds(orderIds);
            return orders;
        }
    }

    public IList<Order> StoreAll(Customer customer, IList<Order> orders)
    {
        foreach (var order in orders)
        {
            if (order.Id == default(Guid))
            {
                order.Id = Guid.NewGuid();
            }
            order.CustomerId = customer.Id;
            if (!customer.Orders.Contains(order.Id))
            {
                customer.Orders.Add(order.Id);
            }

            order.Lines.ForEach(l=>_redisClient
                .IncrementItemInSortedSet(RedisKeys.BestSellingItems,
                                                                 (string) l.Item, (long) l.Quantity));
        }
        var orderIds = orders.Select(o => o.Id.ToString()).ToList();
        using (var transaction = _redisClient.CreateTransaction())
        {
            transaction.QueueCommand(c => c.Store(customer));
            transaction.QueueCommand(c => c.StoreAll(orders));
            transaction.QueueCommand(c => c.AddRangeToSet(RedisKeys
                .GetCustomerOrdersReferenceKey(customer.Id),
                orderIds));
            transaction.Commit();
        }

        return orders;
    }

    public Order Store(Customer customer, Order order)
    {
        IList<Order> result = StoreAll(customer, new List<Order>() { order });
        return result.FirstOrDefault();
    }

    public IDictionary<string, double> GetBestSellingItems(int count)
    {
        return _redisClient
            .GetRangeWithScoresFromSortedSetDesc(RedisKeys.BestSellingItems, 
            0, count - 1);
    }
}

As you can see repositories expose specialized operations. We make use of Redis sorted set type to efficiently store and retrieve best selling products list.

It is worth noting how we implemented Customer -* Orders relation. We store customer's orders (their ids) in a dedicated set so that they can be retrieved quickly without the need for pulling out entire Customer entity.

Client and connection lifecycle management

One of the challenges we will face is connection/client lifecycle management. As you may already know Web API ships with an extensible dependency injection mechanism that can be leveraged to inject and dispose dependencies on per request basis. Instead of writing custom IDependencyResolver implementation from scratch (which is also an option) we can use of of .NET DI libraries such as Ninject, StructureMap, Unity, Windsor or Autofac. The last one is my personal favorite and has good Web API integration that is why I am going to use it in this example.

ServiceStack.Redis ships with IRedisClient factories called client managers:

• BasicRedisClientManager - client factory with load-balancing support,
• PooledRedisClientManager - client factory with load-balancing and connection pooling - useful when working,
• ShardedRedisClientManager - provides sharding of client connections using consistent hashing.

Because these classes are thread-safe we can use one factory instance across all requests.

public class ApiApplication : System.Web.HttpApplication
{
    public IRedisClientsManager ClientsManager;
    private const string RedisUri = "localhost";

    protected void Application_Start()
    {
        ClientsManager = new PooledRedisClientManager(RedisUri);

        AreaRegistration.RegisterAllAreas();

        WebApiConfig.Register(GlobalConfiguration.Configuration);
        ConfigureDependencyResolver(GlobalConfiguration.Configuration);

        FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);
        RouteConfig.RegisterRoutes(RouteTable.Routes);
    }

    private void ConfigureDependencyResolver(HttpConfiguration configuration)
    {
        var builder = new ContainerBuilder();
        builder.RegisterApiControllers(Assembly.GetExecutingAssembly())
            .PropertiesAutowired();

        builder.RegisterType<CustomerRepository>()
            .As<ICustomerRepository>()
            .PropertiesAutowired()
            .InstancePerApiRequest();

        builder.RegisterType<OrderRepository>()
            .As<IOrderRepository>()
            .PropertiesAutowired()
            .InstancePerApiRequest();

        builder.Register<IRedisClient>(c => ClientsManager.GetClient())
            .InstancePerApiRequest();

        configuration.DependencyResolver
            = new AutofacWebApiDependencyResolver(builder.Build());
    }

    protected void Application_OnEnd()
    {
        ClientsManager.Dispose();
    }
}

We are using pooled connection manager as IRedisClientsManager implementation. Every time a request is made a new client instance will be retrieved, injected into repositories and disposed at the end of request.

Controllers

Now that we have repositories let's implement the controllers - one for adding and retrieving the customers and one for managing orders.

public class CustomersController : ApiController
{
    public ICustomerRepository CustomerRepository { get; set; }

    public IOrderRepository OrderRepository { get; set; }

    public IQueryable<Customer> GetAll()
    {
        return CustomerRepository.GetAll().AsQueryable();
    }

    public Customer Get(Guid id)
    {
        var customer = CustomerRepository.Get(id);
        if (customer == null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }

        return customer;
    }

    public HttpResponseMessage Post([FromBody] Customer customer)
    {
        var result = CustomerRepository.Store(customer);
        return Request.CreateResponse(HttpStatusCode.Created, result);
    }

    public HttpResponseMessage Put(Guid id, [FromBody] Customer customer)
    {
        var existingEntity = CustomerRepository.Get(id);
        if (existingEntity == null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        customer.Id = id;
        CustomerRepository.Store(customer);
        return Request.CreateResponse(HttpStatusCode.NoContent);
    }
}

public class OrdersController : ApiController
{
    public IOrderRepository OrderRepository { get; set; }
    public ICustomerRepository CustomerRepository { get; set; }

    public HttpResponseMessage Post([FromBody] Order order)
    {
        var customer = CustomerRepository.Get(order.CustomerId);
        var result = OrderRepository.Store(customer, order);
        return Request.CreateResponse(HttpStatusCode.Created, result);
    }

    [ActionName("top")]
    [HttpGet]
    public IDictionary<string, double> GetBestSellingItems(int count)
    {
        return OrderRepository.GetBestSellingItems(count);
    }

    [ActionName("customer")]
    [HttpGet]
    public IList<Order> GetCustomerOrders(Guid id)
    {
        return OrderRepository.GetCustomerOrders(id);
    }
}

That's about it. We are now using Redis as our data store and dependencies should be auto-wired.

Source is available on Bitbucket.

A couple of months ago I've experimented with running ASP.NET Web API on a Linux box, but ran into blocking issues caused by some functionality missing from Mono. I've decided to give it another go now when more recent versions of the runtime are available.

Getting started

Yes, that is correct you should be able to run Web API services under Linux using recent versions of Mono :). The approach I am taking is to use Visual Studio to write the application and then run it on Linux.

Just a general remark - be advised that copying and running non open source assemblies (like System.Core) is probably not ok from licensing point of view (I am not a legal expert, though). This shouldn't happen under normal circumstances (unless you willingly overwrite Mono assemblies) and ASP.NET Web API is 100% open source, so it is not a problem in this scenario.

I will create a very basic Web API service, having in mind that any unnecessary dependency can create problems. Remember that not all libraries that play nicely with Web API will work happily on Mono.

Start off by adding an empty MVC4 application, remember that you can choose either to use .NET 4.0 or 4.5 when doing that. The latter supports async/await and makes writing message handlers a little bit easier.  Even though Mono 2.11 introduced support for some of 4.5 APIs I ran into issues when trying to run 4.5 Web API app against XSP 2.11. This is why I am going to use .NET 4.0 (which means you should be able to use VS2010 as well).

I am going to create a very simple model and a CRUD controller for testing purposes. Also I will not bother with any kind of IoC container and just go with a static in memory repository.

public class Beer : Entity  
{
    public string Name { get; set; }
    public string Description { get; set; }
    public decimal Price { get; set; }
}

public class Entity  
{
    public Guid Id { get; set; }
}

public class BeersController : ApiController  
{
    private IRepository<Beer> _beerRepository;

    public BeersController()
    {
        _beerRepository = WebApiApplication.BeerRepository;
    }

    public IEnumerable<Beer> Get()
    {
        return _beerRepository.Items.ToArray();
    }

    public Beer Get(Guid id)
    {
        Beer entity = _beerRepository.Get(id);
        if (entity  null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        return entity;
    }

    public HttpResponseMessage Post([FromBody] Beer value)
    {
        var result = _beerRepository.Add(value);
        if (result  null)
        {
            // the entity with this key already exists
            throw new HttpResponseException(HttpStatusCode.Conflict);
        }
        var response = Request.CreateResponse<Beer>(HttpStatusCode.Created, value);
        string uri = Url.Link("DefaultApi", new { id = value.Id });
        response.Headers.Location = new Uri(uri);
        return response;
    }

    public HttpResponseMessage Put(Guid id, Beer value)
    {
        value.Id = id;
        var result = _beerRepository.Update(value);
        if (result  null)
        {
            // entity does not exist
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        return Request.CreateResponse(HttpStatusCode.NoContent);
    }

    public HttpResponseMessage Delete(Guid id)
    {
        var result = _beerRepository.Delete(id);
        if (result  null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        return Request.CreateResponse(HttpStatusCode.NoContent);
    }
}

We can seed the repository with sample data.

public class WebApiApplication : System.Web.HttpApplication  
{
    public static IRepository<Beer> BeerRepository = new InMemoryRepository<Beer>();

    protected void Application_Start()
    {
        BeerRepository.Add(new Beer()
        {
            Name = "Blue Moon",
            Description = "Belgian-style witbier. Orange-amber in color with a cloudy appearance.",
            Id = Guid.NewGuid(),
            Price = 1.99m
        });
        var config = GlobalConfiguration.Configuration;
        WebApiConfig.Register(config);
    }
}

Before continuing make sure that the project runs correctly under ASP.NET and IIS.

Setting up the environment

There are two main things you will need in order to run the service on Linux:

• Working Mono installation including XSP - I will be using version 3.0.5, so this or any above should work,
• HTTP server (unless you want to use Mono's own XSP, which should be enough for testing purposes) that supports FastCGI. I will be using Nginix. Apache should also work (through mod_mono).

Now, there are two main ways of getting Mono - either to compile and install it directly from the source code available at github (or official mono packages available at download site) or to install a package provided by your distribution. Mono support varies among different distributions and to be honest is usually pretty weak when it comes to latest versions. If you just want to experiment and don't have an existing Linux installation I would suggest choosing OpenSUSE or Gentoo (evil grin).

To compile from official tarball package use:

./configure --prefix=/usr/local  
make  
make install

And to clone and compile latest code from master branch:

git clone git://github.com/mono/mono.git  
cd mono  
./autogen.sh --prefix=/usr/local
make  
make install

In this example I will be using Gentoo Linux (as it is my favorite distro :)). Official Gentoo repository (at the time of writing this post) does not contain Mono versions above 2.X so we will need to use layman and dotnet overlay.

emerge -av layman  
layman -a dotnet

Depending on your current configuration you may also need to add USE keywords to the xsp package and unmask mono packages as newest versions are usually considered 'unstable'.

echo "dev-lang/mono ~amd64" >> /etc/portage/package.keywords  
echo "dev-dotnet/xsp ~amd64" >> /etc/portage/package.keywords

echo "dev-dotnet/xsp net40 net45" >> /etc/portage/package.use

Now cross your fingers and emerge mono:

emerge -av =mono-3.0.5

Use a higher version if available as it is likely that it contains bugfixes. This may take a while so go grab a coffee or something ;)

Portage will try to resolve dependencies and if everything goes well you should be able to run mono on your system.

ester ~ # mono --version  
Mono JIT compiler version 3.0.5 (tarball Sat Mar  2 13:04:59 Local time zone must be set--see zic manual page 2013)  
Copyright (C) 2002-2012 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com

If it does not work (e.g. compilation fails) - don't give up, try to search for the solution or use a different package (for example using official source packages instead of cutting edge latest master).

The next step is to install XSP (Mono application server). Again you can use the sources or get the package provided by your distribution. Under gentoo this means:

emerge -av xsp

If both installations were successful go ahead and copy the exported (right click on the project and Publish...) Web API application to some directory on your Linux box (eg. using WinSCP). You can use XSP to test if everything works (remember to use xsp4, which is a .NET 4 version):

xsp4 --root /home/pwalat/Piotr.WebApiMono/ --port 8082

Voila! now use your favorite browser or HTTP debugger to test the service.

As  you can see VS studio compiled Web API code just works under Mono runtime. You don't even need to fiddle with web.config.

Alternatively can choose to compile the sources under Mono.

xbuild Piotr.WebApiMono.sln

Now let's set up Nginx to serve our Web API project as using XSP is good for ad-hoc testing only. First of all install the server and make sure you enable fastcgi support:

echo "www-servers/nginx fastcgi ssl nginx_modules_http_gzip_static" >> /etc/portage/package.use  
emerge -av nginx

Once you have nginx installed you will need to modify virtual host configuration to run off fastcgi (/etc/nginx/nginx.conf):

server {  
 listen   80;
 server_name  domain.com;
 access_log   /var/log/nginx/domain.com.access.log;
 location ~ / { 
   root /var/www/domain.com/;
   index index.html index.htm default.aspx default.htm;
   fastcgi_index /default.htm;
   fastcgi_pass 127.0.0.1:9002;     
   fastcgi_param  PATH_INFO          "";
   fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
   include /etc/nginx/fastcgi_params;
 } 
}

This configures Nginix to pass incoming requests to 127.0.0.1:9002 where Mono FastCGI server will be listening. Of course you can have multiple applications hosted by one Nginx instance (e.g. you can mix ASP.NET with PHP or static pages).

To run fastcgi-mono-server4 we need to provide it a list of applications. We can do it either as a command line parameter or use .webapp config files. Let's do the latter as it is more manageable approach.

mkdir /etc/webapps  
nano /etc/webapps/MonoWebApi.webapp

<apps>  
<web-application>
        <name>MonoWebApi</name>
        <vhost>domain.com</vhost>
        <vport>80</vport>
        <vpath>/</vpath>
        <path>/var/www/domain.com</path>
</web-application>
</apps>

Now we can run both servers:

/etc/init.d/nginx start  
fastcgi-mono-server4 --appconfigdir /etc/webapps /socket=tcp:127.0.0.1:9002

If everything worked correctly your service should run off Nginx and be available at http://domain.com/beers

Instead of having to run your Mono server each time from command line you probably will want to have it started during the boot time. Here is a simple /etc/init.d/mono-fastcgi script to facilitate this:

#!/bin/sh  
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin  
DAEMON=/usr/bin/mono  
NAME=mono-fastcgi  
DESC=mono-fastcgi  
PORT=9002  
MONOSERVER=$(which fastcgi-mono-server4)  
MONOSERVER_PID=$(ps auxf | grep fastcgi-mono-server4.exe | grep -v grep | awk '{print $2}')  
WEBAPPS="/etc/webapps"  
case "$1" in  
start)  
if [ -z "${MONOSERVER_PID}" ]; then  
echo "starting mono server"  
${MONOSERVER} --appconfigdir ${WEBAPPS} /socket=tcp:127.0.0.1:${PORT} &
echo "mono server started"  
else  
echo ${WEBAPPS}  
echo "mono server is running"  
fi  
;;
stop)  
if [ -n "${MONOSERVER_PID}" ]; then  
kill ${MONOSERVER_PID}  
echo "mono server stopped"  
else  
echo "mono server is not running"  
fi  
;;
esac  
exit 0

chmod +x /etc/init.d/mono-fastcgi  
rc-update add mono-fastcgi default  
/etc/init.d/mono-fastcgi start

You should have your ASP.NET Web API service running under Nginix now :)

Just to test that basic Web API functionality and message handling pipeline works correctly I've added delegating handler to calculate content's MD5 checksum and a CSV media formatter (have a look at the source code for implementation). Both work  as expected.

OS X

Generally we need to follow the same procedure as with Linux - i.e. install Mono and then install and configure Nginx.

Installing Mono on Mac should be much easier  than on Linux. Just grab OS X package (I use MDK) from mono site here and run the installer.

Optionally you can also install Xamarin Studio (aka Monodevelop 4.0) to open the solution, develop and then build and Deploy ASP.NET Web API application to a folder.

 Now test the exported application using XSP:

xsp4 --root /Volumes/mc/ExportedApps/Piotr.WebApiMono/ --port 8080

We will install Nginx using MacPorts (if you dont have MacPorts install it from a package available here)

sudo port install nginx  
cp /opt/local/etc/nginx/nginx.conf.example /opt/local/etc/nginx/nginx.conf

If you want to run Nginx during startup execute

sudo port load nginx

Once this is done you can follow Nginx configuration instructions for Linux and configure the server to use fastcgi. The configuration file will be located at /opt/local/etc/nginx/nginx.conf. Then run fastcgi-mono-server4 and you will have ASP.NET Web API service running on OS X.

Please be advised that this example was an experiment and you still may run into issues when digging deeper :) That being said, I am sure that running under Mono will make ASP.NET Web API an appealing choice to even wider group of developers.

HMAC based authentication

HMAC (hash-based message authentication code) provides a relatively simple way to authenticate HTTP messages using a secret that is known to both client and server. Unlike basic authentication it does not require transport level encryption (HTTPS), which makes its an appealing choice in certain scenarios. Moreover, it guarantees message integrity (prevents malicious third parties from modifying contents of the message).

On the other hand proper HMAC authentication implementation requires slightly more work than basic HTTP authentication and not all client platforms support it out of the box (most of them support cryptographic algorithms required to implement it though). My suggestion would be to use it only if HTTPS + basic authentication does not suit your requirements.

One prominent example of HMAC usage is Amazon S3 service.

The basic idea behind HMAC authentication in HTTP can be described as follows:

• both client and server have access to a secret that will be used to generate HMAC - it can be a password (or preferably password hash) created by the user at the time of registration,
• using the secret client generates a message signature using HMAC algorithm (the algorithm is provided by .NET 'for free'),
• signature is attached to the message (eg. as a header) and the message is sent,
• the server receives the message and calculates its own version of the signature using the secret (both client and server use the same HMAC algorithm),
• if the signature computed by the server matches the on the message it means that the message is authorized.

As you can see the secret key (eg. password hash) is only shared between client and server once (eg. during user registration). Noone will be able to produce a valid signature without the access to the secret also any modification of the message (eg. appending content) will result in server calculating a different signature and refusing authorization.

Broadly speaking to create a HMAC authenticated client/server pair using ASP.NET Web API we need:

• method that will return a string representing given http request,
• method that based on secret string and message representation calculates HMAC signature,
• client side - message handler that uses these methods to calculate the signature and attaches it to the request (as HTTP header),
• server side - message handler that calculates signature of incoming request and compares it with the one contained in the header.

Web API client

Ok, so let's start by writing the first piece.

public interface IBuildMessageRepresentation  
{
    string BuildRequestRepresentation(HttpRequestMessage requestMessage);
}

public class CanonicalRepresentationBuilder : IBuildMessageRepresentation  
{
    /// <summary>
    /// Builds message representation as follows:
    /// HTTP METHOD\n +
    /// Content-MD5\n +  
    /// Timestamp\n +
    /// Username\n +
    /// Request URI
    /// </summary>
    /// <returns></returns>
    public string BuildRequestRepresentation(HttpRequestMessage requestMessage)
    {
        bool valid = IsRequestValid(requestMessage);
        if (!valid)
        {
            return null;
        }

        if (!requestMessage.Headers.Date.HasValue)
        {
            return null;
        }
        DateTime date = requestMessage.Headers.Date.Value.UtcDateTime;

        string md5 = requestMessage.Content  null ||
            requestMessage.Content.Headers.ContentMD5  null ?  "" 
            : Convert.ToBase64String(requestMessage.Content.Headers.ContentMD5);

        string httpMethod = requestMessage.Method.Method;
        //string contentType = requestMessage.Content.Headers.ContentType.MediaType;
        if (!requestMessage.Headers.Contains(Configuration.UsernameHeader))
        {
            return null;
        }
        string username = requestMessage.Headers
            .GetValues(Configuration.UsernameHeader).First();
        string uri = requestMessage.RequestUri.AbsolutePath.ToLower();
        // you may need to add more headers if thats required for security reasons
        string representation = String.Join("\n", httpMethod,
            md5, date.ToString(CultureInfo.InvariantCulture),
            username, uri);

        return representation;
    }

    private bool IsRequestValid(HttpRequestMessage requestMessage)
    {
        //for simplicity I am omitting headers check (all required headers should be present)

        return true;
    }
}

A couple of points worth mentioning:

• we construct message representation by concatenating 'important' headers, http method and uri,
• instead of using incorporating the content we use its md5 hash (base64 encoded),
• all parts of the message (eg. headers) that can affect its meaning and have side effects on the server side should be included in the representation (otherwise an attacker would be able to modify them without changing the signature).

Now lets look at that component that will calculate authentication code (signature).

public interface ICalculteSignature  
{
    string Signature(string secret, string value);
}

public class HmacSignatureCalculator : ICalculteSignature  
{
    public string Signature(string secret, string value)
    {
        var secretBytes = Encoding.UTF8.GetBytes(secret);
        var valueBytes = Encoding.UTF8.GetBytes(value);
        string signature;

        using (var hmac = new HMACSHA256(secretBytes))
        {
            var hash = hmac.ComputeHash(valueBytes);
            signature = Convert.ToBase64String(hash);
        }
        return signature;
    }
}

The signature will be encoded using base64 so that we can pass it easily in a header. What header you may ask? Well, unfortunately there is no standard way of  including message authentication codes into the message (as there is no standard way of constructing message representation). We will use Authorization HTTP header for that purpose providing a custom schema (ApiAuth).

Authorization: ApiAuth HMAC_SIGNATURE

The HMAC will be calculated and attached to the request in a custom message handler.

public class HmacSigningHandler : HttpClientHandler  
{
    private readonly ISecretRepository _secretRepository;
    private readonly IBuildMessageRepresentation _representationBuilder;
    private readonly ICalculteSignature _signatureCalculator;

    public string Username { get; set; }

    public HmacSigningHandler(ISecretRepository secretRepository,
                          IBuildMessageRepresentation representationBuilder,
                          ICalculteSignature signatureCalculator)
    {
        _secretRepository = secretRepository;
        _representationBuilder = representationBuilder;
        _signatureCalculator = signatureCalculator;
    }

    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request,
                                 System.Threading.CancellationToken cancellationToken)
    {
        if (!request.Headers.Contains(Configuration.UsernameHeader))
        {
            request.Headers.Add(Configuration.UsernameHeader, Username);
        }
        request.Headers.Date = new DateTimeOffset(DateTime.Now,DateTime.Now-DateTime.UtcNow);
        var representation = _representationBuilder.BuildRequestRepresentation(request);
        var secret = _secretRepository.GetSecretForUser(Username);
        string signature = _signatureCalculator.Signature(secret,
            representation);

        var header = new AuthenticationHeaderValue(Configuration.AuthenticationScheme, signature);

        request.Headers.Authorization = header;
        return base.SendAsync(request, cancellationToken);
    }
}

public class Configuration  
{
    public const string UsernameHeader = "X-ApiAuth-Username";
    public const string AuthenticationScheme = "ApiAuth";
}

public class DummySecretRepository : ISecretRepository  
{
    private readonly IDictionary<string, string> _userPasswords
        = new Dictionary<string, string>()
              {
                  {"username","password"}
              };

    public string GetSecretForUser(string username)
    {
        if (!_userPasswords.ContainsKey(username))
        {
            return null;
        }

        var userPassword = _userPasswords[username];
        var hashed = ComputeHash(userPassword, new SHA1CryptoServiceProvider());
        return hashed;
    }

    private string ComputeHash(string inputData, HashAlgorithm algorithm)
    {
        byte[] inputBytes = Encoding.UTF8.GetBytes(inputData);
        byte[] hashed = algorithm.ComputeHash(inputBytes);
        return Convert.ToBase64String(hashed);
    }
}

public interface ISecretRepository  
{
    string GetSecretForUser(string username);
}

In a real life scenario you could retrieve the hashed password from the a persistent store (a database). If you remember how we constructed our message representation you will notice that we also need to set content MD5 header. We could do it in HmacSigningHandler, but to have separation of concerns and because Web API allows us to combine handlers in a neat way I moved it to a separate (dedicated) handler.

public class RequestContentMd5Handler : DelegatingHandler  
{
    protected async override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request,
                                       System.Threading.CancellationToken cancellationToken)
    {
        if (request.Content  null)
        {
            return await base.SendAsync(request, cancellationToken);
        }

        byte[] content = await request.Content.ReadAsByteArrayAsync();
        MD5 md5 = MD5.Create();
        byte[] hash = md5.ComputeHash(content);
        request.Content.Headers.ContentMD5 = hash;
        var response = await base.SendAsync(request, cancellationToken);
        return response;
    }
}

For simplicity the HMAC handler derives directly from HttpClientHandler. Here is how we would make a request:

static void Main(string[] args)  
{
    var signingHandler = new HmacSigningHandler(new DummySecretRepository(),
                                            new CanonicalRepresentationBuilder(),
                                            new HmacSignatureCalculator());
    signingHandler.Username = "username";

    var client = new HttpClient(new RequestContentMd5Handler()
    {
        InnerHandler = signingHandler
    });
    client.PostAsJsonAsync("http://localhost:48564/api/values","some content").Wait();
}

And that's basically it as far as http client is concerned. Let's have a look at server part.

Web API service

The general logic will be that we will want to authenticate every incoming request (we can us per route handlers to secure only one route for example). Each request's authentication code will be calculated using the very same IBuildMessageRepresentation and ICalculateSignature implementations. If the signature does not match (or the content md5 hash is different from the value in the header) we will immediately return a 401 response.

public class HmacAuthenticationHandler : DelegatingHandler  
{
    private const string UnauthorizedMessage = "Unauthorized request";

    private readonly ISecretRepository _secretRepository;
    private readonly IBuildMessageRepresentation _representationBuilder;
    private readonly ICalculteSignature _signatureCalculator;

    public HmacAuthenticationHandler(ISecretRepository secretRepository,
        IBuildMessageRepresentation representationBuilder,
        ICalculteSignature signatureCalculator)
    {
        _secretRepository = secretRepository;
        _representationBuilder = representationBuilder;
        _signatureCalculator = signatureCalculator;
    }

    protected async Task<bool> IsAuthenticated(HttpRequestMessage requestMessage)
    {
        if (!requestMessage.Headers.Contains(Configuration.UsernameHeader))
        {
            return false;
        }

        if (requestMessage.Headers.Authorization  null 
            || requestMessage.Headers.Authorization.Scheme 
                    != Configuration.AuthenticationScheme)
        {
            return false;
        }

        string username = requestMessage.Headers.GetValues(Configuration.UsernameHeader)
                                .First();
        var secret = _secretRepository.GetSecretForUser(username);
        if (secret  null)
        {
            return false;
        }

        var representation = _representationBuilder.BuildRequestRepresentation(requestMessage);
        if (representation  null)
        {
            return false;
        }

        if (requestMessage.Content.Headers.ContentMD5 != null 
            && !await IsMd5Valid(requestMessage))
        {
            return false;
        }

        var signature = _signatureCalculator.Signature(secret, representation);        

        var result = requestMessage.Headers.Authorization.Parameter  signature;

        return result;
    }

    protected async override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request,
           System.Threading.CancellationToken cancellationToken)
    {
        var isAuthenticated = await IsAuthenticated(request);

        if (!isAuthenticated)
        {
            var response = request
                .CreateErrorResponse(HttpStatusCode.Unauthorized, UnauthorizedMessage);
            response.Headers.WwwAuthenticate.Add(new AuthenticationHeaderValue(
                Configuration.AuthenticationScheme));
            return response;
        }
        return await base.SendAsync(request, cancellationToken);
    }
}

The bulk of work is done by IsAuthenticated() method. Also please note that we do not sign the response, meaning the client will not be able verify the authenticity of the response (although response signing would be easy to do given components that we already have). I have omitted IsMd5Valid() method for brevity, it basically compares content hash with MD5 header value (just remember not to compare byte[] arrays using operator).

Configuration part is simple and can look like that (per route handler):

config.Routes.MapHttpRoute(  
                name: "DefaultApi",
                routeTemplate: "api/{controller}/{id}",
                constraints: null,
                handler: new HmacAuthenticationHandler(new DummySecretRepository(),
                    new CanonicalRepresentationBuilder(), new HmacSignatureCalculator())
                    {
                        InnerHandler = new HttpControllerDispatcher(config)
                    },
                defaults: new { id = RouteParameter.Optional }
            );

Replay attack prevention

There is one very important flaw in the current approach. Imagine a malicious third party intercepts a valid (properly authenticated) HTTP request coming from a legitimate client (eg. using a sniffer). Such a message can be stored and resent to our server at any time enabling attacker to repeat operations performed previously by authenticated users. Please note that new messages still cannot be created as the attacker does not know the secret nor has a way of retrieving it from intercepted data.

To help us fix this issue lets make following three observations/assumptions about dates of  requests in our system:

• requests with different Date header values will have different signatures, thus attacker will not be able to modify the timestamp,
• we assume identical, consecutive messages coming from a user will always have different timestamps - in other words that no client will want to send two or more identical messages at a given point in time,
• we introduce a requirement that no http request can be older than X (eg. 5) minutes - if for any reason the message is delayed for more than that it will have to be resent with a refreshed timestamp.

Once we know the above we can introduce following changes into IsAuthenticated() method:

protected async Task<bool> IsAuthenticated(HttpRequestMessage requestMessage)  
{
    //(...)
    var isDateValid = IsDateValid(requestMessage);
    if (!isDateValid)
    {
        return false;
    }
    //(...)

    //disallow duplicate messages being sent within validity window (5 mins)
    if(MemoryCache.Default.Contains(signature))
    {
        return false;
    }

    var result = requestMessage.Headers.Authorization.Parameter  signature;
    if (result  true)
    {
        MemoryCache.Default.Add(signature, username,
                DateTimeOffset.UtcNow.AddMinutes(Configuration.ValidityPeriodInMinutes));
    }
    return result;
}

private bool IsDateValid(HttpRequestMessage requestMessage)  
{
    var utcNow = DateTime.UtcNow;
    var date = requestMessage.Headers.Date.Value.UtcDateTime;
    if (date >= utcNow.AddMinutes(Configuration.ValidityPeriodInMinutes)
        || date <= utcNow.AddMinutes(-Configuration.ValidityPeriodInMinutes))
    {
        return false;
    }
    return true;
}

For simplicity I didn't test the example for sever and client residing in different timezones (although as long as we normalize the dates to UTC we should be save here).

The code is available as usually on bitbucket.

Hope this article helps some of you!

This is an introductory post that shows examples of arrow functions usage in TypeScript and explains how they differ from standard function expressions in terms of this keyword binding.

If you are familiar with C#, arrow functions have similar look and feel to lambda expressions. Let's start by creating a simple TypeScript function that calculates sum earned by deposited funds.

var calculateInterest = function (amount, interestRate, duration) {  
    return amount * interestRate * duration / 12;
}

Using arrow function expression we can define this function alternatively as follows:

var calculateInterest2 = (amount, interestRat, duration) => {  
    return amount * interestRate * duration / 12;
}

We can simplify this further by using assignment expression instead of function body block:

var calculateInterest3 = (amount, interestRate, duration) => amount * interestRate * duration / 12;

This is a compact form that saves us some typing. It is worth mentioning that all three TypeScript functions will compile to identical JavaScript code.

var calculateInterest = function (amount, interestRate, duration) {  
    return amount * interestRate * duration / 12;
};
var calculateInterest2 = function (amount, interestRate, duration) {  
    return amount * interestRate * duration / 12;
};
var calculateInterest3 = function (amount, interestRate, duration) {  
    return amount * interestRate * duration / 12;
};

As you possibly know TypeScript provides a way of describing object contracts in the form of interfaces. Arrow expressions can be used to define function properties of the object.

interface BankAccount {  
    balance: number;

    withdraw: (amount: number) => void;
    deposit: (amount: number) => void;
    calculateInterest(interestRate: number, duration: number) : number;
}

Just like standard function expressions, arrow functions support optional and default parameters.

var executeStandingOrder  
    = function (amount: number = 100, description?: string) {
        var message = 'Standing order amount = $' + amount;
        console.log(message);
    }

var executeStandingOrder2  
     = (amount: number = 100, description?: string) => {
        var message = 'Standing order amount = $' + amount;
        console.log(message);
    }

This will result in identical JavaScript being generated for both functions.

var executeStandingOrder = function (amount, description) {  
    if (typeof amount === "undefined") { amount = 100; }
    var message = 'Standing order amount = $' + amount;
    console.log(message);
};

At this point you may think that standard and arrow functions are pretty much the same and can always be used interchangeably. This is not true, as there is one subtle but very important difference between the two.

Consider the following implementation of BankAccount interface.

var account: BankAccount = {  
    balance: 2020,
    withdraw: (amount: number) => {
        this.balance -= amount;
    },
    deposit: function (amount: number) {
        this.balance += amount;
    },
    calculateInterest: function (interestRate: number, duration: number) {
        return this.balance * interestRate * duration / 12;
    }
}

Now let's deposit and withdraw some funds.

account.deposit(4000);
account.withdraw(2000);
console.log('Balance = $' + account.balance);

If I got my math right, the resulting balance after the operations should be $4020, but when we execute this code (I am using jQuery to execute after DOM has been loaded) we will see the following output.

Balance = $6020

Well, clearly something is wrong with our logic and even worse there is no console error that would point us in a right direction. If you have some JavaScript experience, you may immediately suspect 'this' keyword usage as a potential culprit. And you are right.

Standard functions (ie. written using function keyword) will dynamically bind this depending on execution context (just like in JavaScript), arrow functions on the other hand will preserve this of enclosing context. This is a conscious design decision as arrow functions in ECMAScript 6 are meant to address some problems associated with dynamically bound this (eg. using function invocation pattern).

To clarify what it means in practice, let's look at generated JavaScript:

var _this = this;  
var account = {  
    balance: 2020,
    withdraw: function (amount) {
        _this.balance -= amount;
    },
    deposit: function (amount) {
        this.balance += amount;
    }
};

As you can see for withdraw arrow function TypeScript uses var that=this pattern/trick to preserve this of enclosing context (which is a global object = window  in this scenario). In other words we are trying to decrease balance property of a global object (window.balance = window.balance - amount). Because window.balance is undefined we end up assigning NaN to the property and no error is being thrown (another nice 'feature' of the language ;)).

Deposit on the other hand is defined using standard function expression, this means that if we invoke it as a method (ie. account.deposit(4000)), this will point to account object and everything will work as expected.

Aforementioned behavior of arrow functions can be particularly useful when dealing with callbacks. Let's add following method to the interface and its implementation:

interface BankAccount {  
    balance: number;
    //(...)
    standingOrder: (amount:number) => void;
}

var account: BankAccount = {  
    balance: 2020,
    //...
    standingOrder: function (amount: number) {
        setTimeout(() => {
            this.balance -= amount;
            console.log('Standing order executed. Current balance is $' 
                                                    + this.balance);
        }, 700);
    }
}

In the example above we are invoking setTimeout function that accepts a callback to be executed after a given delay. If we used standard function statement as the callback, then this would be bound to a global object and the code wouldn't work as expected. However, because we are using arrow function this will be preserved from enclosing context and will point to account object as intended.

In this post I am going to show how to leverage some of OData features introduced to ASP.NET Web API to build example service.

OData

You may be wondering why would you need another http based protocol for your web apps. Aren't simple JSON or XML services good enough? Well in fact OData extends these and is not meant to be a replacement. It can use either XML (ATOM) or JSON to represent resources and what is important adheres to REST principles. In some sense it builds on top of 'simple' REST HTTP services with a very clear aim - to simplify and standardize the way we manipulate and query resources and data sets. If your application is data centric chances are you could benefit from OData. Also if you've struggled to create search/filter or paging api for your REST services, OData has a provides this as well.

Some examples of OData query syntax:

• Entity set - /Artists
• Entity by id - /Artists(1)
• Sorting - /Artists?$orderby=Name
• Filtering - /Artists?$filter=Name eq 'Gridlock'

But that's just a tip of an iceberg.

Instead of talking more, let's write some code. Fortunately, ASP.NET Web API let's us create OData endpoints quite easily.

Creating the project

Let's start by creating a new ASP.NET Web API project. We won't need any MVC related things (views, js libraries, etc.).
OData functionality is provided by a separate assembly that has to be installed separately. Please note that at the time of writing the package is still pre-release and the latest version available on official nuget repository is 0.3 RC (see this blogpost for a detailed overview of the release).
Unfortunately there is an issue when using this package with latest ODataLib preventing some functionality from working (eg. filtering). Moreover newer commits introduced some breaking changes in configuration API. Because there is no point learning deprecated API we will use nightly builds available at http://www.myget.org/F/aspnetwebstacknightly/ nuget source. If you are unsure how to configure nuget to get these, have a look here.

Once you set up nightly build nuget source, you can install latest Web API OData package using Manage NuGet Packages, jus make sure you select 'Include Prerelease' in the dropdown on the top.

Please have in mind that Web API OData support is still work in progress and may miss support for certain OData features. Having said this, it definitely provides an impressive set of functionality.

Data model

We need a simple model to operate on. I will use Entity Framework and SQL CE 4, but Web API's OData implementation does not constraint you to any particular data persistence technology.

CREATE TABLE [Album]  
(
    [AlbumId] INT NOT NULL IDENTITY,
    [Title] NVARCHAR(160) NOT NULL,
    [ArtistId] INT NOT NULL,
    [GenreId] INT NOT NULL,
    [ReleaseDate] DATETIME,
    CONSTRAINT [PK_Album] PRIMARY KEY  ([AlbumId])
);

CREATE TABLE [Artist]  
(
    [ArtistId] INT NOT NULL IDENTITY,
    [Name] NVARCHAR(120),
    CONSTRAINT [PK_Artist] PRIMARY KEY  ([ArtistId])
);

CREATE TABLE [Genre]  
(
    [GenreId] INT NOT NULL IDENTITY,
    [Name] NVARCHAR(120),
    [Description] NVARCHAR(1020),
    CONSTRAINT [PK_Genre] PRIMARY KEY  ([GenreId])
);

ALTER TABLE [Album] ADD CONSTRAINT [FK_AlbumArtistId]  
    FOREIGN KEY ([ArtistId]) REFERENCES [Artist] ([ArtistId]) 
      ON DELETE NO ACTION ON UPDATE NO ACTION;

CREATE INDEX [IFK_AlbumArtistId] ON [Album] ([ArtistId]);

ALTER TABLE [Album] ADD CONSTRAINT [FK_AlbumGenreId]  
    FOREIGN KEY ([GenreId]) REFERENCES [Genre] ([GenreId]) 
      ON DELETE NO ACTION ON UPDATE NO ACTION;

CREATE INDEX [IFK_AlbumGenreId] ON [Album] ([GenreId]);

You can create a new SQL CE database in App_Data folder and use built in explorer to execute SQL code. Please note that it does not support execution of multiple statements so you will need to execute it one by one. Once database schema is in place we can generate Entity Data Model using the wizard provided (it should automatically detect the database created).

In the end we should get a DbContext class that will be used to perform data operations.

$metadata endpoint and IEdmModel

As I mentioned previously OData standard defines a special metadata endpoint that contains a document defining the entity sets, relationships, entity types, and operations. This makes OData service self-describing and enables client libraries to generate client-side code to represent server types and simplify service access (for example by generating proxies). Metadata endpoint should be available under /$metadata. If you are familiar with SOAP services you can think about it roughly as a WSDL analogue.

GET http://services.odata.org/Northwind/Northwind.svc/$metadata

Metadata document uses OData Common Schema Definition Language (CSDL). Fortunately ASP.NET Web API can expose  $metadata endpoint for us, as long as we supply a representation of our model in the form of IEdmModel object.

public class ModelBuilder  
{
    public IEdmModel Build()
    {
        ODataModelBuilder modelBuilder = new ODataConventionModelBuilder();
        modelBuilder.EntitySet<Album>("Albums");
        modelBuilder.EntitySet<Artist>("Artists");
        modelBuilder.EntitySet<Genre>("Genres");

        return modelBuilder.GetEdmModel();
    }
}

You can also build model representation explicitly using ODataModelBuilder to have more fine grained control over generated representation.

public IEdmModel BuildExplicitly()  
{
    ODataModelBuilder modelBuilder = new ODataModelBuilder();
    EntitySetConfiguration<Genre> genres = modelBuilder.EntitySet<Genre>("Genres");
    EntityTypeConfiguration<Genre> genre = genres.EntityType;
    genre.HasKey(g => g.GenreId);
    genre.Property(g => g.Name);
    genre.Property(g => g.Description);

    //(...)

    return modelBuilder.GetEdmModel();
}

Enabling OData

Microsoft.AspNet.WebApi.OData package provides a set of classes that are supposed to plug into Web API extensibility points in order to provide OData support (formatters, path handling, etc.).

The RC version had a single HttpConfiguration.EnableOData(IEdmModel) helper method that did all this in one go. However this approach wasn't really well fitted for scenarios where we wanted to support different models in one application. Because of this latest versions use per route configuration (which is more flexible).

public static class WebApiConfig  
{
    public static void Register(HttpConfiguration config)
    {
        var modelBuilder = new ModelBuilder();
        IEdmModel model = modelBuilder.Build();
        config.Routes.MapODataRoute("OData", null, model);
        config.EnableQuerySupport();
    }
}

This code (executed from Global.asax.cs) does two things:

• registers our model representation (IEdmModel) with a route - we are using null for route prefix, meaning it will be the root route, but we could have specified something like 'albums' making OData Albums services available at /albums instead of / (and thanks to this we could serve more data models in one app)
• enables query support (EnableQuerySupport()) for actions returning IQueryable<T> (we will show what's that about when creating controllers)

Now our service should automagically know how to handle OData ~/$metadata request. Cool, isn't it :) ?

<edmx:Edmx xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx" Version="1.0">  
  <edmx:DataServices xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" m:DataServiceVersion="1.0">
    <Schema xmlns="http://schemas.microsoft.com/ado/2009/11/edm" Namespace="Piotr.ODataWebApiService.Service.Models">
      <EntityType Name="Album">...</EntityType>
      <EntityType Name="Artist">...</EntityType>
      <EntityType Name="Genre">...</EntityType>
      <Association 
        Name="Piotr_ODataWebApiService_Service_Models_Album_Artist_Piotr_ODataWebApiService_Service_Models_Artist_ArtistPartner">...</Association>
      <Association Name="Piotr_ODataWebApiService_Service_Models_Album_Genre_Piotr_ODataWebApiService_Service_Models_Genre_GenrePartner">...</Association>
      <Association Name="Piotr_ODataWebApiService_Service_Models_Artist_Albums_Piotr_ODataWebApiService_Service_Models_Album_AlbumsPartner">...</Association>
      <Association Name="Piotr_ODataWebApiService_Service_Models_Genre_Albums_Piotr_ODataWebApiService_Service_Models_Album_AlbumsPartner">...</Association>
    </Schema>
    <Schema xmlns="http://schemas.microsoft.com/ado/2009/11/edm" Namespace="Default">...</Schema>
  </edmx:DataServices>
</edmx:Edmx>

Controllers

Now it's time to reap the reward. We need to add controllers that will actually expose our entities as OData resources. As you will see this is not very different from writing 'regular' CRUD controllers. It is very easy to expose OData entity set.

[ODataRouting]  
[ODataFormatting]
public class ArtistsController : ApiController  
{
    private AlbumsContext db = new AlbumsContext();

    // GET /Artists
    // GET /Artists?$filter=startswith(Name,'Grid')
    [Queryable]
    public IQueryable<Artist> Get()
    {
        return db.Artists;
    }        

    protected override void Dispose(bool disposing)
    {
        db.Dispose();
        base.Dispose(disposing);
    }
}

/Artists resource should now become available along with all the fancy filtering functionality for entity sets.

OData specific routing and formatting has been provided by controller attributes. Alternatively instead of deriving ArtistsController from ApiController we could have derived from ODataController which is a helper abstract class already decorated with all the necessary attributes (I would recommend this approach as additional functionality may be introduced in the future).

Now we can continue by providing other actions such as add (POST), update (PUT), partial update (PATCH) and delete.

[ODataRouting]  
[ODataFormatting]
public class ArtistsController : ApiController  
{
    private AlbumsContext _db = new AlbumsContext();

    // GET /Artists
    // GET /Artists?$filter=startswith(Name,'Grid')
    [Queryable]
    public IQueryable<Artist> Get()
    {
        return _db.Artists;
    }

    // GET /Artists(2)
    public HttpResponseMessage Get([FromODataUri]int id)
    {
        Artist artist = _db.Artists.SingleOrDefault(b => b.ArtistId  id);
        if (artist  null)
        {
            return Request.CreateResponse(HttpStatusCode.NotFound);
        }

        return Request.CreateResponse(HttpStatusCode.OK, artist);
    }

    public HttpResponseMessage Put([FromODataUri] int id, Artist artist)
    {
        if (!_db.Artists.Any(a => a.ArtistId  id))
        {
            return Request.CreateResponse(HttpStatusCode.NotFound);
        }
        //overwrite any existing id, as url is more explicit
        artist.ArtistId = id;
        _db.Entry(artist).State = EntityState.Modified;

        try
        {
            _db.SaveChanges();
        }
        catch (DbUpdateConcurrencyException)
        {
            return Request.CreateResponse(HttpStatusCode.NotFound);
        }

        return Request.CreateResponse(HttpStatusCode.NoContent);
    }

    public HttpResponseMessage Post(Artist artist)
    {
        var odataPath = Request.GetODataPath();
        if (odataPath  null)
        {
            return Request.CreateErrorResponse(HttpStatusCode.BadRequest,
                "ODataPath not present in the request.");
        }

        var entitySetPathSegment
            = odataPath.Segments.FirstOrDefault() as EntitySetPathSegment;

        if (entitySetPathSegment  null)
        {
            return Request.CreateErrorResponse(HttpStatusCode.BadRequest,
                "ODataPath does not start with entity set path segment");
        }

        Artist addedArtist = _db.Artists.Add(artist);
        _db.SaveChanges();
        var response = Request
            .CreateResponse(HttpStatusCode.Created, addedArtist);

        response.Headers.Location = new Uri(Url.ODataLink(
                              entitySetPathSegment,
                              new KeyValuePathSegment(ODataUriUtils
                            .ConvertToUriLiteral(addedArtist.ArtistId
                            , ODataVersion.V3))));
        return response;
    }

    public HttpResponseMessage Patch([FromODataUri] int id,
        Delta<Artist> artistPatch)
    {
        Artist artist = _db.Artists
            .SingleOrDefault(p => p.ArtistId  id);
        if (artist  null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }

        artistPatch.Patch(artist);
        _db.SaveChanges();

        return Request.CreateResponse(HttpStatusCode.NoContent);
    }

    public HttpResponseMessage Delete([FromODataUri] int id)
    {
        Artist artist = _db.Artists.Find(id);
        if (artist  null)
        {
            return Request.CreateResponse(HttpStatusCode.NotFound);
        }

        _db.Artists.Remove(artist);

        _db.SaveChanges();
        return Request.CreateResponse(HttpStatusCode.Accepted);
    }

    protected override void Dispose(bool disposing)
    {
        _db.Dispose();
        base.Dispose(disposing);
    }
}

Please note that current builds use JSON as default formatting. You can change the format to XML by using appropriate Accept header when making the request.

Security and mass assignment

In the example above we expose our model directly to the user. We also assume that users can modify data entities including all their properties without restrictions (which in this particular case is not a security problem).
In real life scenarios that involve authorization (and authentication) such an approach is very often unacceptable as we may obliviously expose properties that were never supposed to be modified by the given user. This is especially true when our model contains properties that directly affect authorization or authentication process (eg. the infamous isAdmin flag or a property denoting owner of an object). Usually in such case we need an extra layer of security and validation that will ensure a user does not execute an action he is not permitted to execute (eg. modify an object he does not own). If possible we also can introduce a flattened-out DTO model that is mapped to data model (eg. using Automapper). Such DTOs are meant to 'hide' persistable data model.

Testing the service

I will use Fiddler composer to test the service.

Note the Content-Type: application/json header. This should add a new genre. If we wanted to make a partial update to the entity we would PATCH verb as follows.

Now the genre with id=3 will have an updated description.

Finally, let's issue a query against Artists entity set that will sort the results and return their count:
http://localhost:2537/Artists?$orderby=Name&$inlinecount=allpages

As you can see we didn't have to write any special logic to support this feature - all was provided by the framework. If we wanted we also could have provided custom actions in the controllers as we are not limited to OData specific crud operations.

OData is without a doubt an interesting protocol. It feels a little bit like REST services on steroids :)

The source code is available as usually on bitbucket.

XAML's DebugSettings.EnableFrameRateCounter

If you are familiar with XAML development, you might have used DebugSettings.EnableFrameRateCounter flag to display performance and framerate related statistics. This can be enabled in App.xaml.cs by adding one line to the constructor.

public App()  
{
    InitializeComponent();
    Suspending += OnSuspending;
    DebugSettings.EnableFrameRateCounter = true;
}

It is a very useful feature, this however wont work when using full screen SwapChainBackgroundPanel(which is the default technique for MonoGame + XAML Windows Store apps).

Simple MonoGame framerate counter

Frame rate calculation algorithm we use is simple and is based on following principles:

• every time a scene has been drawn using Draw() increase FrameCounter by 1,
• in Update() method measure ElapsedTime which is the time (in milliseconds) since last frame rate update,
• if ElapsedTime is more than 1 second (1000 ms) use FrameCounter as the new FPS value.

public class FrameRate  
{
    /// <summary>
    /// Current FPS
    /// </summary>
    public int Rate;

    /// <summary>
    /// Frame counter
    /// </summary>
    public int Counter;

    /// <summary>
    /// Time elapsed from the last update
    /// </summary>
    public float ElapsedTime;

    /// <summary>
    /// Min FPS
    /// </summary>
    public int MinRate = 999;

    /// <summary>
    /// Seconds elapsed
    /// </summary>
    public int SecondsElapsed;
}

public interface IUpdate<T>  
        where T : class
{
    void Update(GameTime gameTime, T objectToUpdate);
}

public interface IDraw<T>  
        where T:class
{
    void LoadContent(ContentManager contentManager);
    void Draw(DrawingContext context, T objectToDraw);
}

DrawingContext is just a placeholder for SpriteBatch (and GraphicsDevice if you need it). Feel free to pass in SpriteBatch directly - this will simplify IDraw signature a little bit

public class DrawingContext  
{
    public SpriteBatch SpriteBatch { get; set; }
    public GraphicsDevice GraphicsDevice { get; set; }
}

Here is the logic that will be executed in Game.Update() method.

public class FrameRateUpdater : IUpdate<FrameRate>  
{
    public void Update(GameTime gameTime, FrameRate frameRate)
    {
        frameRate.ElapsedTime 
            += (float)gameTime.ElapsedGameTime.TotalMilliseconds;

        if (frameRate.ElapsedTime >= 1000f)
        {
            frameRate.ElapsedTime -= 1000f;
            frameRate.Rate = frameRate.Counter;
            if(frameRate.SecondsElapsed > 0 
                && frameRate.MinRate > frameRate.Rate)
            {
                frameRate.MinRate = frameRate.Rate;
            }
            frameRate.Counter = 0;
            frameRate.SecondsElapsed++;
        }
    }
}

public class FrameRateDrawer : IDraw<FrameRate>  
{
    SpriteFont spriteFont;
    private const string FontName = "MapFont";
    private readonly Vector2 _fpsPositionBlack = new Vector2(20,20);
    private readonly Vector2 _fpsPositionWhite = new Vector2(20,20);
    private readonly Vector2 _minPositionBlack = new Vector2(54, 20);
    private readonly Vector2 _minPositionWhite = new Vector2(55, 20);

    public void LoadContent(ContentManager contentManager)
    {
        spriteFont = contentManager.Load<SpriteFont>(FontName);
    }

    public void Draw(DrawingContext context, FrameRate objectToDraw)
    {
        context.SpriteBatch.DrawString(spriteFont,
            objectToDraw.Rate.ToString(), _fpsPositionBlack, Color.Black);
        context.SpriteBatch.DrawString(spriteFont,
            objectToDraw.Rate.ToString(), _fpsPositionWhite, Color.White);

        context.SpriteBatch.DrawString(spriteFont,
            objectToDraw.MinRate.ToString(), _minPositionBlack, Color.Black);
        context.SpriteBatch.DrawString(spriteFont,
            objectToDraw.MinRate.ToString(), _minPositionWhite, Color.Orange);

        objectToDraw.Counter++;
    }
}

Please note that instead of creating new Vector2 objects used to position the strings every time we call  SpriteBatch.DrawString(), we are reusing instances created earlier. This aims to reduce unnecessary garbage collection and even though this may seem obvious to many, such optimizations can be easily overlooked.

Draw method has twofold purpose - obviously it needs to draw frame-rate strings, but apart from that we need it to increase frame counter. To be more flexible you may want to measure the strings instead of using hard-coded position values.

Now, in your Game class you can add and instantiate the following fields:

private FrameRate _frameRate = new FrameRate();  
private FrameRateDrawer _frameRateDrawer = new FrameRateDrawer();  
private FrameRateUpdater _frameRateUpdater = new FrameRateUpdater();

protected override void Update(GameTime gameTime)  
{
    //...
    _frameRateUpdater.Update(gameTime, _frameRate);
    //...
}

protected override void Draw(GameTime gameTime)  
{
    GraphicsDevice.Clear(Color.Black);
    _spriteBatch.Begin();
    //...     
    _frameRateDrawer.Draw(_drawingContext, _frameRate);
    //...
    _spriteBatch.End();
    //...
}

Also don't forget to load the sprite font and initialize content:

protected override void LoadContent()  
{
    _spriteBatch = new SpriteBatch(GraphicsDevice);
    _background = Content.Load<Texture2D>("background");
    _frameRateDrawer.LoadContent(Content);
    //..
    _drawingContext.SpriteBatch = _spriteBatch;
    _drawingContext.GraphicsDevice = _graphics.GraphicsDevice;
}

The end result will look somewhat like this, it is definitely a no frills solution, but its not something that is normally visible to the end-user anyway.

I imagine that frame-rate data (especially min/avg. FPS) could also be used to gather some performance data directly from the users running the game, especially if you cannot test it on all device types that the game is intended for (this shouldn't be that hard with Windows 8/Windows RT). Although this is a material for another post :)

The code is not Windows 8 specific in any way and should work on all MonoGame supported platforms, but I have only tested it in a Windows 8 XAML scenario.

You can skip next step if you already have certificates and do not need to create self-signed surrogates.

Generating certificates

In order to issue client certificate we will need to create a Certificate Authority (CA) using a self-signed certificate. This will be also used to create server certificate that is imported into IIS (makecert should be available in VS command prompt).

makecert -r -pe -n "CN=Awesome CA" -ss CA -a sha1 -sky signature -cy authority -sv AwesomeCA.pvk AwesomeCA.cer  
makecert -pe -n "CN=127.0.0.1" -a sha1 -sky exchange -eku 1.3.6.1.5.5.7.3.1 -ic AwesomeCA.cer -iv AwesomeCA.pvk -sp "Microsoft RSA SChannel Cryptographic Provider"  -sy 12 -sv LocalServer.pvk LocalServer.cer  
pvk2pfx -pvk LocalServer.pvk -spc LocalServer.cer -pfx LocalServer.pfx

After running these commands and entering password for several times, you should end up with a couple of certificate files. Now you will need to tell your computer to trust the newly created CA (it is generally a good idea to remove that trust after you are finished with testing). To do that start mmc.exe as an Administrator Add/Remove Snap-In (Ctrl+M), and when prompted with certificate store option choose Computer account. Under Certificates (Local Computer) choose AllTasks and Import AwesomeCA.cer. It should become visible in the list along other Trusted Root CAs.

Setting up IIS

The next step consists of setting up SSL in IIS. I am using Windows 8 that runs IIS 8, but instructions for 7/7.5 should be very similar/the same.

Open IIS Manager and go to Server Certificates panel. Then click Import.. and ... import your LocalServer.pfx.

Then under Default Web Site go to Bindings and make sure that the https binding is properly set up (if it doesn't exist, create it) and that newly created certificate is mapped to that binding.

Now create a basic ASP.NET Web API application - the template provides ValuesController by default. Make sure that the user that you are running Visual Studio as has sufficient permissions to create new virtual directories in IIS.

Go to Project properties and make sure you use IIS as server (not IIS Express), also use https:// instead of http:// for Project Url option, create virtual directory if necessary. After you do this go back to IIS Manager and under SSL Settings for a newly create virtual directory check 'Require SSL' and 'Require client certificates'. I am using 127.0.0.1 as host address here, but in real life you probably would use a domain name and not an IP address. Remember that the certificate has been issued for that particular name (for example localhost and not 127.0.0.1).

Now when you go to the newly created ASP.NET Web API service using your web browser you will see 403 error, this is because the application requires client to present a SSL certificate and the browser does not have one.

Let's create client cert signed by our CAs:

makecert -pe -n "CN=piotr@piotrwalat.net" -a sha1 -sky exchange -eku 1.3.6.1.5.5.7.3.2 -ic AwesomeCA.cer -iv AwesomeCA.pvk -sv Client.pvk Client.cer  
pvk2pfx -pvk Client.pvk -spc Client.cer -pfx Client.pfx -po PASSWORD

You should end up with Client.pfx certificate that can be imported to your users store (and later on to Windows 8 app certificate store). Import the file (make sure to mark is as exportable - we will need this later) and navigate to /api/values endpoint - this time you should see the data.

Client certificates in Windows 8

Now, lets move on and create a sample Windows 8 XAML app that will consume the service. Windows 8 apps run in sand-boxed environment - this also means that they get their own certificate stores. By enabling Shared User Certificates capability in Package.appxmanifest you allow app to reach for certificates outside of its store, which is useful for example when dealing with smartcards.

You can include certificates that should ship with your app in Certificates declaration inside of Package.appxmanifest. This is an xml file so you can either edit the source directly or use the designer that ships with VS 2012. Copy over CA .cer file to the project folder and add it to Certificates declaration, use "Root" as store name.

  <Capabilities>  
    <Capability Name="sharedUserCertificates" />
    <Capability Name="enterpriseAuthentication" />
    <Capability Name="privateNetworkClientServer" />
    <Capability Name="internetClient" />
  </Capabilities>
  <Extensions>
    <Extension Category="windows.certificates">
      <Certificates>
        <Certificate StoreName="Root" Content="AwesomeCA.cer" />
        <SelectionCriteria AutoSelect="true" />
      </Certificates>
    </Extension>
  </Extensions>

There is a subtle bug in the way that windows 8 xaml apps handle certificates that we need to apply workaround for (jpsanders mentions it here). Make sure to add the following certigicate policy OID to the cleint cert (i am using mmc.exe + cert snap-in to do that).

Also make sure that Client authentication is selected as one of certificate purposes. Export the certificate as .pfx and copy it to the project directory.

Here is the code that can be used to import the file to app certificate storage.

StorageFolder packageLocation = Windows.ApplicationModel.Package.Current.InstalledLocation;  
StorageFolder certificateFolder = await packageLocation.GetFolderAsync("Certificates");  
StorageFile certificate = await certificateFolder.GetFileAsync("Client.pfx");

IBuffer buffer = await Windows.Storage.FileIO.ReadBufferAsync(certificate);  
string encodedString = Windows.Security.Cryptography.CryptographicBuffer.EncodeToBase64String(buffer);

await CertificateEnrollmentManager.ImportPfxDataAsync(  
    encodedString,
    "PASSWORD",
    ExportOption.NotExportable,
    KeyProtectionLevel.NoConsent,
    InstallOptions.None,
    "Client certificate");

In order for HttpClient to be able to use the certificate, we need to create an instance of HttpClientHandler and tell it to pick the certificate automatically.

HttpClientHandler messageHandler = new HttpClientHandler();  
messageHandler.ClientCertificateOptions = ClientCertificateOption.Automatic;  
HttpClient httpClient = new HttpClient(messageHandler);  
HttpResponseMessage result = await httpClient.GetAsync("https://127.0.0.1/Piotr.Win8CertAuth.Api/api/values");

Once you run this code you should be able to successfully connect to ASP.NET Web API service and retrieve the data.

Adding delegating handler

So far the mechanism wasn't really ASP.NET Web API specific and would have really worked in any ASP.NET application. It is also pretty basic, without any logic to really extend certificate validation or provide any kind of certificate-to-user mapping.

So how do we actually retrieve the certificate in ASP.NET Web API? Actually it turns out that's really easy as there is HttpRequestMessage.GetClientCertificate(); extension method that returns certificate object. Let's create a delegating handler that will intercept the request and inject certificate related logic into the pipeline.

public interface IValidateCertificates  
{
    bool IsValid(X509Certificate2 certificate);
    IPrincipal GetPrincipal(X509Certificate2 certificate2);
}

public class BasicCertificateValidator : IValidateCertificates  
{
    public bool IsValid(X509Certificate2 certificate)
    {
        return certificate.Issuer  "CN=Awesome CA"
               && certificate.GetCertHashString()  "B04AED3DA6CB4BD2F817EE2C726183C00035F4C6";
        //make a better check here (eg. against mapping, verify the chain etc)
    }

    public IPrincipal GetPrincipal(X509Certificate2 certificate2)
    {
        return new GenericPrincipal(
            new GenericIdentity(certificate2.Subject), new[] { "User" });
    }
}

public class CertificateAuthHandler : DelegatingHandler  
{
    public IValidateCertificates CertificateValidator { get; set; }

    public CertificateAuthHandler()
    {
        CertificateValidator = new BasicCertificateValidator();
    }

    protected override System.Threading.Tasks.Task<HttpResponseMessage>
        SendAsync(HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
    {
        X509Certificate2 certificate = request.GetClientCertificate();
        if (certificate == null || !CertificateValidator.IsValid(certificate))
        {
            return Task<HttpResponseMessage>.Factory.StartNew(
                () => request.CreateResponse(HttpStatusCode.Unauthorized));

        }
        Thread.CurrentPrincipal = CertificateValidator.GetPrincipal(certificate);
        return base.SendAsync(request, cancellationToken);
    }
}

Now, we can add the handler instance to global configuration object.

GlobalConfiguration.Configuration.MessageHandlers.Add( new CertificateAuthHandler());

From now on for every user that has a valid client certificate we will create IPrincipal object and assign it to current thread. This means that you can use Authorize attribute to provide more granular authorization to your services.

[Authorize(Roles = "User")]  
public IEnumerable<string> Get()  
{
    return new string[] { "value1", "value2" };
}

Hope you find this post helpful (source code coming soon).

• stream wrapper for FileStream that can return partial data,
• memory mapped files.

Simple file download service

Thanks to StreamContent class, creating basic file download service in ASP.NET Web API is a relatively straightforward task. Let's start by implementing a basic scenario, where files are served from a directory.
Instead of dealing with file system access directly in controllers I usually like to encapsulate this functionality in a dedicated object, which makes unit testing/mocking easier and makes code tidier. For our examples we will create IFileProvider interface that exposes three operations:

public interface IFileProvider  
{
    bool Exists(string name);
    FileStream Open(string name);
    long GetLength(string name);
}

The actual implementation will use app settings in web.config file to configure storage folder location.

public class FileProvider : IFileProvider  
{
    private readonly string _filesDirectory;
    const string DefaultFileLocation = "Files";
    private const string AppSettingsKey = "FileProvider.FilesLocation";

    public FileProvider()
    {
        _filesDirectory = DefaultFileLocation;
        var fileLocation = ConfigurationManager.AppSettings[AppSettingsKey];
        if(!String.IsNullOrWhiteSpace(fileLocation))
        {
            _filesDirectory = fileLocation;
        }
    }

    public bool Exists(string name)
    {
        //make sure we dont access directories outside of our store for security reasons
        string file = Directory.GetFiles(_filesDirectory, name, SearchOption.TopDirectoryOnly)
                .FirstOrDefault();
        return file != null;
    }

    public FileStream Open(string name)
    {
        return File.Open(GetFilePath(name), 
            FileMode.Open, FileAccess.Read);
    }

    public long GetLength(string name)
    {
        return new FileInfo(GetFilePath(name)).Length;
    }

    private string GetFilePath(string name)
    {
        return Path.Combine(_filesDirectory, name);
    }
}

<appSettings>  
    <!-- (...) -->
    <add key="FileProvider.FilesLocation" value="H:\Storage" />
</appSettings>

With file access logic ready we can write code that actually serves the data. A simple Web API controller that streams files will look like this:

public class SimpleFilesController : ApiController  
{
    public IFileProvider FileProvider { get; set; }

    public SimpleFilesController()
    {
        FileProvider = new FileProvider();
    }

    public HttpResponseMessage Get(string fileName)
    {
        if (!FileProvider.Exists(fileName))
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }

        FileStream fileStream = FileProvider.Open(fileName);
        var response = new HttpResponseMessage();
        response.Content = new StreamContent(fileStream);
        response.Content.Headers.ContentDisposition
            = new ContentDispositionHeaderValue("attachment");
        response.Content.Headers.ContentDisposition.FileName = fileName;
        response.Content.Headers.ContentType
            = new MediaTypeHeaderValue("application/octet-stream");
        response.Content.Headers.ContentLength 
                = FileProvider.GetLength(fileName);
        return response;
    }
}

It is a basic version, yet it seems to work fine. If you wanted to use it in more advanced scenarios however, there are a couple of potential problems to face.
First of all when the transfer is interrupted for whatever reason, the client has to start downloading from the beginning. This is unacceptable when serving large files and would be a major annoyance for people using mobile connections that drop often. Another problem is that the implementation above is not very client friendly in terms of http support (eg. HEAD verb).

Adding resume support

There are two main areas that we need to add more logic to in order to introduce pause/resume functionality:

• extend HTTP protocol support - most importantly by handling Range header properly,
• use a Stream that is capable of  returning file portion (from byte A to byte B).

HEAD http://localhost/Piotr.AspNetFileServer/api/files/data.zip HTTP/1.1  
User-Agent: Fiddler  
Host: localhost

HTTP/1.1 200 OK  
Content-Length: 1182367743  
Content-Type: application/octet-stream  
Accept-Ranges: bytes  
Server: Microsoft-IIS/8.0  
Content-Disposition: attachment; filename=data.zip

HEAD http://localhost/Piotr.AspNetFileServer/api/files/data.zip HTTP/1.1  
User-Agent: Fiddler  
Host: localhost  
Range: bytes=0-999

HTTP/1.1 206 Partial Content  
Content-Length: 1000  
Content-Type: application/octet-stream  
Content-Range: bytes 0-999/1182367743  
Accept-Ranges: bytes  
Server: Microsoft-IIS/8.0  
Content-Disposition: attachment; filename=data.zip

This is a helper class used to store some information passed in HTTP headers.

public class ContentInfo  
{
    public long From;
    public long To;
    public bool IsPartial;
    public long Length;
}

The controller itself can look like this:

public class FilesController : ApiController  
{
    public IFileProvider FileProvider { get; set; }

    public FilesController()
    {
        FileProvider = new FileProvider();
    }

    public HttpResponseMessage Head(string fileName)
    {
        if (!FileProvider.Exists(fileName))
        {
            //if file does not exist return 404
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        long fileLength = FileProvider.GetLength(fileName);
        ContentInfo contentInfo = GetContentInfoFromRequest(this.Request, fileLength);

        var response = new HttpResponseMessage();
        response.Content = new ByteArrayContent(new byte[0]);
        SetResponseHeaders(response, contentInfo, fileLength, fileName);
        return response;
    }

    public HttpResponseMessage Get(string fileName)
    {
        if (!FileProvider.Exists(fileName))
        {
            //if file does not exist return 404
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        long fileLength = FileProvider.GetLength(fileName);
        ContentInfo contentInfo 
           = GetContentInfoFromRequest(this.Request, fileLength);
        var stream = new PartialReadFileStream(FileProvider.Open(fileName), 
                                               contentInfo.From, contentInfo.To);
        var response = new HttpResponseMessage();
        response.Content = new StreamContent(stream);
        SetResponseHeaders(response, contentInfo, fileLength, fileName);
        return response;
    }

    private ContentInfo GetContentInfoFromRequest(HttpRequestMessage request, long entityLength)
    {
        var result = new ContentInfo 
                    {
                        From = 0, To = entityLength - 1, 
                        IsPartial = false, Length = entityLength
                    };
        RangeHeaderValue rangeHeader = request.Headers.Range;
        if (rangeHeader != null && rangeHeader.Ranges.Count != 0)
        {
            //we support only one range
            if (rangeHeader.Ranges.Count > 1)
            {
                //we probably return other status code here
                throw new HttpResponseException(HttpStatusCode.RequestedRangeNotSatisfiable);
            }
            RangeItemHeaderValue range = rangeHeader.Ranges.First();
            if (range.From.HasValue && range.From < 0 
                || range.To.HasValue && range.To > entityLength - 1)
            {
                throw new HttpResponseException(HttpStatusCode.RequestedRangeNotSatisfiable);
            }

            result.From = range.From ?? 0;
            result.To = range.To ?? entityLength - 1;
            result.IsPartial = true;
            result.Length = entityLength;
            if (range.From.HasValue && range.To.HasValue)
            {
                result.Length = range.To.Value - range.From.Value + 1;
            }
            else if (range.From.HasValue)
            {
                result.Length = entityLength - range.From.Value + 1;
            }
            else if (range.To.HasValue)
            {
                result.Length = range.To.Value + 1;
            }
        }

        return result;
    }

    private void SetResponseHeaders(HttpResponseMessage response, ContentInfo contentInfo,
                                    long fileLength, string fileName)
    {
        response.Headers.AcceptRanges.Add("bytes");
        response.StatusCode = contentInfo.IsPartial ? HttpStatusCode.PartialContent
                                  : HttpStatusCode.OK;
        response.Content.Headers.ContentDisposition 
          = new ContentDispositionHeaderValue("attachment");
        response.Content.Headers.ContentDisposition.FileName 
          = fileName;
        response.Content.Headers.ContentType 
          = new MediaTypeHeaderValue("application/octet-stream");
        response.Content.Headers.ContentLength = contentInfo.Length;
        if (contentInfo.IsPartial)
        {
            response.Content.Headers.ContentRange
                = new ContentRangeHeaderValue(contentInfo.From, contentInfo.To, fileLength);
        }
    }
}

Another important part of this solution is a stream implementation that can return byte range from a file. This is actually a wrapper for FileStream, please note that this code is largely untested, although gives an idea about approach - you have been warned ;)

internal class PartialReadFileStream : Stream  
{
    private readonly long _start;
    private readonly long _end;
    private long _position;
    private FileStream _fileStream;
    public PartialReadFileStream(FileStream fileStream, long start, long end)
    {
        _start = start;
        _position = start;
        _end = end;
        _fileStream = fileStream;

        if (start > 0)
        {
            _fileStream.Seek(start, SeekOrigin.Begin);
        }
    }

    public override void Flush()
    {
        _fileStream.Flush();
    }

    public override long Seek(long offset, SeekOrigin origin)
    {
        if (origin  SeekOrigin.Begin)
        {
            _position = _start + offset;
            return _fileStream.Seek(_start + offset, origin);
        }
        else if (origin  SeekOrigin.Current)
        {
            _position += offset;
            return _fileStream.Seek(_position + offset, origin);
        }
        else
        {
            throw new NotImplementedException("Seeking from SeekOrigin.End is not implemented");
        }
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        int byteCountToRead = count;
        if (_position + count > _end)
        {
            byteCountToRead = (int)(_end - _position) + 1;
        }
        var result = _fileStream.Read(buffer, offset, byteCountToRead);
        _position += byteCountToRead;
        return result;
    }

    public override IAsyncResult BeginRead(byte[] buffer, int offset, int count,
       AsyncCallback callback, object state)
    {
        int byteCountToRead = count;
        if (_position + count > _end)
        {
            byteCountToRead = (int)(_end - _position);
        }
        var result = _fileStream.BeginRead(buffer, offset,
                           count, (s) =>
                                      {
                                          _position += byteCountToRead;
                                          callback(s);
                                      }, state);
        return result;
    }

    public override int EndRead(IAsyncResult asyncResult)
    {
        return _fileStream.EndRead(asyncResult);
    }

    public override int ReadByte()
    {
        int result = _fileStream.ReadByte();
        _position++;
        return result;
    }

    // ...

    protected override void Dispose(bool disposing)
    {
        if (disposing)
        {
            _fileStream.Dispose();
        }
        base.Dispose(disposing);
    }
}

If  you think about this performance-wise, its not the most optimal approach as every time a file is being requested we need  to read it from the disk and disks are very slow (compared to RAM) and disk access may become bottleneck very fast. It becomes evident that for more advanced scenarios some kind of a caching mechanism would be a good optimization.

Using memory-mapped files

Memory mapped file is a portion of virtual memory that has been mapped to a file. This is not a new concept and has been around in Windows (and other OSes) for many years, but just recently (from NET 4 that is) has been made available to C# programmers as a managed API. Memory mapped files allow processes to modify and read files as if they were reading and writing to the memory. If my memory serves me well IPC in Windows is actually implemented using this feature.

Please note that the files are mapped and not copied into virtual memory, but from program's perspective its transparent as Windows loads parts of physical files as they are accessed by application. Another advantage of MMF is that the system performs transfers in 4K chunks of data (pages) and virtual-memory manager (VMM) decides when it should free those pages up. Windows is highly optimized for page-related IO operations, and it tries to minimize the number of times the hard disk head has to move. In other words by using MMF you have a guarantee that the OS will optimize disk access and additionally you get a form of memory cache.

Because files are mapped to virtual memory, to serve big files we need to run our application in 64 bit mode, otherwise it wouldn't be able to address all space needed.For this example, make sure to change target platform to x64 in project properties.

public class MemMappedFilesController : ApiController  
{
    private const string MapNamePrefix = "FileServerMap";

    public IFileProvider FileProvider { get; set; }

    public MemMappedFilesController()
    {
        FileProvider = new FileProvider();
    }

    private ContentInfo GetContentInfoFromRequest(HttpRequestMessage request, long entityLength)
    {
        //...
    }

    private void SetResponseHeaders(HttpResponseMessage response, ContentInfo contentInfo,
        long fileLength, string fileName)
    {
        //...
    }

    public HttpResponseMessage Head(string fileName)
    {
        //string fileName = GetFileName(name);
        if (!FileProvider.Exists(fileName))
        {
            //if file does not exist return 404
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        long fileLength = FileProvider.GetLength(fileName);
        ContentInfo contentInfo = GetContentInfoFromRequest(this.Request, fileLength);

        var response = new HttpResponseMessage();
        response.Content = new ByteArrayContent(new byte[0]);
        SetResponseHeaders(response, contentInfo, fileLength, fileName);
        return response;
    }

    public HttpResponseMessage Get(string fileName)
    {
        if (!FileProvider.Exists(fileName))
        {
            //if file does not exist return 404
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        long fileLength = FileProvider.GetLength(fileName);
        ContentInfo contentInfo = GetContentInfoFromRequest(this.Request, fileLength);
        string mapName = GenerateMapNameFromName(fileName);

        MemoryMappedFile mmf = null;
        try
        {
            mmf = MemoryMappedFile.OpenExisting(mapName, MemoryMappedFileRights.Read);
        }
        catch (FileNotFoundException)
        {
            //every time we use an exception to control flow a kitten dies

            mmf = MemoryMappedFile
                .CreateFromFile(FileProvider.Open(fileName), mapName, fileLength,
                                MemoryMappedFileAccess.Read, null, HandleInheritability.None,
                                false);
        }
        using (mmf)
        {
            Stream stream
                = contentInfo.IsPartial
                ? mmf.CreateViewStream(contentInfo.From, 
                contentInfo.Length, MemoryMappedFileAccess.Read)
                : mmf.CreateViewStream(0, fileLength, 
                MemoryMappedFileAccess.Read);

            var response = new HttpResponseMessage();
            response.Content = new StreamContent(stream);
            SetResponseHeaders(response, contentInfo, fileLength, fileName);
            return response;
        }
    }

    private string GenerateMapNameFromName(string fileName)
    {
        return String.Format("{0}_{1}", MapNamePrefix, fileName);
    }
}

I've removed code that is identical to FilesController. Please note that we have 1-1 relationship between a file (or its name to be more precise) and a map name. This means we use same map for all requests asking for the same filename.

Both controllers should provide pause/resume function.

Hope you find this post useful, complete source code is available as usually on bitbucket. Enjoy!

In particular I will look into following topics:

• using  Bing Maps SDK for Windows Store apps in Visual Studio 2012 projects,
• zooming and centering map on user's current location using geolocation service,
• adding pushpins,
• drawing polygons,
• adding other UIElements to the map.

Prerequisites

First of all we will need to get Bing Maps SDK for Windows Store apps which is located here. After installing the extension Bing Maps SDK will be available in Windows Store apps in Reference Manager.

Make sure to select Microsoft Visual C++ Runtime Package as it required by maps components. Because Bing Maps native implementation, we need to change Active solution configuration to either ARM, x86 or x64. Otherwise the project will not compile.

There is one last step required to start using the components - you will need a Bing Maps Key for Windows Store apps. There are three types of keys available - Trial, Basic and Enterprise, you can learn about differences between them here. I am using a Trial key, but when writing real world Windows 8 apps you are most likely to use Basic.

In order to generate the key go to https://www.bingmapsportal.com/, login with your Live ID (if you dont have it you will need to create one) and click on Create or view keys under My Account section.

Now we are good to go - let's create a new Windows Store app (I am using Blank App template).

<Page  
    (...)
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:Maps="using:Bing.Maps"
    mc:Ignorable="d">
    <Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}">
        <Maps:Map Credentials="INSERT_YOUR_KEY" x:Name="BingMap">
        </Maps:Map>
    </Grid>
</Page>

Instead of hardcoding the key in pages, it is usually better to have it defined in a resource dictionary (eg. in App.xaml).

Centering and zooming in on current location

Let's start with the basics and show how to center and zoom the map based on current location. Before running this example make sure that Location capability is enabled in Package.appxmanifest.

protected async override void OnNavigatedTo(NavigationEventArgs e)  
{
    Location location = await GetCurrentLocationAsync();
    CenterOnLocation(location);
}

private async Task<Location> GetCurrentLocationAsync()  
{
    var geolocator = new Geolocator();
    Geoposition currentGeoposition = await geolocator.GetGeopositionAsync();
    var location = new Location()
    {
        Latitude = currentGeoposition.Coordinate.Latitude,
        Longitude = currentGeoposition.Coordinate.Longitude,
    };
    return location;
}

private void CenterOnLocation(Location location)  
{
    BingMap.Center = location;
    BingMap.ZoomLevel = ZoomLevel;
}

Using geolocation features of Windows 8 is really straightforward and is further simplified by async/await pattern. After the page gets navigated to, map control should be automatically centered on current location.

Adding pushpins

Pushpins can be added to the map either in XAML or imperatively in code-behind.

<Maps:Map Credentials="YOUR_KEY_HERE"  
          x:Name="BingMap">
    <Maps:Map.Children>
        <Maps:Pushpin Text="1">
            <Maps:MapLayer.Position>
                <Maps:Location Latitude="50.0104955"
                                          Longitude="21.9888709"/>
            </Maps:MapLayer.Position>
        </Maps:Pushpin>
    </Maps:Map.Children>            
</Maps:Map>

Please note that in order to define pushpin position on the map we are setting MapLayer.Position attached property.

private void AddPushpin(Location location, string text)  
{
    var pushpin = new Pushpin()
                          {
                              Text = text,
                          };
    MapLayer.SetPosition(pushpin, location);
    BingMap.Children.Add(pushpin);
}

Default pushpin template should look like this:

In order to add some interactivity to the pushpin, we can handle Tapped event like this:

<Maps:Pushpin Text="1" Tapped="PushpinTapped">  
    <Maps:MapLayer.Position>
        <Maps:Location Latitude="50.0104955" 
                       Longitude="21.9888709"/>
    </Maps:MapLayer.Position>
</Maps:Pushpin>

private async void PushpinTapped(object sender, TappedRoutedEventArgs e)  
{            
    var dialog 
        = new MessageDialog("Congratulations. You tapped a pushpin.");
    await dialog.ShowAsync();
}

Drawing polygons

Bing Maps groups shapes (such as polygons and polylines) into layers represented by MapShapeLayer objects. Polygons are drawn using MapPolygon objects with vertices specified in Locations property. Following example creates a semi-transparent overlay for New Mexico state.

<Maps:Map Credentials="YOUR_KEY_HERE"  
          x:Name="BingMap">    
    <Maps:Map.ShapeLayers>
        <Maps:MapShapeLayer>
            <Maps:MapShapeLayer.Shapes>
                <Maps:MapPolygon FillColor="#5000ff00">
                    <Maps:MapPolygon.Locations>
                        <Maps:Location Latitude="36.9971" Longitude="-109.0448"/>
                        <Maps:Location Latitude="31.3337" Longitude="-109.0489"/>
                        <Maps:Location Latitude="31.3349" Longitude="-108.2140"/>
                        <Maps:Location Latitude="31.7795" Longitude="-108.2071"/>
                        <Maps:Location Latitude="31.7830" Longitude="-106.5317"/>
                        <Maps:Location Latitude="32.0034" Longitude="-106.6223"/>
                        <Maps:Location Latitude="31.9999" Longitude="-103.0696"/>
                        <Maps:Location Latitude="36.9982" Longitude="-103.0023"/>
                        <Maps:Location Latitude="36.9982" Longitude="-109.0475"/>
                    </Maps:MapPolygon.Locations>
                </Maps:MapPolygon>
            </Maps:MapShapeLayer.Shapes>
        </Maps:MapShapeLayer>
    </Maps:Map.ShapeLayers>
</Maps:Map>

And here is the result:

Adding other UIElements to the map

Map.Children property is actually a MapUIElementCollection object, which means we can add any UIElement to it. This opens a whole new set of possibilities, especially if you think of all the neat features XAML has to offer. To position elements we need to set MapLayer.Position attached property, just like in pushpin example (Pushpin is a UIElement as well). Here is an example that uses Image control.

<Maps:Map.Children>  
    <Maps:Pushpin Text="1" Tapped="PushpinTapped">
        <Maps:MapLayer.Position>
            <Maps:Location Latitude="50.0104955" 
                           Longitude="21.9888709"/>
        </Maps:MapLayer.Position>
    </Maps:Pushpin>
    <Image Source="Assets/Rain.png" Stretch="None">
        <Maps:MapLayer.Position>
            <Maps:Location Latitude="47.9097" 
                           Longitude="-122.6331"/>
        </Maps:MapLayer.Position>
    </Image>
</Maps:Map.Children>

This will add a rain icon over Seattle. No pun intended.

Hope you will find this post helpful!

• use of TypeScript to create AngularJS controllers,
• communication with Web API services from TypeScript using AngularJS ajax functionality,
• use of strongly typed TypeScript declarations for AngularJS objects.

I am using Visual Studio 2012/Sublime Text 2 as my IDE in this example. You are fine using any text editor, and if you want to have syntax highlighting / auto-completion features you will need to download packages separately.

For Visual Studio plugin and windows compiler binary (tsc.exe, part of installer package) visit http://www.typescriptlang.org/#Download. If you prefer to use a different editor (eg. Sublime Text or Vim) get the goodies from http://aka.ms/qwe1qu. The only thing that you really need is TypeScript compiler binary.

The services are implemented using ASP.NET Web API (.NET framework for building HTTP services, which i personally like very much), but you can use any other technology as long as it produces valid JSON.

Http services using ASP.NET Web API

For this example I've chosen to deal with a simple model consisting of one entity - Product.

public abstract class Entity  
{
    public Guid Id { get; set; }
}

public class Product : Entity  
{
    public string Name { get; set; }
    public decimal Price { get; set; }
}

We will need a persistence mechanism to store the entities. I am using in-memory storage (thread-safe collection) and a repository pattern. Feel free to change it to anything that suits you.

public interface IRepository<TEntity>  
    where TEntity : Entity
{
    TEntity Add(TEntity entity);
    TEntity Delete(Guid id);
    TEntity Get(Guid id);
    TEntity Update(TEntity entity);
    IQueryable<TEntity> Items { get; }
}

public class InMemoryRepository<TEntity> : IRepository<TEntity> where TEntity : Entity  
{
    private readonly ConcurrentDictionary<Guid, TEntity> _concurrentDictionary 
        = new ConcurrentDictionary<Guid, TEntity>();

    public TEntity Add(TEntity entity)
    {
        if (entity  null)
        {
            //we dont want to store nulls in our collection
            throw new ArgumentNullException("entity");
        }

        if (entity.Id  Guid.Empty)
        {
            //we assume no Guid collisions will occur
            entity.Id = Guid.NewGuid();
        }

        if (_concurrentDictionary.ContainsKey(entity.Id))
        {
            return null;
        }

        bool result = _concurrentDictionary.TryAdd(entity.Id, entity);

        if (result  false)
        {
            return null;
        }
        return entity;
    }

    public TEntity Delete(Guid id)
    {
        TEntity removed;
        if (!_concurrentDictionary.ContainsKey(id))
        {
            return null;
        }
        bool result = _concurrentDictionary.TryRemove(id, out removed);
        if (!result)
        {
            return null;
        }
        return removed;
    }

    public TEntity Get(Guid id)
    {
        if (!_concurrentDictionary.ContainsKey(id))
        {
            return null;
        }
        TEntity entity;
        bool result = _concurrentDictionary.TryGetValue(id, out entity);
        if (!result)
        {
            return null;
        }
        return entity;
    }

    public TEntity Update(TEntity entity)
    {
        if (entity  null)
        {
            throw new ArgumentNullException("entity");
        }
        if (!_concurrentDictionary.ContainsKey(entity.Id))
        {
            return null;
        }
        _concurrentDictionary[entity.Id] = entity;
        return entity;
    }

    public IQueryable<TEntity> Items
    {
        get { return _concurrentDictionary.Values.AsQueryable(); }
    }
}

Once we have a persistence mechanism in place, we can create a HTTP service that will expose a basic set of operations. Because I am using ASP.NET Web API this means I need to create a new controller.

public class ProductsController : ApiController  
{
    public static IRepository<Product> ProductRepository
        = new InMemoryRepository<Product>();

    public IEnumerable<Product> Get()
    {
        return ProductRepository.Items.ToArray();
    }

    public Product Get(Guid id)
    {
        Product entity = ProductRepository.Get(id);
        if (entity  null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        return entity;
    }

    public HttpResponseMessage Post(Product value)
    {
        var result = ProductRepository.Add(value);
        if (result  null)
        {
            // the entity with this key already exists
            throw new HttpResponseException(HttpStatusCode.Conflict);
        }
        var response = Request.CreateResponse<Product>(HttpStatusCode.Created, value);
        string uri = Url.Link("DefaultApi", new { id = value.Id });
        response.Headers.Location = new Uri(uri);
        return response;
    }

    public HttpResponseMessage Put(Guid id, Product value)
    {
        value.Id = id;
        var result = ProductRepository.Update(value);
        if (result  null)
        {
            // entity does not exist
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        return Request.CreateResponse(HttpStatusCode.NoContent);
    }

    public HttpResponseMessage Delete(Guid id)
    {
        var result = ProductRepository.Delete(id);
        if (result  null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        return Request.CreateResponse(HttpStatusCode.NoContent);
    }
}

We are trying to adhere to HTTP standard, hence additional logic to handle responses. After this step we should have a fully functional CRUD HTTP service.

Starting with AngularJS and TypeScript

With services ready for action, we can continue with creating the actual website. This will be plain HTML/CSS/JavaScript (TypeScript is compiled to JavaScript). The template I've started with looks like this:

<!DOCTYPE html>  
<html ng-app>
<head>
    <title>Product list</title>
    <link rel="stylesheet" href="Content/bootstrap.css"/>
    <script type="text/javascript" src="Scripts/bootstrap.js"></script>
    <script type="text/javascript" src="Scripts/angular.js"></script>
    <script type="text/javascript" src="Scripts/Controllers/ProductsController.js"></script>
</head>
    <body>
        <div ng-controller="Products.Controller">
        </div>
    </body>
</html>

Please note that ProductsController.js file will be generated from ProductsController.ts TypeScript source using tsc.exe compiler (I am using command line for compilation step). Let's create the file that will contain AngularJS controller for our page.

module Products {  
    export interface Scope {
        greetingText: string;
    }

    export class Controller {
        constructor ($scope: Scope) {
            $scope.greetingText = "Hello from TypeScript + AngularJS";
        }
    }
}

The Products module will be compiled into a JavaScript namespace and our controller will be available as Products.Controller. Now we can bind greetingText in our page. Please note that in order to leverage TypeScript's features we define contract for $scope object in the form of Scope interface. TypeScript would let us use any type instead as well, but personally I prefer more strict approach as it can help you catch errors at compile time (VS2012 will even underline errors in red as you edit your code, which is very nice). Now we have to compile ProductsController.ts, reference ProductsController.js in html and modify the view to display the message.

<div ng-controller="Products.Controller">  
    <p>{{greetingText}}</p>
</div>

With AngularJS controller stub in place, let's move on and add some additional contract declarations.

Creating model module

Let's create a module called Model that will contain Product class used as a DTO (serialized to JSON) with our HTTP services.

module Model {  
    export class Product {
        Id: string;
        Name: string;
        Price: number;
    }
}

This simple module contains a definition of one type that is exported and ready to use in page controller class.

Ambient declarations

In order to leverage ajax functionality provided by AngularJS we will use $http service passed in to controller constructor:

class Controller {  
    private httpService: any;

    constructor ($scope: Scope, $http: any) {
        this.httpService = $http;
        //...
    }
    //...
}

Because we declared httpService as any type, the compiler will not be able to help us catch potential errors in compile time. To address this we can use ambient declarations. Ambient declarations are used to tell the compiler about elements that will be introduced to the program by external means (in our case through AngularJS) and no JavaScript code will be emitted from them. In other words think of them as a contract for 3rd party libraries. Declaration source files (.d.ts extension) are restricted to contain ambient declarations only.  Here is angular.d.ts file that defines two interfaces used by us to call HTTP services.

declare module Angular {  
    export interface HttpPromise {
        success(callback: Function) : HttpPromise;
        error(callback: Function) : HttpPromise;
    }
    export interface Http {
        get(url: string): HttpPromise;
        post(url: string, data: any): HttpPromise;
        delete(url: string): HttpPromise;
    }
}

Declare keyword is optional, as it is implicitly inferred in all .d.ts files.

TypeScript and AngularJS

In order for compiler to know about newly introduced modules (Model and Angular) we need to add two reference statements to ProductsController.ts. Moreover we want to define Scope interface to include all properties and functions used by the view.

The page will consist of an input form for adding a new Product (name, price textboxes and a button) and will also display a list of all Products with the ability to delete individual entities. For brevity I am skipping, update scenario, but once we have other operations in place update implementation is really easy.

The Scope interface that provides this can look like this.

/// <reference path='angular.d.ts' />  
/// <reference path='model.ts' />

module Products {

    export interface Scope {
        newProductName: string;
        newProductPrice: number;
        products: Model.Product[];
        addNewProduct: Function;
        deleteProduct: Function;
    }
    // ...
}

Any time a product is added or deleted we want to refresh the list by getting all products from the server. Thanks to declarations introduced earlier we can use strongly typed  Angular.Http and Angular.HttpPromise interfaces.

The controller will contain private methods to communicate with our web service (getAllProducts, addProduct and deleteProduct).

export class Controller {  
    private httpService: any;

    constructor ($scope: Scope, $http: any) {
        this.httpService = $http;

        this.refreshProducts($scope);

        var controller = this;

        $scope.addNewProduct = function () {
            var newProduct = new Model.Product();
            newProduct.Name = $scope.newProductName;
            newProduct.Price = $scope.newProductPrice;

            controller.addProduct(newProduct, function () {
                controller.getAllProducts(function (data) {
                    $scope.products = data;
                });
            });
        };

        $scope.deleteProduct = function (productId) {
            controller.deleteProduct(productId, function () {
                controller.getAllProducts(function (data) {
                    $scope.products = data;
                });
            });
        }
    }

    getAllProducts(successCallback: Function): void{
        this.httpService.get('/api/products').success(function (data, status) {
            successCallback(data);
        });
    }

    addProduct(product: Model.Product, successCallback: Function): void {
        this.httpService.post('/api/products', product).success(function () {
            successCallback();
        });
    }

    deleteProduct(productId: string, successCallback: Function): void {
        this.httpService.delete('/api/products/'+productId).success(function () {
            successCallback();
        });
    }

    refreshProducts(scope: Scope) {
        this.getAllProducts(function (data) {
                    scope.products = data;
                });
    }

}

What's really nice is that we don't have to introduce any custom serialization logic. When retrieving products we treat data returned by $http service as a collection of strongly typed Products. Same applies to add operation - we simply pass a Product, it gets serialized and consumed by service in the end.

Creating the view

As a last step we need to create the view that will leverage new controller features. I am using bootstrap to make it a little bit easier.

<!DOCTYPE html>  
<html ng-app>
<head>
    <title>Product list</title>
    <link rel="stylesheet" href="Content/bootstrap.css" />
    <script type="text/javascript" src="Scripts/angular.js"></script>
    <script type="text/javascript" src="Scripts/Controllers/model.js"></script>
    <script type="text/javascript" src="Scripts/Controllers/productsController.js"></script>
</head>
<body>
    <div ng-controller="Products.Controller">
        <form class="form-horizontal" ng-submit="addNewProduct()">
            <input type="text" ng-model="newProductName" size="30"
                placeholder="product name">
            <input type="text" ng-model="newProductPrice" size="5"
                placeholder="product price">
            <button class="btn" type="submit" value="add">
                <i class="icon-plus"></i>
            </button>
        </form>
        <table class="table table-striped table-hover" style="width: 500px;">
            <thead>
                <tr>
                    <th>Name</th>
                    <th>Price</th>
                    <th></th>
                </tr>
            </thead>
            <tbody>
                <tr ng-repeat="product in products">
                    <td>{{product.Name}}</td>
                    <td>${{product.Price}}</td>
                    <td>
                        <button class="btn-small" ng-click="deleteProduct(product.Id)">
                            <i class="icon-trash"></i>
                        </button>
                    </td>
                </tr>
            </tbody>
        </table>
    </div>
</body>
</html>

The end result should look like this

Now our page should be functional and communication with HTTP service should work as expected.

As usually code is available to browse and download on bitbucket.

Edit: It seems that Wordpress Android app managed to somehow overwrite this post with an old, incomplete version. Sorry for that as well as for reposting.

HttpClient vs Background Transfer

C# developers writing Windows Store apps have two main options to download files from network locations over HTTP:

• HttpClient class (and its GetAsync method)
• Background Transfer functionality

Using HttpClient is only viable when dealing with relatively small files (MSDN suggests 'a couple of KB'). By default HttpClient response content buffer size is set to 64KB and if response is bigger than we would get an error. Buffer size can be increased if necessary (MaxResponseContentBufferSize property), but if you need to do that it is likely that you should use Background Transfer feature instead.

Background Transfer has several other advantages. First of all it runs outside of calling application which means that if the app is suspended, the download can still continue in the background. It also provides inherent support for pause, resume operations and can cope with sudden network status changes automatically. When app terminates, existing downloads will paused and persisted. Moreover it also plays nicely with power management (OS can disable downloads when it deems it necessary) and metered networks (via BackgroundDownloader.CostPolicy property) - after all user may not want to download 1GB+ file over roamed data connection. These two features are very important in mobile scenarios and make Background Transfer even more appealing.

 Starting a download

Let's create a simple C#/XAML app that will download a big file from a remote location and store it in the local app data store. Start off with a blank Windows Store app template and modify MainPage.xaml to look like this.

<Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}">  
    <Button Content="Download" Margin="0,150,0,0"
            HorizontalAlignment="Center"  VerticalAlignment="Center" 
            Height="58" Width="145" FontSize="17" Grid.Row="2" 
            Click="DownloadClick"/>
    <ProgressBar HorizontalAlignment="Center"
                 Height="30" Margin="0,-30,0,0" 
                 VerticalAlignment="Center" 
                 Width="400" x:Name="DownloadProgress"/>
</Grid>

The interface is very simple and consists of a button to initiate download as well as a progress bar to indicate progress.

Here is DownloadClick event handler implementation.

private async void DownloadClick(object sender, RoutedEventArgs e)  
{
    const string fileLocation
     = "http://download.thinkbroadband.com/100MB.zip";
    var uri = new Uri(fileLocation);
    var downloader = new BackgroundDownloader();
    StorageFile file = await ApplicationData.Current.LocalFolder.CreateFileAsync("100MB.zip",
        CreationCollisionOption.ReplaceExisting);
    DownloadOperation download = downloader.CreateDownload(uri, file);
    await StartDownloadAsync(download);
}

private void ProgressCallback(DownloadOperation obj)  
{
    double progress 
        = ((double) obj.Progress.BytesReceived / obj.Progress.TotalBytesToReceive);
    DownloadProgress.Value = progress * 100;
    if(progress >= 1.0)
    {
        _activeDownload = null;
        DownloadButton.IsEnabled = true;
    }
}

private async Task StartDownloadAsync(DownloadOperation downloadOperation)  
{
    DownloadButton.IsEnabled = false;
    _activeDownload = downloadOperation;
    var progress = new Progress<DownloadOperation>(ProgressCallback);
    await downloadOperation.StartAsync().AsTask(progress);
}

First of all we need to obtain StorageFile (or more precisely IStorageFile implementation) - in our scenario we use ApplicationData.Current.LocalFolder.CreateFileAsync() to create a file in local app data store. Please note that we use async/await pattern, hence the event handling method is marked as async. We also need to create BackgroundDownloader class instance that is used to actually create new download using CreateDownload() method.

Every download created using Background Tranfser feature is encapsulated in DownloadOperation object. These objects provide basic operations used to manipulate the download. In example above we start the download using StartAsync() that returns IAsyncOperationWithProgress. We make use of AsTask() extension method to cast the returned value to Task and provide progress callback used to update ProgressBar control.

Handling existing downloads

Previous example is a very simple scenario. Once we terminate the app, we lose track of our download, even though its being paused and persisted by Background Transfer automatically. To address this problem we can use BackgroundDownloader.GetCurrentDownloadsAsync() method to retrieve all active DownloadOperation objects for our applciation. Once we do this we can easily re-attach progress handler and any logic handling completed downloads.

async void MainPageLoaded(object sender, RoutedEventArgs e)  
{
    await LoadActiveDownloadsAsync();
}

private async Task LoadActiveDownloadsAsync()  
{
    IReadOnlyList<DownloadOperation> downloads = null;
    downloads = await BackgroundDownloader.GetCurrentDownloadsAsync();
    if(downloads.Count > 0)
    {
        //for simplicity we support only one download
        await ResumeDownloadAsync(downloads.First());
    }
}

private async Task ResumeDownloadAsync(DownloadOperation downloadOperation)  
{
    DownloadButton.IsEnabled = false;
    _activeDownload = downloadOperation;
    var progress = new Progress<DownloadOperation>(ProgressCallback);
    await downloadOperation.AttachAsync().AsTask(progress);
}

Once we have this code in place our download will reattach every time we load the page. For simplicity we support only one active download.

You can find the source on bickbucket .

If you don't own it, don't modify it

A good JavaScript rule says that you shouldn't modify objects you don't own. For example if you decide to override a method chances are that you are breaking libraries that depend on it and you are generating a lot of confusion among other developers.

window.originalAlert = window.alert;  
window.alert = function(msg) {  
    if (typeof msg === "string") {
        return console.log(msg);
    }
    return window.originalAlert(msg);
};

alert('ooh so awesome'); // console  
alert(3.14); //alert

Here we modify windows.alert to log all string values to console instead displaying a message box. For other types we invoke the original function. Regardless of our motivation, the result will be a lot of confusion among developers using alert function. Of course playing with DOM objects and methods like getElementById() can lead to far severe consequences (nasty, nasty bugs).

Modifying objects by adding new methods can also be harmful.

Math.cube = function(n) {  
    return Math.pow(n, 3);
};
console.log(Math.cube(2)); // 8

The biggest problem with this approach are naming collisions that may happen in the future. Even if Math object does not contain cube method now, next iteration of JavaScript standard may introduce it (even though this is unlikely). It will mean that we replace native (that is possible faster, better or simply behaving differently) implementation without even knowing this. This may painful and costly maintenance problems in your applications. A real world example of this problem is document.getElementsByClassName() introduced by Prototype library, just to be later included as a part of the standard.

Unfortunately there is no guarantee that other developers will leave your objects alone. If you provide something that in your opinion should be closed to modifications, you can use new JavaScript feature described below.

 Object.preventExtensions()

You can use Object.preventExtensions() to prevent new methods or properties being added to the object. Please note that existing members can be modified or even deleted. To determine whether an object is extensible or not use Object.isExtensible() method.

var song = {  
    title: 'Hope Leaves',
    artist: 'Opeth'
};

console.log(Object.isExtensible(song)); //true  
Object.preventExtensions(song);  
console.log(Object.isExtensible(song)); //false  
//(...)

song.play = function() {  
    console.log('ahh soo awesome');
}; //silently fails
song.album = 'Damnation'; //silently fails

console.log(song.title);  // Hope Leaves  
console.log(song.artist); //Opeth  
delete song.artist;       // we can delete and modify  
console.log(song.artist); // undefined  
console.log(song.album);  // undefined  
song.play(); //error no play() method defined

In this example if you try to add new members to a locked down object the operation will silently fail. This behavior will be changed if we use strict mode.

"use strict";

var song = {  
    title: 'Hope Leaves',
    artist: 'Opeth'
};
Object.preventExtensions(song);  
// (...)
song.album = 'Damnation'; //TypeError

In strict mode an error will be thrown when we try to add new member.

Object.seal()

Use Object.seal() to seal an object. Every sealed object is non-extensible (so it acts as if we acted with Object.preventExtension() on it ), but additionally none of its existing properties or methods can be removed.

var song = {  
    title: 'Hope Leaves',
    artist: 'Opeth'
};

Object.seal(song);  
console.log(Object.isExtensible(song)); //false  
console.log(Object.isSealed(song)); //true  
//(...)

song.album = 'Damnation'; //silently fails in non-strict mode  
delete song.artist;       //silently fails in non-strict mode  
console.log(song.artist); // 'Opeth'  
console.log(song.album);  // undefined

Again, in strict mode silent failures will be replaced by errors.

Object.freeze()

Frozen objects are considered to be sealed (and non-extensible as well). The additional constraint is that no modifications to existing properties or methods can occur.

var song = {  
    title: 'Hope Leaves',
    artist: 'Opeth',
    getLongTitle: function() {
        return this.artist + " - " + this.title;
    }
};

Object.freeze(song);  
console.log(Object.isExtensible(song)); //false  
console.log(Object.isSealed(song)); //true  
console.log(Object.isFrozen(song)); //true  
//(...)
song.album = 'Damnation'; //silently fails in non-strict mode  
delete song.artist; //silently fails in non-strict mode  
song.getLongTitle = function() {  
    return "foobar";
}; //silently fails in non-strict mode
console.log(song.getLongTitle()); // Opeth - Hope Leaves  
console.log(song.artist); // 'Opeth'  
console.log(song.album); // undefined

In strict mode instead of silent failures we would see errors.

This methods should be supported by any recent version of all major browsers:

•  IE 9+ (this means it should work in WinJS, but haven't tested it),
•  Firefox 4+
•  Safari 5.1+
•  Chrome 7+
•  Opera 12+

Let's start off by creating a simple model for our example.

public abstract class Entity  
{
    public Guid Id { get; set; }
}

public class Expense : Entity  
{
    public string Name { get; set; }
    public DateTime Date { get; set; }
    public decimal Amount { get; set; }
    public string Type { get; set; }
    public string Notes { get; set; }
    public string Account { get; set; }
}

We also want a way to persist our objects, instead of using a database I will just store them in memory (using thread safe collections introduced in .NET 4) and use a repository pattern to abstract actual persistence mechanism.

public interface IRepository<TEntity>  
        where TEntity : Entity
{
    TEntity Add(TEntity entity);
    TEntity Delete(Guid id);
    TEntity Get(Guid id);
    TEntity Update(TEntity entity);
    IQueryable<TEntity> Items { get; }
}

CRUD service in Web API

The next step is to create a simple Web API service that will provide basic CRUD operations.
HTTP/1.1 protocol defines a set of common methods that map to these operations in a following way:

• GET  /api/expenses - get a list of all expenses
• GET  /api/expenses/id - get expense by id
• POST  /api/expenses - create a new expense
• PUT  /api/expenses/id - update an expense
• DELETE  /api/expenses/id - delete an expense

If you have been implementing REST services before it may seem to be quite obvious.
The tricky part is (at least in my opinion) to make our service as HTTP compliant as possible by returning appropriate responses and status codes.

public class ExpensesController : ApiController  
{
    public static IRepository<Expense> ExpenseRepository
        = new InMemoryRepository<Expense>();

    public IEnumerable<Expense> Get()
    {
        var result = ExpenseRepository.Items.ToArray();
        return result;
    }

    public Expense Get(Guid id)
    {
        Expense entity = ExpenseRepository.Get(id);
        if(entity  null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        return entity;
    }

    public HttpResponseMessage Post(Expense value)
    {
        var result = ExpenseRepository.Add(value);
        if(result  null)
        {
            // the entity with this key already exists
            throw new HttpResponseException(HttpStatusCode.Conflict);
        }
        var response = Request.CreateResponse<Expense>(HttpStatusCode.Created, value);
        string uri = Url.Link("DefaultApi", new { id = value.Id });
        response.Headers.Location = new Uri(uri);  
        return response;
    }

    public HttpResponseMessage Put(Guid id, Expense value)
    {
        value.Id = id;
        var result = ExpenseRepository.Update(value);
        if(result  null)
        {
            // entity does not exist
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        return Request.CreateResponse(HttpStatusCode.NoContent);
    }

    public HttpResponseMessage Delete(Guid id)
    {
        var result = ExpenseRepository.Delete(id);
        if(result  null)
        {
            throw new HttpResponseException(HttpStatusCode.NotFound);
        }
        return Request.CreateResponse(HttpStatusCode.NoContent);
    }
}

You can find HTTP/1.1 spec document here (cough@frystyk is a co-author of this document cough). For example a section on POST method states that for resources that can be identified by a URI "the response SHOULD be 201 (Created) and contain an entity which describes the status of the request (...) and [contain] a Location header". Because expenses can be identified by a URL /api/expenses/id (it is a resource),  ideally we should follow this specification. Fortunately HTTP is a first class citizen in Web API and tasks like setting Location header in response are simple. In real world scenario we would also get rid of an unbounded Get() method and use paging or other result limiting mechanism.

Windows 8 app

For the sake of demonstration I am keeping the application very simple (and ugly unfortunately). It will consist of one screen and will be capable of:

• getting and displaying a list of all expenses,
• adding a new expense (with randomly generated properties),
• deleting selected expense,
• modifying selected expense.

All these operations will call our new shiny Web API HTTP service to retrieve and manipulate data.

Communicating with Web API

To consume HTTP service from Windows 8 app we can use an instance of HttpClient or HttpWebRequest class (the latter provides a little bit more features) . It's worth mentioning that out of the box, ASP.NET Web API supports both xml and json as media types for messages being exchanged with clients (you can use accept header to specify which one you want to use). Windows 8 also supports serialization/deserialization for both formats (through XmlSerializer and DataContractJsonSerializer classes respectively), without a need for external libraries. In this app I am going to use HttpClient and json as a message format.

But where should we put the logic to actually communicate with our server and serialize/deserialize entities?
The simplest solution would be to place it directly in code behind in event handlers associated with particular user actions. Even though suitable for such a simple demo app, this approach would backfire on us in more complex scenarios (imagine for example that we wanted to add authorization to our service in the future - we would need update code in many places). Because of this lets abstract actual data manipulation (and communication) logic.

public interface IExpenseService  
{
    Task<IEnumerable<Expense>> GetAll();
    Task Add(Expense expense);
    Task Delete(Guid id);
    Task Update(Expense expense);
}

This interface provides basic data manipulation operations, its implementation that uses Json (DataContractJsonSerializer)  and HTTP to communicate with Web API service will look like this:

public class ExpenseService : IExpenseService  
{
    private const string ServiceUrl = "http://localhost:12898/api/expenses";
    private readonly HttpClient _client = new HttpClient();

    public async Task<IEnumerable<Expense>> GetAll()
    {
        HttpResponseMessage response = await _client.GetAsync(ServiceUrl);
        var jsonSerializer = CreateDataContractJsonSerializer(typeof(Expense[]));
        var stream = await response.Content.ReadAsStreamAsync();
        return (Expense[])jsonSerializer.ReadObject(stream);
    }

    public async Task Add(Expense expense)
    {
        var jsonString = Serialize(expense);
        var content = new StringContent(jsonString, Encoding.UTF8, "application/json");
        var result = await _client.PostAsync(ServiceUrl, content);
    }

    public async Task Delete(Guid id)
    {
        var result = await _client.DeleteAsync(String.Format("{0}/{1}"
                , ServiceUrl, id.ToString()));
    }

    public async Task Update(Expense expense)
    {
        var jsonString = Serialize(expense);
        var content = new StringContent(jsonString, Encoding.UTF8, "application/json");
        var result = await _client.PutAsync(String
            .Format("{0}/{1}", ServiceUrl, expense.Id), content);
    }

    private static DataContractJsonSerializer CreateDataContractJsonSerializer(Type type)
    {
        const string dateFormat = "yyyy-MM-ddTHH:mm:ss.fffffffZ";
        var settings = new DataContractJsonSerializerSettings
                            {
                                DateTimeFormat = new DateTimeFormat(dateFormat)
                            };
        var serializer = new DataContractJsonSerializer(type, settings);
        return serializer;
    }

    private string Serialize(Expense expense)
    {
        var jsonSerializer = CreateDataContractJsonSerializer(typeof(Expense));
        byte[] streamArray = null;
        using (var memoryStream = new MemoryStream())
        {
            jsonSerializer.WriteObject(memoryStream, expense);
            streamArray = memoryStream.ToArray();
        }
        string json = Encoding.UTF8.GetString(streamArray, 0, streamArray.Length);
        return json;
    }
}

The main benefit of DataContractJsonSerializer  is that it is being provided by the platform (no concerns about using 3rd party library). On the other hand it requires a little bit of code to set up and its api is a somewhat clunky.

Here is IExpenseService implementation that uses Json.NET, which is a very popular Json serialization library for .NET. If you want to use it the easiest way is to get a NuGet package.

public class JsonNetExpenseService : IExpenseService  
{
    private const string ServiceUrl = "http://localhost:12898/api/expenses";
    private readonly HttpClient _client = new HttpClient();

    public async Task<IEnumerable<Expense>> GetAll()
    {
        HttpResponseMessage response = await _client.GetAsync(ServiceUrl);
        var jsonString = await response.Content.ReadAsStringAsync();
        return JsonConvert.DeserializeObject<Expense[]>(jsonString);
    }

    public async Task Add(Expense expense)
    {
        var jsonString = Serialize(expense);
        var content = new StringContent(jsonString, Encoding.UTF8, "application/json");
        var result = await _client.PostAsync(ServiceUrl, content);
    }

    public async Task Delete(Guid id)
    {
        var result = await _client.DeleteAsync(String.Format("{0}/{1}"
                                                            , ServiceUrl, id.ToString()));
    }

    public async Task Update(Expense expense)
    {
        var jsonString = Serialize(expense);
        var content = new StringContent(jsonString,
                                        Encoding.UTF8, "application/json");
        var result = await _client.PutAsync(String.Format("{0}/{1}",
                                                            ServiceUrl, expense.Id), content);
    }

    private string Serialize(Expense expense)
    {
        return JsonConvert.SerializeObject(expense);
    }
}

I like it better :).

Please note that we are leveraging async/await support provided by the platform. It will help us reduce code complexity later on.

XAML

IExpenseService  enables us to communicate with our Web API service, but we need to call this logic somewhere and we obviously need UI as well. As I said before, to keep things simple lets use one page only.

<Page>  
    <Page.BottomAppBar>
        <AppBar IsSticky="True" IsOpen="True">
            <Grid>
                <Grid.ColumnDefinitions>
                    <ColumnDefinition/>
                    <ColumnDefinition/>
                </Grid.ColumnDefinitions>
                <StackPanel Orientation="Horizontal">
                    <Button Style="{StaticResource RefreshAppBarButtonStyle}" 
                            AutomationProperties.Name="Refresh Data" Command="{Binding RemoveCommand}"/>
                    <Button Style="{StaticResource AddAppBarButtonStyle}" 
                            AutomationProperties.Name="Add Item" Command="{Binding AddCommand}"/>
                </StackPanel>
                <StackPanel Grid.Column="1" HorizontalAlignment="Right" Orientation="Horizontal">
                    <Button Style="{StaticResource EditAppBarButtonStyle}" 
                            AutomationProperties.Name="Update Selected Item" Command="{Binding UpdateCommand}"/>
                    <Button Style="{StaticResource DeleteAppBarButtonStyle}"
                            AutomationProperties.Name="Delete Selected Item" Command="{Binding DeleteCommand}"/>
                </StackPanel>
            </Grid>
        </AppBar>
    </Page.BottomAppBar>

    <Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}">
        <ListView SelectedItem="{Binding SelectedItem, Mode=TwoWay}" 
                  ItemsSource="{Binding Items}">
            <ListView.ItemTemplate>
                <DataTemplate>
                    ...                    
                </DataTemplate>
            </ListView.ItemTemplate>
        </ListView>
        <ProgressRing IsActive="{Binding IsBusy}" Width="70" Height="70"/>
    </Grid>    
</Page>

Please note that I am using MVVM pattern, but if you dont feel comfortable with it you can just stick IExpenseService calls in code-behind. Expense list will be displayed by ListView. We use bottom AppBar to provide user with buttons and there is a simple activity indicator (ProgressRing) which will be displayed while we are retrieving data from the server.

View model

We also need to create a view model that the page will bind to and that will encapsulate view related logic.

public class MainViewModel : INotifyPropertyChanged  
{
    // ...
    private const string ServiceUrl = "http://localhost:12898/api/expenses";

    public IExpenseService ExpenseService { get; set; }

    public Expense SelectedItem
    {
        get { return _selectedItem; }
        set
        {
            if (_selectedItem != value)
            {
                _selectedItem = value;
                DeleteCommand.RaiseCanExecuteChanged();
                UpdateCommand.RaiseCanExecuteChanged();
                OnPropertyChanged("SelectedItem");
            }
        }
    }

    public bool IsBusy
    {
        // ...
    }

    public IEnumerable<Expense> Items
    {
        // ...
    }

    #region commands

    public RelayCommand AddCommand { get; set; }
    public RelayCommand RefreshCommand { get; set; }
    public RelayCommand DeleteCommand { get; set; }
    public RelayCommand UpdateCommand { get; set; }

    #endregion commands

    public MainViewModel()
    {
        CreateCommands();
        ExpenseService = new ExpenseService();
    }

    private void CreateCommands()
    {
        AddCommand = new RelayCommand(o => AddHandler());
        RefreshCommand = new RelayCommand(o => RefreshHandler());
        DeleteCommand = new RelayCommand(o => DeleteHandler(),
                                         () => SelectedItem != null);
        UpdateCommand = new RelayCommand(o => UpdateHandler(),
                                         () => SelectedItem != null);
    }

    private void UpdateHandler()
    {
        if(SelectedItem != null)
        {
            var random = new Random();
            var amount = GenerateAmount(random);
            SelectedItem.Amount = amount;
            IsBusy = true;
            ExpenseService.Update(SelectedItem);
            IsBusy = false;
            RefreshData();
        }
    }

    private async void DeleteHandler()
    {
        if(SelectedItem != null)
        {
            IsBusy = true;
            await ExpenseService.Delete(SelectedItem.Id);
            IsBusy = false;
            RefreshData();
        }
    }

    private void RefreshHandler()
    {
        RefreshData();
    }

    private async void AddHandler()
    {
        Random random = new Random();
        int account = random.Next(1, 999);
        var amount = GenerateAmount(random);
        Expense newExpense = new Expense()
                                 {
                                     Account = account.ToString(),
                                     Date = DateTime.UtcNow,
                                     Amount = amount,
                                     Name = GenerateName(random),
                                     Notes = "Some notes",
                                     Type = GenerateType(random),
                                 };
        IsBusy = true;
        await ExpenseService.Add(newExpense);
        IsBusy = false;
        RefreshData();
    }

    // ...

    public void Load()
    {
        RefreshData();
    }

    private async void RefreshData()
    {
        IsBusy = true;
        Items = await ExpenseService.GetAll();
        IsBusy = false;
    }

    public event PropertyChangedEventHandler PropertyChanged;

    protected virtual void OnPropertyChanged(string propertyName)
    {
        PropertyChangedEventHandler handler = PropertyChanged;
        if (handler != null) handler(this, new PropertyChangedEventArgs(propertyName));
    }
}

I have removed less important code to make it smaller and easier to read. I am also generating random data instead of providing user editable forms.

There are two patterns worth mentioning here. First of all we are using commands (RelayCommand is an ICommand implementation and you can find it in source code) to react to user actions. For delete and update operations we provide canExecute predicate that determines whether a command can be executed or not. This is used to disable buttons when no item is selected.

The other pattern is the use of IsBusy property to indicate that view model is 'busy' - in our scenario that we are in progress of sending or retrieving data from server. Thanks to async/await support in .NET 4.5 and C# Windows 8 apps we don't have to worry about thread marshaling and the code looks as if it was synchronous.

You can download or view full source code (for both services and client app) on bitbucket (alternatively zip archive here).