image    image    image    image

About Project


Problem:

Traditionally uploading files to Windows Azure blob storage involve one of the following approaches, assuming the keys of your account are not to be made available to the client uploading the file:

    1. Web role uploads file to temporary directory and then uses API to upload file to a blob in parallel.

    2. Web role uses client side script, generally JQuery to split a client file and use an intercepting WCF service to upload the file to blob sequentially.

    3. Web role uses client side script and shared access signature to upload file sequentially to blob storage.

    4. Web role uses Silverlight control to upload files using shared access signature on the container.

None of the solutions are truly up to mark in the issue of uploading files to blob storage from client side (to make it faster) and in parallel (to utilize parallel uploading capabilities of blob storage and make the process even faster).

Solution:

The solution to the problem is two staged:

    1. Use a client side application to manage file operations such as splitting the file in chunks and retrying in case of failure.

    2. Web role uses client side script, generally JQuery to split a client file and use an intercepting WCF service to upload the file to blob sequentially.

Building such a solution is simple with Silverlight. It runs on client and supports threading as well. Let’s proceed step by step to build such a solution (Process flow diagram at the end of document):

    1. Create a cross domain policy for access to blob storage through Silverlight application. This involves adding a policy file to the $root container of your storage account.

    2. Acquire shared access signature on container for a sufficient time in which the file may be uploaded.

    3. Pass this signature to the Silverlight application handling file uploads. I passed it through initialization parameter to Silverlight application.

    4. Inside the Silverlight application split the file into chunks of 1 MB each.

    5. Upload file using a single PUT request of you get a single packet and otherwise upload the file chunks as block blobs in parallel using multi-threading, I used Portable TPL that is an open source abstraction to threading in Silverlight 4. Silverlight 5 would have inbuilt TPL, but the process would remain essentially the same.

    6. In case any of the threads fail to upload its designated content, retry a finite number of times and fail the entire upload process if it keeps on failing.

    7. In case you have successfully uploaded the file as block blobs then issue a PUT request to commit the blocks.

    8. Exit the application.

Please refer the documentation for more details.

 

Silverlight Blob Parallel Upload Control


image    image  image

 

Features Coming Soon


    1. Support to upload multiple files.

    2. More Descriptive Upload Progress Tracker bar. This would display the each MB wise or kb wise uploading progress of the file.

    3. Implement concurrency control with TaskScheduler.

    4. Support for even larger files (>200 Mb).

 

References


    1. http://blog.smarx.com/posts/uploading-windows-azure-blobs-from-silverlight-part-1-shared-access-signatures

    2. http://portabletpl.codeplex.com/

    3. http://watwp.codeplex.com/

Last edited Sep 15, 2011 at 6:15 AM by RahulRai, version 19