Workflow Foundation and Compute Cluster Server

by marc 15. September 2006 15:21

It's been a couple of months since I had a look at this, but I said I'd make some comment on some concept work with WF and CCS so here it is...

You can see a video at the Microsoft site which was used as part of the launch for CCS (if you look really closely you can see me waving my hands about in it...)

So, aside from the specifics of enabling computational fluid dynamics solves on CCS, some of the interesting technical architecture was around the use of WF to access the job scheduler to complete an overall solve workflow.

As it turned out, WF was very well suited to the purpose. So what did we do? Quite a lot.

The things we were trying to achieve were:

  • Harnessing the job scheduler of CCS from a front-end application.
  • Creating workflows of multiple CCS jobs of multiple types.
  • Create an authoring and publishing experience for Workflow generation.
  • Track and persist long running workflows (BIG computations)

To do this, we did the following:

  • Wrapped the XML API for CCS with a WCF layer.
  • Created an NT Service that would be used to host the WF runtime and enabled this service with WCF.
  • Create a DB to hold XAML descriptions of workflow templates.
  • Created a Winforms application that rehosted the WF designer from Visual Studio (with limited access) allowing Workflow 'users' to load a workflow, change parameters and execute the workflow.
  • Used WCF and the XAML definitions to compile and execute the workflows on-the-fly at the NT Service instead of the client application
  • Used and extended the out-of-the-box WF tracking and persistence service to provide a complete trail of all previous and current workflow states.
  • Created a new 'source control provider' inside Visual Studio to be used with the WF design surface allowing WF XAML templates to be stored in the template database and provide a simple publishing experience.

So we have a straightforward tiered design, though handling a very specialised engineering process. The application (or framework really) allows a very simple experience across what is a complex workflow, and provides a number of strategies for introducing data visualisation, optimisation loops, and other business rules and features.

In this instance, we built our own host (the NT service) for WF in order to keep things relatively simple (our Proof of Concept build periods at the MTC are only 3 weeks), but we could have chosen Sharepoint Server which could be worthwhile in a production architecture.

The real strength was the flexibility of WF for hosting, tracking and persistence as aspects of its capability and then the great model for building hierarchies and libraries of custom activities for the construction of workflows to harness CCS.

The unsung hero was CCS itself, which worked well with pre-existing code, and we even recompiled this with the MS MPI stack. Treating CCS as a 'service gateway' - like a database - made a lot of sense as it could be called upon when needed rather than being embedded in the middle of the solution.

Add comment


(Will show your Gravatar icon)  

  Country flag

biuquote
  • Comment
  • Preview
Loading



Powered by BlogEngine.NET 1.4.5.0
Original Theme by Mads Kristensen and adjusted by me