Dryad and DryadLINQ

This forum is being retired. It will removed in the near future.

If you are looking for the Beta version of LINQ to HPC, please go to the Microsoft HPC Pack 2008 R2 Beta program.

In December 2010, the Microsoft High-Performance Computing (HPC) team started a new beta program with the goal to make Dryad, DryadLINQ and related technologies available as supported Microsoft products. You can read the full announcement on the HPC Team blog.

Dryad and DryadLINQ

DryadLINQ is a Microsoft Research project, which aims to make distributed computing on clusters of computers simple enough for all programmers. DryadLINQ combines another Microsoft Research technology, Dryad, with the familiar LINQ technology from the Microsoft .NET framework.

Dryad is a high-performance, general-purpose distributed computing engine that handles some of the most difficult aspects of cluster-based distributed computing such as automatic scheduling of processes on the cluster machines, monitoring, fault-tolerance, and support for efficient data-transfer between processes. Dryad provides excellent performance and scalability, and can handle very large-scale data-parallel computations. Microsoft routinely uses Dryad to analyze petabytes of data on cluster of thousands of computers.

DryadLINQ extends the Language Integrated Query (LINQ) programming model to dramatically simplify the task of writing Dryad applications. With DryadLINQ, the code for a program that can run on hundreds of computers to process terabytes of data looks just like the code of a sequential .NET program using LINQ. Behind the scenes however, the DryadLINQ provider transparently converts a LINQ query into a distributed Dryad application and executes it on a cluster.

In 2009, the Microsoft Research Connections group teamed up with Dryad/DryadLINQ researchers and with the Windows HPC Server group to make Dryad & DryadLINQ available to the research and academic community. This version of the software has been replaced with LINQ to HPC. To try this latest version, consider joining the Beta program led by the HPC team.