Issue in running .Net Map Reduce job on Azure HDInsight Service - by Bhawna Paliwal

Status : 

  Not Reproducible<br /><br />
		The product team could not reproduce this item with the description and steps provided.<br /><br />
		A more detailed explanation for the resolution of this particular item may have been provided in the comments section.


4
0
Sign in
to vote
ID 784893 Comments
Status Closed Workarounds
Type Bug Repros 2
Opened 4/17/2013 4:30:48 AM
Access Restriction Public

Description

Using following links i tried running Map Reduce job using .Net console application:

http://cloud.dzone.com/articles/hdinsight-writing-hadoop-map

It runs fine on local. To run it on Azure i took refernce from following link where i provided required credentials:

http://hadoopsdk.codeplex.com/wikipage?title=Running%20jobs%20on%20Azure%20HDInsight%20service&referringTitle=Map%2fReduce

But It is raising error on Azure.     
Initially it had also thrown an error that remote machine is not responding.

[Original Error Message=A connection attempt failed because the
connected party did not properly respond after a period of time, or
established connection failed because connected host has failed to
respond 168.62.171.51:50070 ]

Now it is returning NULL object only.

I used following approach to connect cluster:

Uri azureCluster = new Uri("https://azureCluster.azurehdinsight.net:563"); string clusterUserName = "{Your cluster user name}"; string clusterPassword = "{Your cluster password}"; // This is the name of the account under which Hadoop will execute jobs. // Normally this is just "Hadoop". string hadoopUserName = "{HadoopAccount}"; // Azure Storage Information. string azureStorageAccount = "{YourStorageAccountName.blob.core.windows.net}"; string azureStorageKey = "{YourAzureStorageKey}"; string azureStorageContainer = "{YourAzureStorageContainer}"; bool createContinerIfNotExist = true; var hadoop = Hadoop.Connect(azureCluster, clusterUserName, hadoopUserName, clusterPassword, azureStorageAccount, azureStorageKey, azureStorageContainer, createContinerIfNotExist); var result = hadoop.MapReduceJob.ExecuteJob<JobType>();

This command is returning NULL object:

hadoop.MapReduceJob.ExecuteJob<NamespaceCounterJob>()

here "NamespaceCounterJob" is Job type

public class NamespaceCounterJob : HadoopJob<NamespaceMapper, NamespaceReducer>

{

..

}

Kindly share if any one has idea to resolve the error or any suggestion to run Map Reduce job on Azure using .Net
Sign in to post a comment.
Posted by Mark J. Miller on 7/17/2013 at 2:57 PM
Source code to duplicate the issue: https://gist.github.com/developmentalmadness/6024923