IOException during reduce operation - by davegmitchell

Status : 

  External<br /><br />
		This item may be valid but belongs to an external system out of the direct control of this product team.<br /><br />
		A more detailed explanation for the resolution of this particular item may have been provided in the comments section.

Sign in
to vote
ID 786096 Comments
Status Closed Workarounds
Type Bug Repros 0
Opened 5/2/2013 3:52:00 AM
Access Restriction Public


We are getting a new failure on HDInsight, whist running a pig script. The error seems to occur during the reduce operation whilst writing to BlobOutputStream.

The pig script used in the job has executed successfully on other occasions, but with different underlying data.

Sign in to post a comment.
Posted by Brian [MSFT] on 9/16/2013 at 7:23 AM
It is possible that this error is the end result of throttling by Azure Storage, though it's difficult to say with certainty based only on the error message (we're working on getting a better error message). To help us determine if this was a likely throttling situation, can you tell us if the job you were running was placing heavy load on Azure Storage (i.e. was it a heavy I/O job?) and/or whether there might have been other processes hitting the same storage account at the same time?