Home Dashboard Directory Help

IOException during reduce operation by davegmitchell


 as External Help for as External

Sign in
to vote
Type: Bug
ID: 786096
Opened: 5/2/2013 3:52:00 AM
Access Restriction: Public
User(s) can reproduce this bug


We are getting a new failure on HDInsight, whist running a pig script. The error seems to occur during the reduce operation whilst writing to BlobOutputStream.

The pig script used in the job has executed successfully on other occasions, but with different underlying data.

Sign in to post a comment.
Posted by Microsoft on 9/16/2013 at 7:23 AM
It is possible that this error is the end result of throttling by Azure Storage, though it's difficult to say with certainty based only on the error message (we're working on getting a better error message). To help us determine if this was a likely throttling situation, can you tell us if the job you were running was placing heavy load on Azure Storage (i.e. was it a heavy I/O job?) and/or whether there might have been other processes hitting the same storage account at the same time?
Sign in to post a workaround.
File Name Submitted By Submitted On File Size  
fullstacktrace.txt 5/2/2013 19 KB
stacktrace.png 5/2/2013 129 KB