bq. Is it the size of a particular row That was likely the cause.
Can you post the full stack trace ? Thanks On Sat, Jul 25, 2015 at 6:28 PM, F. Jerrell Schivers <[email protected] > wrote: > Hello, > > I'm getting the following error when I try to bulk load some data into > an HBase table at the end of a MapReduce job: > > org.apache.hadoop.mapred.YarnChild: Exception running child : > com.google.protobuf.InvalidProtocolBufferException: Protocol message > was too large. May be malicious. Use CodedInputStream.setSizeLimit() > to increase the size limit. > > This process was working fine until recently, so presumably as the > dataset has grown I've hit the default 64MB protobuf message size > limit. > > How can I increase this limit? I'm doing the bulk load > programatically, and I haven't found a way to call > CodedInputStream.setSizeLimit() as suggested. > > Only one reducer is failing, out of 500. Is there any way to figure > out which keys are in that reducer? When this happened once in the > past I was able to trace the problem to one particular key > corresponding to a very wide row. Since I knew that key wasn't > important I simply removed it from the dataset. However I'm having no > luck this time around. > > One last question. Can someone explain what exactly is exceeding this > size limit? Is it the size of a particular row, or something else? > > I'm running HBase 0.98.2. > > Thanks, > Jerrell >
