On Sun, May 25, 2014 at 2:02 AM, Amit Sela <[email protected]> wrote:
> I'm running HBase 0.94.12 (Hadoop 1.0.4). > I have a daily bulk load job that runs for a long time now, suddenly I got > this exception: > > java.io.IOException: Added a key not lexically larger than previous > > key=\x00U20140523]org.cinetux\x09/2013/11/ver-pelicula-charlie-countryman-online-gratis-2013.html\x03gen\x03WMQ\x00\x00\x00\x05\x02MX\x0210\x010\ > x11www.infolinks.com\x08tagcloud\x00\x00\x01F-\x83\xB0\x88\x04, > > lastkey=\x00R20140523_org.cinetux\x09/2013/11/ver-pelicula-caza-al-asesino-online-gratis-2013.html\x03gen\x03WMQ\x00\x00\x00\x05\x02VE\x03901\x010\x0Cfacebook.com\x08tagcloud\x00\x00\x01F-\x83\xB0\x88\x04 > at > > org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:207) > .......... > > I'm running with configureIncrementalLoad() and using > ImmutableBytesWritable as key and Put as value (leads to using > PutSortReducer). > > The weird thing is after I re-executed the same job, it worked just fine, > no exceptions... > > While I'm happy it works now, I really don't understand what could go wrong > there ? > It does indeed look as though keys in the file are not properly ordered. How are you generating the files you are bulk loading? Do you have the file still? IIRC, reading the file with the hfile tool will fail if it is not sorted properly: http://hbase.apache.org/book.html#hfile_tool2 St.Ack
