SequenceFileLogReader uses a reflection hack resulting in runtime failures

[Mikhail Bautin]

Hello,

The following reflection hack is from SequenceFileLogReader.java.

         try {
           Field fIn = FilterInputStream.class.getDeclaredField("in");
           fIn.setAccessible(true);
           Object realIn = fIn.get(this.in);
           Method getFileLength = realIn.getClass().
             getMethod("getFileLength", new Class<?> []{});
           getFileLength.setAccessible(true);
           long realLength = ((Long)getFileLength.
             invoke(realIn, new Object []{})).longValue();
           assert(realLength >= this.length);
           adjust = realLength - this.length;
         } catch(Exception e) {
           SequenceFileLogReader.LOG.warn(
             "Error while trying to get accurate file length.  " +
             "Truncation / data loss may occur if RegionServers die.", e);
         }

In my testing this resulted in the following error:

11/12/01 21:40:07 WARN wal.SequenceFileLogReader: Error while trying to get
accurate file length.  Truncation / data loss may occur if RegionServers
die.
java.lang.NoSuchMethodException:
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.getFileLength()
        at java.lang.Class.getMethod(Class.java:1605)
        at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader$WALReaderFSDataInputStream.getPos(SequenceFileLogReader.java:108)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1434)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)
        at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:57)
        at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:158)
        at
org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:681)
        at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:821)
        at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:734)
        at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFileToTemp(HLogSplitter.java:382)
        at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFileToTemp(HLogSplitter.java:351)
        at
org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:113)
        at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.grabTask(SplitLogWorker.java:266)
        at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.taskLoop(SplitLogWorker.java:197)
        at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.run(SplitLogWorker.java:165)
        at java.lang.Thread.run(Thread.java:680)

Besides, even when it works, the reflection-based solution is probably
_much_ slower than straightforward method access. Should we create a Hadoop
patch to expose the appropriate API call and get rid of the reflection hack?

Thanks,
--Mikhail