Russ is a co-worker.
There are at least 3 variants, depending, if that helps:
Unable to find tablets that overlap file
hdfs://mas1.soaktest.phemi.com:8020/apps/accumulo/data/tables/3/b-00019av/I0001b1u.rf
java.io.IOException: Corrupted Meta region Index Unable to find tablets
that overlap file
hdfs://mas1.soaktest.phemi.com:8020/apps/accumulo/data/tables/3/b-00016t1/I00017xv.rf
java.lang.RuntimeException: Incompatible BCFile fileBCFileVersion.
Unable to find tablets that overlap file
hdfs://mas1.soaktest.phemi.com:8020/apps/accumulo/data/tables/3/b-00016t1/I000173f.rf
java.lang.IndexOutOfBoundsException: Invalid offset/length: 6230/-25
On 16-07-08 11:49 AM, Russ Weeks wrote:
Hi, folks,
Has anybody ever encountered a problem where the RFiles that are
generated by AccumuloFileOutputFormat can't be imported using
TableOperations.importDirectory?
I'm seeing this problem very frequently for small RFiles and
occasionally for larger RFiles. The errors shown in the monitor's log
UI suggest a corrupt file, to me. For instance, the stack trace below
shows a case where the BCFileVersion was incorrect, but sometimes it
will complain about an invalid length, negative offset, or invalid codec.
I'm using HDP Accumulo 1.7.0 (1.7.0.2.3.4.12-1) on an encrypted HDFS
volume, with Kerberos turned on. The RFiles are generated by
AccumuloFileOutputFormat from a Spark job.
A very small RFile that exhibits this problem is available here:
http://firebar.newbrightidea.com/downloads/bad_rfiles/I0000waz.rf
I'm pretty confident that the keys are being written to the RFile in
order. Are there any tools I could use to inspect the internal
structure of the RFile?
Thanks,
-Russ
Unable to find tablets that overlap file
hdfs://[redacted]/accumulo/data/tables/f/b-0000ze9/I0000zeb.rf
java.lang.RuntimeException: Incompatible BCFile fileBCFileVersion.
at
org.apache.accumulo.core.file.rfile.bcfile.BCFile$Reader.<init>(BCFile.java:828)
at
org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile$Reader.init(CachableBlockFile.java:246)
at
org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile$Reader.getBCFile(CachableBlockFile.java:257)
at
org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile$Reader.access$100(CachableBlockFile.java:137)
at
org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile$Reader$MetaBlockLoader.get(CachableBlockFile.java:209)
at
org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile$Reader.getBlock(CachableBlockFile.java:313)
at
org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile$Reader.getMetaBlock(CachableBlockFile.java:368)
at
org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile$Reader.getMetaBlock(CachableBlockFile.java:137)
at org.apache.accumulo.core.file.rfile.RFile$Reader.<init>(RFile.java:843)
at
org.apache.accumulo.core.file.rfile.RFileOperations.openReader(RFileOperations.java:79)
at
org.apache.accumulo.core.file.DispatchingFileFactory.openReader(DispatchingFileFactory.java:69)
at
org.apache.accumulo.server.client.BulkImporter.findOverlappingTablets(BulkImporter.java:644)
at
org.apache.accumulo.server.client.BulkImporter.findOverlappingTablets(BulkImporter.java:615)
at
org.apache.accumulo.server.client.BulkImporter$1.run(BulkImporter.java:146)
at
org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
at org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
at java.lang.Thread.run(Thread.java:745)
--
Josef Roehrl
Senior Software Developer
*PHEMI Systems*
180-887 Great Northern Way
Vancouver, BC V5T 4T5
604-336-1119
Website <http://www.phemi.com/> Twitter
<https://twitter.com/PHEMISystems> Linkedin
<http://www.linkedin.com/company/3561810?trk=tyah&trkInfo=tarId%3A1403279580554%2Ctas%3Aphemi%20hea%2Cidx%3A1-1-1>
