[
https://issues.apache.org/jira/browse/HIVE-26709?focusedWorklogId=823673&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-823673
]
ASF GitHub Bot logged work on HIVE-26709:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Nov/22 16:05
Start Date: 05/Nov/22 16:05
Worklog Time Spent: 10m
Work Description: ayushtkn opened a new pull request, #3733:
URL: https://github.com/apache/hive/pull/3733
### What changes were proposed in this pull request?
Add the RoaringBitmap to hive-exec jar required by iceberg during Runtime
for processing delete files.
### Why are the changes needed?
Queries with Iceberg Table fails
### How was this patch tested?
In Actual env. Doesn't reproduces in Unit Test, Maven puts all the jars on
the class path, but during execution these jars aren't there at Runtime with Tez
Issue Time Tracking
-------------------
Worklog Id: (was: 823673)
Remaining Estimate: 0h
Time Spent: 10m
> Iceberg: Count(*) fails for V2 tables with delete files.
> --------------------------------------------------------
>
> Key: HIVE-26709
> URL: https://issues.apache.org/jira/browse/HIVE-26709
> Project: Hive
> Issue Type: Bug
> Reporter: Ayush Saxena
> Assignee: Ayush Saxena
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Steps to Repro.
> * Create a v2 table
> * Add some Data
> * Delete a Row
> * Do a count(*) on the table
> *Reason:* Missing RoaringBitmap dependency, Iceberg now requires it during
> runtime for Delete files filtering
> StackTrace:
> {noformat}
> Caused by: java.lang.ClassNotFoundException:
> org.roaringbitmap.longlong.Roaring64Bitmap
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 42 more
> , errorMessage=Cannot recover from this error:java.lang.NoClassDefFoundError:
> org/roaringbitmap/longlong/Roaring64Bitmap
> at
> org.apache.iceberg.deletes.BitmapPositionDeleteIndex.<init>(BitmapPositionDeleteIndex.java:28)
> at org.apache.iceberg.deletes.Deletes.toPositionIndex(Deletes.java:102)
> at org.apache.iceberg.deletes.Deletes.toPositionIndex(Deletes.java:97)
> at
> org.apache.iceberg.data.DeleteFilter.applyPosDeletes(DeleteFilter.java:229)
> at org.apache.iceberg.data.DeleteFilter.filter(DeleteFilter.java:132)
> at
> org.apache.iceberg.mr.mapreduce.IcebergInputFormat$IcebergRecordReader.open(IcebergInputFormat.java:376)
> at
> org.apache.iceberg.mr.mapreduce.IcebergInputFormat$IcebergRecordReader.nextTask(IcebergInputFormat.java:266)
> at
> org.apache.iceberg.mr.mapreduce.IcebergInputFormat$IcebergRecordReader.initialize(IcebergInputFormat.java:262)
> at
> org.apache.iceberg.mr.mapred.AbstractMapredIcebergRecordReader.<init>(AbstractMapredIcebergRecordReader.java:40)
> at
> org.apache.iceberg.mr.mapred.MapredIcebergInputFormat$MapredIcebergRecordReader.<init>(MapredIcebergInputFormat.java:89)
> at
> org.apache.iceberg.mr.mapred.MapredIcebergInputFormat.getRecordReader(MapredIcebergInputFormat.java:79)
> at
> org.apache.iceberg.mr.hive.HiveIcebergInputFormat.getRecordReader(HiveIcebergInputFormat.java:169)
> at
> org.apache.hadoop.hive.ql.io.RecordReaderWrapper.create(RecordReaderWrapper.java:72)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:461)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.<init>(TezGroupedSplitsInputFormat.java:145)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getRecordReader(TezGroupedSplitsInputFormat.java:111)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.setupOldRecordReader(MRReaderMapred.java:164)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.setSplit(MRReaderMapred.java:83)
> at
> org.apache.tez.mapreduce.input.MRInput.initFromEventInternal(MRInput.java:706)
> at
> org.apache.tez.mapreduce.input.MRInput.initFromEvent(MRInput.java:665)
> at
> org.apache.tez.mapreduce.input.MRInputLegacy.checkAndAwaitRecordReaderInitialization(MRInputLegacy.java:150)
> at
> org.apache.tez.mapreduce.input.MRInputLegacy.init(MRInputLegacy.java:114)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getMRInput(MapRecordProcessor.java:543)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:189)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
