deniskuzZ commented on code in PR #5549:
URL: https://github.com/apache/hive/pull/5549#discussion_r2036854579
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/vector/HiveDeleteFilter.java:
##########
@@ -85,105 +86,98 @@ protected InputFile getInputFile(String location) {
*/
public CloseableIterable<HiveBatchContext>
filterBatch(CloseableIterable<HiveBatchContext> batches) {
+ CloseableIterator<HiveBatchContext> iterator = new
DeleteFilterBatchIterator(batches);
+
+ return new CloseableIterable<HiveBatchContext>() {
+
+ @Override
+ public CloseableIterator<HiveBatchContext> iterator() {
+ return iterator;
+ }
+
+ @Override
+ public void close() throws IOException {
+ iterator.close();
+ }
+ };
+ }
+
+ // VRB iterator with the delete filter
+ private class DeleteFilterBatchIterator implements
CloseableIterator<HiveBatchContext> {
+
// Delete filter pipeline setup logic:
// A HiveRow iterable (deleteInputIterable) is provided as input iterable
for the DeleteFilter.
// The content in deleteInputIterable is provided by row iterators from
the incoming VRBs i.e. on the arrival of
// a new batch the underlying iterator gets swapped.
- SwappableHiveRowIterable deleteInputIterable = new
SwappableHiveRowIterable();
+ private final SwappableHiveRowIterable deleteInputIterable;
// Output iterable of DeleteFilter, and its iterator
- CloseableIterable<HiveRow> deleteOutputIterable =
filter(deleteInputIterable);
- CloseableIterator<HiveRow> deleteOutputIterator =
deleteOutputIterable.iterator();
+ private final CloseableIterable<HiveRow> deleteOutputIterable;
- return new CloseableIterable<HiveBatchContext>() {
+ private final CloseableIterator<HiveBatchContext> srcIterator;
- @Override
- public CloseableIterator<HiveBatchContext> iterator() {
+ DeleteFilterBatchIterator(CloseableIterable<HiveBatchContext> batches) {
Review Comment:
hi @SourabhBadhya, long time no see :)
This class constructs an iterator with the delete filter on top of a row
batch iterator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
