[ 
https://issues.apache.org/jira/browse/HIVE-30182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18141895#comment-18141895
 ] 

László Bodor commented on HIVE-30182:
-------------------------------------

had this earlier on a cluster, logs are full with it (checking if it's the 
same):

{code}
2026-09-24T13:53:23,939 ERROR [IO-Elevator-Thread-8 ()] 
read.ParquetFilterPredicateConverter: fail to build predicate filter leaf with 
errorsorg.apache.hadoop.hive.ql.metadata.HiveException: The conversion to 
Parquet FilterPredicate is not supported for DECIMAL. Please try to set the 
following configurations at the session level
 set hive.optimize.index.filter=false;
 set hive.optimize.ppd=false;
org.apache.hadoop.hive.ql.metadata.HiveException: The conversion to Parquet 
FilterPredicate is not supported for DECIMAL. Please try to set the following 
configurations at the session level
 set hive.optimize.index.filter=false;
 set hive.optimize.ppd=false;
    at 
org.apache.hadoop.hive.ql.io.parquet.LeafFilterFactory.getLeafFilterBuilderByType(LeafFilterFactory.java:221)
    at 
org.apache.hadoop.hive.ql.io.parquet.read.ParquetFilterPredicateConverter.buildFilterPredicateFromPredicateLeaf(ParquetFilterPredicateConverter.java:131)
    at 
org.apache.hadoop.hive.ql.io.parquet.read.ParquetFilterPredicateConverter.translate(ParquetFilterPredicateConverter.java:112)
    at 
org.apache.hadoop.hive.ql.io.parquet.read.ParquetFilterPredicateConverter.translate(ParquetFilterPredicateConverter.java:98)
    at 
org.apache.hadoop.hive.ql.io.parquet.read.ParquetFilterPredicateConverter.translate(ParquetFilterPredicateConverter.java:87)
    at 
org.apache.hadoop.hive.ql.io.parquet.read.ParquetFilterPredicateConverter.toFilterPredicate(ParquetFilterPredicateConverter.java:58)
    at 
org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.toFilterPredicate(ParquetRecordReaderBase.java:222)
    at 
org.apache.hadoop.hive.llap.io.encoded.ParquetEncodedDataReader.performDataRead(ParquetEncodedDataReader.java:236)
    at 
org.apache.hadoop.hive.llap.io.encoded.ParquetEncodedDataReader.lambda$callInternal$0(ParquetEncodedDataReader.java:196)
    at 
java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
    at java.base/javax.security.auth.Subject.doAs(Subject.java:525)
    at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1953)
    at 
org.apache.hadoop.hive.llap.io.encoded.ParquetEncodedDataReader.callInternal(ParquetEncodedDataReader.java:193)
    at 
org.apache.hadoop.hive.llap.io.encoded.ParquetEncodedDataReader.callInternal(ParquetEncodedDataReader.java:102)
    at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
    at 
org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer$CpuRecordingCallable.call(EncodedDataConsumer.java:88)
    at 
org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer$CpuRecordingCallable.call(EncodedDataConsumer.java:73)
    at 
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:112)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
    at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)

{code}

> Parquet: a leaf without its runtime value drops the whole row-group filter
> --------------------------------------------------------------------------
>
>                 Key: HIVE-30182
>                 URL: https://issues.apache.org/jira/browse/HIVE-30182
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Parquet, Query Processor
>            Reporter: Denys Kuzmenko
>            Assignee: László Bodor
>            Priority: Major
>              Labels: pull-request-available
>
> `ParquetFilterPredicateConverter.toFilterPredicate` returns null (no 
> row-group filter at all) when any leaf of the search argument cannot be 
> built. A leaf whose literal is a runtime value (DynamicValue, e.g. a 
> semijoin's min/max) that is not available when the reader builds its filter 
> is such a leaf: BETWEEN/IN get an empty literal list from 
> SearchArgumentImpl.getLiteralList and FilterPredicateLeafBuilder throws "Not 
> able to build 'between' operation filter with [] which needs two literals"; a 
> single-literal leaf throws NoDynamicValuesException. The static leaves of the 
> same conjunction are lost with it, and every failure logs an ERROR with a 
> stack trace: 45,927 of them in one TPC-DS 10 TB run, all from LLAP 
> IO-Elevator threads reading semijoin-filtered Parquet scans.
> Fix: such a leaf filters nothing. The search argument pushes NOT down to the 
> leaves, so dropping a leaf from a conjunction only weakens the filter; a 
> disjunction containing it still filters nothing.
> Not covered here: the runtime values are not reachable from the LLAP IO 
> threads at all, so a semijoin's min/max never prunes Parquet row groups on 
> that path (separate issue).



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to