[ https://issues.apache.org/jira/browse/HIVE-30182 ]
László Bodor deleted comment on HIVE-30182:
-------------------------------------
was (Author: abstractdog):
had this earlier on a cluster, logs are full with it (checking if it's the
same):
{code}
2026-09-24T13:53:23,939 ERROR [IO-Elevator-Thread-8 ()]
read.ParquetFilterPredicateConverter: fail to build predicate filter leaf with
errorsorg.apache.hadoop.hive.ql.metadata.HiveException: The conversion to
Parquet FilterPredicate is not supported for DECIMAL. Please try to set the
following configurations at the session level
set hive.optimize.index.filter=false;
set hive.optimize.ppd=false;
org.apache.hadoop.hive.ql.metadata.HiveException: The conversion to Parquet
FilterPredicate is not supported for DECIMAL. Please try to set the following
configurations at the session level
set hive.optimize.index.filter=false;
set hive.optimize.ppd=false;
at
org.apache.hadoop.hive.ql.io.parquet.LeafFilterFactory.getLeafFilterBuilderByType(LeafFilterFactory.java:221)
at
org.apache.hadoop.hive.ql.io.parquet.read.ParquetFilterPredicateConverter.buildFilterPredicateFromPredicateLeaf(ParquetFilterPredicateConverter.java:131)
at
org.apache.hadoop.hive.ql.io.parquet.read.ParquetFilterPredicateConverter.translate(ParquetFilterPredicateConverter.java:112)
at
org.apache.hadoop.hive.ql.io.parquet.read.ParquetFilterPredicateConverter.translate(ParquetFilterPredicateConverter.java:98)
at
org.apache.hadoop.hive.ql.io.parquet.read.ParquetFilterPredicateConverter.translate(ParquetFilterPredicateConverter.java:87)
at
org.apache.hadoop.hive.ql.io.parquet.read.ParquetFilterPredicateConverter.toFilterPredicate(ParquetFilterPredicateConverter.java:58)
at
org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.toFilterPredicate(ParquetRecordReaderBase.java:222)
at
org.apache.hadoop.hive.llap.io.encoded.ParquetEncodedDataReader.performDataRead(ParquetEncodedDataReader.java:236)
at
org.apache.hadoop.hive.llap.io.encoded.ParquetEncodedDataReader.lambda$callInternal$0(ParquetEncodedDataReader.java:196)
at
java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
at java.base/javax.security.auth.Subject.doAs(Subject.java:525)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1953)
at
org.apache.hadoop.hive.llap.io.encoded.ParquetEncodedDataReader.callInternal(ParquetEncodedDataReader.java:193)
at
org.apache.hadoop.hive.llap.io.encoded.ParquetEncodedDataReader.callInternal(ParquetEncodedDataReader.java:102)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at
org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer$CpuRecordingCallable.call(EncodedDataConsumer.java:88)
at
org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer$CpuRecordingCallable.call(EncodedDataConsumer.java:73)
at
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:112)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
{code}
> Parquet: a leaf without its runtime value drops the whole row-group filter
> --------------------------------------------------------------------------
>
> Key: HIVE-30182
> URL: https://issues.apache.org/jira/browse/HIVE-30182
> Project: Hive
> Issue Type: Sub-task
> Components: Parquet, Query Processor
> Reporter: Denys Kuzmenko
> Assignee: László Bodor
> Priority: Major
> Labels: pull-request-available
>
> `ParquetFilterPredicateConverter.toFilterPredicate` returns null (no
> row-group filter at all) when any leaf of the search argument cannot be
> built. A leaf whose literal is a runtime value (DynamicValue, e.g. a
> semijoin's min/max) that is not available when the reader builds its filter
> is such a leaf: BETWEEN/IN get an empty literal list from
> SearchArgumentImpl.getLiteralList and FilterPredicateLeafBuilder throws "Not
> able to build 'between' operation filter with [] which needs two literals"; a
> single-literal leaf throws NoDynamicValuesException. The static leaves of the
> same conjunction are lost with it, and every failure logs an ERROR with a
> stack trace: 45,927 of them in one TPC-DS 10 TB run, all from LLAP
> IO-Elevator threads reading semijoin-filtered Parquet scans.
> Fix: such a leaf filters nothing. The search argument pushes NOT down to the
> leaves, so dropping a leaf from a conjunction only weakens the filter; a
> disjunction containing it still filters nothing.
> Not covered here: the runtime values are not reachable from the LLAP IO
> threads at all, so a semijoin's min/max never prunes Parquet row groups on
> that path (separate issue).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)