YanTang Zhai created SPARK-4693:
-----------------------------------
Summary: PruningPredicates may be wrong if predicates contains an
empty AttributeSet() references
Key: SPARK-4693
URL: https://issues.apache.org/jira/browse/SPARK-4693
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: YanTang Zhai
Priority: Minor
The sql "select * from spark_test::for_test where abs(20141202) is not null"
has predicates=List(IS NOT NULL
HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202)) and
partitionKeyIds=AttributeSet(). PruningPredicates is List(IS NOT NULL
HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202)). Then the
exception "java.lang.IllegalArgumentException: requirement failed: Partition
pruning predicates only supported for partitioned tables." is thrown.
The sql "select * from spark_test::for_test_partitioned_table where
abs(20141202) is not null and type_id=11 and platform = 3" with partitioned key
insert_date has predicates=List(IS NOT NULL
HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202), (type_id#12 =
11), (platform#8 = 3)) and partitionKeyIds=AttributeSet(insert_date#24).
PruningPredicates is List(IS NOT NULL
HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202)).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
