[
https://issues.apache.org/jira/browse/SPARK-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Armbrust resolved SPARK-4693.
-------------------------------------
Resolution: Fixed
Fix Version/s: 1.3.0
Issue resolved by pull request 3556
[https://github.com/apache/spark/pull/3556]
> PruningPredicates may be wrong if predicates contains an empty AttributeSet()
> references
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-4693
> URL: https://issues.apache.org/jira/browse/SPARK-4693
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: YanTang Zhai
> Priority: Minor
> Fix For: 1.3.0
>
>
> The sql "select * from spark_test::for_test where abs(20141202) is not null"
> has predicates=List(IS NOT NULL
> HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202)) and
> partitionKeyIds=AttributeSet(). PruningPredicates is List(IS NOT NULL
> HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202)). Then the
> exception "java.lang.IllegalArgumentException: requirement failed: Partition
> pruning predicates only supported for partitioned tables." is thrown.
> The sql "select * from spark_test::for_test_partitioned_table where
> abs(20141202) is not null and type_id=11 and platform = 3" with partitioned
> key insert_date has predicates=List(IS NOT NULL
> HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202), (type_id#12 =
> 11), (platform#8 = 3)) and partitionKeyIds=AttributeSet(insert_date#24).
> PruningPredicates is List(IS NOT NULL
> HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202)).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
