GitHub user SongYadong opened a pull request:
https://github.com/apache/spark/pull/22706
[SPARK-25716][SQL][MINOR] remove unnecessary collection operation in valid
constraints generation
## What changes were proposed in this pull request?
Project logical operator generates valid constraints using two opposite
operations. It substracts child constraints from all constraints, than union
child constraints again. I think it may be not necessary.
Aggregate operator has the same problem with Project.
This PR try to remove these two opposite collection operations.
## How was this patch tested?
Related unit tests:
ProjectEstimationSuite
CollapseProjectSuite
PushProjectThroughUnionSuite
UnsafeProjectionBenchmark
GeneratedProjectionSuite
CodeGeneratorWithInterpretedFallbackSuite
TakeOrderedAndProjectSuite
GenerateUnsafeProjectionSuite
BucketedRandomProjectionLSHSuite
RemoveRedundantAliasAndProjectSuite
AggregateBenchmark
AggregateOptimizeSuite
AggregateEstimationSuite
DecimalAggregatesSuite
DateFrameAggregateSuite
ObjectHashAggregateSuite
TwoLevelAggregateHashMapSuite
ObjectHashAggregateExecBenchmark
SingleLevelAggregateHaspMapSuite
TypedImperativeAggregateSuite
RewriteDistinctAggregatesSuite
HashAggregationQuerySuite
HashAggregationQueryWithControlledFallbackSuite
TypedImperativeAggregateSuite
TwoLevelAggregateHashMapWithVectorizedMapSuite
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/SongYadong/spark generate_constraints
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22706.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22706
----
commit fab5faaa838295affdb9a1bfeae1d613eddfb7a1
Author: SongYadong <song.yadong1@...>
Date: 2018-10-11T14:12:05Z
remove unnecessary collection operation in valid constraints generation
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
