Anderson de Andrade created SPARK-14715:
-------------------------------------------
Summary: Provide a way to mask partitions of a Dataset/Dataframe
Key: SPARK-14715
URL: https://issues.apache.org/jira/browse/SPARK-14715
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.1.0
Reporter: Anderson de Andrade
If a Dataset/Dataframe were to have a custom partitioning by key(s), it would
be very efficient to just mask partitions when filtering by the same key(s).
This feature is already provide by PartitionPruningRDD on RDDs. We need
something similar on the Dataset/Dataframe space.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
