[ https://issues.apache.org/jira/browse/SPARK-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15055189#comment-15055189 ]
Maciej BryĆski edited comment on SPARK-11410 at 12/13/15 10:16 PM: ------------------------------------------------------------------- And what about such example. We can partition table by column. Then we run the query: {code} select * from table where column = value {code} In this case Spark should scan only one partition. PS. Is partitioning is saved in parquet format ? was (Author: maver1ck): And what about such example. We can partition table by column. Then we run the query: {code} select * from table where column = value {code} In this case Spark should scan only one partition. PS. Is partitioning is saved in parquet format ? > Add a DataFrame API that provides functionality similar to HiveQL's > DISTRIBUTE BY > --------------------------------------------------------------------------------- > > Key: SPARK-11410 > URL: https://issues.apache.org/jira/browse/SPARK-11410 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 1.5.1 > Reporter: Nong Li > Assignee: Nong Li > Fix For: 1.6.0 > > > DISTRIBUTE BY allows the user to control the partitioning and ordering of a > data set which can be very useful for some applications. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org