[
https://issues.apache.org/jira/browse/DRILL-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kunal Khatua updated DRILL-3381:
--------------------------------
Fix Version/s: 1.13.0
> Add option to distribute partition keys in CTAS
> -----------------------------------------------
>
> Key: DRILL-3381
> URL: https://issues.apache.org/jira/browse/DRILL-3381
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Steven Phillips
> Assignee: Steven Phillips
> Fix For: 1.13.0
>
> Attachments: DRILL-3381.patch, DRILL-3381.patch
>
>
> The current implementation does not redistribute, which would tend to result
> in a lot of extra files. Specifically, the number of files will be larger by
> a factor equal to the number of fragments in the final stage of the query. On
> even a moderately sized cluster, this number could easily be in the
> thousands, so a table with a 100 different partitions would end up with
> hundreds of thousands of files.
> To allow a workaround for this situation, we should add an option to include
> an extra distribution, so that all the rows for any given partition are
> written from the same writer.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
