[
https://issues.apache.org/jira/browse/IMPALA-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-2807.
-----------------------------------
Resolution: Won't Fix
IMPALA-8125 added an option to control this. Maybe we could be smarter about
picking parallelism to trade-off file size, but I think we need a clearer
problem to solve.
> During insert operation impala creates too many files for a table size <
> block size
> -----------------------------------------------------------------------------------
>
> Key: IMPALA-2807
> URL: https://issues.apache.org/jira/browse/IMPALA-2807
> Project: IMPALA
> Issue Type: Improvement
> Components: Perf Investigation
> Affects Versions: Impala 2.3.0
> Reporter: Dileep Kumar
> Priority: Major
>
> When loading the "customer" table from TPC-DS based schema, total no. of
> files created is 20 (which is equal to number of impala nodes in the cluster).
> The total size of the this table is 204.2 MiB which can fit in a single block
> while it occupies 20 blocks in this case.
> When ran the same insert command with a single impalad running in the cluster
> single block was able to hold all the table data and only one hdfs file was
> created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
