[
https://issues.apache.org/jira/browse/IMPALA-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8125.
-----------------------------------
Fix Version/s: Impala 4.0
Resolution: Fixed
> Limit number of files generated by insert
> -----------------------------------------
>
> Key: IMPALA-8125
> URL: https://issues.apache.org/jira/browse/IMPALA-8125
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Tim Armstrong
> Assignee: Bikramjeet Vig
> Priority: Major
> Labels: multithreading
> Fix For: Impala 4.0
>
>
> One pitfall of multithreaded execution is that, if implemented naively, the
> number of files generated by an unpartitioned insert will be multiplied by
> mt_dop.
> We should provide a mechanism to limit the number of files generated, e.g.
> limit the number of insert fragment instances (note that there a pre-existing
> problem with unpartitioned inserts generating too many files).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
