[
https://issues.apache.org/jira/browse/IMPALA-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-8509.
-----------------------------------
Fix Version/s: Impala 3.4.0
Resolution: Fixed
Most of the eval blocks were converted to use LOAD DATA statements here:
[https://github.com/apache/impala/commit/90ab610d3432a93248486ee38e10f58ec47908e6]
That improved the generate-schema-statements.py time significantly, so I'm
going to close this as resolved.
> Data load schema generation should lazily evaluate shell substitutions
> ----------------------------------------------------------------------
>
> Key: IMPALA-8509
> URL: https://issues.apache.org/jira/browse/IMPALA-8509
> Project: IMPALA
> Issue Type: Improvement
> Components: Infrastructure
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Major
> Fix For: Impala 3.4.0
>
>
> Some of the data loading commands (testescape_* tables in particular) use
> shell statements to execute. These are evaluated by the 'eval_section'
> function in generate-schema-statements.py. However, the 'eval_section' call
> was eagerly evaluated even if the table already existed, which makes
> 'generate-schema-statements' quite slow on every run.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
