[
https://issues.apache.org/jira/browse/IMPALA-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-4534.
-----------------------------------
Fix Version/s: Not Applicable
Resolution: Won't Fix
> Not all of the data load files follow the accepted format when partitioning
> test data tables
> --------------------------------------------------------------------------------------------
>
> Key: IMPALA-4534
> URL: https://issues.apache.org/jira/browse/IMPALA-4534
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
> Affects Versions: Impala 2.8.0
> Reporter: David Knupp
> Priority: Major
> Labels: test-infra
> Fix For: Not Applicable
>
>
> When loading Impala test data, we "generally" partition tables in our data
> load process by adding PARTITION_COLUMNS and ALTER sections to the schema
> template files, e.g. from functional_schema_template.sql:
> {noformat}
> ---- DATASET
> functional
> ---- BASE_TABLE_NAME
> alltypessmall
> ---- COLUMNS
> id int
> bool_col boolean
> tinyint_col tinyint
> smallint_col smallint
> int_col int
> bigint_col bigint
> float_col float
> double_col double
> date_string_col string
> string_col string
> timestamp_col timestamp
> ---- PARTITION_COLUMNS
> year int
> month int
> ---- ALTER
> ALTER TABLE {table_name} ADD IF NOT EXISTS PARTITION(year=2009, month=1);
> ALTER TABLE {table_name} ADD IF NOT EXISTS PARTITION(year=2009, month=2);
> ALTER TABLE {table_name} ADD IF NOT EXISTS PARTITION(year=2009, month=3);
> ALTER TABLE {table_name} ADD IF NOT EXISTS PARTITION(year=2009, month=4);
> {noformat}
> However, some tables forego this, and combine the PARTITION BY clause with
> the CREATE TABLE clause, and they may or may not include an ALTER section.
> This sidesteps logic in generate-schema-statements.py that specifically
> branches based upon whether the PARTITION_COLUMNS and/or ALTER sections have
> been defined.
> We should investigate what effect the omission of these sections has on our
> data load process for those tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
