[
https://issues.apache.org/jira/browse/IMPALA-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17376780#comment-17376780
]
ASF subversion and git services commented on IMPALA-9822:
---------------------------------------------------------
Commit 59d32853ee42886ae683aac95a8be7f9c89b8eb7 in impala's branch
refs/heads/master from ShikhaAsrani
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=59d3285 ]
IMPALA-9822: Add warnings when row format delimiters are ignored on
non-text/sequence tables
When users create a non-text/sequence table the row format gets ignored.
To avoid surprising results in subsequent queries, users should be
warned about the row format being ignored. So users can make sure if
they want to change the file type instead.
Fix: This patch adds a warning message in analyzing Create Table
statement, when row format delimited fields are non empty and file
format is not TEXTFILE/SEQUENCEFILE.
Tests: Added a FE test to verify the warning message.
Change-Id: I6fff8f26556ff3854403a418891a41d2a27f01a9
Reviewed-on: http://gerrit.cloudera.org:8080/17589
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Impala does not notify user that row format delimited fields is only logical
> when using STORED AS TEXTFILE
> ----------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-9822
> URL: https://issues.apache.org/jira/browse/IMPALA-9822
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Alexandra Dunai
> Assignee: Shikha Asrani
> Priority: Minor
> Labels: newbie, ramp-up, usability
>
> When creating a table with added "ROW FORMAT DELIMITED FIELDS", Impala does
> not alert the user that this is only logical when using STORED AS TEXTFILE.
> You only discover that you made a mistake after trying to run a select from
> the table.
> Table creation:
> {code:bash}
> [adunai-1.adunai.root.hwx.site:21000] default> CREATE EXTERNAL TABLE
> sales_fact_1997(product_id INT,time_id INT,customer_id INT,promotion_id
> INT,store_id INT,store_sales DECIMAL(10,4),store_cost
> DECIMAL(10,4),unit_sales DECIMAL(10,4))
> > row format delimited fields terminated by '\011' STORED AS PARQUET
> > location '/user/impala/mondrian/sales_fact_1997';
> Query: CREATE EXTERNAL TABLE sales_fact_1997(product_id INT,time_id
> INT,customer_id INT,promotion_id INT,store_id INT,store_sales
> DECIMAL(10,4),store_cost DECIMAL(10,4),unit_sales DECIMAL(10,4))row format
> delimited fields terminated by '\011' STORED AS PARQUET location
> '/user/impala/mondrian/sales_fact_1997'
>
> +-------------------------+
> | summary |
> +-------------------------+
> | Table has been created. |
> +-------------------------+
> Fetched 1 row(s) in 0.10s
> {code}
>
> Select:
> {code:bash}
> [adunai-1.adunai.root.hwx.site:21000] mondrian> select count(*) from
> agg_c_10_sales_fact_1997;
> Query: select count(*) from agg_c_10_sales_fact_1997
> Query submitted at: 2020-06-03 11:55:06 (Coordinator:
> http://adunai-1.adunai.root.hwx.site:25000)
> Query progress can be monitored at:
> http://adunai-1.adunai.root.hwx.site:25000/query_plan?query_id=d547fafd0162da4e:872a95c100000000
> ERROR: File
> 'hdfs://adunai-2.adunai.root.hwx.site:8020/user/impala/mondrian/agg_c_10_sales_fact_1997/agg_c_10_sales_fact_1997.tsv'
> has an invalid Parquet version number: 717. Please check that it is a valid
> Parquet file. This error can also occur due to stale metadata. If you believe
> this is a valid Parquet file, try running "refresh
> mondrian.agg_c_10_sales_fact_1997".{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
