[
https://issues.apache.org/jira/browse/IMPALA-9923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160193#comment-17160193
]
Fang-Yu Rao commented on IMPALA-9923:
-------------------------------------
Hi [~pvary], we saw the same error again in a recent Impala build.
The query that caused the error is as follows.
{code:java}
INSERT OVERWRITE TABLE tpcds_orc_def.store_sales PARTITION (ss_sold_date_sk)
SELECT ss_sold_time_sk,
ss_item_sk,
ss_customer_sk,
ss_cdemo_sk,
ss_hdemo_sk,
ss_addr_sk,
ss_store_sk,
ss_promo_sk,
ss_ticket_number,
ss_quantity,
ss_wholesale_cost,
ss_list_price,
ss_sales_price,
ss_ext_discount_amt,
ss_ext_sales_price,
ss_ext_wholesale_cost,
ss_ext_list_price,
ss_ext_tax,
ss_coupon_amt,
ss_net_paid,
ss_net_paid_inc_tax,
ss_net_profit,
ss_sold_date_sk
FROM tpcds.store_sales
WHERE ss_sold_date_sk IS NULL;
{code}
The version of Hive we were using is {{3.1.3000.7.2.1.0-287}}, and the file
system is HDFS.
The error we saw is very similar to the one reported in the section of the
description of this JIRA.
{code:java}
ERROR : FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: Fail to get
checksum, since file
/test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451141/base_0000002/_orc_acid_version
is under construction.
INFO : Completed executing
command(queryId=jenkins_20200716163033_4fd19a3b-47b2-4ceb-a7ec-f4781b5c68b2);
Time taken: 80.229 seconds
INFO : OK
Error: Error while compiling statement: FAILED: Execution Error, return code 1
from org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: Fail to get
checksum, since file
/test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451141/base_0000002/_orc_acid_version
is under construction. (state=08S01,code=1)
java.sql.SQLException: Error while compiling statement: FAILED: Execution
Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask.
java.io.IOException: Fail to get checksum, since file
/test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451141/base_0000002/_orc_acid_version
is under construction.
at
org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:401)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:266)
at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1007)
at org.apache.hive.beeline.Commands.execute(Commands.java:1217)
at org.apache.hive.beeline.Commands.sql(Commands.java:1146)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1497)
at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1355)
at org.apache.hive.beeline.BeeLine.executeFile(BeeLine.java:1329)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1127)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1082)
at
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:546)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:528)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Closing: 0: jdbc:hive2://localhost:11050/default;auth=none
{code}
For easy reference, the file containing the SQL statements and the file
containing the resulting logs
([^load-tpcds-core-hive-generated-orc-def-block.sql] and
[^load-tpcds-core-hive-generated-orc-def-block.sql.log]) are also attached.
> Data loading of TPC-DS ORC fails with "Fail to get checksum"
> ------------------------------------------------------------
>
> Key: IMPALA-9923
> URL: https://issues.apache.org/jira/browse/IMPALA-9923
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Reporter: Tim Armstrong
> Assignee: Zoltán Borók-Nagy
> Priority: Critical
> Labels: broken-build, flaky
> Attachments: load-tpcds-core-hive-generated-orc-def-block.sql,
> load-tpcds-core-hive-generated-orc-def-block.sql.log
>
>
> {noformat}
> INFO : Loading data to table tpcds_orc_def.store_sales partition
> (ss_sold_date_sk=null) from
> hdfs://localhost:20500/test-warehouse/managed/tpcds.store_sales_orc_def
> INFO :
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: Fail to get
> checksum, since file
> /test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451646/base_0000003/_orc_acid_version
> is under construction.
> INFO : Completed executing
> command(queryId=ubuntu_20200707055650_a1958916-1e85-4db5-b1bc-cc63d80b3537);
> Time taken: 14.512 seconds
> INFO : OK
> Error: Error while compiling statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: Fail to
> get checksum, since file
> /test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451646/base_0000003/_orc_acid_version
> is under construction. (state=08S01,code=1)
> java.sql.SQLException: Error while compiling statement: FAILED: Execution
> Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask.
> java.io.IOException: Fail to get checksum, since file
> /test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451646/base_0000003/_orc_acid_version
> is under construction.
> at
> org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:401)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:266)
> at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1007)
> at org.apache.hive.beeline.Commands.execute(Commands.java:1217)
> at org.apache.hive.beeline.Commands.sql(Commands.java:1146)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1497)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1355)
> at org.apache.hive.beeline.BeeLine.executeFile(BeeLine.java:1329)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1127)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1082)
> at
> org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:546)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:528)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> Closing: 0: jdbc:hive2://localhost:11050/default;auth=none
> {noformat}
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/11223/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
