[
https://issues.apache.org/jira/browse/IMPALA-9923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155128#comment-17155128
]
Peter Vary commented on IMPALA-9923:
------------------------------------
What is the exact query causing problems?
Which Hive version are you using in these tests? (There are recent changes in
loadDynamicPartitions to run the async)
S3 or HDSF is the environment for these tests?
Taking a look at the exception it seems to me that after moving the files we
would like to collect the new files for creating a Notification Event. When we
do this we try to get data about the _orc_acid_version file. The checksum
creation for this file is not yet finished somehow.
Is the checksum creation is a synchronous task on the FS Impala is using for
testing?
Just slightly related: HIVE-23825. This jira is about avoiding creating this
specific file which causing this specific issue. I am not convinced that after
this there will not be the same issue with actual data files, but might worth a
try when HIVE-23825 is committed.
Thanks,
Peter
> Data loading of TPC-DS ORC fails with "Fail to get checksum"
> ------------------------------------------------------------
>
> Key: IMPALA-9923
> URL: https://issues.apache.org/jira/browse/IMPALA-9923
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Reporter: Tim Armstrong
> Priority: Critical
> Labels: broken-build, flaky
>
> {noformat}
> INFO : Loading data to table tpcds_orc_def.store_sales partition
> (ss_sold_date_sk=null) from
> hdfs://localhost:20500/test-warehouse/managed/tpcds.store_sales_orc_def
> INFO :
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: Fail to get
> checksum, since file
> /test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451646/base_0000003/_orc_acid_version
> is under construction.
> INFO : Completed executing
> command(queryId=ubuntu_20200707055650_a1958916-1e85-4db5-b1bc-cc63d80b3537);
> Time taken: 14.512 seconds
> INFO : OK
> Error: Error while compiling statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: Fail to
> get checksum, since file
> /test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451646/base_0000003/_orc_acid_version
> is under construction. (state=08S01,code=1)
> java.sql.SQLException: Error while compiling statement: FAILED: Execution
> Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask.
> java.io.IOException: Fail to get checksum, since file
> /test-warehouse/managed/tpcds.store_sales_orc_def/ss_sold_date_sk=2451646/base_0000003/_orc_acid_version
> is under construction.
> at
> org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:401)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:266)
> at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1007)
> at org.apache.hive.beeline.Commands.execute(Commands.java:1217)
> at org.apache.hive.beeline.Commands.sql(Commands.java:1146)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1497)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1355)
> at org.apache.hive.beeline.BeeLine.executeFile(BeeLine.java:1329)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1127)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1082)
> at
> org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:546)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:528)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> Closing: 0: jdbc:hive2://localhost:11050/default;auth=none
> {noformat}
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/11223/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
