[
https://issues.apache.org/jira/browse/IMPALA-8841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902053#comment-16902053
]
Csaba Ringhofer commented on IMPALA-8841:
-----------------------------------------
Our theory is that concurrent queries that start Tez sessions try to copy the
.jar at the same time, leading some of them to see different mtime than
expected.
If the copy is done only during the very first sessions when the file doesn't
exist yet, then it should be enough to run a query (that needs Tez) before
starting parallel work, or to simply copy the .jar to its destination.
> Dataload flakiness: hive-exec-*.jar changed on src filesystem
> -------------------------------------------------------------
>
> Key: IMPALA-8841
> URL: https://issues.apache.org/jira/browse/IMPALA-8841
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.3.0
> Reporter: Csaba Ringhofer
> Priority: Major
>
> The following exception appears time to time in Hive 3 +Tez builds:
> {code}
> Failing this attempt.Diagnostics: [2019-07-25 11:26:46.931]Resource
> hdfs://localhost:20500/user/ubuntu/.hiveJars/hive-exec-3.1.0.7.0.0.0-280-f198e31861337336ae45a4c9779fc5db5af5d097a4b2000f8a96876573d03c8e.jar
> changed on src filesystem (expected 1564054001302, was 1564054001832
> java.io.IOException: Resource
> hdfs://localhost:20500/user/ubuntu/.hiveJars/hive-exec-3.1.0.7.0.0.0-280-f198e31861337336ae45a4c9779fc5db5af5d097a4b2000f8a96876573d03c8e.jar
> changed on src filesystem (expected 1564054001302, was 1564054001832
> at
> org.apache.hadoop.yarn.util.FSDownload.verifyAndCopy(FSDownload.java:273)
> at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:67)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:414)
> at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:411)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:411)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:242)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:235)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:223)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
