This is an automated email from the ASF dual-hosted git repository.
pmaheshwari pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/samza.git
The following commit(s) were added to refs/heads/master by this push:
new 8bb6e5f SAMZA-2231: Exception thrown during yarn staging directory
cleanup masks original exception
8bb6e5f is described below
commit 8bb6e5fd2c7f756b790a4dca151ac506e208d52d
Author: thunderstumpges <[email protected]>
AuthorDate: Fri May 31 14:03:15 2019 -0700
SAMZA-2231: Exception thrown during yarn staging directory cleanup masks
original exception
Failures cleaning up the staging directory on another exception were
masking the original exception making troubleshooting difficult. Add some
logging and an extra try/catch around the cleanup.
Author: thunderstumpges <[email protected]>
Reviewers: Daniel Nishimura <[email protected]>
Closes #1059 from thunderstumpges/try-catch-on-staging-cleanup
---
.../src/main/scala/org/apache/samza/job/yarn/YarnJob.scala | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala
b/samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala
index 1d72a88..c374507 100644
--- a/samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala
+++ b/samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala
@@ -71,8 +71,16 @@ class YarnJob(config: Config, hadoopConfig: Configuration)
extends StreamJob {
)
} catch {
case e: Throwable =>
- client.cleanupStagingDir
- throw e
+ logger.error("Exception submitting yarn job.", e )
+ try {
+ // try to clean up. this may throw an exception depending on how far
into launching the job we got.
+ // we don't want to mask the original problem by throwing this.
+ client.cleanupStagingDir
+ } catch {
+ case ce: Throwable => logger.warn("Exception cleaning Staging
Directory after failed launch attempt.", ce)
+ } finally {
+ throw e
+ }
}
this
