spark git commit: SPARK-4300 [CORE] Race condition during SparkWorker shutdown

2015-03-13 Thread srowen 
Repository: spark
Updated Branches:
  refs/heads/branch-1.3 170af49bb - a3493eb77


SPARK-4300 [CORE] Race condition during SparkWorker shutdown

Close appender saving stdout/stderr before destroying process to avoid 
exception on reading closed input stream.
(This also removes a redundant `waitFor()` although it was harmless)

CC tdas since I think you wrote this method.

Author: Sean Owen so...@cloudera.com

Closes #4787 from srowen/SPARK-4300 and squashes the following commits:

e0cdabf [Sean Owen] Close appender saving stdout/stderr before destroying 
process to avoid exception on reading closed input stream


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a3493eb7
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a3493eb7
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a3493eb7

Branch: refs/heads/branch-1.3
Commit: a3493eb77a0aa7d3048e657459ebaa22e98ccf0c
Parents: 170af49
Author: Sean Owen so...@cloudera.com
Authored: Thu Feb 26 14:08:56 2015 -0800
Committer: Sean Owen so...@cloudera.com
Committed: Fri Mar 13 17:54:31 2015 +

--
 .../scala/org/apache/spark/deploy/worker/ExecutorRunner.scala | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/spark/blob/a3493eb7/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
--
diff --git 
a/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala 
b/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
index 2ec10f8..a3ec803 100644
--- a/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
@@ -86,14 +86,13 @@ private[spark] class ExecutorRunner(
 var exitCode: Option[Int] = None
 if (process != null) {
   logInfo(Killing process!)
-  process.destroy()
-  process.waitFor()
   if (stdoutAppender != null) {
 stdoutAppender.stop()
   }
   if (stderrAppender != null) {
 stderrAppender.stop()
   }
+  process.destroy()
   exitCode = Some(process.waitFor())
 }
 worker ! ExecutorStateChanged(appId, execId, state, message, exitCode)


-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: SPARK-4300 [CORE] Race condition during SparkWorker shutdown

2015-02-26 Thread andrewor14 
Repository: spark
Updated Branches:
  refs/heads/branch-1.2 2d83442f2 - 64e0cbc73


SPARK-4300 [CORE] Race condition during SparkWorker shutdown

Close appender saving stdout/stderr before destroying process to avoid 
exception on reading closed input stream.
(This also removes a redundant `waitFor()` although it was harmless)

CC tdas since I think you wrote this method.

Author: Sean Owen so...@cloudera.com

Closes #4787 from srowen/SPARK-4300 and squashes the following commits:

e0cdabf [Sean Owen] Close appender saving stdout/stderr before destroying 
process to avoid exception on reading closed input stream

(cherry picked from commit 3fb53c0298761ba227890525ae79ce4ec6300deb)
Signed-off-by: Andrew Or and...@databricks.com


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/64e0cbc7
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/64e0cbc7
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/64e0cbc7

Branch: refs/heads/branch-1.2
Commit: 64e0cbc731c38028afeac62d881573452a86058f
Parents: 2d83442
Author: Sean Owen so...@cloudera.com
Authored: Thu Feb 26 14:08:56 2015 -0800
Committer: Andrew Or and...@databricks.com
Committed: Thu Feb 26 14:09:03 2015 -0800

--
 .../scala/org/apache/spark/deploy/worker/ExecutorRunner.scala | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/spark/blob/64e0cbc7/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
--
diff --git 
a/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala 
b/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
index acbdf0d..1876408 100644
--- a/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
@@ -84,14 +84,13 @@ private[spark] class ExecutorRunner(
 var exitCode: Option[Int] = None
 if (process != null) {
   logInfo(Killing process!)
-  process.destroy()
-  process.waitFor()
   if (stdoutAppender != null) {
 stdoutAppender.stop()
   }
   if (stderrAppender != null) {
 stderrAppender.stop()
   }
+  process.destroy()
   exitCode = Some(process.waitFor())
 }
 worker ! ExecutorStateChanged(appId, execId, state, message, exitCode)


-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: SPARK-4300 [CORE] Race condition during SparkWorker shutdown

2015-02-26 Thread andrewor14 
Repository: spark
Updated Branches:
  refs/heads/master 5f3238b3b - 3fb53c029


SPARK-4300 [CORE] Race condition during SparkWorker shutdown

Close appender saving stdout/stderr before destroying process to avoid 
exception on reading closed input stream.
(This also removes a redundant `waitFor()` although it was harmless)

CC tdas since I think you wrote this method.

Author: Sean Owen so...@cloudera.com

Closes #4787 from srowen/SPARK-4300 and squashes the following commits:

e0cdabf [Sean Owen] Close appender saving stdout/stderr before destroying 
process to avoid exception on reading closed input stream


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/3fb53c02
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/3fb53c02
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/3fb53c02

Branch: refs/heads/master
Commit: 3fb53c0298761ba227890525ae79ce4ec6300deb
Parents: 5f3238b
Author: Sean Owen so...@cloudera.com
Authored: Thu Feb 26 14:08:56 2015 -0800
Committer: Andrew Or and...@databricks.com
Committed: Thu Feb 26 14:08:56 2015 -0800

--
 .../scala/org/apache/spark/deploy/worker/ExecutorRunner.scala | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/spark/blob/3fb53c02/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
--
diff --git 
a/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala 
b/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
index 6653aca..066d46c 100644
--- a/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
@@ -85,14 +85,13 @@ private[spark] class ExecutorRunner(
 var exitCode: Option[Int] = None
 if (process != null) {
   logInfo(Killing process!)
-  process.destroy()
-  process.waitFor()
   if (stdoutAppender != null) {
 stdoutAppender.stop()
   }
   if (stderrAppender != null) {
 stderrAppender.stop()
   }
+  process.destroy()
   exitCode = Some(process.waitFor())
 }
 worker ! ExecutorStateChanged(appId, execId, state, message, exitCode)


-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org