Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21567#discussion_r195657414
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -354,7 +354,7 @@ private[spark] abstract class BasePythonRunner[IN, OUT](
extends Thread(s"Worker Monitor for $pythonExec") {
/** How long to wait before killing the python worker if a task cannot
be interrupted. */
- private val taskKillTimeout =
env.conf.getTimeAsMs("spark.python.task.killTimeout", "2s")
+ private val taskKillTimeout =
env.conf.getTimeAsSeconds("spark.python.task.killTimeout", "2s") * 1000L
--- End diff --
Nit: Probably, we'd be better to add `Ms` in the suffix, =>
`taskKillTimeoutMs`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
