Merge pull request #210 from haitaoyao/http-timeout add http timeout for httpbroadcast
While pulling task bytecode from HttpBroadcast server, there's no timeout value set. This may cause spark executor code hang and other task in the same executor process wait for the lock. I have encountered the issue in my cluster. Here's the stacktrace I captured : https://gist.github.com/haitaoyao/7655830 So add a time out value to ensure the task fail fast. Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/743a31a7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/743a31a7 Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/743a31a7 Branch: refs/heads/master Commit: 743a31a7ca4421cbbd5b615b773997a06a7ab4ee Parents: fb6875d db998a6 Author: Matei Zaharia <[email protected]> Authored: Wed Nov 27 18:24:39 2013 -0800 Committer: Matei Zaharia <[email protected]> Committed: Wed Nov 27 18:24:39 2013 -0800 ---------------------------------------------------------------------- .../scala/org/apache/spark/broadcast/HttpBroadcast.scala | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) ----------------------------------------------------------------------
