Re: Failing jobs with Spark 2.2 running on Yarn with HDFS

2017-08-31 Thread Jan-Hendrik Zab
he current version. We still have _some_ dropped packets, but so far jobs haven't died because of it. As far as I can tell, it was also only Spark manifesting the problem. The usual Hadoop mapred jobs were running fine. Best, -jhz On Fri, 18 Aug 2017 19:59:26 +0200 Jan-Hendrik Zab &l

Failing jobs with Spark 2.2 running on Yarn with HDFS

2017-08-18 Thread Jan-Hendrik Zab
Hello! I've some weird problems with Spark running on top of Yarn. (Spark 2.2 on Cloudera CDH 5.12) There are a lot of "java.net.SocketException: Network is unreachable" in the executors, part of a log file: http://support.l3s.de/~zab/spark-errors.txt and the jobs also fail at rather random