UnknownHostException with Mesos and custom Jar

2015-09-28 Thread Stephen Hankinson
Hi, Wondering if anyone can help me with the issue I am having. I am receiving an UnknownHostException when running a custom jar with Spark on Mesos. The issue does not happen when running spark-shell. My spark-env.sh contains the following: export MESOS_NATIVE_JAVA_LIBRARY=/usr/local/lib/libme

DataFrame/Dataset join not producing correct results in Spark 2.0/Yarn

2016-10-12 Thread Stephen Hankinson
oop cluster also works. Only when running on top of YARN do we see this issue. This also seems very similar to this issue: https://issues.apache. org/jira/browse/SPARK-10896 Thoughts? *Stephen Hankinson*