Turns out that I was just being idiotic and had assigned so much memory to
Spark that the O/S was ending up continually swapping. Apologies for the
noise.
Phil
On Wed, Dec 24, 2014 at 1:16 AM, Andrew Ash wrote:
> Hi Phil,
>
> This sounds a lot like a deadlock in Hadoop's Configuration object t
Hi Phil,
This sounds a lot like a deadlock in Hadoop's Configuration object that I
ran into a while back. If you jstack the JVM and see a thread that looks
like the below, it could be https://issues.apache.org/jira/browse/SPARK-2546
"Executor task launch worker-6" daemon prio=10 tid=0x7f91f0
I've been attempting to run a job based on MLlib's ALS implementation for a
while now and have hit an issue I'm having a lot of difficulty getting to
the bottom of.
On a moderate size set of input data it works fine, but against larger
(still well short of what I'd think of as big) sets of data, I
