Hi Balakumar
Two things.
One - It seems like your cluster is running out of memory and then
eventually out of disc , likely while materializing the dataframe to write
(what's the volume of data created by the join?)
Two - Your job is running in local mode, and is able to utilize just the
master
Hi ,
While running the following spark code in the cluster with following
configuration it is spread into 3 job Id's
CLUSTER CONFIGURATION
3 NODE CLUSTER
NODE 1 - 64GB 16CORES
NODE 2 - 64GB 16CORES
NODE 3 - 64GB 16CORES
At Job Id 2 job is stuck at the stage 51 of 254 and then it starts
ut