Thank you for your inputs. Will test it out and share my findings
On Thursday, July 14, 2016, CosminC wrote:
> Didn't have the time to investigate much further, but the one thing that
> popped out is that partitioning was no longer working on 1.6.1. This would
> definitely explain the 2x perfo
Didn't have the time to investigate much further, but the one thing that
popped out is that partitioning was no longer working on 1.6.1. This would
definitely explain the 2x performance loss.
Checking 1.5.1 Spark logs for the same application showed that our
partitioner was working correctly, and
You can get some good pointers in this JIRA
https://issues.apache.org/jira/browse/SPARK-15796
Dibyendu
On Thu, Jul 14, 2016 at 12:53 AM, Sunita wrote:
> I am facing the same issue. Upgrading to Spark1.6 is causing hugh
> performance
> loss. Could you solve this issue? I am also attempting mem
I am facing the same issue. Upgrading to Spark1.6 is causing hugh performance
loss. Could you solve this issue? I am also attempting memory settings as
mentioned
http://spark.apache.org/docs/latest/configuration.html#memory-management
But its not making a lot of difference. Appreciate your inputs