Re: How to optimiz and make this code faster using coalesce(1) and mapPartitionIndex

2016-01-13 Thread unk1102
data set size is small I dont understand why it just hangs there. I have seen same code runs fine when dataset is smaller than regular size over weekend. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-optimiz-and-make-this-code-faster-using-coalesce-1

How to optimiz and make this code faster using coalesce(1) and mapPartitionIndex

2016-01-12 Thread unk1102
t.toArray()); rowList.add(updatedRow); } return rowList.iterator(); } }, true); -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-optimiz-and-make-this-code-faster-using-coale