partition id and then do
mapPartitions).
Thanks,
Assaf.
From: Lalwani, Jayesh [mailto:jayesh.lalw...@capitalone.com]
Sent: Thursday, August 03, 2017 5:20 PM
To: user@spark.apache.org
Subject: mapPartitioningWithIndex in Dataframe
Are there any plans to add mapPartitioningWithIndex in the
Are there any plans to add mapPartitioningWithIndex in the Dataframe API? Or is
there any way to implement my own mapPartitionWithIndex for a Dataframe?
I am implementing something which is logically similar to the randomSplit
function. In 2.1, randomSplit internally does df.mapPartitionWithInde
