:
rddObject.mapPartitions(x = {
x.map(getDataFromDB(_))
}, true).saveAsTextFile(hdfs:///some-folder/)
Does that make sense?
On Wed, Oct 1, 2014 at 12:52 AM, Henry Hung ythu...@winbond.com wrote:
Hi All,
A noob question:
How to get SparckContext inside mapPartitions?
Example:
Let’s say I have
Hi All,
A noob question:
How to get SparckContext inside mapPartitions?
Example:
Let's say I have rddObjects that can be split into different partitions to be
assigned to multiple executors, to speed up the export data from database.
Variable sc is created in the main program using
