Hi, I am working on some use case where I want to perform some action on each active executor of application once. How to run some function on each active executor associated with current spark application?
num_executors = len(self._jsc.sc().statusTracker().getExecutorInfos()) - 1 if num_executors > 0: dummyRDD = self.parallelize(range(num_executors), num_executors) dummyRDD.foreachPartition(functionfoo) Will it guarantee that function foo will be executed on each active executor? Or will it miss few executors if there are more than 1 core per executor? Deeply appreciate help and time. *Thanks,Parag Chaudhari*