Dear all, I tried to customize my own RDD. In the getPreferredLocations() function, I used the following code to query anonter RDD, which was used as an input to initialize this customized RDD:
* val results: Array[Array[DataChunkPartition]] = context.runJob(partitionsRDD, (context: TaskContext, partIter: Iterator[DataChunkPartition]) => partIter.toArray, partitions, allowLocal = true)* The problem is that when executing the above code, the task seemed to be suspended. I mean the job just stopped at this code, but no errors and no outputs. What is the reason for it? Thanks, Fei