You can use mapPartition, which allows you to apply the map function elementwise to all elements of a partition. Here you can place custom code around your function invocation that lets you reuse the array.
--sebastian Am 19.11.2013 10:02 schrieb "Wenlei Xie" <[email protected]>: > Hi, > > I am trying to do some tasks with the following style map function: > > rdd.map { e => > val a = new Array[Int](100) > ...Some calculation... > } > > But here the array a is really just used as a temporary buffer and can be > reused. I am wondering if I can avoid constructing it everytime? (As it > might incur some overhead for JVM?) Would use an array outside the closure > work? > > Thank you! > > Best, > Wenlei > >
