Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/276#discussion_r11172504
--- Diff:
graphx/src/main/scala/org/apache/spark/graphx/impl/EdgePartition.scala ---
@@ -84,19 +87,13 @@ class EdgePartition[@specialized(Char, Int, Boolean,
Byte, Long, Float, Double)
* order of the edges returned by `EdgePartition.iterator` and
* should return attributes equal to the number of edges.
*
- * @param f a function from an edge to a new attribute
+ * @param iter an iterator for the new attribute values
* @tparam ED2 the type of the new attribute
- * @return a new edge partition with the result of the function `f`
- * applied to each edge
+ * @return a new edge partition with the attribute values replaced
*/
def map[ED2: ClassTag](iter: Iterator[ED2]): EdgePartition[ED2] = {
- val newData = new Array[ED2](data.size)
- var i = 0
- while (iter.hasNext) {
- newData(i) = iter.next()
- i += 1
- }
- assert(newData.size == i)
+ val newData = iter.toArray
--- End diff --
Can we revert this change? This is where Scala fails us. The while loop is
~ 2x faster than iter.toArray in my micro benchmark.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
