Github user darabos commented on a diff in the pull request:
https://github.com/apache/spark/pull/276#discussion_r11199524
--- Diff:
graphx/src/main/scala/org/apache/spark/graphx/impl/EdgePartition.scala ---
@@ -84,19 +87,13 @@ class EdgePartition[@specialized(Char, Int, Boolean,
Byte, Long, Float, Double)
* order of the edges returned by `EdgePartition.iterator` and
* should return attributes equal to the number of edges.
*
- * @param f a function from an edge to a new attribute
+ * @param iter an iterator for the new attribute values
* @tparam ED2 the type of the new attribute
- * @return a new edge partition with the result of the function `f`
- * applied to each edge
+ * @return a new edge partition with the attribute values replaced
*/
def map[ED2: ClassTag](iter: Iterator[ED2]): EdgePartition[ED2] = {
- val newData = new Array[ED2](data.size)
- var i = 0
- while (iter.hasNext) {
- newData(i) = iter.next()
- i += 1
- }
- assert(newData.size == i)
+ val newData = iter.toArray
--- End diff --
Done. Yeah, I didn't benchmark this, sorry.
I guess toArray must work with a dynamic array, since it does not know the
size in advance. There is an Iterator.copyToArray, but regrettably it does not
return the number of elements copied, so we would lose the important assertion.
I filed a feature request for this.
(https://issues.scala-lang.org/browse/SI-8469) Hopefully I'm not being obtuse.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
