Ankur Dave created SPARK-1552:
---------------------------------
Summary: GraphX performs type comparison incorrectly
Key: SPARK-1552
URL: https://issues.apache.org/jira/browse/SPARK-1552
Project: Spark
Issue Type: Bug
Components: GraphX
Reporter: Ankur Dave
In GraphImpl, mapVertices and outerJoinVertices use a more efficient
implementation when the map function preserves vertex attribute types. This is
implemented by comparing the ClassTags of the old and new vertex attribute
types. However, ClassTags store _erased_ types, so the comparison will return a
false positive for types with different type parameters, such as Option[Int]
and Option[Double].
Demo in the Scala shell:
scala> import scala.reflect.{classTag, ClassTag}
scala> def typesEqual[A: ClassTag, B: ClassTag](a: A, b: B): Boolean =
classTag[A] equals classTag[B]
scala> typesEqual(Some(1), Some(2.0)) // should return false
res2: Boolean = true
We can require richer TypeTags for these methods, or just take a flag from the
caller specifying whether the types are equal.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
