[
https://issues.apache.org/jira/browse/SPARK-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222661#comment-14222661
]
Sean Owen commented on SPARK-1857:
----------------------------------
Here m can easily be a Map.
> map() with lookup() causes exception
> ------------------------------------
>
> Key: SPARK-1857
> URL: https://issues.apache.org/jira/browse/SPARK-1857
> Project: Spark
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Michael Malak
>
> Using map() and lookup() in conjunction throws an exception
> {noformat}
> val a = sc.parallelize(Array(11))
> val m = sc.parallelize(Array((11,21)))
> a.map(m.lookup(_)(0)).collect
> 14/05/14 15:03:35 ERROR Executor: Exception in task ID 23
> scala.MatchError: null
> at org.apache.spark.rdd.PairRDDFunctions.lookup(PairRDDFunctions.scala:551)
> {noformat}
> A workaround is:
> {noformat}
> a.map((_,0)).join(m).map(_._2._2).collect
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
