[ https://issues.apache.org/jira/browse/SPARK-20872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiao Li resolved SPARK-20872. ----------------------------- Resolution: Fixed Fix Version/s: 2.2.0 > ShuffleExchange.nodeName should handle null coordinator > ------------------------------------------------------- > > Key: SPARK-20872 > URL: https://issues.apache.org/jira/browse/SPARK-20872 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.2.0 > Reporter: Kris Mok > Assignee: Kris Mok > Priority: Trivial > Labels: easyfix > Fix For: 2.2.0 > > > A ShuffleExchange's coordinator can be null sometimes, and when we need to do > a toString() on it, it'll go to ShuffleExchange.nodeName() and throw a > MatchError there because of inexhaustive match -- the match only handles Some > and None, but doesn't handle null. > An example of this issue is when trying to inspect a Catalyst physical > operator on the Executor side in an IDE: > {code:none} > child = {WholeStageCodegenExec@13881} Method threw 'scala.MatchError' > exception. Cannot evaluate > org.apache.spark.sql.execution.WholeStageCodegenExec.toString() > {code} > where this WholeStageCodegenExec transitively references a ShuffleExchange. > On the Executor side, this ShuffleExchange instance is deserialized from the > data sent over from the Driver, and because the coordinator field is marked > transient, it's not carried over to the Executor, that's why it can be null > upon inspection. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org