Josh Rosen created SPARK-11080: ---------------------------------- Summary: NamedExpression.newExprId should only be called on driver Key: SPARK-11080 URL: https://issues.apache.org/jira/browse/SPARK-11080 Project: Spark Issue Type: Bug Components: SQL Reporter: Josh Rosen Assignee: Josh Rosen
My understanding of {{NamedExpression.newExprId}} is that it is only intended to be called on the driver. If it is called on executors, then this may lead to scenarios where the same expression id is re-used in two different NamedExpressions. More generally, I think that calling {{NamedExpression.newExprId}} within tasks may be an indicator of potential attribute binding bugs. Therefore, I think that we should prevent {{NamedExpression.newExprId}} from being called inside of tasks by throwing an exception when such calls occur. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org