[jira] [Commented] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271085#comment-16271085 ] Apache Spark commented on SPARK-22641: -- User 'sethah' has created a pull request for this issue:

[jira] [Commented] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-28 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270132#comment-16270132 ] Andrew Duffy commented on SPARK-22641: -- Query plan with the literal: {code} == Parsed Logical Plan

[jira] [Commented] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-28 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270128#comment-16270128 ] Andrew Duffy commented on SPARK-22641: -- So it seems this is only a problem when using literal