[jira] [Commented] (SPARK-23552) Dataset.withColumn does not allow overriding of a struct field
[
https://issues.apache.org/jira/browse/SPARK-23552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383002#comment-16383002
]
David Capwell commented on SPARK-23552:
---
This appears to also be a issue with drop.
> Dataset.withColumn does not allow overriding of a struct field
> --
>
> Key: SPARK-23552
> URL: https://issues.apache.org/jira/browse/SPARK-23552
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 2.2.0
>Reporter: David Capwell
>Priority: Major
>
> We have a Dataset with a schema such as the following:
>
>
> {code:java}
> struct>
> {code}
>
>
> if we do the following, one would expect to override the type of bar, but
> instead a new column gets added
>
>
> {code:java}
> ds.withColumn("foo.bar", ...){code}
>
> This produces the following schema
>
> {code:java}
> struct, foo.bar: ...>{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-23552) Dataset.withColumn does not allow overriding of a struct field
David Capwell created SPARK-23552:
-
Summary: Dataset.withColumn does not allow overriding of a struct
field
Key: SPARK-23552
URL: https://issues.apache.org/jira/browse/SPARK-23552
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.2.0
Reporter: David Capwell
We have a Dataset with a schema such as the following:
{code:java}
struct>
{code}
if we do the following, one would expect to override the type of bar, but
instead a new column gets added
{code:java}
ds.withColumn("foo.bar", ...){code}
This produces the following schema
{code:java}
struct, foo.bar: ...>{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
