[jira] [Commented] (SPARK-25531) new write APIs for data source v2
[
https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028231#comment-17028231
]
Dongjoon Hyun commented on SPARK-25531:
---
I moved this to 3.1.0 for now. If we can resolve this issue, you can change
back to 3.0.0.
> new write APIs for data source v2
> -
>
> Key: SPARK-25531
> URL: https://issues.apache.org/jira/browse/SPARK-25531
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Wenchen Fan
>Priority: Major
>
> The current data source write API heavily depend on {{SaveMode}}, which
> doesn't have a clear semantic, especially when writing to tables.
> We should design a new set of write API without {{SaveMode}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25531) new write APIs for data source v2
[
https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028230#comment-17028230
]
Dongjoon Hyun commented on SPARK-25531:
---
Hi, [~cloud_fan].
Could you resolve this issue or adjust the target version to `3.1.0` ?
> new write APIs for data source v2
> -
>
> Key: SPARK-25531
> URL: https://issues.apache.org/jira/browse/SPARK-25531
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Wenchen Fan
>Priority: Major
>
> The current data source write API heavily depend on {{SaveMode}}, which
> doesn't have a clear semantic, especially when writing to tables.
> We should design a new set of write API without {{SaveMode}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25531) new write APIs for data source v2
[
https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629550#comment-16629550
]
Wenchen Fan commented on SPARK-25531:
-
I want to have a more structured view of the data source v2 project. It's a bad
idea to put everything under SPARK-22386, which is so general that only says
it's improvement. I'm starting to create tickets for some big steps of the data
source v2 project, like this one, like the API refactoring, and potentially the
catalog work, the custom metrics, etc. in the future.
For this particular case, the final goal is to design a new write api, for both
data source and end-users, to get rid of SaveMode. "Standardize SQL logical
plans" is how to achieve this goal IMO.
Note that all of them will be marks as "blocks SPARK-25186 Stabilize Data
Source V2 API".
> new write APIs for data source v2
> -
>
> Key: SPARK-25531
> URL: https://issues.apache.org/jira/browse/SPARK-25531
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 2.5.0
>Reporter: Wenchen Fan
>Priority: Major
>
> The current data source write API heavily depend on {{SaveMode}}, which
> doesn't have a clear semantic, especially when writing to tables.
> We should design a new set of write API without {{SaveMode}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25531) new write APIs for data source v2
[
https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629418#comment-16629418
]
Ryan Blue commented on SPARK-25531:
---
[~cloud_fan], what was the intent for this umbrella issue? You described it as
progress of "Standardize SQL logical plans" but the current description is "new
write APIs" instead. Also, these issues were already tracked under the umbrella
SPARK-22386 to improve DSv2, which covers the new logical plans and other
support issues like adding interfaces for required clustering and sorting
(SPARK-23889).
Is your intent to close the other issue because it is too old?
> new write APIs for data source v2
> -
>
> Key: SPARK-25531
> URL: https://issues.apache.org/jira/browse/SPARK-25531
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 2.5.0
>Reporter: Wenchen Fan
>Priority: Major
>
> The current data source write API heavily depend on {{SaveMode}}, which
> doesn't have a clear semantic, especially when writing to tables.
> We should design a new set of write API without {{SaveMode}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-25531) new write APIs for data source v2
[
https://issues.apache.org/jira/browse/SPARK-25531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16627572#comment-16627572
]
Wenchen Fan commented on SPARK-25531:
-
Hi [~rdblue], I've created this umbrella JIRA ticket to track the progress of
the "Standardize SQL logical plans" project. Feel free to update description
and add more sub-tasks. Thanks!
> new write APIs for data source v2
> -
>
> Key: SPARK-25531
> URL: https://issues.apache.org/jira/browse/SPARK-25531
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 2.5.0
>Reporter: Wenchen Fan
>Priority: Major
>
> The current data source write API heavily depend on {{SaveMode}}, which
> doesn't have a clear semantic, especially when writing to tables.
> We should design a new set of write API without {{SaveMode}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
