[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432727#comment-17432727 ] L. C. Hsieh commented on SPARK-36913: - Sounds a good idea. +1 with [~dongjoon]. > Implement createIndex and IndexExists in JDBC (MySQL dialect) > - > > Key: SPARK-36913 > URL: https://issues.apache.org/jira/browse/SPARK-36913 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Huaxin Gao >Assignee: Huaxin Gao >Priority: Major > Fix For: 3.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432725#comment-17432725 ] Dongjoon Hyun commented on SPARK-36913: --- I agree with [~rxin] that we cannot do those things in general. For the parent ticket, SPARK-36525, why don't we narrow down to `CREATE/DROP BLOOM FILTER INDEX`? > My concern is not about JDBC (I should've commented on the parent ticket) > Implement createIndex and IndexExists in JDBC (MySQL dialect) > - > > Key: SPARK-36913 > URL: https://issues.apache.org/jira/browse/SPARK-36913 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Huaxin Gao >Assignee: Huaxin Gao >Priority: Major > Fix For: 3.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430780#comment-17430780 ] Reynold Xin commented on SPARK-36913: - My concern is not about JDBC (I should've commented on the parent ticket). My concern is that there are *a lot* of RDBMS features and we can't possibly support all of them. It seems like we'd be much better off just having a generic fallback API to execute a command that's passed through by Spark to the underlying data source, and then the underlying data source can decide what to do. Otherwise we will have to add create index, define foreign key, define sequence objects, and 50 other DDL commands in Spark. > Implement createIndex and IndexExists in JDBC (MySQL dialect) > - > > Key: SPARK-36913 > URL: https://issues.apache.org/jira/browse/SPARK-36913 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Huaxin Gao >Assignee: Huaxin Gao >Priority: Major > Fix For: 3.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430778#comment-17430778 ] Huaxin Gao commented on SPARK-36913: [~rxin] Hi Reynold, thank you for taking a look into this and sharing your concerns. Are you concerning about the interface SupportsIndex or the JDBC implementation? The interface I added is very generic and it's up to the data source to implement it. The JDBC implementation just provides me a simple way to do an end to end testing so I don't have to implement the new interface in InMemoryTable to do the test. > Implement createIndex and IndexExists in JDBC (MySQL dialect) > - > > Key: SPARK-36913 > URL: https://issues.apache.org/jira/browse/SPARK-36913 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Huaxin Gao >Assignee: Huaxin Gao >Priority: Major > Fix For: 3.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430324#comment-17430324 ] Reynold Xin commented on SPARK-36913: - I'm actually a bit worried about adding stuff like this to Spark. There are so many different types of indexes out there (and on top of that a lot of other common database features, e.g. create primary key, foreign key, sequence objects). Spark shouldn't and can't become a layer for doing database management across all database features. Are there concrete use cases for create index that we have seen? > Implement createIndex and IndexExists in JDBC (MySQL dialect) > - > > Key: SPARK-36913 > URL: https://issues.apache.org/jira/browse/SPARK-36913 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Huaxin Gao >Assignee: Huaxin Gao >Priority: Major > Fix For: 3.3.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423591#comment-17423591 ] Apache Spark commented on SPARK-36913: -- User 'huaxingao' has created a pull request for this issue: https://github.com/apache/spark/pull/34164 > Implement createIndex and IndexExists in JDBC (MySQL dialect) > - > > Key: SPARK-36913 > URL: https://issues.apache.org/jira/browse/SPARK-36913 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Huaxin Gao >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423590#comment-17423590 ] Apache Spark commented on SPARK-36913: -- User 'huaxingao' has created a pull request for this issue: https://github.com/apache/spark/pull/34164 > Implement createIndex and IndexExists in JDBC (MySQL dialect) > - > > Key: SPARK-36913 > URL: https://issues.apache.org/jira/browse/SPARK-36913 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.3.0 >Reporter: Huaxin Gao >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
