[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15196 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user sureshthalamati commented on the issue: https://github.com/apache/spark/pull/15196 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15196 OK, let's leave the more conservative change here. I myself don't know what works or not on these dialects. I didn't add the Cassandra dialect; I think the idea is that a user could add such a dialect, for now, but only if this bug is fixed so that this query can be overridden. Later incorporating it into Spark is another question. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user sureshthalamati commented on the issue: https://github.com/apache/spark/pull/15196 @srowen Thanks for the clarification. My understanding is because read path takes query, "select * from $table where 1!=0 is preferred when it works . Hope is that source will optimize the query , and does not execute the user specified query completely. getTableExistsQuery() is primarily used to check the table in the data frame write path, there is no user specified query in this case. "select 1 from $table limit 1" performance might be ok. If we find select * from $table where 1 !=0 is better, then this query can be changed; otherwise leaving it as is may be the safest bet. I did not see CassandraDialect in the change set. Are you planning to add one ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15196 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15196 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65928/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15196 **[Test build #65928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65928/consoleFull)** for PR 15196 at commit [`c8789b5`](https://github.com/apache/spark/commit/c8789b50d9250c714ffb855d073b1982dcac7dfa). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15196 **[Test build #65928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65928/consoleFull)** for PR 15196 at commit [`c8789b5`](https://github.com/apache/spark/commit/c8789b50d9250c714ffb855d073b1982dcac7dfa). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15196 Sounds fine. I guess I'm surprised these wouldn't have to be overridden in the same way, but, on the other hand I suppose the existing query works today for MySQL et al? this is a more conservative change now, yes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15196 @srowen, I think that @sureshthalamati is right and that we shouldn't override for those other dialects until we know that there's a problem with them. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15196 @sureshthalamati I'm assuming it's required because the existing `getTableExistsQuery` is overridden in the same way. @JoshRosen if this is what you had in mind, I'll merge it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user sureshthalamati commented on the issue: https://github.com/apache/spark/pull/15196 Just curious , Is overwrite of getSchemaQuery() for Posgres, and MySQL dialects required ? I thought default query works fine for My SQL, and Postgres also. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15196 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65819/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15196 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15196 **[Test build #65819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65819/consoleFull)** for PR 15196 at commit [`aa64b82`](https://github.com/apache/spark/commit/aa64b82eba92a0e131f368a76154dc240a77f76c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15196 **[Test build #65819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65819/consoleFull)** for PR 15196 at commit [`aa64b82`](https://github.com/apache/spark/commit/aa64b82eba92a0e131f368a76154dc240a77f76c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15196 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15196 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65766/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15196 **[Test build #65766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65766/consoleFull)** for PR 15196 at commit [`27fe87d`](https://github.com/apache/spark/commit/27fe87d3b50b5465d1b94057c1fc8a0162c7bfb8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***) use t...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15196 **[Test build #65766 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65766/consoleFull)** for PR 15196 at commit [`27fe87d`](https://github.com/apache/spark/commit/27fe87d3b50b5465d1b94057c1fc8a0162c7bfb8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
