[GitHub] [spark] AmplabJenkins removed a comment on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging
AmplabJenkins removed a comment on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging URL: https://github.com/apache/spark/pull/25564#issuecomment-524182465 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14670/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging
AmplabJenkins removed a comment on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging URL: https://github.com/apache/spark/pull/25564#issuecomment-524182462 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging
SparkQA commented on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging URL: https://github.com/apache/spark/pull/25564#issuecomment-524182921 **[Test build #109616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109616/testReport)** for PR 25564 at commit [`0b0e02e`](https://github.com/apache/spark/commit/0b0e02e2c9ccd011a797eefe56bab254cb8babfa). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging
dongjoon-hyun commented on a change in pull request #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging URL: https://github.com/apache/spark/pull/25564#discussion_r316983236 ## File path: dev/merge_spark_pr.py ## @@ -495,7 +495,24 @@ def main(): else: title = pr["title"] -body = pr["body"] +modified_body = re.sub(re.compile(r'\n?', re.DOTALL), '', pr["body"]).lstrip() Review comment: `lstrip()` is used to remove the first PR template comments and new lines. `strip()` has false positive because it will modify all PR description (which doesn't have comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging
dongjoon-hyun commented on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging URL: https://github.com/apache/spark/pull/25564#issuecomment-524182631 Could you review this PR, @gatorsmile , @cloud-fan , @HyukjinKwon ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging
AmplabJenkins commented on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging URL: https://github.com/apache/spark/pull/25564#issuecomment-524182462 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging
AmplabJenkins commented on issue #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging URL: https://github.com/apache/spark/pull/25564#issuecomment-524182465 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14670/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #25453: [SPARK-28730][SQL] Configurable type coercion policy for table insertion
cloud-fan commented on issue #25453: [SPARK-28730][SQL] Configurable type coercion policy for table insertion URL: https://github.com/apache/spark/pull/25453#issuecomment-524181937 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #25453: [SPARK-28730][SQL] Configurable type coercion policy for table insertion
cloud-fan closed pull request #25453: [SPARK-28730][SQL] Configurable type coercion policy for table insertion URL: https://github.com/apache/spark/pull/25453 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
SparkQA commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524181306 **[Test build #109615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109615/testReport)** for PR 25464 at commit [`b0ae689`](https://github.com/apache/spark/commit/b0ae689fe86f36b6620d5709adeb429c514f40d4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun opened a new pull request #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging
dongjoon-hyun opened a new pull request #25564: [SPARK-28857][INFRA] Clean up the comments of PR template during merging URL: https://github.com/apache/spark/pull/25564 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce any user-facing change? No. (only for committers) ### How was this patch tested? Manually with Python2/Python3. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524180963 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14669/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524180960 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524180963 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14669/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524180960 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path
AmplabJenkins commented on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path URL: https://github.com/apache/spark/pull/25556#issuecomment-524180405 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109605/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables"
dongjoon-hyun commented on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables" URL: https://github.com/apache/spark/pull/25563#issuecomment-524180456 cc @shahidki31 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path
AmplabJenkins removed a comment on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path URL: https://github.com/apache/spark/pull/25556#issuecomment-524180405 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109605/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path
AmplabJenkins commented on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path URL: https://github.com/apache/spark/pull/25556#issuecomment-524180401 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path
AmplabJenkins removed a comment on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path URL: https://github.com/apache/spark/pull/25556#issuecomment-524180401 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait
AmplabJenkins removed a comment on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait URL: https://github.com/apache/spark/pull/25535#issuecomment-524180126 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait
AmplabJenkins commented on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait URL: https://github.com/apache/spark/pull/25535#issuecomment-524180128 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109606/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait
AmplabJenkins removed a comment on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait URL: https://github.com/apache/spark/pull/25535#issuecomment-524180128 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109606/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait
AmplabJenkins commented on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait URL: https://github.com/apache/spark/pull/25535#issuecomment-524180126 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path
SparkQA commented on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path URL: https://github.com/apache/spark/pull/25556#issuecomment-524180071 **[Test build #109605 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109605/testReport)** for PR 25556 at commit [`9aca171`](https://github.com/apache/spark/commit/9aca1719235985bb9068745941734c3158de68c2). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path
SparkQA removed a comment on issue #25556: [SPARK-28853][SQL] Support conf to organize file partitions by file path URL: https://github.com/apache/spark/pull/25556#issuecomment-52416 **[Test build #109605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109605/testReport)** for PR 25556 at commit [`9aca171`](https://github.com/apache/spark/commit/9aca1719235985bb9068745941734c3158de68c2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait
SparkQA removed a comment on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait URL: https://github.com/apache/spark/pull/25535#issuecomment-524144471 **[Test build #109606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109606/testReport)** for PR 25535 at commit [`1ca8607`](https://github.com/apache/spark/commit/1ca86079cfd31a9344ee3059360e04af05d4f4cf). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait
SparkQA commented on issue #25535: [SPARK-28835][SQL][TEST] Add TPCDSSchema trait URL: https://github.com/apache/spark/pull/25535#issuecomment-524179800 **[Test build #109606 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109606/testReport)** for PR 25535 at commit [`1ca8607`](https://github.com/apache/spark/commit/1ca86079cfd31a9344ee3059360e04af05d4f4cf). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #25525: [SPARK-28786][DOC][SQL]Document INSERT statement in SQL Reference
dilipbiswal commented on a change in pull request #25525:
[SPARK-28786][DOC][SQL]Document INSERT statement in SQL Reference
URL: https://github.com/apache/spark/pull/25525#discussion_r316979104
##
File path: docs/sql-ref-syntax-dml-insert-into.md
##
@@ -0,0 +1,65 @@
+---
+layout: global
+title: INSERT INTO
+displayTitle: INSERT INTO
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+### Description
+
+The `INSERT INTO` statement inserts new rows into a table. The inserted rows
can be specified by value expressions, or resulted from a query.
+
+### Syntax
+{% highlight sql %}
+INSERT INTO [TABLE] [db_name.]table_name [partition_spec] value_clause | query
Review comment:
@huaxingao
Should it be { value_clause | query }
or
( value_clause | query )
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #25525: [SPARK-28786][DOC][SQL]Document INSERT statement in SQL Reference
dilipbiswal commented on a change in pull request #25525:
[SPARK-28786][DOC][SQL]Document INSERT statement in SQL Reference
URL: https://github.com/apache/spark/pull/25525#discussion_r316979104
##
File path: docs/sql-ref-syntax-dml-insert-into.md
##
@@ -0,0 +1,65 @@
+---
+layout: global
+title: INSERT INTO
+displayTitle: INSERT INTO
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+### Description
+
+The `INSERT INTO` statement inserts new rows into a table. The inserted rows
can be specified by value expressions, or resulted from a query.
+
+### Syntax
+{% highlight sql %}
+INSERT INTO [TABLE] [db_name.]table_name [partition_spec] value_clause | query
Review comment:
@huaxingao
Should it be { value_clause | query }
or
( value_clause | query ) ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true
AmplabJenkins removed a comment on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true URL: https://github.com/apache/spark/pull/25520#issuecomment-524177587 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109607/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true
AmplabJenkins removed a comment on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true URL: https://github.com/apache/spark/pull/25520#issuecomment-524177576 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true
AmplabJenkins commented on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true URL: https://github.com/apache/spark/pull/25520#issuecomment-524177587 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109607/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true
AmplabJenkins commented on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true URL: https://github.com/apache/spark/pull/25520#issuecomment-524177576 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true
SparkQA removed a comment on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true URL: https://github.com/apache/spark/pull/25520#issuecomment-524147391 **[Test build #109607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109607/testReport)** for PR 25520 at commit [`47c7186`](https://github.com/apache/spark/commit/47c7186eb58b93ec1c82a41db59c8e58ed028ad8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true
SparkQA commented on issue #25520: [SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true URL: https://github.com/apache/spark/pull/25520#issuecomment-524177403 **[Test build #109607 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109607/testReport)** for PR 25520 at commit [`47c7186`](https://github.com/apache/spark/commit/47c7186eb58b93ec1c82a41db59c8e58ed028ad8). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JoshRosen commented on issue #25503: [SPARK-28702][SQL] Display useful error message (instead of NPE) for invalid Dataset operations
JoshRosen commented on issue #25503: [SPARK-28702][SQL] Display useful error message (instead of NPE) for invalid Dataset operations URL: https://github.com/apache/spark/pull/25503#issuecomment-524175550 Merged to master. Thanks @shivusondur! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JoshRosen closed pull request #25503: [SPARK-28702][SQL] Display useful error message (instead of NPE) for invalid Dataset operations
JoshRosen closed pull request #25503: [SPARK-28702][SQL] Display useful error message (instead of NPE) for invalid Dataset operations URL: https://github.com/apache/spark/pull/25503 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files
AmplabJenkins commented on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files URL: https://github.com/apache/spark/pull/25488#issuecomment-524174912 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files
AmplabJenkins commented on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files URL: https://github.com/apache/spark/pull/25488#issuecomment-524174917 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109604/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files
AmplabJenkins removed a comment on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files URL: https://github.com/apache/spark/pull/25488#issuecomment-524174917 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109604/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files
AmplabJenkins removed a comment on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files URL: https://github.com/apache/spark/pull/25488#issuecomment-524174912 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files
SparkQA removed a comment on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files URL: https://github.com/apache/spark/pull/25488#issuecomment-524137571 **[Test build #109604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109604/testReport)** for PR 25488 at commit [`376b7eb`](https://github.com/apache/spark/commit/376b7eb102532c87e27e2008866f868fa87951b8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files
SparkQA commented on issue #25488: [SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking crc files URL: https://github.com/apache/spark/pull/25488#issuecomment-524174582 **[Test build #109604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109604/testReport)** for PR 25488 at commit [`376b7eb`](https://github.com/apache/spark/commit/376b7eb102532c87e27e2008866f868fa87951b8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables"
AmplabJenkins commented on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables" URL: https://github.com/apache/spark/pull/25563#issuecomment-524173869 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables"
AmplabJenkins commented on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables" URL: https://github.com/apache/spark/pull/25563#issuecomment-524173878 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14668/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables"
AmplabJenkins removed a comment on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables" URL: https://github.com/apache/spark/pull/25563#issuecomment-524173869 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables"
AmplabJenkins removed a comment on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables" URL: https://github.com/apache/spark/pull/25563#issuecomment-524173878 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14668/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
SparkQA commented on issue #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#issuecomment-524172932 **[Test build #109614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109614/testReport)** for PR 25410 at commit [`74a7fec`](https://github.com/apache/spark/commit/74a7fecd82c410be0351998689914ae6b1e55eab). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables"
SparkQA commented on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables" URL: https://github.com/apache/spark/pull/25563#issuecomment-524172935 **[Test build #109613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109613/testReport)** for PR 25563 at commit [`9a265d7`](https://github.com/apache/spark/commit/9a265d7d448f23aad42dc73aff641f8e887a5f54). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
AmplabJenkins removed a comment on issue #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#issuecomment-524172574 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14667/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile opened a new pull request #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables"
gatorsmile opened a new pull request #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables" URL: https://github.com/apache/spark/pull/25563 This reverts commit 485ae6d1818e8756a86da38d6aefc8f1dbde49c2. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
AmplabJenkins commented on issue #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#issuecomment-524172568 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
AmplabJenkins commented on issue #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#issuecomment-524172574 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14667/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables"
gatorsmile commented on issue #25563: Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHdfs` in data source tables" URL: https://github.com/apache/spark/pull/25563#issuecomment-524172714 cc @wangyum @dongjoon-hyun @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates
AmplabJenkins removed a comment on issue #25410: [SPARK-28690][SQL] Add `date_part` function for timestamps/dates URL: https://github.com/apache/spark/pull/25410#issuecomment-524172568 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #25562: [SPARK-27330][SS] call foreach writer close on abort
HeartSaVioR commented on a change in pull request #25562: [SPARK-27330][SS] call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#discussion_r316974056 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/sources/ForeachWriterSuite.scala ## @@ -154,6 +154,9 @@ class ForeachWriterSuite extends StreamTest with SharedSQLContext with BeforeAnd val errorEvent = allEvents(0)(2).asInstanceOf[ForeachWriterSuite.Close] assert(errorEvent.error.get.isInstanceOf[RuntimeException]) assert(errorEvent.error.get.getMessage === "ForeachSinkSuite error") + Review comment: nit: unnecessary newline This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #25562: [SPARK-27330][SS] call foreach writer close on abort
HeartSaVioR commented on a change in pull request #25562: [SPARK-27330][SS] call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#discussion_r316973937 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/sources/ForeachWriterSuite.scala ## @@ -258,6 +261,35 @@ class ForeachWriterSuite extends StreamTest with SharedSQLContext with BeforeAnd query.stop() } } + Review comment: nit: unnecessary newline This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #25562: [SPARK-27330][SS] call foreach writer close on abort
HeartSaVioR commented on a change in pull request #25562: [SPARK-27330][SS] call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#discussion_r316973836 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/ForeachWriterProvider.scala ## @@ -112,6 +113,8 @@ class ForeachDataWriter[T]( // If open returns false, we should skip writing rows. private val opened = writer.open(partitionId, epochId) + private var closeCalled: Boolean = false + Review comment: nit: unnecessary newline This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25560: [SPARK-28642][SQL][2.4] Hide credentials in show create table
dongjoon-hyun commented on issue #25560: [SPARK-28642][SQL][2.4] Hide credentials in show create table URL: https://github.com/apache/spark/pull/25560#issuecomment-524172140 cc @gatorsmile , too. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #25562: [SPARK-27330][SS] call foreach writer close on abort
HeartSaVioR commented on issue #25562: [SPARK-27330][SS] call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#issuecomment-524171604 1. Please add `[BRANCH-2.4]` after `[SS]` in PR's title. 2. It would save reviewers' time to link the origin PR when you're proposing PR for backport. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #23952: [SPARK-26929][SQL]fix table owner use user instead of principal when create table through spark-sql or beeline
HeartSaVioR commented on a change in pull request #23952: [SPARK-26929][SQL]fix
table owner use user instead of principal when create table through spark-sql
or beeline
URL: https://github.com/apache/spark/pull/23952#discussion_r316973255
##
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##
@@ -191,7 +193,12 @@ private[hive] class HiveClientImpl(
/** Returns the configuration for the current session. */
def conf: HiveConf = state.getConf
- private val userName = conf.getUser
+ private val userName: String = try {
+val ugi = HiveUtils.getUGI
+ugi.getShortUserName
Review comment:
Looks like `Utils` is accessible from here, so if we just want to leverage
Hive code and call `getShortUserName` instead, we can just do it with one-liner.
```
private val userName =
org.apache.hadoop.hive.shims.Utils.getUGI.getShortUserName
```
Here I intended to not catch LoginException as we don't do anything but
throw it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #23952: [SPARK-26929][SQL]fix table owner use user instead of principal when create table through spark-sql or beeline
HeartSaVioR commented on a change in pull request #23952: [SPARK-26929][SQL]fix
table owner use user instead of principal when create table through spark-sql
or beeline
URL: https://github.com/apache/spark/pull/23952#discussion_r316973255
##
File path:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##
@@ -191,7 +193,12 @@ private[hive] class HiveClientImpl(
/** Returns the configuration for the current session. */
def conf: HiveConf = state.getConf
- private val userName = conf.getUser
+ private val userName: String = try {
+val ugi = HiveUtils.getUGI
+ugi.getShortUserName
Review comment:
Looks like `Utils` is accessible from here, so if we just want to leverage
Hive code and call `getShortUserName` instead, we can just do it with one-liner.
```
private val userName =
org.apache.hadoop.hive.shims.Utils.getUGI.getShortUserName
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
SparkQA commented on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#issuecomment-524170508 **[Test build #109612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109612/testReport)** for PR 25001 at commit [`bb0be67`](https://github.com/apache/spark/commit/bb0be67a4f7fe9c8ed52a596347a51ca291b4a8d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
AmplabJenkins removed a comment on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#issuecomment-524170187 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
AmplabJenkins removed a comment on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#issuecomment-524170191 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14666/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
AmplabJenkins commented on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#issuecomment-524170187 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
AmplabJenkins commented on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#issuecomment-524170191 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14666/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524170059 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109603/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524170055 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524170059 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109603/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
AmplabJenkins commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524170055 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions
AmplabJenkins removed a comment on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions URL: https://github.com/apache/spark/pull/20965#issuecomment-524169571 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109602/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions
AmplabJenkins removed a comment on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions URL: https://github.com/apache/spark/pull/20965#issuecomment-524169568 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
SparkQA commented on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524169766 **[Test build #109603 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109603/testReport)** for PR 25464 at commit [`4fbf34a`](https://github.com/apache/spark/commit/4fbf34a7bb92cc6cbd6110f1222ed159f7a6a26f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries
SparkQA removed a comment on issue #25464: [SPARK-28746][SQL] Add partitionby hint for sql queries URL: https://github.com/apache/spark/pull/25464#issuecomment-524133467 **[Test build #109603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109603/testReport)** for PR 25464 at commit [`4fbf34a`](https://github.com/apache/spark/commit/4fbf34a7bb92cc6cbd6110f1222ed159f7a6a26f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions
AmplabJenkins commented on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions URL: https://github.com/apache/spark/pull/20965#issuecomment-524169568 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions
AmplabJenkins commented on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions URL: https://github.com/apache/spark/pull/20965#issuecomment-524169571 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109602/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
beliefer commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#discussion_r316971787 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ## @@ -83,33 +83,37 @@ abstract class StringRegexExpression extends BinaryExpression % matches zero or more characters in the input (similar to .* in posix regular expressions) - The escape character is '\'. If an escape character precedes a special symbol or another - escape character, the following character is matched literally. It is invalid to escape - any other character. - Since Spark 2.0, string literals are unescaped in our SQL parser. For example, in order to match "\abc", the pattern should be "\\abc". When SQL config 'spark.sql.parser.escapedStringLiterals' is enabled, it fallbacks to Spark 1.6 behavior regarding string literal parsing. For example, if the config is enabled, the pattern to match "\abc" should be "\abc". + * escape - a optional string. The default escape character is the '\' . If an escape character + precedes a special symbol or another escape character, the following character is matched + literally. It is invalid to escape any other character. Review comment: Of course, I will add it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #25525: [SPARK-28786][DOC][SQL]Document INSERT statement in SQL Reference
dilipbiswal commented on a change in pull request #25525: [SPARK-28786][DOC][SQL]Document INSERT statement in SQL Reference URL: https://github.com/apache/spark/pull/25525#discussion_r316964289 ## File path: docs/sql-ref-syntax-dml-insert-into.md ## @@ -0,0 +1,65 @@ +--- +layout: global +title: INSERT INTO +displayTitle: INSERT INTO +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +### Description + +The `INSERT INTO` statement inserts new rows into a table. The inserted rows can be specified by value expressions, or resulted from a query. Review comment: @huaxingao Wondering if this would sound better ? The input rows for the `INSERT` statement can be produced by one of two ways : - By value expressions - By a query This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions
SparkQA removed a comment on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions URL: https://github.com/apache/spark/pull/20965#issuecomment-524130828 **[Test build #109602 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109602/testReport)** for PR 20965 at commit [`ae1b086`](https://github.com/apache/spark/commit/ae1b086a2d4d354653e657179bfd9a5ba2599a97). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions
SparkQA commented on issue #20965: [SPARK-21870][SQL] Split aggregation code into small functions URL: https://github.com/apache/spark/pull/20965#issuecomment-524169315 **[Test build #109602 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109602/testReport)** for PR 20965 at commit [`ae1b086`](https://github.com/apache/spark/commit/ae1b086a2d4d354653e657179bfd9a5ba2599a97). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow
maropu commented on a change in pull request #25461: [SPARK-28741][SQL]Throw
exceptions when casting to integers causes overflow
URL: https://github.com/apache/spark/pull/25461#discussion_r316970632
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -1805,9 +1805,9 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
- val ARITHMETIC_OPERATIONS_FAIL_ON_OVERFLOW =
-buildConf("spark.sql.arithmeticOperations.failOnOverFlow")
- .doc("If it is set to true, all arithmetic operations on non-decimal
fields throw an " +
+ val FAIL_ON_INTEGER_OVERFLOW =
+buildConf("spark.sql.failOnIntegerOverFlow")
Review comment:
failOnIntegerOverFlow -> failOnIntegralTypeOverFlow? To me, Integer is a bit
ambiguous.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow
AmplabJenkins removed a comment on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow URL: https://github.com/apache/spark/pull/25461#issuecomment-524167848 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109601/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow
AmplabJenkins removed a comment on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow URL: https://github.com/apache/spark/pull/25461#issuecomment-524167845 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow
AmplabJenkins commented on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow URL: https://github.com/apache/spark/pull/25461#issuecomment-524167845 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow
AmplabJenkins commented on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow URL: https://github.com/apache/spark/pull/25461#issuecomment-524167848 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109601/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow
SparkQA removed a comment on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow URL: https://github.com/apache/spark/pull/25461#issuecomment-524129603 **[Test build #109601 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109601/testReport)** for PR 25461 at commit [`977d509`](https://github.com/apache/spark/commit/977d50969eab560ffab7d0efca2c8e0c70e9b463). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow
maropu commented on a change in pull request #25461: [SPARK-28741][SQL]Throw
exceptions when casting to integers causes overflow
URL: https://github.com/apache/spark/pull/25461#discussion_r316970371
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
##
@@ -1182,6 +1233,78 @@ case class Cast(child: Expression, dataType: DataType,
timeZoneId: Option[String
(c, evPrim, evNull) => code"$evPrim = $c != 0;"
}
+ private[this] def castTimestampToIntegerCode(
+ ctx: CodegenContext,
+ integralType: String): CastFunction = {
+if (failOnIntegerOverflow) {
+ val longValue = ctx.freshName("longValue")
+ (c, evPrim, evNull) =>
+code"""
+ long $longValue = ${timestampToLongCode(c)};
+ if ($longValue == ($integralType) $longValue) {
+$evPrim = ($integralType) $longValue;
+ } else {
+throw new ArithmeticException("Casting $c to $integralType causes
overflow");
+ }
+"""
+} else {
+ (c, evPrim, evNull) => code"$evPrim = ($integralType)
${timestampToLongCode(c)};"
+}
+ }
+
+ private[this] def castDecimalToIntegerCode(
+ ctx: CodegenContext,
+ integralType: String): CastFunction = {
+(c, evPrim, evNull) => code"$evPrim =
$c.to${integralType.capitalize}($failOnIntegerOverflow);"
+ }
+
+ private[this] def castIntegerToIntegerExactCode(integralType: String):
CastFunction = {
Review comment:
(oh, I didn't know this functionality, ```suggestion, cool)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow
SparkQA commented on issue #25461: [SPARK-28741][SQL]Throw exceptions when casting to integers causes overflow URL: https://github.com/apache/spark/pull/25461#issuecomment-524167593 **[Test build #109601 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109601/testReport)** for PR 25461 at commit [`977d509`](https://github.com/apache/spark/commit/977d50969eab560ffab7d0efca2c8e0c70e9b463). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] eyalzit commented on issue #24382: [SPARK-27330][SS] support task abort in foreach writer
eyalzit commented on issue #24382: [SPARK-27330][SS] support task abort in foreach writer URL: https://github.com/apache/spark/pull/24382#issuecomment-524166530 > @eyalzit could you open a PR for 2.4? I can help if you don't have time. I did a quick check. There are just some minor import conflicts. https://github.com/apache/spark/pull/25562 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] eyalzit commented on issue #25562: [SPARK-27330][SS] call foreach writer close on abort
eyalzit commented on issue #25562: [SPARK-27330][SS] call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#issuecomment-524166438 @zsxwing @cloud-fan can someone please review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25562: SPARK-27330[SS] call foreach writer close on abort
AmplabJenkins removed a comment on issue #25562: SPARK-27330[SS] call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#issuecomment-524165296 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25562: SPARK 27330 call foreach writer close on abort
SparkQA commented on issue #25562: SPARK 27330 call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#issuecomment-524165792 **[Test build #109611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/109611/testReport)** for PR 25562 at commit [`e26a0bc`](https://github.com/apache/spark/commit/e26a0bc5a0e39d01f630332484fa0d4ed39d38bc). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25562: SPARK 27330 call foreach writer close on abort
AmplabJenkins removed a comment on issue #25562: SPARK 27330 call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#issuecomment-524165444 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14665/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25562: SPARK 27330 call foreach writer close on abort
AmplabJenkins removed a comment on issue #25562: SPARK 27330 call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#issuecomment-524165439 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25562: SPARK 27330 call foreach writer close on abort
AmplabJenkins commented on issue #25562: SPARK 27330 call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#issuecomment-524165439 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25562: SPARK 27330 call foreach writer close on abort
AmplabJenkins commented on issue #25562: SPARK 27330 call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#issuecomment-524165444 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14665/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25562: SPARK 27330 call foreach writer close on abort
AmplabJenkins commented on issue #25562: SPARK 27330 call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562#issuecomment-524165296 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] eyalzit opened a new pull request #25562: SPARK 27330 call foreach writer close on abort
eyalzit opened a new pull request #25562: SPARK 27330 call foreach writer close on abort URL: https://github.com/apache/spark/pull/25562 ### What changes were proposed in this pull request? in order to address cases where foreach writer task is failing without calling the close() method, (for example when a task is interrupted) abort() method will be called when the task is aborted. the abort will call writer.close() ### How was this patch tested? update existing unit tests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
