[GitHub] [spark] AmplabJenkins commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
AmplabJenkins commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507006361 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
AmplabJenkins commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507006362 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12239/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
AmplabJenkins removed a comment on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507006362 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12239/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
AmplabJenkins removed a comment on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507006361 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
SparkQA commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507006436 **[Test build #107046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107046/testReport)** for PR 24972 at commit [`1d7e899`](https://github.com/apache/spark/commit/1d7e89926287ff6d4e017dbbbcc906c69e500b58). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mgaido91 commented on a change in pull request #25011: [SPARK-28170][ML][PYTHON] Uniform Vectors and Matrix documentation
mgaido91 commented on a change in pull request #25011: [SPARK-28170][ML][PYTHON] Uniform Vectors and Matrix documentation URL: https://github.com/apache/spark/pull/25011#discussion_r298818695 ## File path: python/pyspark/ml/linalg/__init__.py ## @@ -386,14 +386,14 @@ def squared_distance(self, other): def toArray(self): """ -Returns an numpy.ndarray +Returns the underlying numpy.ndarray Review comment: In case of `DenseVector` it means that there is a vector containing the values...and it think makes sense to report it to the users, in order that they know what happens if they modify it... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25016: [SPARK-28200] Decimal overflow handling
AmplabJenkins commented on issue #25016: [SPARK-28200] Decimal overflow handling URL: https://github.com/apache/spark/pull/25016#issuecomment-507010257 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25016: [SPARK-28200] Decimal overflow handling
AmplabJenkins removed a comment on issue #25016: [SPARK-28200] Decimal overflow handling URL: https://github.com/apache/spark/pull/25016#issuecomment-507010152 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
SparkQA commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507007256 **[Test build #107046 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107046/testReport)** for PR 24972 at commit [`1d7e899`](https://github.com/apache/spark/commit/1d7e89926287ff6d4e017dbbbcc906c69e500b58). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
AmplabJenkins commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507007272 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
SparkQA removed a comment on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507006436 **[Test build #107046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107046/testReport)** for PR 24972 at commit [`1d7e899`](https://github.com/apache/spark/commit/1d7e89926287ff6d4e017dbbbcc906c69e500b58). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
AmplabJenkins removed a comment on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507007273 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107046/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
AmplabJenkins commented on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507007273 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107046/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool
AmplabJenkins removed a comment on issue #24972: [SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#issuecomment-507007272 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25016: [SPARK-28200] Decimal overflow handling
AmplabJenkins removed a comment on issue #25016: [SPARK-28200] Decimal overflow handling URL: https://github.com/apache/spark/pull/25016#issuecomment-507010137 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25016: [SPARK-28200] Decimal overflow handling
AmplabJenkins commented on issue #25016: [SPARK-28200] Decimal overflow handling URL: https://github.com/apache/spark/pull/25016#issuecomment-507010152 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25016: [SPARK-28200] Decimal overflow handling
AmplabJenkins commented on issue #25016: [SPARK-28200] Decimal overflow handling URL: https://github.com/apache/spark/pull/25016#issuecomment-507010137 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24964: [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed
SparkQA commented on issue #24964: [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed URL: https://github.com/apache/spark/pull/24964#issuecomment-507005118 **[Test build #4813 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4813/testReport)** for PR 24964 at commit [`d2330cc`](https://github.com/apache/spark/commit/d2330cc8b4a74378556f7ae5924b3bb9388219d2). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24964: [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed
SparkQA removed a comment on issue #24964: [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed URL: https://github.com/apache/spark/pull/24964#issuecomment-506998921 **[Test build #4813 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4813/testReport)** for PR 24964 at commit [`d2330cc`](https://github.com/apache/spark/commit/d2330cc8b4a74378556f7ae5924b3bb9388219d2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mickjermsurawong-stripe opened a new pull request #25016: [SPARK-28200] Decimal overflow handling
mickjermsurawong-stripe opened a new pull request #25016: [SPARK-28200] Decimal overflow handling URL: https://github.com/apache/spark/pull/25016 ## What changes were proposed in this pull request? - In [SPARK-23179](https://github.com/apache/spark/pull/20350), an option to throw exception on decimal overflow was introduced. The option was applied only to sql Decimal operation and `RowEncoder` but not `ExpressionEncoder`. - The problem now is round-tripping overflowing java/scala BigDecimal/BigInteger currently return null results. - The serializer encode java/scala BigDecimal to to sql Decimal, which still has the underlying the former. - When writing out to UnsafeRow, `changePrecision` will be false and row has null value. https://github.com/apache/spark/blob/24e1e41648de58d3437e008b187b84828830e238/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java#L202-L206 - This PR adds the option to throw when detecting overflowing BigDecimal/BigInteger. This gives a consistent behavior between decimal arithmetic on sql expression (DecimalPrecision), and getting decimal from dataframe (RowEncoder) ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) Please review https://spark.apache.org/contributing.html before opening a pull request. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
SparkQA commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506981375 **[Test build #107044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107044/testReport)** for PR 25010 at commit [`4928330`](https://github.com/apache/spark/commit/4928330c928e988f71002a5f5f17ae9283232e2a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils
SparkQA removed a comment on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils URL: https://github.com/apache/spark/pull/25014#issuecomment-506970649 **[Test build #107040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107040/testReport)** for PR 25014 at commit [`d4589be`](https://github.com/apache/spark/commit/d4589be4a8fe15b04f3779a301173b2a048480e9). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils
SparkQA commented on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils URL: https://github.com/apache/spark/pull/25014#issuecomment-506982422 **[Test build #107040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107040/testReport)** for PR 25014 at commit [`d4589be`](https://github.com/apache/spark/commit/d4589be4a8fe15b04f3779a301173b2a048480e9). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils
AmplabJenkins commented on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils URL: https://github.com/apache/spark/pull/25014#issuecomment-506982534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107040/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils
AmplabJenkins commented on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils URL: https://github.com/apache/spark/pull/25014#issuecomment-506982533 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506988061 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107043/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC
gatorsmile commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-506996085 Thanks! Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LantaoJin commented on issue #24964: [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed
LantaoJin commented on issue #24964: [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed URL: https://github.com/apache/spark/pull/24964#issuecomment-506997893 Gentle ping @srowen This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506980805 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506980805 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506980806 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12236/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite
SparkQA commented on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite URL: https://github.com/apache/spark/pull/25013#issuecomment-506980834 **[Test build #107038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107038/testReport)** for PR 25013 at commit [`a31522b`](https://github.com/apache/spark/commit/a31522bd15ea84b249052b6010a051e850231379). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506980806 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12236/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506988059 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506988061 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107043/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
SparkQA removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506980445 **[Test build #107043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107043/testReport)** for PR 24978 at commit [`228ed58`](https://github.com/apache/spark/commit/228ed58e0b79f1647247977982f0bd13f0fd0bc8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506988059 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
SparkQA commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506988026 **[Test build #107043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107043/testReport)** for PR 24978 at commit [`228ed58`](https://github.com/apache/spark/commit/228ed58e0b79f1647247977982f0bd13f0fd0bc8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC
gatorsmile commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-506996039 @WangGuangxin Could you submit a follow-up PR for updating the document? See the example https://spark.apache.org/docs/latest/sql-data-sources-parquet.html#schema-merging This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] vanzin commented on issue #24982: [SPARK-28181][CORE] Add a filter interface to KVStore to speed up the entities retrieve
vanzin commented on issue #24982: [SPARK-28181][CORE] Add a filter interface to KVStore to speed up the entities retrieve URL: https://github.com/apache/spark/pull/24982#issuecomment-506998016 > The target is getting all tasks with particular status from all tasks. Right. That means a view with the "status" index, configured with both "first" and 'last" to match the desired status. That will return all the tasks with the status you're looking for. And doesn't require deserializing anything to find the desired tasks. Unless I'm not understanding what exactly it is you're trying to find. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24964: [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed
SparkQA commented on issue #24964: [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed URL: https://github.com/apache/spark/pull/24964#issuecomment-506998921 **[Test build #4813 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4813/testReport)** for PR 24964 at commit [`d2330cc`](https://github.com/apache/spark/commit/d2330cc8b4a74378556f7ae5924b3bb9388219d2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on issue #25012: [SPARK-28215][SQL][R] as_tibble was removed from Arrow R API
viirya commented on issue #25012: [SPARK-28215][SQL][R] as_tibble was removed from Arrow R API URL: https://github.com/apache/spark/pull/25012#issuecomment-50779 cc @HyukjinKwon This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
SparkQA commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506978686 **[Test build #107042 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107042/testReport)** for PR 25010 at commit [`90b0fd2`](https://github.com/apache/spark/commit/90b0fd2042d54ae1a0fa29e2057e0684301ad64a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
SparkQA commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506980445 **[Test build #107043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107043/testReport)** for PR 24978 at commit [`228ed58`](https://github.com/apache/spark/commit/228ed58e0b79f1647247977982f0bd13f0fd0bc8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506981261 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506981264 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12237/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506981264 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12237/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506981261 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
SparkQA commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506981759 **[Test build #107045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107045/testReport)** for PR 24978 at commit [`7f0397a`](https://github.com/apache/spark/commit/7f0397a11958fe5712352d58cc4938df4b926b3c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25015: [SPARK-28217][SQL] Allow a pluggable statistics plan visitor for a logical plan.
AmplabJenkins commented on issue #25015: [SPARK-28217][SQL] Allow a pluggable statistics plan visitor for a logical plan. URL: https://github.com/apache/spark/pull/25015#issuecomment-506984834 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25015: [SPARK-28217][SQL] Allow a pluggable statistics plan visitor for a logical plan.
AmplabJenkins commented on issue #25015: [SPARK-28217][SQL] Allow a pluggable statistics plan visitor for a logical plan. URL: https://github.com/apache/spark/pull/25015#issuecomment-506984870 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25015: [SPARK-28217][SQL] Allow a pluggable statistics plan visitor for a logical plan.
AmplabJenkins removed a comment on issue #25015: [SPARK-28217][SQL] Allow a pluggable statistics plan visitor for a logical plan. URL: https://github.com/apache/spark/pull/25015#issuecomment-506984581 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25015: [SPARK-28217][SQL] Allow a pluggable statistics plan visitor for a logical plan.
AmplabJenkins removed a comment on issue #25015: [SPARK-28217][SQL] Allow a pluggable statistics plan visitor for a logical plan. URL: https://github.com/apache/spark/pull/25015#issuecomment-506984834 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506989222 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506989222 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506989224 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107045/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
SparkQA removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506981759 **[Test build #107045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107045/testReport)** for PR 24978 at commit [`7f0397a`](https://github.com/apache/spark/commit/7f0397a11958fe5712352d58cc4938df4b926b3c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
SparkQA commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506989182 **[Test build #107045 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107045/testReport)** for PR 24978 at commit [`7f0397a`](https://github.com/apache/spark/commit/7f0397a11958fe5712352d58cc4938df4b926b3c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
SparkQA removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506981375 **[Test build #107044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107044/testReport)** for PR 25010 at commit [`4928330`](https://github.com/apache/spark/commit/4928330c928e988f71002a5f5f17ae9283232e2a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506991602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107044/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506991600 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
SparkQA commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506991499 **[Test build #107044 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107044/testReport)** for PR 25010 at commit [`4928330`](https://github.com/apache/spark/commit/4928330c928e988f71002a5f5f17ae9283232e2a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #24972: [WIP][SPARK-28167][SQL] Show global temporary view in database tool
gatorsmile commented on a change in pull request #24972: [WIP][SPARK-28167][SQL] Show global temporary view in database tool URL: https://github.com/apache/spark/pull/24972#discussion_r298813556 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -775,32 +775,61 @@ class SessionCatalog( * Note that, if the specified database is global temporary view database, we will list global * temporary views. */ - def listTables(db: String): Seq[TableIdentifier] = listTables(db, "*") Review comment: Yes. Definitely, we should not change the semantics of listTable API. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen commented on issue #24998: [SPARK-28202] [Core] [Test] Avoid noises of system props in SparkConfSuite
srowen commented on issue #24998: [SPARK-28202] [Core] [Test] Avoid noises of system props in SparkConfSuite URL: https://github.com/apache/spark/pull/24998#issuecomment-506999082 This test doesn't fail in a normal test run -- why would you customize the test env? that seems like the problem? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WangGuangxin commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC
WangGuangxin commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-507001494 > Thanks for your work @WangGuangxin ! > > What is your JIRA account? We need to assign the assignee field to your JIRA account. https://issues.apache.org/jira/browse/SPARK-11412 Thanks. My jira account is EdisonWang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WangGuangxin commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC
WangGuangxin commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-507001560 > @WangGuangxin Could you submit a follow-up PR for updating the document? See the example https://spark.apache.org/docs/latest/sql-data-sources-parquet.html#schema-merging ok This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Tonix517 commented on issue #24994: [SPARK-28133] Adding inverse hyperbolic functions in SQL
Tonix517 commented on issue #24994: [SPARK-28133] Adding inverse hyperbolic functions in SQL URL: https://github.com/apache/spark/pull/24994#issuecomment-507002652 Copied the link from the JIRA ticket: https://www.postgresql.org/docs/12/functions-math.html#FUNCTIONS-MATH-HYP-TABLE Just tried Hive and these functions were not supported indeed. > These are very niche functions. Does any other DB support it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506978721 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107042/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506978718 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
SparkQA removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506973258 **[Test build #107042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107042/testReport)** for PR 25010 at commit [`90b0fd2`](https://github.com/apache/spark/commit/90b0fd2042d54ae1a0fa29e2057e0684301ad64a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506978721 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107042/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins commented on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506978718 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite
AmplabJenkins removed a comment on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite URL: https://github.com/apache/spark/pull/25013#issuecomment-506980954 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite
AmplabJenkins removed a comment on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite URL: https://github.com/apache/spark/pull/25013#issuecomment-506980956 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107038/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite
SparkQA removed a comment on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite URL: https://github.com/apache/spark/pull/25013#issuecomment-506969043 **[Test build #107038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/107038/testReport)** for PR 25013 at commit [`a31522b`](https://github.com/apache/spark/commit/a31522bd15ea84b249052b6010a051e850231379). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite
AmplabJenkins commented on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite URL: https://github.com/apache/spark/pull/25013#issuecomment-506980956 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107038/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite
AmplabJenkins commented on issue #25013: [SPARK-28077][SQL][FOLLOW-UP] Add overlay and placing to TableIdentifierParserSuite URL: https://github.com/apache/spark/pull/25013#issuecomment-506980954 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506982143 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins commented on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506982146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12238/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506982146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12238/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506982143 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils
AmplabJenkins removed a comment on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils URL: https://github.com/apache/spark/pull/25014#issuecomment-506982534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107040/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils
AmplabJenkins removed a comment on issue #25014: [SPARK-28216][SQl][TEST] Move getDataSize from StatisticsCollectionTestBase to SQLTestUtils URL: https://github.com/apache/spark/pull/25014#issuecomment-506982533 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25015: [SPARK-28217][SQL] Allow a custom statistics logical plan visitor to be plugged in.
AmplabJenkins commented on issue #25015: [SPARK-28217][SQL] Allow a custom statistics logical plan visitor to be plugged in. URL: https://github.com/apache/spark/pull/25015#issuecomment-506984581 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 opened a new pull request #25015: [SPARK-28217][SQL] Allow a custom statistics logical plan visitor to be plugged in.
imback82 opened a new pull request #25015: [SPARK-28217][SQL] Allow a custom
statistics logical plan visitor to be plugged in.
URL: https://github.com/apache/spark/pull/25015
## What changes were proposed in this pull request?
Spark currently has two built-in statistics plan visitor:
SizeInBytesOnlyStatsPlanVisitor and BasicStatsPlanVisitor. However, this is a
bit limited since there is no way to plug in a custom plan visitor - from which
a custom query optimizer can benefit from.
This PR allowers the user to specify a custom stat plan visitor via a Spark
conf to override the built-in one:
```Scala
// First create your custom stat plan visitor.
class MyStatsPlanVisitor extends LogicalPlanVisitor[Statistics] {
// Implement LogicalPlanVisitor[Statistics] trait
}
// Set the visitor via Spark conf.
spark.conf.set("spark.sql.catalyst.statsPlanVisitorClass",
"MyStatsPlanVisitor")
// Now, stat() on a LogicalPlan object will use MyStatsPlanVisitor as a stat
plan visitor.
```
## How was this patch tested?
Existing and new unit tests.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution
AmplabJenkins removed a comment on issue #24978: [SPARK-28177][SQL] Adjust post shuffle partition number in adaptive execution URL: https://github.com/apache/spark/pull/24978#issuecomment-506989224 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107045/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506991600 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow
AmplabJenkins removed a comment on issue #25010: [SPARK-28201][SQL] Revisit MakeDecimal behavior on overflow URL: https://github.com/apache/spark/pull/25010#issuecomment-506991602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107044/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Tonix517 commented on a change in pull request #24994: [SPARK-28133] Adding inverse hyperbolic functions in SQL
Tonix517 commented on a change in pull request #24994: [SPARK-28133] Adding
inverse hyperbolic functions in SQL
URL: https://github.com/apache/spark/pull/24994#discussion_r298813645
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala
##
@@ -557,6 +578,27 @@ case class Sin(child: Expression) extends
UnaryMathExpression(math.sin, "SIN")
""")
case class Sinh(child: Expression) extends UnaryMathExpression(math.sinh,
"SINH")
+@ExpressionDescription(
+ usage = """
+_FUNC_(expr) - Returns inverse hyperbolic sine of `expr`.
+ """,
+ arguments = """
+Arguments:
+ * expr - hyperbolic angle
+ """,
+ examples = """
+Examples:
+ > SELECT _FUNC_(0);
+ 0.0
+ """)
+case class Asinh(child: Expression)
+ extends UnaryMathExpression((x: Double) => math.log(x + math.sqrt(x * x +
1.0)), "ASINH") {
+ override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
+defineCodeGen(ctx, ev, c =>
+ s"${ev.value} = java.lang.Math.log($c + java.lang.Math.sqrt($c * $c +
1.0));")
Review comment:
took a closer look at asinh\atanh implementation in FastMath. the negative
handling inside is to apply calculation optimization on abs(double a) and then
apply proper sign at the end of the function to return, which should produce
the same value as the original formula i'm using here
(https://en.wikipedia.org/wiki/Inverse_hyperbolic_functions#Definitions_in_terms_of_logarithms).
But FastMath runs faster than the original functions though.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC
gatorsmile commented on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-506996308 Thanks for your work @WangGuangxin ! What is your JIRA account? We need to assign the author to your JIRA account. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile edited a comment on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC
gatorsmile edited a comment on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-506996308 Thanks for your work @WangGuangxin ! What is your JIRA account? We need to assign the assignee field to your JIRA account. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile edited a comment on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC
gatorsmile edited a comment on issue #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043#issuecomment-506996308 Thanks for your work @WangGuangxin ! What is your JIRA account? We need to assign the assignee field to your JIRA account. https://issues.apache.org/jira/browse/SPARK-11412 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile closed pull request #24043: [SPARK-11412][SQL] Support merge schema for ORC
gatorsmile closed pull request #24043: [SPARK-11412][SQL] Support merge schema for ORC URL: https://github.com/apache/spark/pull/24043 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #25011: [SPARK-28170][ML][PYTHON] Uniform Vectors and Matrix documentation
srowen commented on a change in pull request #25011: [SPARK-28170][ML][PYTHON] Uniform Vectors and Matrix documentation URL: https://github.com/apache/spark/pull/25011#discussion_r298815289 ## File path: python/pyspark/ml/linalg/__init__.py ## @@ -386,14 +386,14 @@ def squared_distance(self, other): def toArray(self): """ -Returns an numpy.ndarray +Returns the underlying numpy.ndarray Review comment: I think this is OK, but just wondering how much to promise that it's the underlying array This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24964: [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed
AmplabJenkins removed a comment on issue #24964: [SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed URL: https://github.com/apache/spark/pull/24964#issuecomment-505377517 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen commented on issue #24996: [SPARK-28199][SS] Remove usage of deprecated ProcessingTime in Spark codebase
srowen commented on issue #24996: [SPARK-28199][SS] Remove usage of deprecated ProcessingTime in Spark codebase URL: https://github.com/apache/spark/pull/24996#issuecomment-506999280 Why introduce a new abstraction? This is what `Trigger.ProcessingTime` is meant to be. Just move the implementation to that class, and have the deprecated impl use it, rather than the other way around? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #24992: [SPARK-28194][SQL] Refactor code to prevent None.get in EnsureRequirements
srowen commented on a change in pull request #24992: [SPARK-28194][SQL]
Refactor code to prevent None.get in EnsureRequirements
URL: https://github.com/apache/spark/pull/24992#discussion_r298815400
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala
##
@@ -231,14 +231,15 @@ case class EnsureRequirements(conf: SQLConf) extends
Rule[SparkPlan] {
val keysAndIndexes = currentOrderOfKeys.zipWithIndex
expectedOrderOfKeys.foreach(expression => {
- val index = keysAndIndexes.find { case (e, idx) =>
+ keysAndIndexes.find { case (e, idx) =>
// As we may have the same key used many times, we need to filter out
its occurrence we
// have already used.
e.semanticEquals(expression) && !pickedIndexes.contains(idx)
- }.map(_._2).get
- pickedIndexes += index
- leftKeysBuffer.append(leftKeys(index))
- rightKeysBuffer.append(rightKeys(index))
+ }.map(_._2).map(index => {
Review comment:
This has to be `foreach`, not `map`, and can be `.foreach { index =>`
However I wonder if there's some other bug if the code really doesn't expect
to not find a value?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile closed pull request #24985: [SPARK-28184][SQL][TEST] Avoid creating new sessions in SparkMetadataOperationSuite
gatorsmile closed pull request #24985: [SPARK-28184][SQL][TEST] Avoid creating new sessions in SparkMetadataOperationSuite URL: https://github.com/apache/spark/pull/24985 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen commented on issue #24991: SPARK-28188 Materialize Dataframe API
srowen commented on issue #24991: SPARK-28188 Materialize Dataframe API URL: https://github.com/apache/spark/pull/24991#issuecomment-506999407 I don't think we should add this. It's already very common to `.count()` or `.mapPartitions` with a no-op to do this. I do think there are use cases for proactively materializing, but, it's overused too. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile closed pull request #24995: [SPARK-28196][SQL] Add a new `listTables` and `listLocalTempViews` APIs for SessionCatalog
gatorsmile closed pull request #24995: [SPARK-28196][SQL] Add a new `listTables` and `listLocalTempViews` APIs for SessionCatalog URL: https://github.com/apache/spark/pull/24995 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
