[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524947360 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN" export MAVEN_CLI_OPTS="--no-transfer-progress" -mkdir -p ~/.m2 Review comment: Sure, I ran the build without cache at https://github.com/apache/spark/pull/30394. If it passes, I'll keep it. If it fails, I will recover it back with a comment :-). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon opened a new pull request #30394: Check SPARK-33464
HyukjinKwon opened a new pull request #30394: URL: https://github.com/apache/spark/pull/30394 Check if .m2 is required. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-728754094 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
SparkQA commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-728754079 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35804/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
dongjoon-hyun commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524945402 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN" export MAVEN_CLI_OPTS="--no-transfer-progress" -mkdir -p ~/.m2 Review comment: If GitHub Action passed with no prior-cache status, we can remove it. Otherwise, we need to keep this still. It's up to your testing~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
SparkQA commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728753070 **[Test build #131207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131207/testReport)** for PR 30391 at commit [`15ea6b5`](https://github.com/apache/spark/commit/15ea6b5d6bd67510d88f31d662df7a082c96fd07). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524945013 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
dongjoon-hyun commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524944927 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN" export MAVEN_CLI_OPTS="--no-transfer-progress" -mkdir -p ~/.m2 Review comment: Ur, this also has a history. Maven doesn't create this at that time. Cache created it. So, when there is no cache, it failed. Please double check it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
dongjoon-hyun commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524943408 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment: Thank you! You can add some comments there if you want~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524943842 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN" export MAVEN_CLI_OPTS="--no-transfer-progress" -mkdir -p ~/.m2 Review comment: @dongjoon-hyun, I removed this though. Maven seems creating it automatically (whether the cache exists or not). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728751123 Addressed! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728750100 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35805/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524942149 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment: Oh, sure. Then it makes sense to keep `install`. +1 for keeping it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #30372: [SPARK-33172][SQL] Adding support for UserDefinedType for Spark SQL Code generator
maropu commented on a change in pull request #30372:
URL: https://github.com/apache/spark/pull/30372#discussion_r524941668
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
##
@@ -1734,15 +1734,17 @@ object CodeGenerator extends Logging {
* Returns the specialized code to access a value from a column vector for a
given `DataType`.
*/
def getValueFromVector(vector: String, dataType: DataType, rowId: String):
String = {
-if (dataType.isInstanceOf[StructType]) {
- // `ColumnVector.getStruct` is different from `InternalRow.getStruct`,
it only takes an
- // `ordinal` parameter.
- s"$vector.getStruct($rowId)"
-} else {
- getValue(vector, dataType, rowId)
+dataType match {
+ case udt: UserDefinedType[_] => getValueFromVector(vector, udt.sqlType,
rowId)
+ case _ => if (dataType.isInstanceOf[StructType]) {
+// `ColumnVector.getStruct` is different from `InternalRow.getStruct`,
it only takes an
+// `ordinal` parameter.
+s"$vector.getStruct($rowId)"
+ } else {
+getValue(vector, dataType, rowId)
+ }
}
}
-
Review comment:
nit: revert this (unnecessary change)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #30372: [SPARK-33172][SQL] Adding support for UserDefinedType for Spark SQL Code generator
maropu commented on a change in pull request #30372:
URL: https://github.com/apache/spark/pull/30372#discussion_r524941488
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
##
@@ -1734,15 +1734,17 @@ object CodeGenerator extends Logging {
* Returns the specialized code to access a value from a column vector for a
given `DataType`.
*/
def getValueFromVector(vector: String, dataType: DataType, rowId: String):
String = {
-if (dataType.isInstanceOf[StructType]) {
- // `ColumnVector.getStruct` is different from `InternalRow.getStruct`,
it only takes an
- // `ordinal` parameter.
- s"$vector.getStruct($rowId)"
-} else {
- getValue(vector, dataType, rowId)
+dataType match {
+ case udt: UserDefinedType[_] => getValueFromVector(vector, udt.sqlType,
rowId)
Review comment:
Does this issue only happens when using
`spark-bigquery-with-dependencies`? In the current spark codebase, it seems
dataType cannot be an user-defined type in this method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
dongjoon-hyun commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728749215 Please check https://github.com/apache/spark/pull/30391#discussion_r524939916 . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
dongjoon-hyun commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524940363 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment: Do we have another new place to test `install`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
dongjoon-hyun commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524939916 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment: @sarutak and @HyukjinKwon . I'd like to keep this `install` test coverage. This is a contribution from @wangyum to check a valid error check from JDK8 age. Please see e5c176a243b76b3953cc03b28e6c281658da93c8 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
AmplabJenkins removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728747106 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
AmplabJenkins commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728747106 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
SparkQA removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728724756 **[Test build #131202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131202/testReport)** for PR 28386 at commit [`3aa97f3`](https://github.com/apache/spark/commit/3aa97f369c3b3cb468a73d6bebc8b8c235094469). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728746848 **[Test build #131202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131202/testReport)** for PR 28386 at commit [`3aa97f3`](https://github.com/apache/spark/commit/3aa97f369c3b3cb468a73d6bebc8b8c235094469). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
AmplabJenkins removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743942 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131204/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
SparkQA commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728746560 **[Test build #131206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131206/testReport)** for PR 30391 at commit [`8f6d64f`](https://github.com/apache/spark/commit/8f6d64fa9767eb567b3e2080d69f22e6079fab17). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29999: [SPARK-33045][SQL] Support build-in function like_all and fix StackOverflowError issue.
AmplabJenkins removed a comment on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-728745887 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead
mridulm commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728746062 I am not sure I follow - the `reduce` will reduce it at driver - based on the individual priority queues per partition - while `treeReduce` will progressively reduce it in executors before pulling final pq result to driver. What is the concern here ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
AmplabJenkins removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728745377 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29999: [SPARK-33045][SQL] Support build-in function like_all and fix StackOverflowError issue.
AmplabJenkins commented on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-728745887 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
AmplabJenkins removed a comment on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728745183 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/35802/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
AmplabJenkins commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728745377 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728745363 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35803/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
AmplabJenkins commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728745179 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
AmplabJenkins removed a comment on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728745179 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
SparkQA commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728745161 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35802/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29999: [SPARK-33045][SQL] Support build-in function like_all and fix StackOverflowError issue.
SparkQA commented on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-728744819 **[Test build #131189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131189/testReport)** for PR 2 at commit [`97c1c73`](https://github.com/apache/spark/commit/97c1c7389e537f0d38f1b6a17bbe9ba70c9bc6ea). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29999: [SPARK-33045][SQL] Support build-in function like_all and fix StackOverflowError issue.
SparkQA removed a comment on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-728649075 **[Test build #131189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131189/testReport)** for PR 2 at commit [`97c1c73`](https://github.com/apache/spark/commit/97c1c7389e537f0d38f1b6a17bbe9ba70c9bc6ea). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
AmplabJenkins removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743938 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #30164: [SPARK-32919][SHUFFLE][test-maven][test-hadoop2.7] Driver side changes for coordinating push based shuffle by selecting external shuffle serv
mridulm commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-728744238 Thanks for the +1 and review @Ngone51 ! I dont have any further comments, the recent changes proposed look good to me too. Any other review comments @tgravescs, @attilapiros, @Victsm, @otterc ? I will leave it for a couple of days to give more time for review before merging. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
AmplabJenkins removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743881 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
AmplabJenkins commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743938 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
AmplabJenkins commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743881 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
SparkQA commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728743346 **[Test build #131205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131205/testReport)** for PR 30391 at commit [`9b65e62`](https://github.com/apache/spark/commit/9b65e624fedb12a9fde9f3ae0c9ca5f3b61787f4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead
AmplabJenkins commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728740932 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
SparkQA commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-728740967 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35804/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead
AmplabJenkins removed a comment on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728740932 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead
SparkQA commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728740916 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35801/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
SparkQA commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728740285 **[Test build #131204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131204/testReport)** for PR 30391 at commit [`848dea0`](https://github.com/apache/spark/commit/848dea02a32bde58dbd0a4b2f9bd116917092b1a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow
AmplabJenkins removed a comment on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728738673 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131203/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow
SparkQA removed a comment on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728737413 **[Test build #131203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131203/testReport)** for PR 30393 at commit [`a92af2f`](https://github.com/apache/spark/commit/a92af2fe5e48239b154d997e055e32d34b2c61ce). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow
AmplabJenkins commented on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728738666 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow
SparkQA commented on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728738655 **[Test build #131203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131203/testReport)** for PR 30393 at commit [`a92af2f`](https://github.com/apache/spark/commit/a92af2fe5e48239b154d997e055e32d34b2c61ce). * This patch **fails Python style tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow
AmplabJenkins removed a comment on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728738666 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] BryanCutler commented on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow
BryanCutler commented on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728737383 ping @HyukjinKwon to please take a look when you can, thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow
SparkQA commented on pull request #30393: URL: https://github.com/apache/spark/pull/30393#issuecomment-728737413 **[Test build #131203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131203/testReport)** for PR 30393 at commit [`a92af2f`](https://github.com/apache/spark/commit/a92af2fe5e48239b154d997e055e32d34b2c61ce). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728736268 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35803/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] tanelk edited a comment on pull request #30040: [WIP][SPARK-33124][SQL][DOCS] Adds a group tag in all the expressions for built-in functions
tanelk edited a comment on pull request #30040: URL: https://github.com/apache/spark/pull/30040#issuecomment-728736015 > @tanelk Are you still working on this? I haven't work on it for a while, but I'll get back to it this week. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] tanelk commented on pull request #30040: [WIP][SPARK-33124][SQL][DOCS] Adds a group tag in all the expressions for built-in functions
tanelk commented on pull request #30040: URL: https://github.com/apache/spark/pull/30040#issuecomment-728736015 > @tanelk Are you still working on this? I havent work on it for a while, but I'll get back to it this week. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead
viirya commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728735611 I will run benchmark later if the test is passed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30325: [SPARK-33416][SQL] Avoid Hive metastore stack overflow when InSet predicate have many values
AmplabJenkins removed a comment on pull request #30325: URL: https://github.com/apache/spark/pull/30325#issuecomment-728735429 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
SparkQA commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728735386 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35802/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30325: [SPARK-33416][SQL] Avoid Hive metastore stack overflow when InSet predicate have many values
AmplabJenkins commented on pull request #30325: URL: https://github.com/apache/spark/pull/30325#issuecomment-728735429 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30325: [SPARK-33416][SQL] Avoid Hive metastore stack overflow when InSet predicate have many values
SparkQA removed a comment on pull request #30325: URL: https://github.com/apache/spark/pull/30325#issuecomment-728642372 **[Test build #131186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131186/testReport)** for PR 30325 at commit [`231cd6f`](https://github.com/apache/spark/commit/231cd6f15f3fa71d677aae7170e1e404b82cef2a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30325: [SPARK-33416][SQL] Avoid Hive metastore stack overflow when InSet predicate have many values
SparkQA commented on pull request #30325: URL: https://github.com/apache/spark/pull/30325#issuecomment-728734485 **[Test build #131186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131186/testReport)** for PR 30325 at commit [`231cd6f`](https://github.com/apache/spark/commit/231cd6f15f3fa71d677aae7170e1e404b82cef2a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] BryanCutler opened a new pull request #30393: [SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow
BryanCutler opened a new pull request #30393: URL: https://github.com/apache/spark/pull/30393 ### What changes were proposed in this pull request? This change adds MapType support for PySpark with Arrow, if using pyarrow >= 2.0.0. ### Why are the changes needed? MapType was previous unsupported with Arrow. ### Does this PR introduce _any_ user-facing change? User can now enable MapType for `createDataFrame()`, `toPandas()` with Arrow optimization, and with Pandas UDFs. ### How was this patch tested? Added new PySpark tests for createDataFrame(), toPandas() and Scalar Pandas UDFs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal
AmplabJenkins removed a comment on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-728733454 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal
AmplabJenkins commented on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-728733454 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
AmplabJenkins removed a comment on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728732814 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
SparkQA commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728732787 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35800/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal
SparkQA removed a comment on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-728642439 **[Test build #131187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131187/testReport)** for PR 30145 at commit [`6fa8115`](https://github.com/apache/spark/commit/6fa811589c5731aed22a45f1dd67ed8643a1d318). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
AmplabJenkins commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728732814 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal
SparkQA commented on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-728732426 **[Test build #131187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131187/testReport)** for PR 30145 at commit [`6fa8115`](https://github.com/apache/spark/commit/6fa811589c5731aed22a45f1dd67ed8643a1d318). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead
SparkQA commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728732186 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35801/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
AmplabJenkins removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728731347 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131194/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
AmplabJenkins commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728731332 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
AmplabJenkins removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728731332 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LuciferYang commented on a change in pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports
LuciferYang commented on a change in pull request #30351:
URL: https://github.com/apache/spark/pull/30351#discussion_r524923149
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
##
@@ -894,10 +893,6 @@ trait ScalaReflection extends Logging {
import universe._
- // The Predef.Map is scala.collection.immutable.Map.
- // Since the map values can be mutable, we explicitly import
scala.collection.Map at here.
- import scala.collection.Map
-
Review comment:
Manual test `SPARK-4052: scala.collection.Map as value type of MapType`
, sql/catalyst module in Scala 2.13 passed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LuciferYang commented on a change in pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports
LuciferYang commented on a change in pull request #30351:
URL: https://github.com/apache/spark/pull/30351#discussion_r524923149
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
##
@@ -894,10 +893,6 @@ trait ScalaReflection extends Logging {
import universe._
- // The Predef.Map is scala.collection.immutable.Map.
- // Since the map values can be mutable, we explicitly import
scala.collection.Map at here.
- import scala.collection.Map
-
Review comment:
Manual test `SPARK-4052: scala.collection.Map as value type of MapType`
, sql/catalyst module , sql/core module in Scala 2.13 passed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
SparkQA removed a comment on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728683503 **[Test build #131194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131194/testReport)** for PR 30391 at commit [`08f7a1b`](https://github.com/apache/spark/commit/08f7a1bffb5dfc8a4f56ab8488fb39dfea63009c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
SparkQA commented on pull request #30391: URL: https://github.com/apache/spark/pull/30391#issuecomment-728730647 **[Test build #131194 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131194/testReport)** for PR 30391 at commit [`08f7a1b`](https://github.com/apache/spark/commit/08f7a1bffb5dfc8a4f56ab8488fb39dfea63009c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MichaelChirico commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
MichaelChirico commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728726963 > Adding magrittr as a dependency Best saved for a different thread, but TL;DR: yes, I think that's a good idea, `magrittr` will help writing `SparkR` code that's more similar to the Python API (replace `.` with `%>%`). It's a lightweight dependency too. However, do note that R is planning to implement a native pipe in the not-so-distant future, see [here](https://www.tidyverse.org/blog/2020/08/magrittr-2-0/) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524918338 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment: That works to me. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #30378: [SPARK-33454][INFRA] Add GitHub Action job for Hadoop 2
HyukjinKwon commented on pull request #30378: URL: https://github.com/apache/spark/pull/30378#issuecomment-728725027 There's a discussion thread in the mailing list about JDK 14 FYI: http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-on-JDK-14-td30348.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728724756 **[Test build #131202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131202/testReport)** for PR 28386 at commit [`3aa97f3`](https://github.com/apache/spark/commit/3aa97f369c3b3cb468a73d6bebc8b8c235094469). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MichaelChirico commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
MichaelChirico commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728724666 > Ideally, we'd like both to behave the same way (possibly resolving the problem with mutate), that's however a breaking change. A reasonable request (though definitely out of scope for this PR). My pushback would be that I assume `mutate` is designed to behave ~like `dplyr`, while the SQL auto-naming is designed to behave ~like SQL (whatever that means, the upshot is that I think users would expect consistency vis-a-vis Python/other APIs). So some divergence may be inevitable here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
AmplabJenkins removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728723077 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131200/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
AmplabJenkins removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728723065 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
SparkQA removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728718374 **[Test build #131200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131200/testReport)** for PR 28386 at commit [`d379971`](https://github.com/apache/spark/commit/d379971ca672f7f6cd3853238184132e3524f0be). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
AmplabJenkins commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728723065 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728723045 **[Test build #131200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131200/testReport)** for PR 28386 at commit [`d379971`](https://github.com/apache/spark/commit/d379971ca672f7f6cd3853238184132e3524f0be). * This patch **fails R style tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
SparkQA commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-728721434 **[Test build #131201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131201/testReport)** for PR 28647 at commit [`4b55575`](https://github.com/apache/spark/commit/4b555750488a5c5c77077dbb0aa98514eb04b03f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jsoref commented on pull request #30323: Spelling
jsoref commented on pull request #30323: URL: https://github.com/apache/spark/pull/30323#issuecomment-728721289 ok, so, part one is merged. `bin`, `repl`, and `streaming` seem to be the next least recently touched... What's do you recommend doing next? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
SparkQA commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728720126 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35800/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-728718374 **[Test build #131200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131200/testReport)** for PR 28386 at commit [`d379971`](https://github.com/apache/spark/commit/d379971ca672f7f6cd3853238184132e3524f0be). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MichaelChirico commented on a change in pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
MichaelChirico commented on a change in pull request #28386:
URL: https://github.com/apache/spark/pull/28386#discussion_r524910801
##
File path: R/pkg/R/DataFrame.R
##
@@ -3444,7 +3445,8 @@ setMethod("as.data.frame",
#' @note attach since 1.6.0
setMethod("attach",
Review comment:
good idea; added
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #30342: [MINOR][GRAPHX][3.0] Correct typos in the sub-modules: graphx, external, and examples
maropu commented on pull request #30342: URL: https://github.com/apache/spark/pull/30342#issuecomment-728715902 Thanks! Merged to branch-3.0. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu closed pull request #30342: [MINOR][GRAPHX][3.0] Correct typos in the sub-modules: graphx, external, and examples
maropu closed pull request #30342: URL: https://github.com/apache/spark/pull/30342 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30368: [SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
SparkQA commented on pull request #30368: URL: https://github.com/apache/spark/pull/30368#issuecomment-728715339 **[Test build #131199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131199/testReport)** for PR 30368 at commit [`45ab8b5`](https://github.com/apache/spark/commit/45ab8b549df1bdcc9f41fd4408c361a4bd58b97c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MichaelChirico commented on a change in pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate
MichaelChirico commented on a change in pull request #28386:
URL: https://github.com/apache/spark/pull/28386#discussion_r524908881
##
File path: R/pkg/R/DataFrame.R
##
@@ -3444,7 +3445,8 @@ setMethod("as.data.frame",
#' @note attach since 1.6.0
setMethod("attach",
signature(what = "SparkDataFrame"),
- function(what, pos = 2L, name = deparse(substitute(what), backtick =
FALSE),
+ function(what, pos = 2L,
+ name = paste(deparse(substitute(what), backtick = FALSE),
collapse = " "),
Review comment:
Sorry I think I responded to this in the main conversation thread rather
than here. Resolving.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30392: [SPARK-33465][CORE] RDD.takeOrdered should get rid of usage of reduce or use treeReduce instead
SparkQA commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-728712269 **[Test build #131198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131198/testReport)** for PR 30392 at commit [`59179f4`](https://github.com/apache/spark/commit/59179f47412e13411da2bc6010b9623f333498d4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]
AmplabJenkins removed a comment on pull request #30387: URL: https://github.com/apache/spark/pull/30387#issuecomment-728710246 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/35798/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
