[GitHub] [spark] AmplabJenkins commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632498700 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins removed a comment on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632498700 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
SparkQA removed a comment on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632444893 **[Test build #122959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122959/testReport)** for PR 28596 at commit [`2be2ca7`](https://github.com/apache/spark/commit/2be2ca728640ff68f4c9ff271d22ae67cf6db44f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
SparkQA commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632497810 **[Test build #122959 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122959/testReport)** for PR 28596 at commit [`2be2ca7`](https://github.com/apache/spark/commit/2be2ca728640ff68f4c9ff271d22ae67cf6db44f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] GuoPhilipse commented on pull request #28605: [SPARK-31790][DOCS] cast(long as timestamp) show different result between Hive and Spark
GuoPhilipse commented on pull request #28605: URL: https://github.com/apache/spark/pull/28605#issuecomment-632494495 :) have corrected and added examples in JIRA This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join
AmplabJenkins removed a comment on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632492175 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join
AmplabJenkins commented on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632492175 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join
SparkQA commented on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632491833 **[Test build #122966 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122966/testReport)** for PR 28556 at commit [`f720bdf`](https://github.com/apache/spark/commit/f720bdff8797bcc221e7ddb0cb4304d98319870c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
AmplabJenkins removed a comment on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632490453 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
AmplabJenkins commented on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632490453 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
SparkQA removed a comment on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632408912 **[Test build #122953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122953/testReport)** for PR 28604 at commit [`4615923`](https://github.com/apache/spark/commit/4615923e35f5f603cb5be9fc18c4a4228867bace). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
SparkQA commented on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632489847 **[Test build #122953 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122953/testReport)** for PR 28604 at commit [`4615923`](https://github.com/apache/spark/commit/4615923e35f5f603cb5be9fc18c4a4228867bace). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join
viirya commented on a change in pull request #28556:
URL: https://github.com/apache/spark/pull/28556#discussion_r429048929
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/SchemaPruningSuite.scala
##
@@ -338,6 +349,93 @@ abstract class SchemaPruningSuite
}
}
+ testSchemaPruning("select one deep nested complex field after repartition") {
+val query = sql("select * from contacts")
+ .repartition(100)
+ .where("employer.company.address is not null")
+ .selectExpr("employer.id as employer_id")
+checkScan(query,
+ "struct>>")
+checkAnswer(query, Row(0) :: Nil)
+ }
+
+ testSchemaPruning("select one deep nested complex field after repartition by
expression") {
+val query1 = sql("select * from contacts")
+ .repartition(100, col("id"))
+ .where("employer.company.address is not null")
+ .selectExpr("employer.id as employer_id")
+checkScan(query1,
+ "struct>>")
+checkAnswer(query1, Row(0) :: Nil)
+
+val query2 = sql("select * from contacts")
+ .repartition(100, col("employer"))
+ .where("employer.company.address is not null")
+ .selectExpr("employer.id as employer_id")
+checkScan(query2,
+
"struct>>")
+checkAnswer(query2, Row(0) :: Nil)
+
+val query3 = sql("select * from contacts")
+ .repartition(100, col("employer.company"))
+ .where("employer.company.address is not null")
+ .selectExpr("employer.company as employer_company")
+checkScan(query3,
+ "struct>>")
+checkAnswer(query3, Row(Row("abc", "123 Business Street")) :: Nil)
+
+val query4 = sql("select * from contacts")
+ .repartition(100, col("employer.company.address"))
+ .where("employer.company.address is not null")
+ .selectExpr("employer.company.address as employer_company_addr")
+checkScan(query4,
+ "struct>>")
+checkAnswer(query4, Row("123 Business Street") :: Nil)
+ }
+
+ testSchemaPruning("select one deep nested complex field after join") {
+val query1 = sql("select contacts.name.middle from contacts, departments
where " +
+"contacts.id = departments.contactId")
+checkScan(query1,
+ "struct>",
+"struct")
+checkAnswer(query1, Row("X.") :: Row("Y.") :: Nil)
+
+val query2 = sql("select contacts.name.middle from contacts, departments
where " +
Review comment:
Seems all tests in this test suite are using lowercases. Changing all
tests seems too bothering... :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28606: [MINOR][YARN]False report isAllNodeBlacklisted when RM is having issue
AmplabJenkins commented on pull request #28606: URL: https://github.com/apache/spark/pull/28606#issuecomment-632486820 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28606: [MINOR][YARN]False report isAllNodeBlacklisted when RM is having issue
AmplabJenkins removed a comment on pull request #28606: URL: https://github.com/apache/spark/pull/28606#issuecomment-632486456 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28606: [MINOR][YARN]False report isAllNodeBlacklisted when RM is having issue
AmplabJenkins commented on pull request #28606: URL: https://github.com/apache/spark/pull/28606#issuecomment-632486456 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cnZach opened a new pull request #28606: [MINOR][YARN]False report isAllNodeBlacklisted when RM is having issue
cnZach opened a new pull request #28606: URL: https://github.com/apache/spark/pull/28606 ### What changes were proposed in this pull request? Improve the check logic on if all node managers are really being backlisted. ### Why are the changes needed? I observed when the AM is out of sync with ResourceManager, or RM is having issue report back with current number of available NMs, something like below happens: ... 20/05/13 09:01:21 INFO RetryInvocationHandler: java.io.EOFException: End of File Exception between local host is: "client.zyx.com/x.x.x.124"; destination host is: "rm.zyx.com":8030; : java.io.EOFException; For more details see: http://wiki.apache.org/hadoop/EOFException, while invoking ApplicationMasterProtocolPBClientImpl.allocate over rm543. Trying to failover immediately. ... 20/05/13 09:01:28 WARN AMRMClientImpl: ApplicationMaster is out of sync with ResourceManager, hence resyncing. ... then the spark job would suddenly run into AllNodeBlacklisted state: ... 20/05/13 09:01:31 INFO ApplicationMaster: Final app status: FAILED, exitCode: 11, (reason: Due to executor failures all available nodes are blacklisted) ... but actually there's no black listed nodes in currentBlacklistedYarnNodes, and I do not see any blacklisting message from: https://github.com/apache/spark/blob/master/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala#L119 We should only return isAllNodeBlacklisted =true when we see there are >0 numClusterNodes AND 'currentBlacklistedYarnNodes.size >= numClusterNodes'. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? A minor change. No changes on tests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
AmplabJenkins commented on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632481978 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
AmplabJenkins removed a comment on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632481978 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
SparkQA commented on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632481523 **[Test build #122965 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122965/testReport)** for PR 28604 at commit [`ea9a527`](https://github.com/apache/spark/commit/ea9a5273abe4b9427e58d7e8082bbb255e4123cc). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
maropu commented on a change in pull request #28604: URL: https://github.com/apache/spark/pull/28604#discussion_r429042439 ## File path: sql/core/src/test/resources/sql-tests/inputs/string-functions.sql ## @@ -48,4 +48,8 @@ SELECT trim(LEADING 'xyz' FROM 'zzzytestxyz'); SELECT trim(LEADING 'xy' FROM 'xyxXxyLAST WORD'); SELECT trim(TRAILING 'xyz' FROM 'testxxzx'); SELECT trim(TRAILING 'xyz' FROM 'xyztestxxzx'); -SELECT trim(TRAILING 'xy' FROM 'TURNERyxXxy'); \ No newline at end of file +SELECT trim(TRAILING 'xy' FROM 'TURNERyxXxy'); + +-- Check lpad/rpad throw an exception for invalid length input Review comment: Ur, my bad. It looks better. I'll update it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
cloud-fan commented on a change in pull request #28604: URL: https://github.com/apache/spark/pull/28604#discussion_r429040810 ## File path: sql/core/src/test/resources/sql-tests/inputs/string-functions.sql ## @@ -48,4 +48,8 @@ SELECT trim(LEADING 'xyz' FROM 'zzzytestxyz'); SELECT trim(LEADING 'xy' FROM 'xyxXxyLAST WORD'); SELECT trim(TRAILING 'xyz' FROM 'testxxzx'); SELECT trim(TRAILING 'xyz' FROM 'xyztestxxzx'); -SELECT trim(TRAILING 'xy' FROM 'TURNERyxXxy'); \ No newline at end of file +SELECT trim(TRAILING 'xy' FROM 'TURNERyxXxy'); + +-- Check lpad/rpad throw an exception for invalid length input Review comment: it only throws exception under ansi mode. how about ``` -- Check lpad/rpad with invalid length parameter ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator
cloud-fan commented on a change in pull request #28600:
URL: https://github.com/apache/spark/pull/28600#discussion_r429039911
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala
##
@@ -423,13 +434,21 @@ case class IntegralDivide(
private lazy val div: (Any, Any) => Any = {
val integral = left.dataType match {
+ // if it is of Integer type than cast it to Long
+ case _: IntegerType =>
Review comment:
We need to handle byte/short as well. I think it's easier to update the
type coercion rules to do the cast at the analysis time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
AmplabJenkins removed a comment on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632474789 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
AmplabJenkins commented on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632474789 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
SparkQA commented on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632474270 **[Test build #122956 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122956/testReport)** for PR 28601 at commit [`4327940`](https://github.com/apache/spark/commit/43279402fbd10ee1cfecbc01d3fdec46f7cbc7d2). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
SparkQA removed a comment on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632421071 **[Test build #122956 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122956/testReport)** for PR 28601 at commit [`4327940`](https://github.com/apache/spark/commit/43279402fbd10ee1cfecbc01d3fdec46f7cbc7d2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sandeep-katta commented on a change in pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator
sandeep-katta commented on a change in pull request #28600:
URL: https://github.com/apache/spark/pull/28600#discussion_r429034719
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala
##
@@ -337,11 +340,19 @@ trait DivModLike extends BinaryArithmetic {
} else {
s"${eval2.value} == 0"
}
-val javaType = CodeGenerator.javaType(dataType)
+val isIntegralDiv = this.isInstanceOf[IntegralDivide]
+// From SPARK-16323 IntegralDivision returns Long data type
+val javaType = if (isIntegralDiv) JAVA_LONG else
CodeGenerator.javaType(dataType)
+val operandJavaType = if (isIntegralDiv) operandsDataType match {
+ case _: IntegerType => JAVA_LONG
Review comment:
okay will update the PR with `TypeCoercion` implementation
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator
HyukjinKwon commented on a change in pull request #28600:
URL: https://github.com/apache/spark/pull/28600#discussion_r429034529
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala
##
@@ -337,11 +340,19 @@ trait DivModLike extends BinaryArithmetic {
} else {
s"${eval2.value} == 0"
}
-val javaType = CodeGenerator.javaType(dataType)
+val isIntegralDiv = this.isInstanceOf[IntegralDivide]
+// From SPARK-16323 IntegralDivision returns Long data type
+val javaType = if (isIntegralDiv) JAVA_LONG else
CodeGenerator.javaType(dataType)
+val operandJavaType = if (isIntegralDiv) operandsDataType match {
+ case _: IntegerType => JAVA_LONG
Review comment:
Okay, allowing it seems making sense. It will also keep the
compatibility.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sandeep-katta commented on a change in pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator
sandeep-katta commented on a change in pull request #28600:
URL: https://github.com/apache/spark/pull/28600#discussion_r429034382
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala
##
@@ -337,11 +340,19 @@ trait DivModLike extends BinaryArithmetic {
} else {
s"${eval2.value} == 0"
}
-val javaType = CodeGenerator.javaType(dataType)
+val isIntegralDiv = this.isInstanceOf[IntegralDivide]
+// From SPARK-16323 IntegralDivision returns Long data type
+val javaType = if (isIntegralDiv) JAVA_LONG else
CodeGenerator.javaType(dataType)
+val operandJavaType = if (isIntegralDiv) operandsDataType match {
+ case _: IntegerType => JAVA_LONG
Review comment:
Behaviour is mixed w.r.t other DBMSes
since `IntegralDivide` returns Long, shall I cast to Long in `TypeCoercion` ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sandeep-katta commented on a change in pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator
sandeep-katta commented on a change in pull request #28600:
URL: https://github.com/apache/spark/pull/28600#discussion_r429033795
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala
##
@@ -337,11 +340,19 @@ trait DivModLike extends BinaryArithmetic {
} else {
s"${eval2.value} == 0"
}
-val javaType = CodeGenerator.javaType(dataType)
+val isIntegralDiv = this.isInstanceOf[IntegralDivide]
+// From SPARK-16323 IntegralDivision returns Long data type
+val javaType = if (isIntegralDiv) JAVA_LONG else
CodeGenerator.javaType(dataType)
+val operandJavaType = if (isIntegralDiv) operandsDataType match {
+ case _: IntegerType => JAVA_LONG
Review comment:
I ran the `select -2147483648 / -1 ` query in some of the databases,
results are as below
MySQL => 2147483648
PostgresSQl9.6 => Out of Integer Range exception
mariaDB => 2147483648
MS Sql Server 2017 => 2147483648
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28605: [SPARK-31790][DOCS] Document cast(ts as long) difference between Hive and Spark
HyukjinKwon commented on pull request #28605: URL: https://github.com/apache/spark/pull/28605#issuecomment-632467539 @GuoPhilipse, feel free to fix the PR title and PR description with examples. It's your PR :-) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
dongjoon-hyun edited a comment on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632467085 `WorkerDecommissionSuite` seems to be a flaky test which is irrelevant to this PR. ``` org.apache.spark.scheduler.WorkerDecommissionSuite.verify a task with all workers decommissioned succeeds ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
dongjoon-hyun commented on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632467085 `WorkerDecommissionSuite` is irrelevant to this PR. ``` org.apache.spark.scheduler.WorkerDecommissionSuite.verify a task with all workers decommissioned succeeds ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632465011 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ms-lolo commented on pull request #17455: [Spark-20044][Web UI] Support Spark UI behind front-end reverse proxy using a path prefix
ms-lolo commented on pull request #17455:
URL: https://github.com/apache/spark/pull/17455#issuecomment-632465049
So I'm definitely confused by the spark docs everywhere… is this still not
possible? I have a single domain with many spark clusters sitting behind it and
I would like to reach them using a path like `/spark/{some-id}`. There are
multiple settings in the config docs mentioning proxies but they just cause
chaos and make no sense to me. Can anyone point me in the right direction or
simply tell me this isn't possible? So far `spark.ui.proxyBase` seems like the
closest thing to what I want but this setting isn't even in the docs even
though it's mentioned as part of other configs (including brand new configs
making their way into 3.0!)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632465011 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
SparkQA commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632464712 **[Test build #122964 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122964/testReport)** for PR 28592 at commit [`8141ef9`](https://github.com/apache/spark/commit/8141ef946aed9d7fe97f1918d4f11698c3ece4c6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins removed a comment on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632463076 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632463093 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632463093 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632463076 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
SparkQA commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632462808 **[Test build #122963 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122963/testReport)** for PR 28592 at commit [`052bfad`](https://github.com/apache/spark/commit/052bfada8d7a0469c76f0e9aa4386c8bc5a304c1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
SparkQA commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632462754 **[Test build #122962 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122962/testReport)** for PR 28596 at commit [`021c0a5`](https://github.com/apache/spark/commit/021c0a50d0f6414a2c2c2f680d9ef508b4da2cdb). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files
SparkQA commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-632459220 **[Test build #122961 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122961/testReport)** for PR 28363 at commit [`fb4ce2c`](https://github.com/apache/spark/commit/fb4ce2c01f7347f6d3808d8d0943540a93b3162b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28579: [SPARK-31757][CORE] Improve HistoryServerDiskManager.updateAccessTime()
AmplabJenkins commented on pull request #28579: URL: https://github.com/apache/spark/pull/28579#issuecomment-632458377 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28579: [SPARK-31757][CORE] Improve HistoryServerDiskManager.updateAccessTime()
AmplabJenkins removed a comment on pull request #28579: URL: https://github.com/apache/spark/pull/28579#issuecomment-632458377 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files
AmplabJenkins removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-632457402 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28579: [SPARK-31757][CORE] Improve HistoryServerDiskManager.updateAccessTime()
SparkQA removed a comment on pull request #28579: URL: https://github.com/apache/spark/pull/28579#issuecomment-632390622 **[Test build #122951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122951/testReport)** for PR 28579 at commit [`f212f33`](https://github.com/apache/spark/commit/f212f33e9fe7f38e490f35f61e1e67fd466e5949). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files
AmplabJenkins commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-632457402 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28579: [SPARK-31757][CORE] Improve HistoryServerDiskManager.updateAccessTime()
SparkQA commented on pull request #28579: URL: https://github.com/apache/spark/pull/28579#issuecomment-632457408 **[Test build #122951 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122951/testReport)** for PR 28579 at commit [`f212f33`](https://github.com/apache/spark/commit/f212f33e9fe7f38e490f35f61e1e67fd466e5949). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files
AmplabJenkins removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-632455427 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122955/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] GuoPhilipse commented on pull request #28605: [SPARK-31790][DOCS] Document cast(ts as long) difference between Hive and Spark
GuoPhilipse commented on pull request #28605: URL: https://github.com/apache/spark/pull/28605#issuecomment-632455796 @HyukjinKwon cast(ts as long) may change to cast(long as timestamp) ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files
AmplabJenkins removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-632455422 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files
SparkQA removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-632417393 **[Test build #122955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122955/testReport)** for PR 28363 at commit [`d2e7ab3`](https://github.com/apache/spark/commit/d2e7ab35ac44dd461373e51a015b7a12e2c3af6e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files
AmplabJenkins commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-632455422 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files
SparkQA commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-632455277 **[Test build #122955 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122955/testReport)** for PR 28363 at commit [`d2e7ab3`](https://github.com/apache/spark/commit/d2e7ab35ac44dd461373e51a015b7a12e2c3af6e). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
Ngone51 commented on a change in pull request #28596:
URL: https://github.com/apache/spark/pull/28596#discussion_r429021273
##
File path:
core/src/test/scala/org/apache/spark/scheduler/BarrierTaskContextSuite.scala
##
@@ -69,12 +69,12 @@ class BarrierTaskContextSuite extends SparkFunSuite with
LocalSparkContext with
// Pass partitionId message in
val message: String = context.partitionId().toString
val messages: Array[String] = context.allGather(message)
- messages.toList.iterator
+ Iterator.single(messages.toList)
}
// Take a sorted list of all the partitionId messages
val messages = rdd2.collect().head
// All the task partitionIds are shared
-for((x, i) <- messages.view.zipWithIndex) assert(x.toString == i.toString)
+assert(messages === List("0", "1", "2", "3"))
Review comment:
make sense.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 commented on a change in pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
WeichenXu123 commented on a change in pull request #28596:
URL: https://github.com/apache/spark/pull/28596#discussion_r429020982
##
File path:
core/src/test/scala/org/apache/spark/scheduler/BarrierTaskContextSuite.scala
##
@@ -69,12 +69,12 @@ class BarrierTaskContextSuite extends SparkFunSuite with
LocalSparkContext with
// Pass partitionId message in
val message: String = context.partitionId().toString
val messages: Array[String] = context.allGather(message)
- messages.toList.iterator
+ Iterator.single(messages.toList)
}
// Take a sorted list of all the partitionId messages
val messages = rdd2.collect().head
// All the task partitionIds are shared
-for((x, i) <- messages.view.zipWithIndex) assert(x.toString == i.toString)
+assert(messages === List("0", "1", "2", "3"))
Review comment:
We'd better verify the whole rdd collect array, not only the head of
result array?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28605: [SPARK-31790][DOCS] Document cast(ts as long) difference between Hive and Spark
AmplabJenkins removed a comment on pull request #28605: URL: https://github.com/apache/spark/pull/28605#issuecomment-632444721 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
AmplabJenkins removed a comment on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632449921 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122952/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28602: Revert "[SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0"
AmplabJenkins removed a comment on pull request #28602: URL: https://github.com/apache/spark/pull/28602#issuecomment-632449546 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122948/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator
AmplabJenkins removed a comment on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-632445096 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
SparkQA removed a comment on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632399864 **[Test build #122952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122952/testReport)** for PR 28601 at commit [`4327940`](https://github.com/apache/spark/commit/43279402fbd10ee1cfecbc01d3fdec46f7cbc7d2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
AmplabJenkins removed a comment on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632451598 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
AmplabJenkins removed a comment on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632449911 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28602: Revert "[SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0"
SparkQA removed a comment on pull request #28602: URL: https://github.com/apache/spark/pull/28602#issuecomment-632388260 **[Test build #122948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122948/testReport)** for PR 28602 at commit [`4a72dff`](https://github.com/apache/spark/commit/4a72dffbc71723196874ee3cc68a26d1a0e8064b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins removed a comment on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632445170 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28602: Revert "[SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0"
AmplabJenkins removed a comment on pull request #28602: URL: https://github.com/apache/spark/pull/28602#issuecomment-632449539 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
SparkQA commented on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632453260 **[Test build #122960 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122960/testReport)** for PR 28604 at commit [`fae1cf4`](https://github.com/apache/spark/commit/fae1cf4b67a7d747136ed854af9c3a6618cf6b5e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on pull request #28603: [SPARK-31788][CORE] Fix UnionRDD of PairRDDs
HyukjinKwon edited a comment on pull request #28603: URL: https://github.com/apache/spark/pull/28603#issuecomment-632437124 Shell we also add a unit test? Also, please describe before/after this fix in "Does this PR introduce any user-facing change?". Technically I think this IS a user-facing behaviour changes from error to working case. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
AmplabJenkins commented on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632451598 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] liangz1 closed pull request #28504: [WIP][SPARK-20007] Make SparkR apply() functions robust to workers that return empty data.frame
liangz1 closed pull request #28504: URL: https://github.com/apache/spark/pull/28504 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
AmplabJenkins commented on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632449911 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28602: Revert "[SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0"
AmplabJenkins commented on pull request #28602: URL: https://github.com/apache/spark/pull/28602#issuecomment-632449539 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2
SparkQA commented on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632449476 **[Test build #122952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122952/testReport)** for PR 28601 at commit [`4327940`](https://github.com/apache/spark/commit/43279402fbd10ee1cfecbc01d3fdec46f7cbc7d2). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
HeartSaVioR commented on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-632449300 Hi @maropu, one question from curiosity. I guess you've already got +1 from @kiszk, so unless you haven't made non-trivial change afterwards why don't you just go merge? Is there something you'd like to get reviewed, or is there someone you want to get reviewed before merging? If you are waiting for anything it would be nice to describe what you're waiting for. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28602: Revert "[SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0"
SparkQA commented on pull request #28602: URL: https://github.com/apache/spark/pull/28602#issuecomment-632449083 **[Test build #122948 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122948/testReport)** for PR 28602 at commit [`4a72dff`](https://github.com/apache/spark/commit/4a72dffbc71723196874ee3cc68a26d1a0e8064b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] GuoPhilipse commented on pull request #28605: add docs for sql migration-guide
GuoPhilipse commented on pull request #28605: URL: https://github.com/apache/spark/pull/28605#issuecomment-632447975 fine, i will create a JIRA for it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input
maropu commented on a change in pull request #28604:
URL: https://github.com/apache/spark/pull/28604#discussion_r429014787
##
File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
##
@@ -49,6 +49,24 @@ class SQLQuerySuite extends QueryTest with
SharedSparkSession with AdaptiveSpark
setupTestData()
+ test("lpad/rpad should throw an exception for invalid length input if the
ANSI mode enabled") {
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator
AmplabJenkins commented on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-632445096 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632445170 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28605: add docs for sql migration-guide
AmplabJenkins commented on pull request #28605: URL: https://github.com/apache/spark/pull/28605#issuecomment-632445014 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
SparkQA commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632444893 **[Test build #122959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122959/testReport)** for PR 28596 at commit [`2be2ca7`](https://github.com/apache/spark/commit/2be2ca728640ff68f4c9ff271d22ae67cf6db44f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator
SparkQA removed a comment on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-632331581 **[Test build #122943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122943/testReport)** for PR 28600 at commit [`cac15d9`](https://github.com/apache/spark/commit/cac15d90f43ace50984699ac6c781f5ec25ef704). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28605: add docs for sql migration-guide
AmplabJenkins commented on pull request #28605: URL: https://github.com/apache/spark/pull/28605#issuecomment-632444721 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator
SparkQA commented on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-63287 **[Test build #122943 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122943/testReport)** for PR 28600 at commit [`cac15d9`](https://github.com/apache/spark/commit/cac15d90f43ace50984699ac6c781f5ec25ef704). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28605: add docs for sql migration-guide
HyukjinKwon commented on pull request #28605: URL: https://github.com/apache/spark/pull/28605#issuecomment-632443991 @GuoPhilipse, let's file a JIRA although it's doc only, it's user-facing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] GuoPhilipse opened a new pull request #28605: add docs for sql migration-guide
GuoPhilipse opened a new pull request #28605: URL: https://github.com/apache/spark/pull/28605 ### What changes were proposed in this pull request? add docs for sql migration-guide ### Why are the changes needed? let user know more about the scenarios in which Hive and Spark generate different results ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? no need to test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins removed a comment on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632443163 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632443163 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
Ngone51 commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632443041 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins removed a comment on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632441981 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122949/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins removed a comment on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632441975 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"
AmplabJenkins commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632441975 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28585: [SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0
AmplabJenkins removed a comment on pull request #28585: URL: https://github.com/apache/spark/pull/28585#issuecomment-632441762 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122950/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28585: [SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0
AmplabJenkins removed a comment on pull request #28585: URL: https://github.com/apache/spark/pull/28585#issuecomment-632441755 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28585: [SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0
AmplabJenkins commented on pull request #28585: URL: https://github.com/apache/spark/pull/28585#issuecomment-632441755 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28585: [SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0
SparkQA removed a comment on pull request #28585: URL: https://github.com/apache/spark/pull/28585#issuecomment-632390619 **[Test build #122950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122950/testReport)** for PR 28585 at commit [`f53735e`](https://github.com/apache/spark/commit/f53735e2e52a8cf3d70f162c3317c10383ae1aff). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28585: [SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0
SparkQA commented on pull request #28585: URL: https://github.com/apache/spark/pull/28585#issuecomment-632441264 **[Test build #122950 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122950/testReport)** for PR 28585 at commit [`f53735e`](https://github.com/apache/spark/commit/f53735e2e52a8cf3d70f162c3317c10383ae1aff). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
