[GitHub] [spark] AmplabJenkins removed a comment on pull request #27983: [SPARK-32105][SQL]Refactor current ScriptTransformationExec code

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #27983:
URL: https://github.com/apache/spark/pull/27983#issuecomment-655304581







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


SparkQA commented on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655304305


   **[Test build #125304 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125304/testReport)**
 for PR 28676 at commit 
[`126ee53`](https://github.com/apache/spark/commit/126ee53705fc72f2be8c93a086ae8cf814549184).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #27983: [SPARK-32105][SQL]Refactor current ScriptTransformationExec code

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #27983:
URL: https://github.com/apache/spark/pull/27983#issuecomment-655304581







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #27983: [SPARK-32105][SQL]Refactor current ScriptTransformationExec code

2020-07-07 Thread GitBox


SparkQA commented on pull request #27983:
URL: https://github.com/apache/spark/pull/27983#issuecomment-655304196


   **[Test build #125305 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125305/testReport)**
 for PR 27983 at commit 
[`f52f376`](https://github.com/apache/spark/commit/f52f376cb890043145517d36d05bbbe7cf4937cd).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] maropu commented on pull request #27983: [SPARK-32105][SQL]Refactor current ScriptTransformationExec code

2020-07-07 Thread GitBox


maropu commented on pull request #27983:
URL: https://github.com/apache/spark/pull/27983#issuecomment-655302337


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655301980







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655301980







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28683:
URL: https://github.com/apache/spark/pull/28683#issuecomment-655301976







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28683:
URL: https://github.com/apache/spark/pull/28683#issuecomment-655301976







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] maropu commented on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


maropu commented on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655301534


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-07 Thread GitBox


SparkQA commented on pull request #28683:
URL: https://github.com/apache/spark/pull/28683#issuecomment-655301621


   **[Test build #125303 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125303/testReport)**
 for PR 28683 at commit 
[`dbdab2d`](https://github.com/apache/spark/commit/dbdab2d72762af0e5e3301ee863f2e99d5bf28cd).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655301172


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125300/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28983:
URL: https://github.com/apache/spark/pull/28983#issuecomment-655301480


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655301165







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655301165


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


SparkQA commented on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655301119


   **[Test build #125300 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125300/testReport)**
 for PR 28676 at commit 
[`126ee53`](https://github.com/apache/spark/commit/126ee53705fc72f2be8c93a086ae8cf814549184).
* This patch **fails to generate documentation**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


SparkQA removed a comment on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655294644


   **[Test build #125300 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125300/testReport)**
 for PR 28676 at commit 
[`126ee53`](https://github.com/apache/spark/commit/126ee53705fc72f2be8c93a086ae8cf814549184).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] maropu commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-07 Thread GitBox


maropu commented on pull request #28683:
URL: https://github.com/apache/spark/pull/28683#issuecomment-655301155


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28983:
URL: https://github.com/apache/spark/pull/28983#issuecomment-655301480







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #29021: [WIP][SPARK-32201][SQL] More general skew join pattern matching

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #29021:
URL: https://github.com/apache/spark/pull/29021#issuecomment-655299960


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125293/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #29021: [WIP][SPARK-32201][SQL] More general skew join pattern matching

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #29021:
URL: https://github.com/apache/spark/pull/29021#issuecomment-655299953


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #29021: [WIP][SPARK-32201][SQL] More general skew join pattern matching

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #29021:
URL: https://github.com/apache/spark/pull/29021#issuecomment-655299953







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #29021: [WIP][SPARK-32201][SQL] More general skew join pattern matching

2020-07-07 Thread GitBox


SparkQA removed a comment on pull request #29021:
URL: https://github.com/apache/spark/pull/29021#issuecomment-655262748


   **[Test build #125293 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125293/testReport)**
 for PR 29021 at commit 
[`80bef0d`](https://github.com/apache/spark/commit/80bef0d2c22bf91d216784f29839b95f22fb230f).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #29021: [WIP][SPARK-32201][SQL] More general skew join pattern matching

2020-07-07 Thread GitBox


SparkQA commented on pull request #29021:
URL: https://github.com/apache/spark/pull/29021#issuecomment-655299619


   **[Test build #125293 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125293/testReport)**
 for PR 29021 at commit 
[`80bef0d`](https://github.com/apache/spark/commit/80bef0d2c22bf91d216784f29839b95f22fb230f).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28683:
URL: https://github.com/apache/spark/pull/28683#issuecomment-655298768


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125262/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #29026: [SPARK-28067][SPARK-32018] Fix decimal overflow issues

2020-07-07 Thread GitBox


SparkQA commented on pull request #29026:
URL: https://github.com/apache/spark/pull/29026#issuecomment-655299207


   **[Test build #125302 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125302/testReport)**
 for PR 29026 at commit 
[`3717fc6`](https://github.com/apache/spark/commit/3717fc618548d80464788d07a9b1f81f4418ed81).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28683:
URL: https://github.com/apache/spark/pull/28683#issuecomment-655298760


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28683:
URL: https://github.com/apache/spark/pull/28683#issuecomment-655298760







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] HyukjinKwon commented on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype

2020-07-07 Thread GitBox


HyukjinKwon commented on pull request #28833:
URL: https://github.com/apache/spark/pull/28833#issuecomment-655298519


   Thanks guys, sure. I will make a followup.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #29026: [SPARK-28067][SPARK-32018] Fix decimal overflow issues

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #29026:
URL: https://github.com/apache/spark/pull/29026#issuecomment-655297303







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-07 Thread GitBox


SparkQA removed a comment on pull request #28683:
URL: https://github.com/apache/spark/pull/28683#issuecomment-655165176


   **[Test build #125262 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125262/testReport)**
 for PR 28683 at commit 
[`f3d030f`](https://github.com/apache/spark/commit/f3d030ff28d4078fa9a6ea5a3a98fb58cf3cd11a).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #29026: [SPARK-28067][SPARK-32018] Fix decimal overflow issues

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #29026:
URL: https://github.com/apache/spark/pull/29026#issuecomment-655297303







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #29026: [SPARK-28067][SPARK-32018] Fix decimal overflow issues

2020-07-07 Thread GitBox


SparkQA commented on pull request #29026:
URL: https://github.com/apache/spark/pull/29026#issuecomment-655297096


   **[Test build #125301 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125301/testReport)**
 for PR 29026 at commit 
[`3717fc6`](https://github.com/apache/spark/commit/3717fc618548d80464788d07a9b1f81f4418ed81).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

2020-07-07 Thread GitBox


SparkQA commented on pull request #28683:
URL: https://github.com/apache/spark/pull/28683#issuecomment-655297380


   **[Test build #125262 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125262/testReport)**
 for PR 28683 at commit 
[`f3d030f`](https://github.com/apache/spark/commit/f3d030ff28d4078fa9a6ea5a3a98fb58cf3cd11a).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] HyukjinKwon commented on pull request #29026: [SPARK-28067][SPARK-32018] Fix decimal overflow issues

2020-07-07 Thread GitBox


HyukjinKwon commented on pull request #29026:
URL: https://github.com/apache/spark/pull/29026#issuecomment-655296767


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] HyukjinKwon edited a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


HyukjinKwon edited a comment on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655233411


   ~~One question 
https://github.com/apache/spark/pull/28993#discussion_r451234819. Otherwise, I 
am okay considering that it's still under development.~~ I misread. Looks good.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] HyukjinKwon edited a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


HyukjinKwon edited a comment on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655233411


   ~~One question 
https://github.com/apache/spark/pull/28993#discussion_r451234819. Otherwise, I 
am okay considering that it's still under development. ~~ I misread. Looks good.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] HyukjinKwon commented on a change in pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


HyukjinKwon commented on a change in pull request #28993:
URL: https://github.com/apache/spark/pull/28993#discussion_r451292098



##
File path: 
sql/catalyst/src/test/scala/org/apache/spark/sql/connector/InMemoryTable.scala
##
@@ -78,10 +92,44 @@ class InMemoryTable(
 throw new IllegalArgumentException(s"Unsupported type, 
${dataType.simpleString}")
 }
   } else {
-value
+(value, schema(index).dataType)
   }
 }
-partCols.map(fieldNames => extractor(fieldNames, schema, row))
+
+partitioning.map {
+  case IdentityTransform(ref) =>
+extractor(ref.fieldNames, schema, row)._1
+  case YearsTransform(ref) =>
+extractor(ref.fieldNames, schema, row) match {
+  case (days: Int, DateType) =>
+ChronoUnit.YEARS.between(EPOCH_LOCAL_DATE, 
DateTimeUtils.daysToLocalDate(days))
+  case (micros: Long, TimestampType) =>
+val localDate = 
DateTimeUtils.microsToInstant(micros).atZone(UTC).toLocalDate
+ChronoUnit.YEARS.between(EPOCH_LOCAL_DATE, localDate)

Review comment:
   Oh! It was in the test!





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #29026: [SPARK-28067][SPARK-32018] Fix decimal overflow issues

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #29026:
URL: https://github.com/apache/spark/pull/29026#issuecomment-655295039







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #29026: [SPARK-28067][SPARK-32018] Fix decimal overflow issues

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #29026:
URL: https://github.com/apache/spark/pull/29026#issuecomment-655295039







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655295026







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655295026







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype

2020-07-07 Thread GitBox


dongjoon-hyun edited a comment on pull request #28833:
URL: https://github.com/apache/spark/pull/28833#issuecomment-655295006


   Could you make a follow-up(full revert or partial revert) as what you 
suggest, @HyukjinKwon ?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] dongjoon-hyun commented on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype

2020-07-07 Thread GitBox


dongjoon-hyun commented on pull request #28833:
URL: https://github.com/apache/spark/pull/28833#issuecomment-655295006


   Could you make a follow-up as what you suggest, @HyukjinKwon ?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


SparkQA commented on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655294644


   **[Test build #125300 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125300/testReport)**
 for PR 28676 at commit 
[`126ee53`](https://github.com/apache/spark/commit/126ee53705fc72f2be8c93a086ae8cf814549184).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on pull request #29026: [SPARK-28067][SPARK-32018] Fix decimal overflow issues

2020-07-07 Thread GitBox


cloud-fan commented on pull request #29026:
URL: https://github.com/apache/spark/pull/29026#issuecomment-655294615


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] imback82 commented on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


imback82 commented on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655293059


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype

2020-07-07 Thread GitBox


cloud-fan commented on pull request #28833:
URL: https://github.com/apache/spark/pull/28833#issuecomment-655292960


   `NullType` is a stable public class, I don't think we can drop it.
   
   The intention is to only allow parsing `NullType` for the type string of 
legacy hive tables. But @HyukjinKwon is right that it also affects places like 
`from_csv`. Let's revert this part and think of a better solution.
   
   We don't document NullType in SQL reference. I think it's better to hide 
NullType from end-users. It's usually type-coercioned to other official types, 
and this PR forbids `NullType` if it leaks to the end (top columns). `df.show` 
is still OK to have `NullType` though. I agree that `NullType.simpleString` 
update can be put in a separate PR and discussed separately.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype

2020-07-07 Thread GitBox


dongjoon-hyun edited a comment on pull request #28833:
URL: https://github.com/apache/spark/pull/28833#issuecomment-655291242


   First of all, it's not a good idea to add `NullType` into a new Spark 
datatype officially. Not only the exposure causes more complexity, but also 
what can we do in Spark SQL world 
(https://spark.apache.org/docs/latest/sql-ref-datatypes.html) if that is an 
official type?
   > If we'll still care and have NullType, let's make it a proper type in 
Spark.
   
   Previously, this was supported until Apache Spark 2.0.0. After that, Apache 
Spark didn't support void. This PR also tried to forbid `VOID`. `AstBuilder` 
provides a way for graceful warning. Currently, we are very careful even in the 
error message, we didn't mention `void type`. We called it `unknown type`. I 
believe this PR is one way to implement your idea, too. Of course, we can add 
more messages, too.
   > If we're going to treat void as Hive legacy, let's don't support it at all 
and make the direction to deprecate and remove NullType away.
   
   In any way, since this is a legitimate suggestion from @HyukjinKwon , cc 
@gatorsmile , too.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] dongjoon-hyun commented on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype

2020-07-07 Thread GitBox


dongjoon-hyun commented on pull request #28833:
URL: https://github.com/apache/spark/pull/28833#issuecomment-655291242


   First of all, it's not a good idea to add `NullType` into a new Spark 
datatype officially. If that is an official type, what can we do in Spark SQL 
world (https://spark.apache.org/docs/latest/sql-ref-datatypes.html)?
   > If we'll still care and have NullType, let's make it a proper type in 
Spark.
   
   Previously, this was supported until Apache Spark 2.0.0. After that, Apache 
Spark didn't support void. This PR also tried to forbid `VOID`. `AstBuilder` 
provides a way for graceful warning. Currently, we are very careful even in the 
error message, we didn't mention `void type`. We called it `unknown type`. I 
believe this PR is one way to implement your idea, too. Of course, we can add 
more messages, too.
   > If we're going to treat void as Hive legacy, let's don't support it at all 
and make the direction to deprecate and remove NullType away.
   
   In any way, since this is a legitimate suggestion from @HyukjinKwon , cc 
@gatorsmile , too.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655289285


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125297/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655289277







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


SparkQA removed a comment on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655282058


   **[Test build #125297 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125297/testReport)**
 for PR 28993 at commit 
[`2efb84c`](https://github.com/apache/spark/commit/2efb84cf1abab6a67718501e79d00c4f63ecc8aa).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655289277


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


SparkQA commented on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655289237


   **[Test build #125297 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125297/testReport)**
 for PR 28993 at commit 
[`2efb84c`](https://github.com/apache/spark/commit/2efb84cf1abab6a67718501e79d00c4f63ecc8aa).
* This patch **fails to generate documentation**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28971: [WIP][SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28971:
URL: https://github.com/apache/spark/pull/28971#issuecomment-655287078


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125276/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28971: [WIP][SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28971:
URL: https://github.com/apache/spark/pull/28971#issuecomment-655287069


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28971: [WIP][SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28971:
URL: https://github.com/apache/spark/pull/28971#issuecomment-655287069







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28971: [WIP][SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13

2020-07-07 Thread GitBox


SparkQA removed a comment on pull request #28971:
URL: https://github.com/apache/spark/pull/28971#issuecomment-655227046


   **[Test build #125276 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125276/testReport)**
 for PR 28971 at commit 
[`89d19c6`](https://github.com/apache/spark/commit/89d19c60ace7fd2f74aa34e2ac8fe79c20883b41).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28971: [WIP][SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13

2020-07-07 Thread GitBox


SparkQA commented on pull request #28971:
URL: https://github.com/apache/spark/pull/28971#issuecomment-655285706


   **[Test build #125276 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125276/testReport)**
 for PR 28971 at commit 
[`89d19c6`](https://github.com/apache/spark/commit/89d19c60ace7fd2f74aa34e2ac8fe79c20883b41).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #29013: [SPARK-32196][SQL] Extract In convertible part if it is not convertible

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #29013:
URL: https://github.com/apache/spark/pull/29013#issuecomment-655284605







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #29022: [SPARK-32207][SQL] Support 'F'-suffixed Float Literals

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #29022:
URL: https://github.com/apache/spark/pull/29022#issuecomment-655284556







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #29022: [SPARK-32207][SQL] Support 'F'-suffixed Float Literals

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #29022:
URL: https://github.com/apache/spark/pull/29022#issuecomment-655284556







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #29013: [SPARK-32196][SQL] Extract In convertible part if it is not convertible

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #29013:
URL: https://github.com/apache/spark/pull/29013#issuecomment-655284605







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #29022: [SPARK-32207][SQL] Support 'F'-suffixed Float Literals

2020-07-07 Thread GitBox


SparkQA commented on pull request #29022:
URL: https://github.com/apache/spark/pull/29022#issuecomment-655284214


   **[Test build #125298 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125298/testReport)**
 for PR 29022 at commit 
[`4439bb6`](https://github.com/apache/spark/commit/4439bb63653cf756d24a2dfe6ab4b1609026a329).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #29013: [SPARK-32196][SQL] Extract In convertible part if it is not convertible

2020-07-07 Thread GitBox


SparkQA commented on pull request #29013:
URL: https://github.com/apache/spark/pull/29013#issuecomment-655284188


   **[Test build #125299 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125299/testReport)**
 for PR 29013 at commit 
[`9bf23cc`](https://github.com/apache/spark/commit/9bf23cc95e1a58044ce5c4e41348f93d99c705a6).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] sarutak commented on a change in pull request #28803: [SPARK-31971][WEBUI] Add pagination support for all jobs timeline

2020-07-07 Thread GitBox


sarutak commented on a change in pull request #28803:
URL: https://github.com/apache/spark/pull/28803#discussion_r451282622



##
File path: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala
##
@@ -184,20 +188,52 @@ private[ui] class AllJobsPage(parent: JobsTab, store: 
AppStatusStore) extends We
 val eventArrayAsStr =
   (jobEventJsonAsStrSeq ++ executorEventJsonAsStrSeq).mkString("[", ",", 
"]")
 
-
-  
-  
-Event Timeline
-  
- ++
-
-  
-
-  
-  Enable zooming
+if (totalPages > 0) {
+  
+
+
+  Event Timeline
+
+   ++
+
+  
+
+  
+  Enable zooming
+
+
+  
+Jobs:
+  {totalJobs}
+  .
+  {totalPages}
+  Pages. Jump to
+

[GitHub] [spark] gengliangwang commented on pull request #28803: [SPARK-31971][WEBUI] Add pagination support for all jobs timeline

2020-07-07 Thread GitBox


gengliangwang commented on pull request #28803:
URL: https://github.com/apache/spark/pull/28803#issuecomment-655283658


   @sarutak I am really sorry I missed your ping last week.
   The perf problem is resolved in https://github.com/apache/spark/pull/28806. 
If we want to avoid perf issue in the long term, I think we can avoid rendering 
the timeline by default. 
   I am +0 with the proposal. The pagination makes the visualization simpler, 
but also makes the UI usage more complex. If you insist, let's merge this one.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] sarutak commented on a change in pull request #28803: [SPARK-31971][WEBUI] Add pagination support for all jobs timeline

2020-07-07 Thread GitBox


sarutak commented on a change in pull request #28803:
URL: https://github.com/apache/spark/pull/28803#discussion_r451282321



##
File path: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala
##
@@ -184,20 +188,52 @@ private[ui] class AllJobsPage(parent: JobsTab, store: 
AppStatusStore) extends We
 val eventArrayAsStr =
   (jobEventJsonAsStrSeq ++ executorEventJsonAsStrSeq).mkString("[", ",", 
"]")
 
-
-  
-  
-Event Timeline
-  
- ++
-
-  
-
-  
-  Enable zooming
+if (totalPages > 0) {
+  
+
+
+  Event Timeline
+
+   ++
+
+  
+
+  
+  Enable zooming
+
+
+  

Review comment:
   Thanks. I'll consider it.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] gengliangwang commented on a change in pull request #28803: [SPARK-31971][WEBUI] Add pagination support for all jobs timeline

2020-07-07 Thread GitBox


gengliangwang commented on a change in pull request #28803:
URL: https://github.com/apache/spark/pull/28803#discussion_r451280315



##
File path: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala
##
@@ -184,20 +188,52 @@ private[ui] class AllJobsPage(parent: JobsTab, store: 
AppStatusStore) extends We
 val eventArrayAsStr =
   (jobEventJsonAsStrSeq ++ executorEventJsonAsStrSeq).mkString("[", ",", 
"]")
 
-
-  
-  
-Event Timeline
-  
- ++
-
-  
-
-  
-  Enable zooming
+if (totalPages > 0) {
+  
+
+
+  Event Timeline
+
+   ++
+
+  
+
+  
+  Enable zooming
+
+
+  

Review comment:
   nit: shall we move the style to webui.css?

##
File path: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala
##
@@ -184,20 +188,52 @@ private[ui] class AllJobsPage(parent: JobsTab, store: 
AppStatusStore) extends We
 val eventArrayAsStr =
   (jobEventJsonAsStrSeq ++ executorEventJsonAsStrSeq).mkString("[", ",", 
"]")
 
-
-  
-  
-Event Timeline
-  
- ++
-
-  
-
-  
-  Enable zooming
+if (totalPages > 0) {
+  
+
+
+  Event Timeline
+
+   ++
+
+  
+
+  
+  Enable zooming
+
+
+  
+Jobs:
+  {totalJobs}
+  .
+  {totalPages}
+  Pages. Jump to
+

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655282455







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #29022: [SPARK-32207][SQL] Support 'F'-suffixed Float Literals

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #29022:
URL: https://github.com/apache/spark/pull/29022#issuecomment-655281923


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125287/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28998:
URL: https://github.com/apache/spark/pull/28998#issuecomment-655282443







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] HyukjinKwon commented on pull request #29022: [SPARK-32207][SQL] Support 'F'-suffixed Float Literals

2020-07-07 Thread GitBox


HyukjinKwon commented on pull request #29022:
URL: https://github.com/apache/spark/pull/29022#issuecomment-655282638


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655282455







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28998:
URL: https://github.com/apache/spark/pull/28998#issuecomment-655282443







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28998:
URL: https://github.com/apache/spark/pull/28998#issuecomment-655282144







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #29022: [SPARK-32207][SQL] Support 'F'-suffixed Float Literals

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #29022:
URL: https://github.com/apache/spark/pull/29022#issuecomment-655281916


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-07 Thread GitBox


SparkQA commented on pull request #28998:
URL: https://github.com/apache/spark/pull/28998#issuecomment-655282159


   **[Test build #125296 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125296/testReport)**
 for PR 28998 at commit 
[`04c18a0`](https://github.com/apache/spark/commit/04c18a0d88db9878de707b6b0d467f32da70145a).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28998:
URL: https://github.com/apache/spark/pull/28998#issuecomment-655282144







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] MaxGekk commented on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-07 Thread GitBox


MaxGekk commented on pull request #28998:
URL: https://github.com/apache/spark/pull/28998#issuecomment-655281835


   jenkins, retest this, please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #29022: [SPARK-32207][SQL] Support 'F'-suffixed Float Literals

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #29022:
URL: https://github.com/apache/spark/pull/29022#issuecomment-655281916







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


SparkQA commented on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655282058


   **[Test build #125297 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125297/testReport)**
 for PR 28993 at commit 
[`2efb84c`](https://github.com/apache/spark/commit/2efb84cf1abab6a67718501e79d00c4f63ecc8aa).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


cloud-fan commented on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655281706


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


cloud-fan commented on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655281669


   > python/pyspark/mllib/tests/test_streaming_algorithms.py", line 461, in 
condition
   >self.assertGreater(errors[1] - errors[-1], 2)
   > AssertionError: 1.672640157855923 not greater than 2
   
   seems like a flaky test. @huaxingao  can you take a look?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655281116


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125256/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #29022: [SPARK-32207][SQL] Support 'F'-suffixed Float Literals

2020-07-07 Thread GitBox


SparkQA removed a comment on pull request #29022:
URL: https://github.com/apache/spark/pull/29022#issuecomment-655256843


   **[Test build #125287 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125287/testReport)**
 for PR 29022 at commit 
[`4439bb6`](https://github.com/apache/spark/commit/4439bb63653cf756d24a2dfe6ab4b1609026a329).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-07 Thread GitBox


SparkQA removed a comment on pull request #28998:
URL: https://github.com/apache/spark/pull/28998#issuecomment-655112434


   **[Test build #125253 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125253/testReport)**
 for PR 28998 at commit 
[`04c18a0`](https://github.com/apache/spark/commit/04c18a0d88db9878de707b6b0d467f32da70145a).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655281107


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] cloud-fan commented on a change in pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


cloud-fan commented on a change in pull request #28993:
URL: https://github.com/apache/spark/pull/28993#discussion_r451280565



##
File path: 
sql/catalyst/src/test/scala/org/apache/spark/sql/connector/InMemoryTable.scala
##
@@ -78,10 +92,44 @@ class InMemoryTable(
 throw new IllegalArgumentException(s"Unsupported type, 
${dataType.simpleString}")
 }
   } else {
-value
+(value, schema(index).dataType)
   }
 }
-partCols.map(fieldNames => extractor(fieldNames, schema, row))
+
+partitioning.map {
+  case IdentityTransform(ref) =>
+extractor(ref.fieldNames, schema, row)._1
+  case YearsTransform(ref) =>
+extractor(ref.fieldNames, schema, row) match {
+  case (days: Int, DateType) =>
+ChronoUnit.YEARS.between(EPOCH_LOCAL_DATE, 
DateTimeUtils.daysToLocalDate(days))
+  case (micros: Long, TimestampType) =>
+val localDate = 
DateTimeUtils.microsToInstant(micros).atZone(UTC).toLocalDate
+ChronoUnit.YEARS.between(EPOCH_LOCAL_DATE, localDate)

Review comment:
   It's a testing implementation. There is no default behavior. 
Partitioning expression just indicates how the scan can be faster with specific 
pushed filters. 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-07 Thread GitBox


SparkQA commented on pull request #28998:
URL: https://github.com/apache/spark/pull/28998#issuecomment-655281050


   **[Test build #125253 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125253/testReport)**
 for PR 28998 at commit 
[`04c18a0`](https://github.com/apache/spark/commit/04c18a0d88db9878de707b6b0d467f32da70145a).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
 * `sealed trait UTCTimestamp extends BinaryExpression with 
ImplicitCastInputTypes with NullIntolerant `



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #29022: [SPARK-32207][SQL] Support 'F'-suffixed Float Literals

2020-07-07 Thread GitBox


SparkQA commented on pull request #29022:
URL: https://github.com/apache/spark/pull/29022#issuecomment-655281081


   **[Test build #125287 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125287/testReport)**
 for PR 29022 at commit 
[`4439bb6`](https://github.com/apache/spark/commit/4439bb63653cf756d24a2dfe6ab4b1609026a329).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655281107







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] HyukjinKwon edited a comment on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype

2020-07-07 Thread GitBox


HyukjinKwon edited a comment on pull request #28833:
URL: https://github.com/apache/spark/pull/28833#issuecomment-655277714


   Re: https://github.com/apache/spark/pull/28833#discussion_r448165084
   
   Sorry I read the comments just now. So the decision here is we allow to 
parse `void` as `NullType` but doesn't allow it in some commands like `CREATE 
TABLE`s.
   
   How about other cases when we directly use DDL-formatted string as its type? 
These simple type strings can be used in many places such as `from_csv` (schema 
as DDL formatted string), `from_json` (schema as DDL formatted string), 
`createDataFrame` (Python), etc. However, `StructType.simpleString` cannot 
still be parsed as the valid types.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28803: [SPARK-31971][WEBUI] Add pagination support for all jobs timeline

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28803:
URL: https://github.com/apache/spark/pull/28803#issuecomment-655280269







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655280088


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125290/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


SparkQA removed a comment on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655258874


   **[Test build #125290 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125290/testReport)**
 for PR 28676 at commit 
[`126ee53`](https://github.com/apache/spark/commit/126ee53705fc72f2be8c93a086ae8cf814549184).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

2020-07-07 Thread GitBox


SparkQA removed a comment on pull request #28993:
URL: https://github.com/apache/spark/pull/28993#issuecomment-655129680


   **[Test build #125256 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125256/testReport)**
 for PR 28993 at commit 
[`2efb84c`](https://github.com/apache/spark/commit/2efb84cf1abab6a67718501e79d00c4f63ecc8aa).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28803: [SPARK-31971][WEBUI] Add pagination support for all jobs timeline

2020-07-07 Thread GitBox


AmplabJenkins commented on pull request #28803:
URL: https://github.com/apache/spark/pull/28803#issuecomment-655280269







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-07-07 Thread GitBox


AmplabJenkins removed a comment on pull request #28676:
URL: https://github.com/apache/spark/pull/28676#issuecomment-655280081


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



  1   2   3   4   5   6   7   8   9   10   >