[GitHub] [spark] MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603977578 jenkins, retest this, please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603977426 I think the failure doesn't related to my changes: https://github.com/apache/spark/pull/28016#issuecomment-603958162 . I will update the PR desciption shortly, so the PR will be ready for review. /cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603976469 > So, does this happen only in vectorized reader @dongjoon-hyun Correct, the regular reader uses `DateTimeUtils.fromJavaDate`: https://github.com/apache/spark/blob/300ec1a74cb14867c22e616e657566d510426331/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcDeserializer.scala#L111 And the PRs #27807 and #27980 introduced date rebasing in `fromJavaDate()`. Also, I fixed ORC writer, and added a round trip test for both vectorized and non-vectorized readers. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
