[GitHub] spark pull request #15147: [SPARK-17545] [SQL] Handle additional time offset...
Github user nbeyer closed the pull request at: https://github.com/apache/spark/pull/15147 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...
Github user nbeyer commented on the issue: https://github.com/apache/spark/pull/15147 @HyukjinKwon Yeah, I agree, this PR doesn't really apply anymore. I'll work up a different JIRA and PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...
Github user nbeyer commented on the issue: https://github.com/apache/spark/pull/15147 @HyukjinKwon Based on my further reading of the code, I'd like to suggest that add a deprecation to the stringToTime method and then update the stringToTimestamp method, specifically here https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala#L356 to handle the "no colon" case. It is the stringToTimestamp method that is used by the 'cast'. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...
Github user nbeyer commented on the issue: https://github.com/apache/spark/pull/15147 @HyukjinKwon I'll have to try out the 2.0.1/master changes. Is this DateTimeUtils method no longer used anywhere then? The other scenario where I ran into trouble was using 'cast' on Column class. If the changes you mention fix all places where StringType containing ISO8601 variants can be massaged to TimestampType, then that may take care of everything. However, if there is still use of this DateTimeUtils, then I'd like to push forward with this PR as a suggestion. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #15147: [SPARK-17545] [SQL] Handle additional time offset...
GitHub user nbeyer opened a pull request: https://github.com/apache/spark/pull/15147 [SPARK-17545] [SQL] Handle additional time offset formats of ISO 8601 ## What changes were proposed in this pull request? Allows flexibility in handling additional ISO 8601 time offset variants. The current parsing of ISO 8601 is exclusive to W3C's datetime note (and XML Schema datetime). This change will allow offset to be handled as "HH:MM", "HHMM" and "HH". This is one suggested approach to handling these variants. The other suggestions are to switch back to SimpleDateFormat and utilize the 'X' pattern flag. Another suggestion is to wait for a future release of commons-lang. Either of these suggestions can be implemented later over top of this suggestion. ## How was this patch tested? The patch was tested by running all existing unit tests and by augmenting the existing tests with additional assertions. You can merge this pull request into a Git repository by running: $ git pull https://github.com/nbeyer/spark master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/15147.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #15147 commit db2e1f0c0b6bbf4683bbc2b876500141961becc0 Author: Nathan Beyer <nbe...@gmail.com> Date: 2016-09-19T02:23:05Z Handle additional time offset formats of ISO 8601 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
