[
https://issues.apache.org/jira/browse/DRILL-7989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402912#comment-17402912
]
ASF GitHub Bot commented on DRILL-7989:
---------------------------------------
paul-rogers commented on a change in pull request #2299:
URL: https://github.com/apache/drill/pull/2299#discussion_r693601366
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/VectorOutput.java
##########
@@ -248,8 +249,8 @@ public void writeTimestamp(boolean isNull) throws
IOException {
ts.writeTimeStamp(dt.getMillis());
break;
case VALUE_STRING:
- OffsetDateTime originalDateTime =
OffsetDateTime.parse(parser.getValueAsString(), DateUtility.isoFormatTimeStamp);
- OffsetDateTime utcDateTime =
OffsetDateTime.of(originalDateTime.toLocalDateTime(), ZoneOffset.UTC); //
strips the time zone from the original
+ LocalDateTime localDateTime =
LocalDateTime.parse(parser.getValueAsString(), DateUtility.UTC_FORMATTER);
+ OffsetDateTime utcDateTime = OffsetDateTime.of(localDateTime,
ZoneOffset.UTC); // strips the time zone from the original
Review comment:
This one is going to be tricky! The incoming date is in UTC. So, we have
to use the offset for the local time zone to convert the UTC date/time to local
time. We then write local time into the value vector. See [this
file](https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/values/TimestampValueListener.java)
for how the new JSON parser does this task.
##########
File path: exec/java-exec/src/test/java/org/apache/drill/TestFrameworkTest.java
##########
@@ -219,7 +221,8 @@ public void
testBaselineValsVerificationWithComplexAndNulls() throws Exception {
mapOf("pink", "purple")),
listOf(4l, 2l),
listOf(listOf(2l, 1l),
- listOf(4l, 6l)))
+ listOf(4l, 6l)),
+ LocalDateTime.parse("2019-09-30T20:47:43.123Z",
DateUtility.UTC_FORMATTER))
Review comment:
This isn't right. The data file has this same timestamp in UTC. The date
that should appear in Drill is that same time converted to the local time zone.
See [this
test](https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/drill/exec/store/easy/json/loader/TestExtendedTypes.java#L180)
for how the work is done for the new JSON parser.
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/VectorOutput.java
##########
@@ -248,8 +249,8 @@ public void writeTimestamp(boolean isNull) throws
IOException {
ts.writeTimeStamp(dt.getMillis());
break;
case VALUE_STRING:
- OffsetDateTime originalDateTime =
OffsetDateTime.parse(parser.getValueAsString(), DateUtility.isoFormatTimeStamp);
- OffsetDateTime utcDateTime =
OffsetDateTime.of(originalDateTime.toLocalDateTime(), ZoneOffset.UTC); //
strips the time zone from the original
+ LocalDateTime localDateTime =
LocalDateTime.parse(parser.getValueAsString(), DateUtility.UTC_FORMATTER);
+ OffsetDateTime utcDateTime = OffsetDateTime.of(localDateTime,
ZoneOffset.UTC); // strips the time zone from the original
Review comment:
Also, please add a comment to say why we're doing things this way.
Perhaps reference the Mongo specs. Links are in the file mentioned in the
previous comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Use the UTC formatter in the JSON reader
> ----------------------------------------
>
> Key: DRILL-7989
> URL: https://issues.apache.org/jira/browse/DRILL-7989
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Cong Luo
> Assignee: Cong Luo
> Priority: Major
> Fix For: 1.20.0
>
>
> MongoDB use the UTC format to specify the date value by default. But the JSON
> reader (old version) use the fixed date formatter :
> "yyyy-MM-dd'T'HH:mm:ss.SSSXX". Need to change to the
> "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'" format.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
