[
https://issues.apache.org/jira/browse/DRILL-7989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17404287#comment-17404287
]
ASF GitHub Bot commented on DRILL-7989:
---------------------------------------
luocooong commented on pull request #2299:
URL: https://github.com/apache/drill/pull/2299#issuecomment-905316748
@paul-rogers Thank you for the question. In short, the mongo storage plugin
of Drill supported both the BSON loader and JSON loader. The BSON loader is the
default handler, unless user set the `store.mongo.bson.record.reader` to
`false` (In this case, Drill will use the JSON loader to parse the JSON string).
https://github.com/apache/drill/blob/7d4d75a9bb9b4b697ef5bab74c2156810e31dc9b/contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java#L197-L204
We see that mongo driver will convert the data structure to JSON string
(enabled the JSON loader).
In the mongo-spec, the driver supported two JSON text output format :
`Canonical Format` and `Relaxed Format`. Notably, the Java driver use the
`Relaxed` format as output when call the `cursor.next().toJson()`. So we don't
see both of the following date format at the same time.
```java
// Canonical
{"$date":{"$numberLong":"1565546054692"}}
// Relaxed
{"$date":"2019-08-11T17:54:14.692Z"}
```
If application need the `Canonical` format, the mongo reader code is that :
```java
JsonWriterSettings settings =
JsonWriterSettings.builder().outputMode(JsonMode.EXTENDED).build();
String doc = cursor.next().toJson(settings);
```
It is showed that the JSON output format is depending on the application.
```java
// Canonical output
{"_id": {"$oid": "61226ac33b29ff6306b36aa9"}, "date": {"$date":
{"$numberLong": "1629645507258"}}}
// Relaxed output
{"_id": {"$oid": "61226ac33b29ff6306b36aa9"}, "date": {"$date":
"2021-08-22T15:18:27.258Z"}}
```
I think I've replied the first three items. Once the toJson() is done, the
next operation (parse & save to vector value) is no difference than query the
JSON files. The only issue is that we are now using an old version of JSON
loader, the new JSON loader contains more complete unit tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Use the UTC formatter in the JSON reader
> ----------------------------------------
>
> Key: DRILL-7989
> URL: https://issues.apache.org/jira/browse/DRILL-7989
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Cong Luo
> Assignee: Cong Luo
> Priority: Major
> Fix For: 1.20.0
>
>
> MongoDB use the UTC format to specify the date value by default. But the JSON
> reader (old version) use the fixed date formatter :
> "yyyy-MM-dd'T'HH:mm:ss.SSSXX". Need to change to the
> "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'" format.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
