[
https://issues.apache.org/jira/browse/ORC-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920183#comment-16920183
]
Yukihiro Okada edited comment on ORC-526 at 9/1/19 3:14 AM:
------------------------------------------------------------
Hi [~fgroffenorcl] ,
I sent a PR. Is this PR the same as your intention?
Hi [~owen.omalley],
Could you please take a look this PR if you get a chance?
[https://github.com/apache/orc/pull/425]
{code:java}
% cat t/some.json
{"myid":1,"time":"2019-10-28 07:34:07.123456"}
{"myid":2,"time":"2019-10-29 07:20:57.567891"}
% if [ -f output.orc ]; then rm -f output.orc ; fi && java -jar
tools/target/orc-tools-1.6.0-SNAPSHOT-uber.jar convert --schema
"struct<id:int,time:timestamp>" --timestampformat "yyyy-MM-dd HH:mm:ss.SSSSSS"
t/some.json && java -jar tools/target/orc-tools-1.6.0-SNAPSHOT-uber.jar data
output.orc
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
Processing t/some.json
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
Processing data file output.orc [length: 327]
{"id":null,"time":"2019-10-28 07:34:07.123456"}
{"id":null,"time":"2019-10-29 07:20:57.567891"}
________________________________________________________________________________________________________________________
{code}
was (Author: yuokada):
Hi [~fgroffenorcl] ,
I send a PR. Is this PR the same as your intention?
Hi [~owen.omalley],
Could you please take a look this PR if you get a chance?
[https://github.com/apache/orc/pull/425]
{code:java}
% cat t/some.json
{"myid":1,"time":"2019-10-28 07:34:07.123456"}
{"myid":2,"time":"2019-10-29 07:20:57.567891"}
% if [ -f output.orc ]; then rm -f output.orc ; fi && java -jar
tools/target/orc-tools-1.6.0-SNAPSHOT-uber.jar convert --schema
"struct<id:int,time:timestamp>" --timestampformat "yyyy-MM-dd HH:mm:ss.SSSSSS"
t/some.json && java -jar tools/target/orc-tools-1.6.0-SNAPSHOT-uber.jar data
output.orc
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
Processing t/some.json
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
Processing data file output.orc [length: 327]
{"id":null,"time":"2019-10-28 07:34:07.123456"}
{"id":null,"time":"2019-10-29 07:20:57.567891"}
________________________________________________________________________________________________________________________
{code}
> orc-tools convert does not respect second fractions
> ---------------------------------------------------
>
> Key: ORC-526
> URL: https://issues.apache.org/jira/browse/ORC-526
> Project: ORC
> Issue Type: Bug
> Components: tools
> Affects Versions: 1.2.0, 1.5.5, 1.6.0
> Reporter: Fabian Groffen
> Priority: Trivial
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{% cat timestamp2.csv }}
> {{2019-01-01 00:00:00.0000}}
> {{2015-01-01 00:00:00.0001}}
> {{2015-01-01 00:00:00.0000}}
> {{2014-12-31 23:59:59.9999}}
> {{1970-01-01 00:00:00.0001}}
> {{1970-01-01 00:00:00.0000}}
> {{1969-12-31 23:59:59.9999}}
> {{1969-12-31 23:59:59.0001}}
> {{1969-12-31 23:59:59.0000}}
> {{1969-12-31 23:59:58.9999}}
> {{% java -jar tools/target/orc-tools-1.6.0-SNAPSHOT-uber.jar convert --schema
> timestamp --timestampformat "yyyy-MM-dd HH:mm:ss.SSSS" timestamp2.csv -o
> timestamp2-tools16.orc}}
> Reading back the file generated above excludes the second fractions (nanos).
>
> Reason for this is that tools/convert/CsvReader.java (as well as
> tools/convert/JsonReader.java) do not consider the nanos. They just create a
> Timestamp and never call setNanos() method on it (in this case with the
> result of a getNano() call to the ZonedDateTime. This needs to be done in 4
> places. Unfortunately I am not allowed to make a contribution at this point,
> so I can only describe the problem. I can confirm with a change/fix like
> that, the nanos are stored in the resulting ORC file.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
