cxzl25 commented on code in PR #1902:
URL: https://github.com/apache/orc/pull/1902#discussion_r1596459546
##########
java/bench/core/src/java/org/apache/orc/bench/core/convert/parquet/ParquetReader.java:
##########
@@ -54,8 +56,19 @@ public boolean nextBatch(VectorizedRowBatch batch) throws
IOException {
break;
}
int row = batch.size++;
- for(int c=0; c < converters.length; ++c) {
- converters[c].convert(batch.cols[c], row, value.get(c));
+ for (int c = 0; c < converters.length; ++c) {
+ Object o = value.get(c);
+ if (convertMicroseconds[c] == null) {
+ String fieldSchema =
value.getSchema().getFields().get(c).schema().toString();
+ convertMicroseconds[c] = fieldSchema.equals(
+
"[\"null\",{\"type\":\"long\",\"logicalType\":\"timestamp-micros\"}]");
+ }
+ if (convertMicroseconds[c]) {
+ // The data format of taxi is parquet, and timestamp uses
microseconds,
+ // which is inconsistent with the milliseconds of Java's
java.sql.Timestamp.
+ o = (Long) o / 1000;
Review Comment:
I implemented it in another way at https://github.com/apache/orc/pull/1930,
which will be simpler than the current changes.
Write and read json, convert timestamp type to long type instead of string
type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
