AnthonyPoncet opened a new issue, #2897:
URL: https://github.com/apache/arrow-rs/issues/2897
**Describe the bug**
Using the file::reader::SerializedFileReader in the parquet crate, I could
not convert field of type timestamp with negative value (timestamp before UNIX
epoch) to string. Issue exists with both micros and millis types. And I guess
issue exist also while converting to JSON as it used same functions.
It seems to me that the i64 timestamp is converted into the Field enum as a
u64. Then while using chrono to format it as string, it convert it back to i64
**after** divinding by 1000 where it should be before (see proposed fix bellow).
**To Reproduce**
In test *test_convert_timestamp_to_string* in file
*parquet/src/record/api.rs*, change any tested date by something before 1970.
e.g: `check_datetime_conversion(1969, 1, 2, 13, 12, 54);`
-> Panic with the error `No such local time`
You could also create a parquet file with timestamp field and value before
1970 and then run
```
let file = File::open("<path>")?;
let reader = SerializedFileReader::new(file)?;
let mut row_iter = reader.get_row_iter(None)?;
while let Some(row) = row_iter.next() {
let _ = row.get_column_iter()
.map(|c| c.1.to_string())
.collect::<Vec<String>>();
}
```
**Expected behavior**
Handling date before UNIX epoch but also do not panic.
**Additional context**
In the function `convert_timestamp_millis_to_string` changing `let dt =
Utc.timestamp((value / 1000) as i64, 0);` to `let dt = Utc.timestamp(value as
i64 / 1000, 0);` solve the issue.
Now I do not know the codeline enough to know if the fix should be the fix
above or changing Field type to store a i64?
I could reproduce using both rust 1.63.0 and 1.64.0 on both Linux and
Windows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
