arjunsr1 commented on issue #13847:
URL: https://github.com/apache/arrow/issues/13847#issuecomment-1211418093
@drin , I was unable to use the cast method to change the date64 column to
type :timestamp or Arrow::TimeUnit::MILLI. It doesn't allow conversion to
TimeUnit. However, I believe that the issue may actually begin in
`csv_loader.rb`. I noticed that after loading from CSV, I have quantities
within the rows that look like this:
2022-08-10T09:50:04-07:00
But the header for that row ends up becoming a date64 [ms] quantity, which I
think is actually already losing the data. When I run this code:
`new_data_table['created_time'].data.chunks.collect { |chunk| chunk.to_s }` I
get the below as an output
`new_data_table['created_time'].data.chunks.collect { |chunk| chunk.to_s }`
["[\n 2022-08-10,\n 2022-08-10,\n 2022-08-10,\n 2022-08-10,\n
2022-08-10,\n 2022-08-10,\n 2022-08-10,\n 2022-08-10,\n 2022-08-10,\n
2022-08-10,\n 2022-08-10\n]"]
As you can see, the data chunk by chunk is already losing the time units.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
