Hello,

I integrated the arrow library to a larger project, and was testing doing 
exports/imports of the same tables to see if it behaved well. Doing this, I 
became aware that arrow DURATION types were exported as INT64 (as the 
corresponding number of µs if I remember correctly) in the parquet export, and 
then imported as INT64 types. So the parquet export loses the type for the 
DURATION fields.
Would not it be better to export the DURATION type as the parquet logical type 
"TIME_MICROS" (meaning TIME wit precision micro, as TIME_MICROS seems to be 
somewhat deprecated 
(https://apache.googlesource.com/parquet-format/+/refs/heads/bloom-filter/LogicalTypes.md))
 as is doing matlab (see 
https://fr.mathworks.com/help/matlab/import_export/datatype-mappings-matlab-parquet.html)
 ?

Best regards,
Louis C

Reply via email to