Hi Antoine, Thanks for the clarification! I'll update the pull request based on your feedback.
Thanks again! Best, Sarah ________________________________ From: Antoine Pitrou <anto...@python.org> Sent: Thursday, August 21, 2025 11:42 AM To: d...@parquet.incubator.apache.org <d...@parquet.incubator.apache.org> Subject: Re: Purpose of the isAdjustedToUTC TIME parameter Hi Sarah, My answer in two points: 1) Yes, it is fine to add an option so that all situations can be covered. 2) I don't think converted type compatibility is important anymore (logical types were added to the Parquet spec in 2017!), so my vote is for the default value of the option to be `false`. Regards Antoine. On Thu, 21 Aug 2025 14:00:12 +0000 Sarah Gilmore <sgilm...@mathworks.com.INVALID> wrote: > Hi Antoine, > > Thank you for the quick response and for clarifying the use case! > > I agree that it would be more accurate to write the Arrow Time type to > Parquet with `isAdjustedToUTC=false`. My understanding is that this is not > the default behavior due to the requirements specified by the Parquet > Compatibility Guidelines regarding the deprecation of TIME_MILLIS and > TIME_MICROS[1]. At least, that was my takeaway from this discussion[2], but > please correct me if I'm wrong. > > Given this context, it seems adding a flag to the Arrow Parquet writer to let > users control the `isAdjustedToUTC` value would be a reasonable enhancement. > > Thanks again! > > Best, > > Sarah > > [1] > https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#deprecated-time-convertedtype<https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#deprecated-time-convertedtype> > [2] > https://github.com/apache/arrow/issues/41476#issuecomment-2088094499<https://github.com/apache/arrow/issues/41476#issuecomment-2088094499> > ________________________________________ > From: Antoine Pitrou <antoine-+zn9apsxkcednm+yrof...@public.gmane.org> > Sent: Thursday, August 21, 2025 5:55 AM > To: d...@parquet.incubator.apache.org <d...@parquet.incubator.apache.org> > Subject: Re: Purpose of the isAdjustedToUTC TIME parameter > > > Hi Sarah, > > Thanks for bringing this to our attention. I had missed the Arrow PR. > > As for the Parquet spec, the purpose of the `isAdjustedToUTC` flag > seems clear: > * if `isAdjustedToUTC` is true, the TIME value is expressed in > reference to midnight in the UTC timezone > * if `isAdjustedToUTC` is false, the TIME value is expressed in > reference to midnight in an unknown timezone (presumably the "local > timezone" where the measurements were made) > > As for the Arrow behavior, the Arrow Time type actually doesn't > reference UTC. So, as I've argued on the PR, the Arrow Time type should > probably be written to Parquet with `isAdjustedToUTC=false`. > > Regards > > Antoine. > > > > On Wed, 20 Aug 2025 14:18:59 +0000 > Sarah Gilmore > <sgilm...@mathworks.com.INVALID> wrote: > > Hi all, > > I recently opened a pull request[1] in the apache/arrow project to extend > > the Arrow Parquet writer to support writing TIME data with > > isAdjustedToUTC=false. For your reference, the Arrow Parquet writer > > defaults to exporting Arrow time data as Parquet TIME data with > > isAdjustedToUTC=true, even though Arrow time types are timezone-agnostic, > > in order to follow the Parquet Compatibility Guidelines[2] regarding the > > deprecation of ConvertedTypes TIME_MILLIS and TIME_MICROS. > > However, progress on this pull request has stalled because the Arrow > > community is confused about the purpose of the isAdjustedToUTC > > parameter[3], and some members think the best strategy moving forward would > > be to deprecate it[4]. > > Could the Parquet community provide some background about why the TIME type > > is parameterized on isAdjustedToUTC? Are there plans to deprecate this > > parameter in the future? > > Any information the community can provide would be much appreciated, > > because it will help the Arrow community decide whether it makes sense to > > add a flag for controlling this parameter to the Arrow Parquet writer. > > I appreciate the community's time and thoughts on this issue. > > Thank you! > > Best, > > Sarah Gilmore > > [1] > > https://github.com/apache/arrow/pull/47316<https://github.com/apache/arrow/pull/47316> > > [2] > > https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#deprecated-time-convertedtype<https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#deprecated-time-convertedtype> > > [3] > > https://github.com/apache/arrow/pull/43268#discussion_r1686692052<https://github.com/apache/arrow/pull/43268#discussion_r1686692052> > > [4] > > https://github.com/apache/arrow/pull/47316#issuecomment-3190380012<https://github.com/apache/arrow/pull/47316#issuecomment-3190380012> > > > > > > > >
