For the record, the pull request was now merged in Arrow/Parquet C++. Thanks a lot Sarah!
Regards Antoine. On Fri, 22 Aug 2025 13:54:30 +0000 Sarah Gilmore <sgilm...@mathworks.com.INVALID> wrote: > Hi Antoine, > > Thanks for the clarification! I'll update the pull request based on your > feedback. > > Thanks again! > > Best, > Sarah > ________________________________ > From: Antoine Pitrou <anto...@python.org> > Sent: Thursday, August 21, 2025 11:42 AM > To: d...@parquet.incubator.apache.org <d...@parquet.incubator.apache.org> > Subject: Re: Purpose of the isAdjustedToUTC TIME parameter > > > Hi Sarah, > > My answer in two points: > > 1) Yes, it is fine to add an option so that all situations can be > covered. > 2) I don't think converted type compatibility is important anymore > (logical types were added to the Parquet spec in 2017!), so my vote is > for the default value of the option to be `false`. > > Regards > > Antoine. > > > On Thu, 21 Aug 2025 14:00:12 +0000 > Sarah Gilmore > <sgilm...@mathworks.com.INVALID> wrote: > > Hi Antoine, > > > > Thank you for the quick response and for clarifying the use case! > > > > I agree that it would be more accurate to write the Arrow Time type to > > Parquet with `isAdjustedToUTC=false`. My understanding is that this is not > > the default behavior due to the requirements specified by the Parquet > > Compatibility Guidelines regarding the deprecation of TIME_MILLIS and > > TIME_MICROS[1]. At least, that was my takeaway from this discussion[2], but > > please correct me if I'm wrong. > > > > Given this context, it seems adding a flag to the Arrow Parquet writer to > > let users control the `isAdjustedToUTC` value would be a reasonable > > enhancement. > > > > Thanks again! > > > > Best, > > > > Sarah > > > > [1] > > https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#deprecated-time-convertedtype<https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#deprecated-time-convertedtype> > > [2] > > https://github.com/apache/arrow/issues/41476#issuecomment-2088094499<https://github.com/apache/arrow/issues/41476#issuecomment-2088094499> > > ________________________________________ > > From: Antoine Pitrou <antoine-+zn9apsxkcednm+yrof...@public.gmane.org> > > Sent: Thursday, August 21, 2025 5:55 AM > > To: d...@parquet.incubator.apache.org <d...@parquet.incubator.apache.org> > > Subject: Re: Purpose of the isAdjustedToUTC TIME parameter > > > > > > Hi Sarah, > > > > Thanks for bringing this to our attention. I had missed the Arrow PR. > > > > As for the Parquet spec, the purpose of the `isAdjustedToUTC` flag > > seems clear: > > * if `isAdjustedToUTC` is true, the TIME value is expressed in > > reference to midnight in the UTC timezone > > * if `isAdjustedToUTC` is false, the TIME value is expressed in > > reference to midnight in an unknown timezone (presumably the "local > > timezone" where the measurements were made) > > > > As for the Arrow behavior, the Arrow Time type actually doesn't > > reference UTC. So, as I've argued on the PR, the Arrow Time type should > > probably be written to Parquet with `isAdjustedToUTC=false`. > > > > Regards > > > > Antoine. > > > > > > > > On Wed, 20 Aug 2025 14:18:59 +0000 > > Sarah Gilmore > > <sgilm...@mathworks.com.INVALID> wrote: > > > Hi all, > > > I recently opened a pull request[1] in the apache/arrow project to extend > > > the Arrow Parquet writer to support writing TIME data with > > > isAdjustedToUTC=false. For your reference, the Arrow Parquet writer > > > defaults to exporting Arrow time data as Parquet TIME data with > > > isAdjustedToUTC=true, even though Arrow time types are timezone-agnostic, > > > in order to follow the Parquet Compatibility Guidelines[2] regarding the > > > deprecation of ConvertedTypes TIME_MILLIS and TIME_MICROS. > > > However, progress on this pull request has stalled because the Arrow > > > community is confused about the purpose of the isAdjustedToUTC > > > parameter[3], and some members think the best strategy moving forward > > > would be to deprecate it[4]. > > > Could the Parquet community provide some background about why the TIME > > > type is parameterized on isAdjustedToUTC? Are there plans to deprecate > > > this parameter in the future? > > > Any information the community can provide would be much appreciated, > > > because it will help the Arrow community decide whether it makes sense to > > > add a flag for controlling this parameter to the Arrow Parquet writer. > > > I appreciate the community's time and thoughts on this issue. > > > Thank you! > > > Best, > > > Sarah Gilmore > > > [1] > > > https://github.com/apache/arrow/pull/47316<https://github.com/apache/arrow/pull/47316> > > > [2] > > > https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#deprecated-time-convertedtype<https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#deprecated-time-convertedtype> > > > [3] > > > https://github.com/apache/arrow/pull/43268#discussion_r1686692052<https://github.com/apache/arrow/pull/43268#discussion_r1686692052> > > > [4] > > > https://github.com/apache/arrow/pull/47316#issuecomment-3190380012<https://github.com/apache/arrow/pull/47316#issuecomment-3190380012> > > > > > > > > > > > > > > > > >
