rok commented on issue #50667: URL: https://github.com/apache/arrow/issues/50667#issuecomment-5102500436
This is a large regression. Could you help narrow it down by: - Trying pd.read_parquet(path, engine="pyarrow", pre_buffer=False) and pre_buffer=True, and reporting the timings? The ParquetFile default changed between 24 and 25, although pd.read_parquet ultimately uses pq.read_table, whose default was already True in Arrow 24. - Reporting the pandas version and whether the DateTimeIndex is timezone-aware? Arrow 25 changed reconstructed named timezones from pytz to zoneinfo.ZoneInfo with pandas 3. If the cache compares timezone objects directly, cached ranges could be rejected even though the timestamps are equivalent. It would be useful to print type(df.index.tz), repr(df.index.tz), and the corresponding values for the requested timezone. - Clarifying whether “Query Length (s)” is operation duration or the time range requested from the upstream API? If it is the latter, this may indicate changed cache-hit behavior rather than slower Parquet I/O. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
