Adam's issues seemed related to the ListingTable API which Comet doesn't go through. That said, I can't reproduce the issue in DataFusion alone, so I'll retract my -1 and just defer the Comet upgrade until we can understand it. If it is a true performance regression in DF, I suspect we can do a 54.1.
On Fri, Jun 5, 2026 at 1:28 PM Andrew Lamb <[email protected]> wrote: > I believe Adam reported something similar in Vortex that was related to the > new limits on the metadata cache size: > https://github.com/apache/datafusion/issues/21080#issuecomment-4633904202 > > On Fri, Jun 5, 2026 at 1:24 PM Matt Butrovich <[email protected]> > wrote: > > > -1 (non-binding). > > > > Comet is seeing large performance regressions in TPC-H and TPC-DS SF1000 > > that seems related to metadata parsing in Parquet. At first I was > wondering > > if it had something to do with the predicate caching added in DF54, but I > > have no conclusions yet. If we can find a config to change and add to the > > upgrade guide, I could be swayed to remove my -1. I will continue to > > investigate today and try to reproduce in just DataFusion. > > > > > https://github.com/apache/datafusion/issues/21080#issuecomment-4633642413 > > > > On Fri, Jun 5, 2026 at 5:14 AM Marko Milenković <[email protected]> > > wrote: > > > > > +1 (binding) > > > > > > Thanks Matt, Andrew and all others involved > > > > > > > > > -- > > > Marko Milenković > > > > > > > > > > > > > > > > On 4 Jun 2026, at 00:32, Andrew Lamb <[email protected]> wrote: > > > > > > > > Hi, > > > > > > > > I would like to propose a release of Apache DataFusion version > 54.0.0. > > > > > > > > This release candidate is based on commit: > > > > 45d943dfb8699dc9cb9ef2320e955b73e3e6c03b [1] > > > > The proposed release tarball and signatures are hosted at [2]. > > > > The changelog is located at [3]. > > > > > > > > Please download, verify checksums and signatures, run the unit tests, > > and > > > > vote > > > > on the release. The vote will be open for at least 72 hours. > > > > > > > > Only votes from PMC members are binding, but all members of the > > community > > > > are > > > > encouraged to test the release and vote with "(non-binding)". > > > > > > > > The standard verification procedure is documented at > > > > > > > > > > https://github.com/apache/datafusion/blob/main/dev/release/README.md#verifying-release-candidates > > > > . > > > > > > > > [ ] +1 Release this as Apache DataFusion 54.0.0 > > > > [ ] +0 > > > > [ ] -1 Do not release this as Apache DataFusion 54.0.0 because... > > > > > > > > Here is my vote: > > > > > > > > +1 > > > > > > > > [1]: > > > > > > > > > > https://github.com/apache/datafusion/tree/45d943dfb8699dc9cb9ef2320e955b73e3e6c03b > > > > [2]: > > > > > > > > > > https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-54.0.0-rc1 > > > > [3]: > > > > > > > > > > https://github.com/apache/datafusion/blob/45d943dfb8699dc9cb9ef2320e955b73e3e6c03b/dev/changelog/54.0.0.md > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > >
