+1 make sense to me as well. We should of course be fast for security upgrades, but make sense to avoid such eager upgrades for the rest of the hundreds of Spark dependencies, due to the increased supply chain attack risks in the ecosystem.
Thanks Szehon On Tue, Apr 21, 2026 at 3:32 AM Wenchen Fan <[email protected]> wrote: > Thanks for starting this discussion! I did a data analysis a while ago but > didn't have time to act on it. The analysis shows: > > *58* maven dep upgrades in the last 3 months. > *46%* (27/58) within 7 days of release > ≤7d : 27 / 58 (47%) > 8d–30d : 12 / 58 (21%) > >30d : 19 / 58 (32%) > > You can find the raw data in the attached file. This does look a bit > aggressive. I build Spark locally everyday, and I believe I'm not the only > one. Having a couple of weeks as the buffer time is a good idea to protect > developers like me from potential supply chain attacks. > > On Tue, Apr 21, 2026 at 6:24 AM Hyukjin Kwon <[email protected]> wrote: > >> SGTM I think it's good practice to give a couple of weeks before the >> upgrade >> >> On Tue, 21 Apr 2026 at 07:13, Tian Gao via dev <[email protected]> >> wrote: >> >>> Hi, I want to start a discussion about our dependency upgrade policy for >>> active development. >>> >>> Our current dependency upgrade (mostly for Java, but Python should be >>> included too) is a bit spontaneous. People find that a dependency has a new >>> version available and we just do the upgrade. >>> >>> This raises concerns about potential supply chain attacks. We already >>> established a few sets of rules (including pinning the github action >>> versions) to avoid the supply chain attack, but manually upgrading the >>> dependency version too eagerly could also be risky. >>> >>> It normally takes time for a bad release to be recognized, so I think we >>> should set a buffer time before upgrading to the latest version. For >>> example, we can wait a week or two after the latest release before we set >>> our development dependency to it. This could reduce the possibility of >>> being impacted by malicious releases, or just give them enough time to fix >>> their own severe bugs. >>> >>> The cost for this policy is very low - it barely impacts us if we can’t >>> use the “latest” version of dependencies. >>> >>> Of course, there should be exceptions when dependency upgrades include >>> security fixes for known vulnerabilities; we should upgrade as fast as >>> possible. >>> >>> Tian >>> >> > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected]
