Any thoughts on deleting the package-lock.json during the preparation of the release package, so that it's not included in the source code package?
I think keeping the package-lock.json is important in the main and other long-lived branches for development purposes, but I'm still a bit concerned about it remaining in the source code package. The main concern is that if it stays in the package and contains a pinned vulnerable sub-dependency, we'd need to address that issue as soon as possible. Meanwhile, for end-users who would install the package through the npm registry (the preferred method) would have received the patched sub-dependency would already have been patched. In this case, the current published package would have zero vulnerabilities as there was no lock file. Just to make note, for end-users that had already installed the package, they would need to run npm upgrade or possibly just reinstall the package. Preparing a new source code package would then require bumping the version number. But what's the point of publishing a new version to the npm registry if it only includes a version bump and no actual code changes? Not saying this is the case all the time but there are cases where packages are pretty stable and do not actively receive any code changes to promote a release. I have an idea on how to remove the package-lock.json while keeping it in the main and long-lived branches, but it would result in an orphaned commit. Might increase steps for release and probably make automation harder. > In which case our response will be > "here is the link that explains how you can do it yourself, we never > release old versions because dependencies that were "latest" at the moment > of release have some vulnerabilities I'm not sure if this is a valid approach, but if we decide to keep the package-lock.json, we could include a note in all release blog posts or announcements suggesting that anyone using the source code package run the npm upgrade command to resolve any potential issues in the lock file, since the lock file is intended for development use only and may become stale over time. On Sat, Nov 1, 2025 at 2:32 AM Jarek Potiuk <[email protected]> wrote: > That is a very good set of questions - and I am glad we start discussing > it, > > I had a lot of thoughts about that recently - because of Airflow NPM > dependencies behaviour when comparing to - for example Python. I think we > largely figured good aproach for it in Python > > *Tl;DR; NPM and lock mechanism is a bit problematic for us and I think even > for convenience packages we look for ways to give our users ways to update > their dependencies for released convenience packages.* > > *First a bit of context - how we do Python (and convenience images) in > Airflow:* > > Let me explain how we do it in Airflow for Python, and how I am thinking we > can do something similar for NPM. > > For Airflow Python we almost always use lower-bound but non-upper-bound > dependencies - i.e we specify "pytorch>x.y.a" but we very, very rarely > (only when we have a good reason do `sqlalchemy>x.y.z,<2` - when we > **know** we are not compatible with Sqlalchemy 2. This usually happens when > our dependencies have a well defined release process, and usually > semver-like guarantees, and it also means that 1.x has patchlevel releases > with security fixes we can depend on. Whenever we upper-bound a dependency > we make a comment why, we open an issue describing it and add link to the > issue to the comment and the issue explains conditions and what is blocking > us from removing the upper-binding (and we periodically review and remove > those when conditions are met or even run a project to do so - like it > happens at the very moment with sqlalchemy 2 upgrade: [1] > > This means that our users are usually absolutely free to upgrade the > dependencies (after sufficient testing) - we are **not** blocking them. > This is a very strong policy we have and the main idea is that when we > release our software - applying any "future" fixes to CVEs in our > dependencies might heppen in two ways: > > * our future release will include those fixed versions (automatically as we > do not upper-bind and we are testing latest versions and highest > resolutions of our dependencies in canary build 8 times a day to make sure > we fix all incompatibilities > * USER is responsible to upgrade and test older releases and get those > dependencies upgraded > > Even in CRA - we are steward, not manufacturer - so it's manufacturers > responsibility to "put software on the market without known, exploitable > vulnerabilities" - so it's on THEM not us to make sure whenever a > vulnerability is found in one of our deps, to upgrade those. > > There are some nuances - of course any **new** version of such upper-bound > dependency might break airflow - that's why we produce a "golden set" of > dependencies that we knew as "working" during the release - and we publish > them as "constraints" that we recommend our users to use when they want to > install released airflow version in a reproducible way. Then, after they do > it, and if they want to upgrade to a new version of dependency that has > fixed a vulnerability they can do it "on top of that". The first paragraph > of our "installing from PyPI" describes this process here [2] > > Effectively it means that the users of ours have two choices: > > * if they want to upgrade dependencies in released software - in vast > majority of cases we are not blocking them from that and it's on them to > make sure it happens > * or they can wait until we release a new version of Airflow with (pretty > much automatically) those deps upgraded by us > > There are edge-cases of course that are handled individually but those are > exceptions rather than a rule even if in Airflow we have in total 750+ > dependencies - it works nicely with this setup if you ask me. And it not > only works for "sources" we release but also for convenience packages > (whl/sdist) that we also publish. > > *How is NPM different ?* > > Now - NPM is a different beast altogether - because by package.lock and > minifying the javascript with it, we are effectively blocking our users > from doing similar things - upgrading the npm deps - without actually > taking the sources, modifying them and rebuilding the packages. Which 95%+ > of our users will never do. I understand (and I am happy with) the fact > that ASF releases sources and the rest is just "convenience packages" - yet > many of our users (including the vast majority of future manufacturers who > will release Airflow) are mainly consuming our .whl package. We make them > reproducible and we also publish them at "https://downloads.apache.org"; - > similarly as our sources, so users can easily do binary artifact > comparison with our signed/checksummed binaries) - but they will have > really hard time if we tell them - if you want to upgrade NPM dependencies > you need to rebuild all packages from the sources and you need to modify > the package.lock in those sources before. That's mostly a non-go. They will > expect ASF to release updated versions with those dependencies upgraded and > even more - they will likely **demand** us to release patchlevels for old > releases (2.7.45, where 3.2.1 is already available) with those dependencies > bumped. I do not like it. Our policy for releases is that we release only > the latest MINOR branch with patchlevel fixes. Full stop. If a user wants > to update a dependency in an older version we want THEM to do it (and > manufacturers will be on a hook for that) - but we do not want us to have > to do anything other than tell manufacturers how they can do it and send > them to the page which explains how. And it should be easy for end users to > apply it. > > I want to figure out if that would be possible with NPM. My initial idea is > to produce a separate convenience source package with only things that npm > needs and make it possible to build such a "patch" package by anyone and > add a procedure explaining how you can build and test such a patch with > latest dependencies. Very similarly as we do with our docker containers. We > explain our users already - that when they want to update some of the > container images they use, we will generally **never** re-build those > released images, but they are absolutely free to rebuild the containers > themselves and use latest dependencies from debian available - also they > can extend such images with updated new version of python packages. We > have a dedicated chapter about it in our documentation [3] > > *Why do I think such a "npm-only rebuild" instructions are needed ?* > > Now - maybe such a tooling (to easily rebuild and replace packages for NPM > packages already exists ? Or maybe from our side it's just a matter of > writing some custom package builder script to do so? Such a tooling also > should include some rudimentary (or even quite comprehensive) regression > tests that will allow such a user to be reasonably sure that the newly > released dependencies do not break their build. > > I still have not looked at details - but for me having this is quite a game > changer that gives you worry-free release mechanism: > > * make sure that when you release - you bump deps to latest (fairly easy) > * when you release a version - freeze the latest "minor" release - and do > not worry about 3rd-party released in-between - you explicitly explain that > you "freeze" the versions (as we do with docker container) > * then, put it on the users and make it easy so that they can build and use > a hassle-free way (replacement of compiled code with NPM) that contains the > SAME code we released in that version but with updated NPM dependencies. > > This will give us a very powerful leverage to discuss things with > manufacturers who - undoubtedly - will start banging our doors at the day > CRA is going to be effective with "Release a new version of Airflow with > those dependency issues fixed ASAP:". In which case our response will be > "here is the link that explains how you can do it yourself, we never > release old versions because dependencies that were "latest" at the moment > of release have some vulnerabilities - but here is the instruction on how > you can do it yourself'. And BTW - We accept any donation to us if you > want our security team to work even better as currently it's purely > volunteers and your money as manufacturer would be very helpful to improve > our processes". > > I hope it might be helpful in the discussion (though probably far too long > for anyone to get thus far). > > Links referred to: > > [1] Issue with sqlalchemy 2 upgrade : > https://github.com/apache/airflow/issues/56735 > [2] Installing from PyPI using "golden set" + upgrading to latest Python > dependencies > > https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html > [3] What should I do if my security scan shows critical and high > vulnerabilities in the image? > > https://airflow.apache.org/docs/docker-stack/index.html#what-should-i-do-if-my-security-scan-shows-critical-and-high-vulnerabilities-in-the-image > > > > > > > On Fri, Oct 31, 2025 at 10:48 AM Piotr P. Karwasz < > [email protected]> > wrote: > > > Hi Bryan, > > > > On 31.10.2025 07:57, Bryan Ellis wrote: > > > Question: > > > > > > If a Dependabot PR appears during the middle of a release vote — and it > > > updates a sub-dependency to fix a security issue — **does that > invalidate > > > or terminate the vote?** > > > > > > No, only the release manager has the authority to cancel a vote. > > > > > What exactly are we voting on? > > > > > > 1. **The "release vote package"** — the tarball uploaded to `dist/dev`, > > > created using `npm pack`, representing what a consumer would receive > when > > > installing the package. > > > 2. **The Git release tag** — the tag created in the repository, which a > > > consumer could also install directly (e.g., `npm install > > > github:apache/project-repo#1.0.0`). > > > 3. **The repository clone** — a developer clones the repo, checks out > the > > > release tag, and runs `npm install` within the repository. > > > > > > This is consistent across the ASF: votes are cast on the source release > > package, which must be sufficient for a user to build and test the > > release [1]. Only the source package constitutes an official act of the > > Foundation. > > > > Compiled artifacts (such as those published to NPM) are provided as a > > convenience to users, but they are not considered an official ASF > release. > > > > These rules exist because, in compiled languages, it’s difficult to > > verify that a compiled artifact precisely matches the source. However, > > thanks to reproducible-build efforts, projects like Log4j also verify > > the reproducibility of JARs whenever possible. This verification is a > > prerequisite for allowing releases to be built via GitHub Actions (i.e., > > on infrastructure not owned or directly controlled by the release > manager). > > > > ## Vulnerabilities in dependencies > > > > If a vulnerability is disclosed in any dependency of the project, I > > generally follow this rule of thumb: > > > > - If you ship the dependency, you effectively own it. You should make a > > reasonable effort to release versions without “known exploitable > > vulnerabilities”. > > - If you do not ship the dependency, it’s sufficient to document its > > presence. You don’t need to block or reissue a release solely because > > of it. > > > > If your packages include an `npm-shrinkwrap.json` file, I would pause > > the release to assess the potential impact of the vulnerability. Without > > it, you can assume that users will receive the same Dependabot PR and > > handle the update independently. > > > > > > Piotr > > > > [1] https://www.apache.org/legal/release-policy.html#artifacts > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > [email protected] > > For additional commands, e-mail: > > [email protected] > > > > >
