> On Mar 27, 2026, at 12:31 PM, Holden Karau <[email protected]> wrote: > > One possibility would be to make the pinned version optional (eg > pyspark[pinned]) or publish a separate constraints file for people to > optionally use with -c?
Perhaps I am misunderstanding your proposal, Holden, but this is possible today for people using modern Python packaging workflows that use lock files. In fact, it happens automatically; all transitive dependencies are pinned in the lock file, and this is by design. Furthermore, it is straightforward to add additional restrictions to your project spec (i.e. pyproject.toml) so that when the packaging tool builds the lock file, it does it with whatever restrictions you want that are specific to your project. That could include specific versions or version ranges of libraries to exclude, for example. I had to do this, for example, on a personal project that used PySpark Connect but which was pulling in a version of grpc that was generating a lot of log noise <https://github.com/grpc/grpc/issues/38336#issuecomment-2588422915>. I pinned the version of grpc in my project file and let the packaging tool resolve all the requirements across PySpark Connect and my custom restrictions. Nick
