Hi Hugo > Hi Reza, > > On 1/26/26 22:21, Reza Housseini wrote: >> Hi Guixers >> >> I want to start again the discussion of converting propagated-inputs of >> Python packages to normal inputs. > > Basic question as a relative newcomer: why would you want to convert the > propagated-inputs to normal inputs? What would be the benefits? > > This is an honest question, because I (probably naively) see some drawbacks. > > From the manual: > > > propagated-inputs is similar to inputs, but the specified packages > > will be automatically installed to profiles alongside the package > > they belong to > > Isn't that always what a user wants? E.g. if I install scipy, I assume > I'll get numpy along with it.
This is still the case with my proposed solution, but instead of declaring this dependencies globally through merging of all the site-packages paths of all packages in the profile and setting GUIX_PYTHONPATH to this search paths, all dependencies of a package are "hard-coded" (in some way or another) in the package itself. > More importantly, it is my understanding that different packages in the > same profile would be able to use their own version of a specific > dependency. Because those inputs cannot conflict because they never end > up in the same profile. > > But I don't see how that feature would be useful in practice. E.g. I > definitely want all installed Python libraries to use the exact same > numpy installation. I can imagine a whole load of problems with passing > data around between the-same-but-different-versions of a package. In general this boils down to define dependencies globally (propagated-inputs) or per package (inputs). I argue doing things per package reduces complexity. It may be true that most Python packages should use the same lower dependency versions in a profile, but I think first that this it is not the role of propagated-inputs to enforce this behavior. Second, that Python packages are normally tested against a specific dependency version and should strive to use this version and this may easily conflict with other packages and third that there are valid reasons to mix package versions in the same profile, e.g. for testing reasons. So in short, this gives us flexibility to use different package versions if we need to, but still allows us to enforce a "global" dependency version if we want to. > Or would all libraries use the numpy version of whichever library was > imported first? Since the dependency will then be cached somewhere. That > would be even worse. Caching is the tricky part. I think we need to disable caching for guix python packages or alter the caching mechanism to make this work. > How would typing work? E.g. isinstance()? > > Those dependency problems are exactly the kind of problems I'm trying to > get away from with Guix. > > To be clear: maybe your suggestion for using normal inputs is great; I'm > too inexperienced to argue either way. My main question comes from > ignorance: why would it be useful to use normal inputs of > propagated-inputs for Python packages? > > Hugo > > >> I remembered that there was a >> discussion once on the mailing list but I could not find it anymore. >> >> The general idea to make propagated inputs, normal inputs is the >> following: Every Python package hard codes it's inputs dependency paths >> in a file in the site-packages directory (could also be in METADATA or >> any other similar file). We add an additional finder to the import >> machinery, either to sys.meta_path or sys.path_hooks or we modify the >> existing PathFinder. The new or modified finder has one addition: if an >> import happens from a package residing inside the guix store, it first >> checks if the file or location of the hard coded dependecies are present >> and loads the module from there before processing with the standard >> import process. >> >> I think the actual difficult part is to single out imports coming from >> files inside the store and than finding the package root to lookup the >> hardcoded dependency paths and to make this process somewhat fast >> (although once the module is loaded, we can rely on the caching of >> sys.modules). The hooking into the import needs to be patched for each >> Python version we support (probably somewhere here [0]) but it may not >> be that immersive. >> >> What do you think? This needs probably a GCD if we ever want to go this >> path, but I am not sure if this suggestion is completely wrong or maybe >> undesired or was already suggested and discarded. >> >> Best, >> Reza >> >> [0] >> https://github.com/python/cpython/blob/main/Lib/importlib/_bootstrap_external.py >>
