On Tue, 26 Sep 2017, Gary E. Miller via devel wrote (different thread): > On Tue, 26 Sep 2017 22:04:53 -0400 (EDT) > "Eric S. Raymond via devel" <devel@ntpsec.org> wrote: > > > 2. Keep Fred's patch. Ship 1.0 with FHS non-conformance as a known > > and documented bug. > > Gack. Opening a tech support nightmare. > > We constanlty have issues with conflicting system installed and user > installed ntpd. it will be a lot of fun when the distro updates > ntpd and breaks the user installed ntpd. That was not a problem > before this patch.
Interestingly enough, this last paragraph is both completely irrelevant to the issue at hand, and simultaneously gets to the crux of the matter. :-) The whole issue being discussed here is the install location for *Python libraries*. Nothing else. Since the classic NTP package doesn't use Python at all, a conflict in this area is impossible. There might, of course, be conflicts in *programs*, *config files*, or whatever, but that has absolutely nothing to do with the Python library path. *However*, once distros start including ntpsec instead of classic ntpd, this sort of conflict will become possible, and FHS compliance will matter. But if we can assume that no distro is including a pre-1.0 ntpsec, then the conflict doesn't exist for 1.0 by definition. If we can punt on FHS compliance for 1.0, then that removes the time pressure for figuring out the right way to get Linux Python to play nicely with FHS. > > 3. Hold 1.0 until we can write a replacement get_python_lib() that > > works right (e.g. produces an FHS-conformant path set by default.) > > Before someone rewrites get_python_lib() we better agree on what it > should do. I suspect the solution will require much more than just > changes to get_python_lib(). Agreed. On Thu, 28 Sep 2017, Gary E. Miller via devel wrote: > On Wed, 27 Sep 2017 19:02:23 -0700 (PDT) > Fred Wright via devel <devel@ntpsec.org> wrote: > > > One of the ways to do #1 is to use the path returned by > > get_python_lib() without the prefix option. This is what GPSD has > > done for years. > > And caused subtl problems for years. Can you name any actual problems, aside from offending the FHS gods? If it caused so many problems, why were you apparently unaware of it until I pointed it out? :-) > > 2.2) Make use of some sort of hook to augment sys.path, which is what > > pip appears to do. So far, this approach doesn't look promising, > > partly because the only such mechanisms seem to require that the hook > > itself be in a "non-local" location in order to be seen by Python > > before being applied. A chicken-and-egg problem. > > Huh? Looks simple to me. Looks like the preferred solution. > > Since pip, and many other programs do it, that is a lot of good precendent. Precedent, yes. Full understanding of the mechanism and constraints, no. > > 2.3) Add code to the programs to augment sys.path prior to the import. > > Gack. Breaks in a large number of ways. It is very common for > packaers to build to one path, package the results, then install > elsewhere. It would naturally be based on the *final* install location, not the intermediate. I believe packaging systems normally know the final location at package build time, but I could be wrong about that. If the final install location isn't known at the time the package is built, then *any* kind of sys.path adjustment would have to be handled by the packaging system, since the ntpsec build scripts would have no way to know what to do. This applies whether one is patching code, generating special hook files, or whatever. I'm not saying tat I'm especially fond of this solution, but it is something that would work. Another possibility that occurred to me is placing the library directory in the *program* directory, e.g., /usr/local/bin/ntp/, since module directories parallel to a Python program are always recognized automatically by Python. This would work, but it's rather nonstandard. If ntpsec assumes that it "owns" all paths of the form <prefix>/bin/ntp*, then this wouldn't create any new conflicts, and AFAIK the NTP suite has never had a program just called "ntp", anyway. Again, a not very flavorful, but functional solution. I attempted to see if there's any precedent for this, but the systems I have here don't seem to be terribly useful examples. My Ubuntu install has exactly one program in /usr/local/bin/. My CentOS and Fedora installs don't even have /usr/local/bin/. My Debian installs have lots of things in /usr/local/bin/, but the only Python code is either from ntpsec or GPSD, so it's not really an "outside opinion". One of the this I've discovered in looking at this stuff is that having code that relies on looking at sys.path would be fragile, for at least two reasons: 1) Some directories get added by installs of other packages, but relying on any such directories would break if the relevant packages were removed. 2) Directories that don't exist don't get added. But the install process creates directories as needed. So if one came up with a directory that Python knows about but doesn't currently exist, it would appear from looking at sys.path at configure time that Python doesn't know about it, even though it might work perfectly well after the install. Note that a workaround for #1 is to launch Python with the -S option to suppress processing site.py, but that also may exclude some directories that *aren't* package-specific. BTW, it's worth pointing out that this entire argument is over the *default* value of PYTHONDIR (and technically also PYTHONARCHDIR, but that's currently unused as noted in the comment in pylib/wscript). Any user or packaging system is free to supply a different value, either via environment or via option. At that point, it becomes the responsibility of the user or packaging system to ensure that the specified directory is either valid by default, or will be made valid when it's needed. In fact, I suspect packaging systems would tend to specify a lot of such things explicitly in their own scripts, anyway, so this is mostly about getting reasonable default behavior for a "bare" configure/build/install. Fred Wright _______________________________________________ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
