OK, I was able to revert everything to a version before I applied the OpenEmbedded/Yocto patch I proposed to them, that contained the dpkg patch I proposed here and rerun the entire scenario until I was able to reproduce the issue. It's an even more complicated set of circumstances than I suspected.
The return value from scandir when it actually fails in the wild is EACCES, not ENOTDIR. When I was troubleshooting the method I used to simulate the problem didn't quite match the real scenario, but was close enough to "fix" (e.g. workaround) both. Sorry for the confusion. This is made even more weird because when the failure occurs dpkg is running under apt-get and both are running under a pseudo/fakeroot where a LDPRELOAD shim tricks the user processes into thinking they are root (https://www.yoctoproject.org/tools-resources/projects/pseudo). I'm not sure if this is why we're getting an unusual return value from scandir or if the manual page is inaccurate (POSIX has EACCES listed), but at a minimum it is something to be aware of. In this case the entire directory tree was removed AND some of the higher level directories did not have group permissions sufficient for the other user to traverse. I don't think its fair to complain if dpkg does something wrong when running in this highly unusual environment, but I really appreciate the offer to add a feature for our cross-builds to make it more robust. It will avoid having either OE/Yocto or me carry around a downstream patch of, as you point out, dubious robustness. Conclusions: 1. ignore this patch request except for history/discussion purposes 2. the willingness of dpkg to ignore "weird" errors from reading the config directories is a permissive as it should be. I'm especially convinced by the argument that you shouldn't continue with a 1/2 baked configuration. Presumably and empty cfg.d is by design, but permissions and other errors should cause a failure. 3. rather than a workaround to allow dpkg-native to continue past the error in our very unusual cross environment, a better approach would be to fix the root cause and avoid the hard-coded path to another users or a obsolete or random cfg.d directory in the first place by adding a command-line option. I found that the original Yocto list I posted my patch (which contained this patch) to was the incorrect list as dpkg is pulled in from OpenEmbedded core (http://lists.openembedded.org/pipermail/openembedded-core). Now that I have a basic idea of a direction, I will post the details problem over there to make sure I'm on the right track with the desired fix. I did some looking and I think the best approach is more along the lines of adding a --configdir vs. --sysroot. On Thu, Dec 15, 2016 at 9:21 AM, Anders Oleson <[email protected]> wrote: > On Thu, Dec 15, 2016 at 8:16 AM, Ian Jackson > <[email protected]> wrote: >> Anders Oleson writes ("Re: PATCH: dpkg: allow missing /etc/dpkg/dpkg.d >> directory"): >>> But they expressed a desire to upstream patches (in general), so I >>> thought I'd see what you thought. If don't-care/don't fail was >>> acceptable, >>> then why not start the discussion there. >> >> I wanted to reply to this. I absolutely encourage you to come and try >> to upstream your patches. You will get the benefit of our orneriness >> about error handling :-). By which I meank people like me can give >> you advice on how to solve your practical problems in a more robust >> way. > Actually I agree with 100% of all of this. It isn't good for anyone to have > the baked in paths reused - it's not supposed to be that way. > > So the first question was just whether we actually *want* to check the > errors that way - asked and answered. No surprises really. I independently > debated nearly all the same points with myself. > >> >> And we will hopefully get the benefit of your code contribution. >> So, thanks. >> >> Regards, >> Ian. > > Either way the best long term option would be to make dpkg support floating > around in sysroots without hardcoded paths baked in. My thoughts exactly on > if we had an override, we would set the baked-in paths to something neutral > and illegal same as you suggested to catch stuff like this. I don't like > peoples names floating around baked into executables either even if only > used for the cross-build. > > The biggest help would be to get design guidance. I know Yocto passes > --instdir and --admindir. If we were to add a --sysroot or a --configdir > option at a high level where would we need to touch? There are multiple > executables, perl is involved somewhere, etc. etc. What has to be > considered? What should get touched? > > I am not a frequent contributor, but I am an experienced developer and user > of many open source tools and I maintain, debug and hack on them internally > quite a lot. And lurk vs. post mostly too. It is rare to actually find > something that is actually worth sending upstream, e.g. not so specific or > not already fixed, etc. But the point is that both Yocto and dpkg are of > huge value to me and my way of saying thanks is to offer to pitch in when I > do find something. Give me a few hints and I'll take this on. > > If that goes OK, in addition, I can probably look at the other patches that > Yocto is applying. They have a system where the have a list of patches that > are applied during the build. See: > http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/dpkg/dpkg > > Thanks! > >> >> -- >> Ian Jackson <[email protected]> These opinions are my own. >> >> If I emailed you from an address @fyvzl.net or @evade.org.uk, that is >> a private address which bypasses my fierce spamfilter.
