Re: [Fink-devel] Use-binary-dist flakiness
Dave Vasilevsky wrote: On Aug 5, 2005, at 12:05 AM, Dave Vasilevsky wrote: Perhaps apt only gets confused if it doesn't see the .deb on your disk at all, for example if you haven't run scanpackages in a while? That would explain why I don't see this error much anymore Ok, I did some testing and my hunch was right. The bug of apt-get downloading and 'upgrading' to same version as you already have occurs only if: You have a package installed from a .deb you built with Fink, AND that same version/revision is available in the bindist, AND you haven't done a scanpackages lately so apt doesn't know about the .deb on your hard drive. I am getting this in another situation, even after scanpackages: for installed packages where no deb is available. I assume these packages have been installed from the bindist server (some earlier bindist?). I am observing this here on 10.3.9 and I suspect that the packages concerned haven't had revision rises for quite a while, so they were probably installed from a bindist where fink computed the Installed-Size differently or not at all. As an example: gnome-libs-1.4.2-19: Within this same version and revision over more than 18 months, the maintainer has changed, the BuildDepends list has been changed, and a CompileScript has been added. In addition, the earlier (installed) version has no Installed-Size field. I would say apt-get has a point in proposing an update in this case and the bug is rather that the revision number was never increased. The other 42 packages that "apt-get dist-upgrade" wants to install on this machine despite "fink list -o" not listing anything are probably in a similar state. So in this situation, it is perhaps a good thing to let apt-get do what it wants. This variant of the problem will also be less serious on Tiger, because the Installed-Size is now calculated in a consistent way, so that differences between successive bindists or even home-made bindists will concern fewer packages. There is no guarantee, however, that the calculated Installed-Size will be the same on different machines for all packages. There are, for instance, still quite a few packages that write the build directory path into lib*.la files, so that their size will depend on the precise name of the build directory. There exist several other examples of such mild bugs or instabilities that have an influence on the size of binaries or helper files. So I guess we will have to live with this behavior after every appearance of a new bindist. -- Martin --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Fink-devel mailing list Fink-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fink-devel
Re: [Fink-devel] Use-binary-dist flakiness
On Aug 5, 2005, at 12:05 AM, Dave Vasilevsky wrote: Perhaps apt only gets confused if it doesn't see the .deb on your disk at all, for example if you haven't run scanpackages in a while? That would explain why I don't see this error much anymore Ok, I did some testing and my hunch was right. The bug of apt-get downloading and 'upgrading' to same version as you already have occurs only if: You have a package installed from a .deb you built with Fink, AND that same version/revision is available in the bindist, AND you haven't done a scanpackages lately so apt doesn't know about the .deb on your hard drive. The good news is that Fink 0.25.x will include much better support for apt. It has support for using apt-ftparchive (if that package is installed) for 'fink scanpackages'. This is *much* faster than the current system, making it actually practical to run scanpackages. So Fink 0.25.x will also include a 'AutoScanpackages' option in fink.conf, which will automatically do a scanpackages/'apt-get update' every time it finishes building some new packages. :-) Another solution is to use apt "pinning" to tell apt that you prefer installed packages to those available from the bindist. See 'man apt_preferences' for details. Installed packages have a priority of 100, so the following /sw/etc/apt/preferences works for me: Package: * Pin: release o=Fink Pin-Priority: 99 I'm not sure if pinning would have any negative side effects though, someone more familiar with Debian would know more. We'll have to discuss whether we want one or both of these to become the default, so the bug can go away. Dave PGP.sig Description: This is a digitally signed message part
Re: [Fink-devel] Use-binary-dist flakiness
On Aug 4, 2005, at 7:12 PM, Martin Costabel wrote: This one comes from the "Installed-Size:" field in the DEBIAN/ control file. apt-get compares package versions not only by their version-revision number, but also by a VersionHash number which is computed by taking into account several fields of the control file (in apt-pkg/deb/deblistparser.cc), in particular the Installed-Size field. Therefore two copies of the same package with the same version, but different Installed-Size values are considered different, and in the apt-cache commands "dump", "show", "showpkg" they are listed as separate versions. Thank you for figuring out this much, it should help a lot :-) If the problem is that apt considers the .deb on the server and the .deb on my hard drive to be different, one thing that confuses me is why we only see this problem in one direction. IE: Apt will download a package to replace the one I have installed via Fink, but I've never seen apt install a .deb on my disk to overwrite a downloaded one. Is there something special about one or the other situation? Perhaps apt only gets confused if it doesn't see the .deb on your disk at all, for example if you haven't run scanpackages in a while? That would explain why I don't see this error much anymore. If I understand correctly how fink determines if a package is apt- gettable, it can trip over the problem with different Installed- Size fields, too: It looks at the output of "apt-cache dump" and if the remote version has a different Installed-Size from the locally installed version, the dump shows the same version with several different "File:" fields; if the last one of these is /sw/var/lib/ dpkg/status, the version is marked as not apt-gettable, although the version may exist on the remote server. I don't understand, however, how this can happen for a package version that is not yet locally installed, which is where I am seeing this happen. Perhaps something to do with the apt cache files in /sw/var/cache/apt/ installed by the Fink installer. I'll try looking into this. Dave PGP.sig Description: This is a digitally signed message part
Re: [Fink-devel] Use-binary-dist flakiness
I think I am finally closing in on this old bug. At least I believe I understand now the mechanism of the bug where "apt-get upgrade" wants to install packages although they are already installed. The other one where "fink -b install" doesn't want to realize that files are available on the server is most likely related, but I haven't yet understood the mechanism completely. Dave Vasilevsky wrote: On Jun 29, 2005, at 8:07 AM, Martin Costabel wrote: Using a local server for unstable debs, I have a long-standing quibble with apt-get: It often doesn't understand that a package is already installed and "apt-get dist-upgrade" downloads and installs many packages, sometimes hundreds of packages, whose exact same version is already installed. I have complained about this on the lists and have a bug tracker item open, but apparently no one else understands this either, although it has been observed by others already years ago, at least every time a new bindist comes out. Martin, We know that this happens, and it kinda sucks. But I've never been able to figure it out exactly. I usually end up accidentally letting an apt-get upgrade finish, and then the problem goes away. Since I don't know how to reproduce it, it just leaves me kinda confused and without any way to test further. If anybody can reproduce it consistently, please tell me how so I can debug it. This one comes from the "Installed-Size:" field in the DEBIAN/control file. apt-get compares package versions not only by their version-revision number, but also by a VersionHash number which is computed by taking into account several fields of the control file (in apt-pkg/deb/deblistparser.cc), in particular the Installed-Size field. Therefore two copies of the same package with the same version, but different Installed-Size values are considered different, and in the apt-cache commands "dump", "show", "showpkg" they are listed as separate versions. The Installed-Size field is computed by fink, and until a year ago fink changed its mind several times about the way how (and if) to compute this value. I find quite a few packages on my systems that don't have any Installed-Size field at all. In addition, packages compiled on different machines can often have different values for this field. The result is that when a package is installed locally, but the same version on the bindist server has a different Installed-Size, apt-get upgrade will download and install the remote version. I have a symmetrical problem with fink --use-binary-dist: It very often, and apparently randomly, refuses to use the binary dist, although the package it is going to install does exist there. If I understand correctly how fink determines if a package is apt-gettable, it can trip over the problem with different Installed-Size fields, too: It looks at the output of "apt-cache dump" and if the remote version has a different Installed-Size from the locally installed version, the dump shows the same version with several different "File:" fields; if the last one of these is /sw/var/lib/dpkg/status, the version is marked as not apt-gettable, although the version may exist on the remote server. I don't understand, however, how this can happen for a package version that is not yet locally installed, which is where I am seeing this happen. Perhaps something to do with the apt cache files in /sw/var/cache/apt/ installed by the Fink installer. So it looks like not much can be done about these annoying problems, unless of course we remove the Installed-Size field from the VersionHash computation in apt. This would be a small one-line patch that would probably save us a lot of trouble in the future, but I can also see reasons not to do it and to learn to live with the annoying behavior. -- Martin --- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf ___ Fink-devel mailing list Fink-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fink-devel
Re: [Fink-devel] Use-binary-dist flakiness
On Jun 29, 2005, at 8:07 AM, Martin Costabel wrote: Using a local server for unstable debs, I have a long-standing quibble with apt-get: It often doesn't understand that a package is already installed and "apt-get dist-upgrade" downloads and installs many packages, sometimes hundreds of packages, whose exact same version is already installed. I have complained about this on the lists and have a bug tracker item open, but apparently no one else understands this either, although it has been observed by others already years ago, at least every time a new bindist comes out. Martin, We know that this happens, and it kinda sucks. But I've never been able to figure it out exactly. I usually end up accidentally letting an apt-get upgrade finish, and then the problem goes away. Since I don't know how to reproduce it, it just leaves me kinda confused and without any way to test further. If anybody can reproduce it consistently, please tell me how so I can debug it. I have a symmetrical problem with fink --use-binary-dist: It very often, and apparently randomly, refuses to use the binary dist, although the package it is going to install does exist there. [snip] I have *not* experienced this problem, I'd like some more information. Note first that the --use-binary-dist option to fink (or in fink.conf) will only get packages from the bindist that exist BOTH as .info files and in the bindist. This restriction makes things easier on Fink and should generally not be a problem, but it may be the cause of your issue. Please verify! If this is not your problem, could you please tell your users that if they observe fink starting to buildthings at this stage instead of downloading, to stop it immediately, run a few commands, and send you the output; that way we can look at what's going on. Commands that would be useful are: uname -a fink -V find /sw/fink/dists/ -name fink.info | xargs grep Version apt-cache policy fink And any others you can think of including. Hopefully we can figure out what's going on. Dave PGP.sig Description: This is a digitally signed message part
[Fink-devel] Use-binary-dist flakiness
Using a local server for unstable debs, I have a long-standing quibble with apt-get: It often doesn't understand that a package is already installed and "apt-get dist-upgrade" downloads and installs many packages, sometimes hundreds of packages, whose exact same version is already installed. I have complained about this on the lists and have a bug tracker item open, but apparently no one else understands this either, although it has been observed by others already years ago, at least every time a new bindist comes out. I have a symmetrical problem with fink --use-binary-dist: It very often, and apparently randomly, refuses to use the binary dist, although the package it is going to install does exist there. The latest fight I have with this is related to a little script that I want to distribute to our local Fink users. The idea is that new Fink users or those who just have upgraded to Tiger use Fink's official 0.8.0 binary installer and then run this script (which is double-clickable). It installs versions of /sw/etc/fink.conf and /sw/etc/apt/sources.list that are adapted to our local situation and contain in particular the address of our local unstable deb server, and fink.conf has unstable enabled and UseBinaryDist: true. At the end it runs fink selfupdate-rsync to bring the Fink distribution from the stable one contained in the Fink binary installer to an up-to-date unstable-enabled state. This command should download the latest versions of fink and the base packages in binary form from the server. Often it doesn't do this and starts to compile things from source which is very annoying, because instead of a couple of seconds, it then takes almost 20 minutes even on a dual G5. This is also more or less random, in that it works as intended when I test it, and it works also most of the time when users run the commands from the command line, but when they use the script, it annoyingly starts compiling almost all the time. I have preceded the fink selfupdate by "fink scanpackages", "apt-get update" and by "fink update fink" (which always does use the binary dist as it should), but to no avail. I could, of course, use apt-get directly, or update the base packages one by one, but since this is clearly a bug in Fink, I am still hoping that someone who knows about these things will be able to repair it. I don't believe that I am the only one seeing this. I rather suspect that others have given up on --use-binary-dist because of this flaky behavior and haven't said anything, because it isn't considered very important anyway. At least that's what I have done until now. Any help welcome -- Martin --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click ___ Fink-devel mailing list Fink-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fink-devel
