Bug#834682: mina2: FTBFS in testing (failing tests)
On Fri, Sep 16, 2016 at 07:35:58PM +0200, Santiago Vila wrote: > Hello. > > I tried building this package 20 times this morning. > > It failed 19 times. The build logs are attached in a single tarball. > > There is something which may help in this bug: there is a new upstream > release of mina available (2.0.14). > > So I'd like to propose that you or anybody in the team maintaining > this package do this: > > * Raise this to serious (where it belongs, it's a FTBFS bug). > > * Disable completely the tests for this version (trivial patch attached). > The upload doing that would naturally close this bug. > > Disabling only the tests that fail would be also an option, but the tests > failing for me are not the same failing for Gregor, and it would not be > really productive or useful to determine which of the tests are ok and > which ones are not when we have a new upstream release available that may > have some or all of the failing tests already fixed. > > * Try to upload mina version 2.0.14 for unstable (but this time without the > patch) > to see if things improve with the new version. Hopefully this version does > not > fail anymore, or maybe it fails for you and for me at the same time. > > Would this plan be acceptable to you? > > If yes, please go ahead and stop reading. Thanks a lot! > > If not: there is a tag called stretch-ignore which is the right way to > make a serious bug like this one not to be RC: > > https://release.debian.org/stretch/rc_policy.txt > > Please read the relevant paragraph: You would have to ask the Release > Managers for permission first, you can't just downgrade a FTBFS bug on > your own as a way to make it not RC. > > But I really believe we can skip all that. Believe me, it is not my > intention or desire to see this package autoremoved from Debian or > anything like that, everything I ask is that it builds ok, and that's > very easy to achieve with the attached path. > > Thanks. Hi Santiago, Ugh - thank you for putting so much effort into this and for providing logs. I have tried to get the build to fail for me locally and can't, which is also frustrating. However, I feel like we have an established precedent for disabling non-deterministic test suites, since flaky tests can be worse than no tests at all, so I am applying your patch and preparing an upload. We can always enable tests down the road, as you suggest, perhaps with the upload of the next upstream version. Cheers, tony signature.asc Description: PGP signature
Bug#834682: mina2: FTBFS in testing (failing tests)
On Thu, 8 Sep 2016, gregor herrmann wrote: > Logs attached. Unfortunately I don't have any idea on how to fix the > problem, but if you want me to do something else just shout. Well, if you don't mind burning a bunch of additional CPU cycles, I would continue to build it several times until it fails again. (If you tried three times and it failed once, it seems likely that it happens again). Then it would be interesting to see if the test which fails is the same it failed the first time, which, in your case, was this one: Failed tests: DatagramBindTest>AbstractBindTest.testUnbindDisconnectsClients:187 expected:<5> but was:<4> I have not analyzed my build logs, but I guess this is unfortunately not the same test which fails for me. BTW: I try to be objective by never reporting something when it only happens to me. This is why I sometimes say "here is a build log and it also fails in reproducible builds" or also "here you have several build logs from several different autobuilders". I think it would be just fair to ask the maintainer to be objective as well and consider whether it is really justified to downgrade the severity of a FTBFS bug when everybody else get a FTBFS too and his computer is the only one in which it may not be reproduced. If it is certainly frustrating when one receives a bug which is random and difficult to reproduce, imagine the frustration of the bug reporter who sees his report being downgraded or even closed (!) (see Bug #834964 for a very recent example) when it's quite clear that the package does not really "build from source". Thanks a lot.
Bug#834682: mina2: FTBFS in testing (failing tests)
On Wed, 7 Sep 2016, Markus Koschany wrote: > I am attaching two build logs (cowbuilder and sbuild) that show no > issues at all. The build succeeds. Thanks a lot for trying to reproduce this. There are still a lot of differences, we should better remove them all, until our building environments are the same. Then it should fail for you too (hopefully). Would you please add this to the mix?: * Use a stretch chroot (as reported). * Build only arch-independent packages (as reported). This is the equivalent to "dpkg-buildpackage -A". In sbuild it's done with options "--arch-all --no-arch-any". * The chroot should be as small as possible. Well, I don't do exactly that because most packages build-depend on debhelper, and I want to save disk I/O so I have debhelper by default, but on the other side I don't have gnupg installed. To simplify: Would you please install the packages in package-list.txt (attached) and only those? [ This is of course only the starting point, sbuild should then automatically download and install the required packages ]. * I'm also using --resolve-alternatives option of sbuild. Don't know if it's relevant but we should try to minimize differences. There is at least one package that FTBFS if I didn't use this option, which is otherwise the default (AFAIK) in the official buildds. * Build with only one CPU. This is achieved by putting this in .sbuildrc: $ENV{'DEB_BUILD_OPTIONS'} = 'parallel=1'; * Finally, I'm building on machines with not a lot of memory, because monitoring /proc/meminfo tells me that I don't need more for this particular package (but maybe I'm wrong). Please try on a virtual machine having only 1GB RAM and 1 GB swap. If you do all this, the probability that you will reproduce the bug may only increase (but of course it may also continue to be zero). In either case, we should try. Thanks.adduser apt autoconf automake autopoint autotools-dev base-files base-passwd bash binutils bsdmainutils bsdutils build-essential bzip2 coreutils cpp cpp-6 dash debconf debhelper debian-archive-keyring debianutils dh-autoreconf dh-strip-nondeterminism diffutils dmsetup dpkg dpkg-dev e2fslibs e2fsprogs eatmydata fakeroot file findutils g++ g++-6 gcc gcc-5-base gcc-6 gcc-6-base gettext gettext-base gpgv grep groff-base gzip hostname init-system-helpers initscripts insserv intltool-debian less libacl1 libapt-pkg5.0 libarchive-zip-perl libasan3 libatomic1 libattr1 libaudit-common libaudit1 libblkid1 libbsd0 libbz2-1.0 libc-bin libc-dev-bin libc-l10n libc6 libc6-dev libcap-ng0 libcc1-0 libcilkrts5 libcomerr2 libcroco3 libdb5.3 libdebconfclient0 libdevmapper1.02.1 libdpkg-perl libeatmydata1 libfakeroot libfdisk1 libffi6 libfile-stripnondeterminism-perl libgcc-6-dev libgcc1 libgcrypt20 libgdbm3 libglib2.0-0 libgmp10 libgomp1 libgpg-error0 libicu57 libisl15 libitm1 liblsan0 liblz4-1 liblzma5 libmagic-mgc libmagic1 libmount1 libmpc3 libmpfr4 libmpx2 libncurses5 libncursesw5 libpam-modules libpam-modules-bin libpam-runtime libpam0g libpcre3 libperl5.22 libpipeline1 libquadmath0 libselinux1 libsemanage-common libsemanage1 libsepol1 libsigsegv2 libsmartcols1 libss2 libstdc++-6-dev libstdc++6 libsystemd0 libtimedate-perl libtinfo5 libtool libtsan0 libubsan0 libudev1 libunistring0 libustr-1.0-1 libuuid1 libxml2 linux-libc-dev locales login lsb-base m4 make man-db mawk mount multiarch-support nano ncurses-base ncurses-bin passwd patch perl perl-base perl-modules-5.22 po-debconf sed sensible-utils startpar sysv-rc sysvinit-utils tar tzdata util-linux xz-utils zlib1g
Bug#834682: mina2: FTBFS in testing (failing tests)
On 07.09.2016 14:33, Santiago Vila wrote: > retitle 834682 mina2: FTBFS in testing (failing tests) > thanks > > On Fri, 26 Aug 2016, Markus Koschany wrote: > >> I have rebuilt mina2 ten times in a row now but I can't reproduce the >> failing test or the build failure in general. > > Congratulations. I attach five build logs in five different autobuilders. > > Apparently, it fails for me all the time. > > [ So, instead of you asking me for a way to reproduce it, perhaps I > should be asking you for a way *not* to reproduce it... ] > >> It appears this issue is not random > > After my last five builds yesterday, that's what it seems, yes. > Now it fails all the time, so it's not random anymore. > > I'm changing the subject accordingly. I am attaching two build logs (cowbuilder and sbuild) that show no issues at all. The build succeeds. I have fixed previous packages with random failing tests because I think those tests are not helpful for us in general but I won't disable them if I can't reproduce the randomness of the issue myself. I always try to reproduce bug reports but I am getting more and more frustrated by the way how you interact with us. It is just annoying and disrespectful. mina2_2.0.13-1_amd64.build.gz Description: application/gzip mina2_2.0.13-1_amd64-2016-09-07T12:40:38Z.build.gz Description: application/gzip signature.asc Description: OpenPGP digital signature