Bug#986527: Patches for flaky build and cython unavailability

2021-08-04 Thread Ahzo 

Aug 3, 2021, 11:12 by than...@debian.org:

> Thanks a lot for the patches Ahzo. Especially fixing the file handle leak 
> should help a lot.
>

Actually that patch was rather a workaround than a fix.

The real cause of the "Too many open files" issue is a behavior change of the 
multiprocessing.Process class in python3.
It now opens a pipe internally, which it did not do in python2.

The solution is to call the close() method of multiprocessing.Process to close 
the internal pipe.
Attached u2-close-the-internal-pipe-of-multiprocessing.Process.patch does just 
that.
However, this method was only added in python 3.7, so attempting to use it 
fails in earlier versions of python3 (and also in python2).

Regards,
Ahzo
>From 27ee32c1fc773fbbb7e54036acc5df6453c10131 Mon Sep 17 00:00:00 2001
From: Ahzo 
Date: Wed, 4 Aug 2021 23:23:20 +0200
Subject: [PATCH] close the internal pipe of multiprocessing.Process

Every finished, but not yet logged worker holds an open fd.
Thus when following a long running worker, so many finished tests can accumulate,
that the open files limit (ulimit -n) is reached.
This then causes the test suite to fail with 'OSError: [Errno 24] Too many open files'.

The open fd is due to a behavior change of the multiprocessing.Process class in python3.
It now opens a pipe internally, which it did not do in python2.

The solution is to call the close() method of multiprocessing.Process to close the internal pipe.
However, this method is only available since python 3.7.
---
 sage/src/sage/doctest/forker.py | 12 ++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/sage/src/sage/doctest/forker.py b/sage/src/sage/doctest/forker.py
index 045975c1b5..170ce6824d 100644
--- a/sage/src/sage/doctest/forker.py
+++ b/sage/src/sage/doctest/forker.py
@@ -1891,6 +1891,14 @@ class DocTestDispatcher(SageObject):
 # report(), parallel testing can easily fail
 # with a "Too many open files" error.
 w.save_result_output()
+# In python3 multiprocessing.Process also
+# opens a pipe internally, which has to be
+# closed here, as well.
+# But afterwards, exitcode and pid are
+# no longer available.
+w.copied_exitcode = w.exitcode
+w.copied_pid = w.pid
+w.close()
 finished.append(w)
 workers = new_workers
 
@@ -1910,10 +1918,10 @@ class DocTestDispatcher(SageObject):
 self.controller.reporter.report(
 w.source,
 w.killed,
-w.exitcode,
+w.copied_exitcode,
 w.result,
 w.output,
-pid=w.pid)
+pid=w.copied_pid)
 
 pending_tests -= 1
 
-- 
2.30.2

Bug#986527: Patches for flaky build and cython unavailability

2021-07-31 Thread Ahzo 
Control: tags -1 patch

Hi,

the main problem making the sagemath testsuite flaky is that it randomly aborts 
due to 'Too many open files'.
Thus only a small part of the test suite gets actually run, when the build is 
heavily parallelized.
This can be seen by reporting not only the number of failed, but also that of 
run tests, which shows significant fluctuations.

The problem occurs, because every finished, but not yet logged worker, holds an 
open fd (a pipe used to read the output of the child actually doing the tests).
Thus when following a long running worker, i.e. logging its messages, while it 
is still running, so many finished tests can accumulate, that the open files 
limit (ulimit -n) is reached.

However, there should be no open pipe per finished worker, as the test suite 
calls 'os.close(self.rmessages)' before waiting for logging the messages.
So this seems to be caused by something that python does behind the scenes.
Removing the single line 'finished.append(w)' in src/sage/doctest/forker.py 
prevents the open fd increase, though at the cost of hardly logging any test 
output.

This problem can be avoided by simply logging every finished test, but no 
running one.

With only the 0001-Report-the-number-of-total-tests-run.patch, the result is 
something like:
Success: 5 of 71435 tests failed, up to 200 failures are tolerated

Adding the dt-Do-not-follow-a-running-worker.patch, the result becomes:
Success: 194 of 361139 tests failed, up to 200 failures are tolerated

These 194 failures are pretty close to the threshold of 200, so it is not 
particularly surprising, that this can fail in some environments.
Slightly passing this threshold triggered the build failure in this bug and 
also the one in bug #983931.

Increasing the threshold to 300 should make that rather unlikely, though.
And considering that there are more than 360 thousand tests, less then 300 
failures means more than 99.9 % of the tests succeeded.

The "cython: not found" issue is trivial to fix and important, because 
otherwise 'sage --cython' does not work and there is no '--cython3' option 
(unlike e.g. the '--ipython3' option).

After adding the 0002-Tolerate-up-to-300-failing-tests.patch and the 
u2-Adapt-to-python2-removal.patch the test result is:
Success: 189 of 361139 tests failed, up to 300 failures are tolerated

It would also be a good idea to include a backport of commit 5cf493ca51 ("Avoid 
libgmp's new lazy allocation") in the next sagemath upload, as that fixes a 
severe memory leak (see bug #964848).

As to the crashes, I can't reproduce any crash when testing 
interfaces/singular.py:
sage -t --long --random-seed=0 src/sage/interfaces/singular.py
    [404 tests, 3.87 s]

This crash also does not always happen for the reproducible builds either, e.g. 
the following log shows it first crashing and then passing this test:
https://tests.reproducible-builds.org/debian/rbuild/bullseye/amd64/sagemath_9.2-2.rbuild.log.gz
[...]
sage -t --long --random-seed=0 src/sage/interfaces/singular.py
    Killed due to segmentation fault
[...]
sage -t --long --random-seed=0 src/sage/interfaces/singular.py
    [404 tests, 21.06 s]
[...]

However, a number of other crashes happen during every test run, but only one 
of them causes a test failure:
sage -t --long --random-seed=0 src/sage/interfaces/tests.py
**
File "src/sage/interfaces/tests.py", line 34, in sage.interfaces.tests
Failed example:
    subprocess.call("echo syntax error | ecl", **kwds) in (0, 255)
Expected:
    True
Got:
    False
**

Similar crashes sometimes also occur when testing interfaces/lisp.py, but 
without causing the test to fail.
This is a problem in ecl, which crashes when both stdout and stderr are full, 
see bug #710953.

Then there is a crash in nauty-gentourng triggered by 
src/sage/graphs/digraph_generators.py.
For details see bug #991750.

There are also two SIGABRT crashes in mwrank triggered by 
src/sage/interfaces/mwrank.py.
These seem to be intentional due to invalid input.

Finally, there are some python crashes (5 SIGQUIT, 1 SIGABRT, 1 SIGSEGV) that 
are all caused intentionally by the test suite.

So none of these crashes are problems in sagemath itself.

Regards,
Ahzo
>From 5c741b0066c861504483b5ed66915b01ddd078b0 Mon Sep 17 00:00:00 2001
From: Ahzo 
Date: Sat, 31 Jul 2021 13:15:51 +0200
Subject: [PATCH 1/2] Report the number of total tests run

This makes it easier to notice when tests get skipped.
---
 debian/rules| 10 ++
 debian/tests.mk |  4 
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/debian/rules b/debian/rules
index f984695..9e904e2 100755
--- a/debian/rules
+++ b/debian/rules
@@ -163,11 +163,12 @@ endif
 had-few-failures:
 	if ! test -f $(LOGFILE); then echo "Error: log file $(LOGFILE) not found"; false; fi
 	N_TE

Bug#710953: ecl: segfault when both stdout and stderr are full

2021-07-31 Thread Ahzo 
Control: retitle -1 ecl: segfault when both stdout and stderr are full
Control: found -1 ecl/20.4.24+ds-2
Control: forwarded -1 https://gitlab.com/embeddable-common-lisp/ecl/-/issues/634
Control: affects -1 sagemath

Hi,

the originally reported problem is already fixed, i.e. only redirecting stderr 
to /dev/full no longer causes a crash:
$ echo "syntax error" | ecl 2>/dev/full
ECL (Embeddable Common-Lisp) 20.4.24 (git:UNKNOWN)
Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
Copyright (C) 1993 Giuseppe Attardi
Copyright (C) 2013 Juan J. Garcia-Ripoll
Copyright (C) 2018 Daniel Kochmanski
Copyright (C) 2020 Daniel Kochmanski and Marius Gerbershagen
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help. 
Top level in: #.
>
Available restarts:

1. (RESTART-TOPLEVEL) Go back to Top-Level REPL.

Broken at SI:BYTECODES. [Evaluation of: SYNTAX] In: #.
>>
$ echo $?
0

However, the sagemath testsuite still triggers a related crash in ecl:
sage -t --long --random-seed=0 src/sage/interfaces/tests.py
**
File "src/sage/interfaces/tests.py", line 34, in sage.interfaces.tests
Failed example:
    subprocess.call("echo syntax error | ecl", **kwds) in (0, 255)
Expected:
    True
Got:
    False
**

This crash is easy to reproduce:
$ ecl &> /dev/full
Segmentation fault (core dumped)

Occasionally, similar crashes also occur in other sagemath tests, e.g. 
src/sage/interfaces/lisp.py.
This can be reproduced as follows:
$ sage
sage: l = Lisp()
sage: l._start()
sage: l.quit()

The problem is that ecl crashes when it cannot write to both stdout and stderr.
Therefore I'm reusing this bug for that related problem.

There is already an open upstream issue about this (see the forwarded URL).

Regards,
Ahzo

Bug#991750: nauty: segfaults during sagemath testsuite

2021-07-31 Thread Ahzo 
Package: nauty
Version: 2.7r1+ds-2
Severity: important
Control: affects -1 sagemath

Hi,

during the sagemath testsuite, a nauty crash occurs. Nonetheless, the sagemath 
test passes:
sage -t --long --random-seed=0 src/sage/graphs/digraph_generators.py
    [150 tests, 6.44 s]

The crash can be easily reproduced:
$ nauty-gentourng -d0 -D1 -c 2
>A nauty-gentourng -cd1D0 n=2
Segmentation fault (core dumped)

Anyway, nauty should not crash like this, particularly since the input seems 
valid.

Regards,
Ahzo

Bug#975355: libopenni2-0: missing Multi-Arch support

2020-11-21 Thread Ahzo 
Hi Jochen,

Nov 21, 2020, 07:48 by jspri...@debian.org:

> * Ahzo  [2020-11-20 23:03]:
>
>> --- a/debian/libopenni2-0.install
>> +++ b/debian/libopenni2-0.install
>> @@ -1,3 +1,3 @@
>> -Bin/*-Release/libOpenNI2.so.* usr/lib/
>> -Bin/*-Release/OpenNI2/Drivers/lib*.so.* usr/lib/OpenNI2/Drivers/
>> +Bin/*-Release/libOpenNI2.so.* usr/lib/${DEB_HOST_MULTIARCH}
>> +Bin/*-Release/OpenNI2/Drivers/lib*.so.* 
>> usr/lib/${DEB_HOST_MULTIARCH}/OpenNI2/Drivers/
>>
>
> Did you check that this works?
> AFAIR these are plugins loaded at runtime, so the path needs to be correctly 
> encoded. I don't have access to the hardware currently, so would be great if 
> you (or someone else) could test this.
>
I haven't tested the runtime functionality, but according to the comment in 
OpenNI.ini, this should just work, as the driver-loading is based on a relative 
path by default:
$ cat /etc/openni2/OpenNI.ini | tail -n 5
[Drivers]
; Location of the drivers specified by a relative path based on OpenNI's shared 
library or an absolute path.
; Path separator "/" can be used to be portable for any platforms.
; Default - OpenNI2/Drivers
;Repository=OpenNI2/Drivers

Regards,
Ahzo

Bug#975357: nmu: xoreos-tools_0.0.5-2

2020-11-20 Thread Ahzo 
Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: binnmu

Dear Release Team,

the last upload of xoreos-tools included binaries for amd64, which were built 
with outdated dependencies.
Thus xoreos-tools:amd64 is currently uninstallable, as it depends on boost 
1.67, and was thus removed from testing.

Since a simple rebuild with current dependencies fixes this, please schedule 
this binNMU:

nmu xoreos-tools_0.0.5-2 . amd64 . unstable . -m "Rebuild against boost 1.71"

Thanks in advance,
Ahzo

Bug#975355: libopenni2-0: missing Multi-Arch support

2020-11-20 Thread Ahzo 
Package: libopenni2-0
Version: 2.2.0.33+dfsg-13
Severity: normal
Tags: patch
Affects: gstreamer1.0-plugins-bad

Dear Maintainer,

recently gstreamer1.0-plugins-bad started depending on libopenni2-0. Since then 
that package is no longer multi-arch co-installable, because libopenni2-0 is 
not 'Multi-Arch: same'.

The first attached patch multi-archifies openni2 to fix this.

The second, optional, patch fixes parallel-build support for openni2 to improve 
build-time.

Please include at least the first patch in your next upload, to make 
gstreamer1.0-plugins-bad again multi-arch co-installable.

Thanks in advance,
Ahzo
>From e82c6d4aca383258c829700ac45782a24ba080bd Mon Sep 17 00:00:00 2001
From: Ahzo 
Date: Fri, 20 Nov 2020 22:14:05 +0100
Subject: [PATCH 1/2] multi-archify the library

Switch to debhelper-compat 13 for support of DEB_HOST_MULTIARCH in the
install files.
---
 debian/control| 5 -
 debian/libopenni2-0.install   | 4 ++--
 debian/libopenni2-dev.install | 4 ++--
 3 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/debian/control b/debian/control
index a0c8157..7aa47a5 100644
--- a/debian/control
+++ b/debian/control
@@ -4,7 +4,7 @@ Section: libs
 Maintainer: Debian Multimedia Maintainers 
 Uploaders: Nobuhiro Iwamatsu ,
Jochen Sprickerhof 
-Build-Depends: debhelper-compat (= 12),
+Build-Depends: debhelper-compat (= 13),
freeglut3-dev,
libusb-1.0-0-dev,
python3,
@@ -21,6 +21,7 @@ Vcs-Browser: https://salsa.debian.org/multimedia-team/openni2
 
 Package: libopenni2-0
 Architecture: any
+Multi-Arch: same
 Depends: ${shlibs:Depends},
  ${misc:Depends}
 Description: framework for sensor-based 'Natural Interaction'
@@ -34,6 +35,7 @@ Description: framework for sensor-based 'Natural Interaction'
 Package: openni2-utils
 Section: utils
 Architecture: any
+Multi-Arch: foreign
 Depends: ${shlibs:Depends},
  ${misc:Depends}
 Description: debug and test utilities OpenNI2 framework
@@ -50,6 +52,7 @@ Description: debug and test utilities OpenNI2 framework
 Package: libopenni2-dev
 Section: libdevel
 Architecture: any
+Multi-Arch: same
 Depends: ${misc:Depends},
  libopenni2-0 (= ${binary:Version})
 Suggests: openni2-doc
diff --git a/debian/libopenni2-0.install b/debian/libopenni2-0.install
index 7258244..89fcb95 100644
--- a/debian/libopenni2-0.install
+++ b/debian/libopenni2-0.install
@@ -1,3 +1,3 @@
-Bin/*-Release/libOpenNI2.so.* usr/lib/
-Bin/*-Release/OpenNI2/Drivers/lib*.so.* usr/lib/OpenNI2/Drivers/
+Bin/*-Release/libOpenNI2.so.* usr/lib/${DEB_HOST_MULTIARCH}
+Bin/*-Release/OpenNI2/Drivers/lib*.so.* usr/lib/${DEB_HOST_MULTIARCH}/OpenNI2/Drivers/
 Config/*.ini etc/openni2/
diff --git a/debian/libopenni2-dev.install b/debian/libopenni2-dev.install
index 43f895c..aad904e 100644
--- a/debian/libopenni2-dev.install
+++ b/debian/libopenni2-dev.install
@@ -1,4 +1,4 @@
 Include/* usr/include/openni2/
 debian/libopenni2.pc usr/share/pkgconfig
-Bin/*-Release/libOpenNI2.so usr/lib/
-Bin/*-Release/OpenNI2/Drivers/lib*.so usr/lib/OpenNI2/Drivers/
+Bin/*-Release/libOpenNI2.so usr/lib/${DEB_HOST_MULTIARCH}
+Bin/*-Release/OpenNI2/Drivers/lib*.so usr/lib/${DEB_HOST_MULTIARCH}/OpenNI2/Drivers/
-- 
2.29.2

>From d07e71e869150026834b138c00f2043d3be7dfd9 Mon Sep 17 00:00:00 2001
From: Ahzo 
Date: Fri, 20 Nov 2020 22:14:46 +0100
Subject: [PATCH 2/2] use dh_auto_build to properly support parallel builds

---
 debian/rules | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/rules b/debian/rules
index 57ccdd3..194df62 100755
--- a/debian/rules
+++ b/debian/rules
@@ -4,7 +4,7 @@
 	dh $@ --buildsystem=makefile
 
 override_dh_auto_build:
-	make ALLOW_WARNINGS=1
+	dh_auto_build -- ALLOW_WARNINGS=1
 	cd Source/Documentation && python3 ./Runme.py
 	cd Bin/*-Release/ && mv NiViewer NiViewer2
 
-- 
2.29.2

Bug#972898: hplip: no network scanner detection with 3.20.9

2020-10-27 Thread Ahzo 
Oct 26, 2020, 14:58 by claremont...@gmail.com:

> Thank you for your report, Ahzo. I have forwarded it upstream. Please
> review what I have written and correct or add to it as you see fit.
>

Thanks for forwarding the report, as Lauchpad still doesn't work for me.
Your summary in the upstream report covers the main aspects of the problem.

Regards,
Ahzo

Bug#972898: hplip: no network scanner detection with 3.20.9

2020-10-25 Thread Ahzo 
Package: hplip
Version: 3.20.9+dfsg0-4
Severity: important
Tags: patch

Dear Maintainer,

hplip version 3.20.9 replaced the custom mDNS implementation for scanner 
discovery (protocol/discovery/mdns.c) with using avahi 
(protocol/discovery/avahiDiscovery.c).
While this is a welcome change in principle, unfortunately the new code does 
not work.

The main problem is that it does not wait until all callbacks are called before 
stopping via avahi_simple_poll_quit.
Thus there is a 50/50 chance whether the '_scanner._tcp.local' or the 
'_uscan._tcp.local' mDNS reply gets processed (in the browse_callback) and it 
is practically impossible that any scanner gets processed (in the 
resolve_callback), because avahi_simple_poll_quit is called when the first mDNS 
reply has been processed.
It seems like this code was never really tested with an actual scanner on the 
network.

Attached is a patch fixing this by implementing a check_terminate function 
similar to the one used by avahi-browse.

The second problem is that the code expects the 'ty' part of the mDNS reply, 
which contains the device name, to start with 'HP'. However this is not always 
the case.
Previous versions of hplip would simply remove the first three letters of the 
scanner name and continue, which could be worked around by also removing these 
three letters in the models.dat. That issue has been reported two years ago 
upstream without response from HP. (see: 
https://bugs.launchpad.net/hplip/+bug/1797501)
The new code however  discards the scanner if its 'ty' name does not start with 
'HP', breaking that workaround.
Fortunately, since the new code now uses avahi, a proper fix is relatively 
simple: If the 'ty' part of the mDNS response does not start with 'HP', check 
whether the 'mfg' part does.

The second attached patch implements this fix.

While debugging this I also noticed that the log level for one message is 
wrong, causing spurious errors to be reported.

The third attached patch changes the log level for this message from BUG to DBG 
to fix this.

I hope HP solves this eventually, but until then please include these patches 
in the Debian package, so that scanner detection works again.

Thanks in advance,
Ahzo

PS: I tried to report this upstream, but my Launchpad login attempts always 
fail, because "something just went wrong in Launchpad".

>From 67dbad25f503e1d8c6794efba3f17718c77848ea Mon Sep 17 00:00:00 2001
From: Ahzo 
Date: Sun, 25 Oct 2020 22:17:04 +0100
Subject: [PATCH 1/3] avahiDiscovery: wait for all callbacks

When calling avahi_simple_poll_quit too early, not all callbacks will be called.
---
 protocol/discovery/avahiDiscovery.c | 20 +++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/protocol/discovery/avahiDiscovery.c b/protocol/discovery/avahiDiscovery.c
index 8d325ffc0..395aec088 100644
--- a/protocol/discovery/avahiDiscovery.c
+++ b/protocol/discovery/avahiDiscovery.c
@@ -28,6 +28,7 @@ char ipAddressBuff[MAX_IP_ADDR_LEN]={'\0'};
 static int aBytesRead = 0;
 static AvahiSimplePoll *aSimplePoll = NULL;
 static int aMemAllocated = 0;
+static int n_all_for_now = 0, n_resolving = 0;
 
 /*
 This function will fill the dictionary arguments for the dbus function call
@@ -237,6 +238,15 @@ static bool getHPScannerModel(AvahiStringList *iStrList, const char *ikey,char *
 return aValueFound;  
 }
 
+static void check_terminate() {
+
+assert(n_all_for_now >= 0);
+assert(n_resolving >= 0);
+
+if (n_all_for_now <= 0 && n_resolving <= 0)
+avahi_simple_poll_quit(aSimplePoll);
+}
+
 /*
 This function will gets called whenever a service has been resolved successfully or timed out
 */
@@ -300,6 +310,9 @@ static void resolve_callback(
 }
 }
 //avahi_service_resolver_free(r);
+assert(n_resolving > 0);
+n_resolving--;
+check_terminate();
 }
 /* Called whenever a new services becomes available on the LAN or is removed from the LAN */
 static void browse_callback(
@@ -337,11 +350,14 @@ static void browse_callback(
 
 if (!(avahi_service_resolver_new(c, interface, protocol, name, type, domain, AVAHI_PROTO_INET, (AvahiLookupFlags)0, resolve_callback, c)))
 BUG( "Failed to resolve service '%s': %s\n", name, avahi_strerror(avahi_client_errno(c)));
+else
+n_resolving++;
 
 break;
 
 case AVAHI_BROWSER_ALL_FOR_NOW:
- avahi_simple_poll_quit(aSimplePoll);
+ n_all_for_now--;
+ check_terminate();
  break;
 }
 }
@@ -444,6 +460,7 @@ static void avahi_setup(const int iCommandType, const char* iHostName)
 goto fail;
 }
}
+   n_all_for_now++;
 
/* Create the service browser */
if (!(sb = avahi_service_browser_new(client, AVAHI_IF_UNSPEC, AVAHI_PROTO_INET, "_scanner._tcp", NULL, (AvahiLookupFlags)0, browse_callback, client))

Bug#971027: gcc-10: regression in 10.2.0-9 causes segmentation fault in vlc

2020-09-29 Thread Ahzo 


Sep 29, 2020, 07:14 by rguent...@suse.de:

> I've filed > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97236
>
> Someone needs to create a testcase or provide instructions how to
> reproduce the bug.
>

Thanks for taking care of this issue upstream.

Sep 29, 2020, 15:18 by d...@debian.org:

> On 9/29/20 12:30 PM, Matthias Klose wrote:
> upstream now has a reduced test case.
>
> 10.2.0-12 is uploaded, reverting that commit for now.
>
Thanks for the quick upload. This should prevent further fallout, until 
upstream finds a better solution.

Regards,
Ahzo

Bug#971027: gcc-10: regression in 10.2.0-9 causes segmentation fault in vlc

2020-09-28 Thread Ahzo 
08  mov    %r9,0x8(%rsp,%rcx,1)
   a8cab:   39 c6   cmp    %eax,%esi
   a8cad:   7e 27   jle    a8cd6 
   a8caf:   48 98   cltq  
   a8cb1:   48 8d 70 01 lea    0x1(%rax),%rsi
   a8cb5:   48 c1 e0 05 shl    $0x5,%rax
   a8cb9:   4c 8b 94 03 b0 00 00mov    0xb0(%rbx,%rax,1),%r10
   a8cc0:   00
   a8cc1:   48 8b 84 03 b8 00 00mov    0xb8(%rbx,%rax,1),%rax
   a8cc8:   00
   a8cc9:   48 c1 e6 04 shl    $0x4,%rsi
   a8ccd:   4c 89 14 34 mov    %r10,(%rsp,%rsi,1)
   a8cd1:   48 89 44 34 08  mov    %rax,0x8(%rsp,%rsi,1)


The specific cause of the vlc crash is that the 'mov %rdi,0x20(%rsp)' at a8c38 
writes a bogus value to the res.p[1].p_pixels pointer, as rdi contains 
picture->p[0].i_lines and picture->p[0].i_pitch. The rdi register is correctly 
used by 'mov %rdi,0x18(%rsp)' at a8c3d to write them to res.p[0].i_lines and 
res.p[0].i_pitch.
When that p_pixels pointer is later accessed, it results in a segmentation 
fault.


The problem in gcc is triggered by this 
commit:https://gcc.gnu.org/git/?p=gcc.git;a=patch;h=e93428a8b056aed83a7678d4dc8272131ab671ba
>From e93428a8b056aed83a7678d4dc8272131ab671ba Mon Sep 17 00:00:00 2001
From: Richard Biener 
Date: Mon, 14 Sep 2020 11:25:04 +0200
Subject: [PATCH] tree-optimization/97043 - fix latent wrong-code with SLP
vectorization

When the unrolling decision comes late and would have prevented
eliding a SLP load permutation we can end up generating aligned
loads when the load is in fact unaligned.  Most of the time
alignment analysis figures out the load is in fact unaligned
but that cannot be relied upon.

The following removes the SLP load permutation eliding based on
the still premature vectorization factor.

2020-09-14  Richard Biener  

PR tree-optimization/97043
* tree-vect-slp.c (vect_analyze_slp_instance): Do not
elide a load permutation if the current vectorization
factor is one.
---
gcc/tree-vect-slp.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/gcc/tree-vect-slp.c b/gcc/tree-vect-slp.c
index f6331eeea86..3fdf56f9335 100644
--- a/gcc/tree-vect-slp.c
+++ b/gcc/tree-vect-slp.c
@@ -2309,9 +2309,8 @@ vect_analyze_slp_instance (vec_info *vinfo,
  /* The load requires permutation when unrolling exposes
 a gap either because the group is larger than the SLP
 group-size or because there is a gap between the groups.  */
-     && (known_eq (unrolling_factor, 1U)
-     || (group_size == DR_GROUP_SIZE (first_stmt_info)
-     && DR_GROUP_GAP (first_stmt_info) == 0)))
+     && group_size == DR_GROUP_SIZE (first_stmt_info)
+     && DR_GROUP_GAP (first_stmt_info) == 0)
{
  SLP_TREE_LOAD_PERMUTATION (load_node).release ();
  continue;
--
2.18.4


After reverting this commit on top of gcc-10 version 10.2.0-9 and recompiling 
vlc, the crash does not happen.

Given that a rather simple loop gets miscompiled, this might affect many other 
packages, as well. Thus a higher severity seems warranted.

The problematic gcc commit should be reverted or gcc otherwise be fixed.
Afterwards, vlc simply needs to be recompiled to work again properly.

Regards,
Ahzo

Bug#866974: [Aptitude-devel] Bug#866974: Patch fixing 'Assertion "resman->resolver_exists()" failed.'

2020-09-20 Thread Ahzo 
Hi Axel,

thanks for your quick and positive reply.

Indeed, for some reason the upgrade works in the TUI.
To reproduce the assertion failure in the TUI one can try to install test-b 
after having installed test-a version 2.
That corresponds to the last four steps:
# # Reproduce problem with TUI:
# apt full-upgrade
# sed -i 's/repo2/repo1/' /etc/apt/sources.list
# apt update
# aptitude    # select 'test-b' for installation, press g
Uncaught exception: ../../src/ui.cc:1549: void auto_fix_broken(): Assertion 
"resman->resolver_exists()" failed.

The equivalent CLI operation looks like:
# aptitude install test-b
The following NEW packages will be installed:
  test-b
0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/2164 B of archives. After unpacking 9216 B will be used.
The following packages have unmet dependencies:
test-a : Breaks: test-provide (< 2) which is a virtual package, provided by:
  - test-b (1), but 1 is to be installed

*** ERROR: search aborted by fatal exception.  You may continue
   searching, but some solutions will be unreachable.

I want to resolve dependencies, but no dependency resolver was created.The 
following NEW packages will be installed:
  test-b
0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/2164 B of archives. After unpacking 9216 B will be used.
aptitude failed to find a solution to these dependencies.  You can solve them 
yourself by hand or type 'n' to quit.
The following packages have unmet dependencies:
test-a : Breaks: test-provide (< 2) which is a virtual package, provided by:
  - test-b (1), but 1 is to be installed

Resolve these dependencies by hand? [N/+/-/_/:/?]

Regards,
Ahzo

Bug#866974: Patch fixing 'Assertion "resman->resolver_exists()" failed.'

2020-09-19 Thread Ahzo 
Control: tag -1 + patch

Hi,

the failure can be reliably reproduced with both the CLI (fatal exception) and 
the TUI (assertion failure) in a minimal chroot:
# apt install aptitude equivs apt-utils
# cd /tmp && mkdir test
# cat < test/test-a1.ctl
Package: test-a
Version: 1
Description: test package a
EOF
# cat < test/test-b1.ctl
Package: test-b
Version: 1
Provides: test-provide (= 1)
Description: test package b
EOF
# cat < test/test-a2.ctl
Package: test-a
Version: 2
Breaks: test-provide (<< 2)
Description: test package a
EOF
# mkdir build && cd build
# for ctl in ../test/*.ctl; do equivs-build "$ctl"; done; cd ..
# mkdir test-repo1; mv build/*1*.deb test-repo1
# cd test-repo1; apt-ftparchive packages ./ > Packages; cd ..
# mkdir test-repo2; mv build/*2*.deb test-repo2
# cd test-repo2; apt-ftparchive packages ./ > Packages; cd ..
# cat <> /etc/apt/sources.list
deb [ trusted=yes ] file:/tmp/test-repo1 ./
EOF
# apt update
# apt install test-a test-b
# sed -i 's/repo1/repo2/' /etc/apt/sources.list
# apt update
# # Reproduces problem with CLI:
# aptitude upgrade
The following packages will be upgraded:
  test-a{b}
1 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/2164 B of archives. After unpacking 0 B will be used.
The following packages have unmet dependencies:
test-a : Breaks: test-provide (< 2) which is a virtual package, provided by:
  - test-b (1), but 1 is installed

*** ERROR: search aborted by fatal exception.  You may continue
   searching, but some solutions will be unreachable.

I want to resolve dependencies, but no dependency resolver was created.The 
following packages will be upgraded:
  test-a{b}
1 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/2164 B of archives. After unpacking 0 B will be used.
aptitude failed to find a solution to these dependencies.  You can solve them 
yourself by hand or type 'n' to quit.
The following packages have unmet dependencies:
test-a : Breaks: test-provide (< 2) which is a virtual package, provided by:
  - test-b (1), but 1 is installed

Resolve these dependencies by hand? [N/+/-/_/:/?]
# # Reproduce problem with TUI:
# apt full-upgrade
# sed -i 's/repo2/repo1/' /etc/apt/sources.list
# apt update
# aptitude    # select 'test-b' for installation, press g
Uncaught exception: ../../src/ui.cc:1549: void auto_fix_broken(): Assertion 
"resman->resolver_exists()" failed.


The problem is that aptitude explicitly ignores provides when trying to find 
solutions for versioned Breaks/Conflicts.
Thus it proposes a "solution" that actually does not resolve the dependencies, 
as aptitude later notices, hence the fatal exception/assertion failure.

Attached is a patch which fixes the problem by removing two checks for 
!TargetVer() in the resolver code responsible for examining Provides.
It seems as if these were overlooked, when introducing support for versioned 
provides.

I've also attached a patch fixing the missing newline, as suggested by 積丹尼 Dan 
Jacobson.

Regards,
Ahzo
>From c9a00b1e9950227032bfb23ca706c73b5d06b67d Mon Sep 17 00:00:00 2001
From: Ahzo 
Date: Sat, 19 Sep 2020 22:09:12 +0200
Subject: [PATCH 1/2] cmdline: Add missing newline in error message.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Thanks to 積丹尼 Dan Jacobson for suggesting this.
---
 src/cmdline/cmdline_resolver.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/cmdline/cmdline_resolver.cc b/src/cmdline/cmdline_resolver.cc
index a20d5da5..a5fe4305 100644
--- a/src/cmdline/cmdline_resolver.cc
+++ b/src/cmdline/cmdline_resolver.cc
@@ -814,7 +814,7 @@ aptitude_solution calculate_current_solution(bool suppress_message,
 
   if(!resman->resolver_exists())
 {
-  const std::string msg = _("I want to resolve dependencies, but no dependency resolver was created.");
+  const std::string msg = _("I want to resolve dependencies, but no dependency resolver was created.\n");
 
   throw CmdlineSearchAbortedException(msg);
 }
-- 
2.28.0

>From 205fcf1b32e633a228795939f9670d23576cec20 Mon Sep 17 00:00:00 2001
From: Ahzo 
Date: Sat, 19 Sep 2020 22:11:44 +0200
Subject: [PATCH 2/2] resolver: support breaks and conflicts on versioned
 provides

Previously, provides were not considered for versioned breaks/conflicts.

Versioned provides are allowed since policy version 4.4.0.

Closes: #866974, #867006
---
 src/generic/apt/aptitude_resolver_universe.cc | 6 ++
 src/generic/apt/aptitude_resolver_universe.h  | 3 ---
 2 files changed, 2 insertions(+), 7 deletions(-)

diff --git a/src/generic/apt/aptitude_resolver_universe.cc b/src/generic/apt/aptitude_resolver_universe.cc
index 7ff50f0d..05477c81 100644
--- a/src/generic/apt/aptitude_resolver_universe.cc
+++ b/src/generic/apt/aptitude_resolver_universe.cc

Bug#922038: zulucrypt-cli does not work

2019-03-13 Thread Ahzo 
This bug has also been reported upstream:
https://github.com/mhogomchungu/zuluCrypt/issues/110 


It is fixed by this commit:
https://github.com/mhogomchungu/zuluCrypt/commit/25ab6fc33cba8600e8ceb6e3cd46c7b3ca68ff85.patch
 


A workaround is preloading libgcrypt:
# LD_PRELOAD=/lib/x86_64-linux-gnu/libgcrypt.so.20 zuluCrypt-cli