Bug#952464: clazy: flaky arm64 autopkgtest: unable to execute command: Segmentation fault

2020-02-25 Thread Pino Toscano 
In data martedì 25 febbraio 2020 22:05:21 CET, Paul Gevers ha scritto:
> On 25-02-2020 20:41, Pino Toscano wrote:
> > The test is not flaky.
> 
> I can see why you say that now, but from my PoV (the release team) it is.

As I wrote, the conditions for this test to fail in the way it was
reported in bug are not definitely how a release is:

| Putting all the pieces together: why the autopkgtest can fail?
| The answer is simple: the version of src:llvm-defaults in the
| environment of the test is different than the one used to build clazy.

This is either:
- temporary, like what happens with the src:llvm-defaults bump to a
  newer LLVM version with rebuilds related to that (including clazy)
- a big problem for the Debian release altogether, because it would mean
  that src:llvm-defaults is stuck in unstable for any reason, while
  things built with it migrate happily to testing

> > Although, as I said, the issue "fixed itself" until the next
> > src:llvm-defaults switch, this is slightly less problematic.
> 
> llvm-defaults and gcc-10 were blocked for some days by the clazy
> regression on arm64. Can you explain why gcc-10 wasn't blocked by this
> issue on amd64? Also, the failures on arm64 started before the
> llvm-defaults upload [1] blocking glibc for some days. Do you also
> understand that?
> 
> [1]
> https://ci.debian.net/data/autopkgtest/testing/arm64/c/clazy/4223149/log.gz

Let's check this:

[FAIL] old-style-connect (Failed to build test. Check 
old-style-connect/namespaces.cpp.out for details)
---
Contents of old-style-connect/namespaces.cpp.out:
In file included from old-style-connect/namespaces.cpp:2:
old-style-connect/namespaces.h:22:9: warning: Old Style Connect 
[-Wclazy-old-style-connect]
connect(obj, SIGNAL(signal1()), obj1, SIGNAL(signal1()));
^~~
 ::Bar::signal1   ::Bar::signal1
old-style-connect/namespaces.cpp:32:5: warning: Old Style Connect 
[-Wclazy-old-style-connect]
QObject::connect(o1, SIGNAL(signal1()), o1, SLOT(slot1())); // Warning
^~  ~
 ::signal1::slot1
old-style-connect/namespaces.cpp:33:5: warning: Old Style Connect 
[-Wclazy-old-style-connect]
QObject::connect(o1, SIGNAL(signal1()), o2, SLOT(separateNSSlot())); // 
Warning
^~  ~~
 ::signal1::separateNSSlot
old-style-connect/namespaces.cpp:42:5: warning: Old Style Connect 
[-Wclazy-old-style-connect]
QObject::connect(o1, SIGNAL(signal1()), o1, SLOT(slot1())); // Warning
^~  ~
 ::MyObj::signal1   ::MyObj::slot1
old-style-connect/namespaces.cpp:43:5: warning: Old Style Connect 
[-Wclazy-old-style-connect]
QObject::connect(o1, SIGNAL(signal1()), o2, SLOT(separateNSSlot())); // 
Warning
^~  ~~
 ::MyObj::signal1   ::MyObj2::separateNSSlot
5 warnings generated.
/usr/bin/ld: cannot find -lgcc_s
clang: error: linker command failed with exit code 1 (use -v to see invocation)

This looks to me like bug #951086 of src:gcc-10, which was indeed a
legit bug in gcc-10. Also, this is a totally different case than the
build log posted when opening this bug.

-- 
Pino Toscano

signature.asc
Description: This is a digitally signed message part.

Bug#952464: clazy: flaky arm64 autopkgtest: unable to execute command: Segmentation fault

2020-02-25 Thread Paul Gevers 
Hi Pino,

Thanks for the well written response.

On 25-02-2020 20:41, Pino Toscano wrote:
> The test is not flaky.

I can see why you say that now, but from my PoV (the release team) it is.

>> With a recent upload of gcc-10 to unstable, the autopkgtest of clazy
>> failed on arm64 in testing when that autopkgtest was run with the binary
>> packages of gcc-10 from unstable.
> 
> The failures have nothing to do with gcc-10.

Sorry if it wasn't more clear from the next line that I didn't imply that.

> Although, as I said, the issue "fixed itself" until the next
> src:llvm-defaults switch, this is slightly less problematic.

llvm-defaults and gcc-10 were blocked for some days by the clazy
regression on arm64. Can you explain why gcc-10 wasn't blocked by this
issue on amd64? Also, the failures on arm64 started before the
llvm-defaults upload [1] blocking glibc for some days. Do you also
understand that?

> In the meanwhile: because of what I said above, I'm demoting the
> severity of this bug to important. Also, Paul, please re-enable the
> autopkgtest of clazy on ci.debian.net, as they will pass now.

I didn't disable the autopkgtest on ci.d.n. I told our migration
software (britney) to ignore the regression on arm64. I have removed
that hint.

Paul

[1]
https://ci.debian.net/data/autopkgtest/testing/arm64/c/clazy/4223149/log.gz



signature.asc
Description: OpenPGP digital signature

Bug#952464: clazy: flaky arm64 autopkgtest: unable to execute command: Segmentation fault

2020-02-25 Thread Pino Toscano 
severity 952464 important
thanks

In data lunedì 24 febbraio 2020 21:10:53 CET, Paul Gevers ha scritto:
> Source: clazy
> Version: 1.6-2
> Severity: serious
> Tags: sid bullseye
> X-Debbugs-CC: debian...@lists.debian.org
> User: debian...@lists.debian.org
> Usertags: flaky

The test is not flaky.

> Dear maintainer(s),
> 
> With a recent upload of gcc-10 to unstable, the autopkgtest of clazy
> failed on arm64 in testing when that autopkgtest was run with the binary
> packages of gcc-10 from unstable.

The failures have nothing to do with gcc-10.

> I looked into the history of your autopkgtest and it fails very often.

Err, not really, no need to exagerate things that are not like that.

The situation is more complex than that, and it needs a longer
explanation. No TL;DR is provided.

clazy is a clang plugin to detect more issues in Qt-based software,
and because of that it uses (and links to) the LLVM/clang libraries.
What is shipped is the following:

(1) /usr/bin/clazy
(2) /usr/bin/clazy-standalone
(3) /usr/lib/MULTI-ARCH/ClazyPlugin.so

(3) is the actual plugin, which is used like:
  $ clang -Xclang -load -Xclang ClazyPlugin.so -Xclang \
  -add-plugin -Xclang clazy [etc...]
This is links to the LLVM and clang libraries.

(2) is a similar version of (1), made as standalone executable rather
than a compiler plugin. It is used as compiler, so e.g.:
  $ clazy [...]

(1) is a shells script that wraps the usage of (3), with extra
parameters for version/help, and list the actual checks available.

clazy supports various LLVM versions (clazy 1.6 requires LLVM >= 5),
and it is generally updated whenever a new LLVM version is released.
For this reason, I let it built with the default LLVM version (that is
src:llvm-defaults), so using the unversioned llvm/clang -dev packages.
OTOH, (1) as shipped upstream calls the unversioned "clang++"
executable: because of this, we have local changes that record the path
to the clang++ tool of the clang version used during the built (i.e.
the Debian default one). It is easy to check this:

  $ dpkg -l | grep clazy
  ii  clazy   1.6-2+b1  amd64Clang 
plugin for additional warnings
  $ grep CLANGXX /usr/bin/clazy
  ${CLANGXX:-/usr/lib/llvm-9/bin/clang++} --version | head -1 | awk 
'{printf("clang version: %s\n",$3)}'
${CLANGXX:-/usr/lib/llvm-9/bin/clang++} -Qunused-arguments -Xclang -load 
-Xclang $ClazyPluginLib -Xclang -add-plugin -Xclang clazy $ExtraClangOptions 
"$@"

In addition to that, the clang-X package used is recorded as dependency:

  $ apt-cache show clazy | grep Depends
  Depends: libc6 (>= 2.14), libllvm9 (>= 1:9~svn298832-1~), libstdc++6 (>= 9), 
clang-9

So clazy pulls whichever version of clang it was built with, and thus
(1) works OOTB.

Now let's take a look at the autopkgtest structure:

  $ cat debian/tests/control 
  Tests: run-tests
  Depends: @, clang, clang-tools, python3, qtbase5-dev, qtdeclarative5-dev
  Restrictions: rw-build-tree, allow-stderr

  $ cat debian/tests/run-tests 
  #!/bin/sh
  
  set -e
  
  # show some facts about clang/clang++, so it is easier to debug issues
  clang -E -x c - -v < /dev/null
  clang++ -E -x c++ - -v < /dev/null
  
  cd tests
  ./run_tests.py --verbose

The upstream tests/run_tests.py executable executes the tests twice:
with (2) and (3). We can understand easily that the tests with (2) work
fine in all the cases, and indeed they pass flawlessy. The tests with
(3) use "clang++" as compiler name by default, and as such it is the
Debian default.

Putting all the pieces together: why the autopkgtest can fail?
The answer is simple: the version of src:llvm-defaults in the
environment of the test is different than the one used to build clazy.

In this particular case: clazy was rebuilt when src:llvm-defaults was
switched from 8 to 9, and because LLVM 9 was already in testing, the
binNMU migrated instantly to testing. However, src:llvm-defaults took
its 5 days to migrate to testing, so "clang" was still 8.
I see src:llvm-defaults migrated to testing today: this means that the
failures "disappear", since in both suites the versions of
(a) src:llvm-defaults used when building clazy
(b) src:llvm-defaults present
are the same.

What I will not do: switch away from the unversioned llvm version.
This means manually changing the used llvm version, which is a PITA,
and generally not needed for what clazy needs.

It seems like run_tests.py allows changing the "clang" executable used
with the CLANGXX environment variable. This seems a promising move,
however detecting what was the clang version used to build clazy only
by looking at the installed clazy seems hard. I will try to come up
with something in the next days to allow testing with the right
version. Although, as I said, the issue "fixed itself" until the next
src:llvm-defaults switch, this is slightly less problematic.

In the meanwhile: because of what I said above, I'm demoting the
severity of this bug to important. Also,

Processed: Re: Bug#952464: clazy: flaky arm64 autopkgtest: unable to execute command: Segmentation fault

2020-02-25 Thread Debian Bug Tracking System 
Processing commands for cont...@bugs.debian.org:

> severity 952464 important
Bug #952464 [src:clazy] clazy: flaky arm64 autopkgtest: unable to execute 
command: Segmentation fault
Severity set to 'important' from 'serious'
> thanks
Stopping processing here.

Please contact me if you need assistance.
-- 
952464: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=952464
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#952464: clazy: flaky arm64 autopkgtest: unable to execute command: Segmentation fault

2020-02-25 Thread Lisandro Damián Nicanor Pérez Meyer 
On 20/02/25 01:46, Lisandro Damián Nicanor Pérez Meyer wrote:
> On 20/02/24 09:10, Paul Gevers wrote:
> > Source: clazy
> > Version: 1.6-2
> > Severity: serious
> > Tags: sid bullseye
> > X-Debbugs-CC: debian...@lists.debian.org
> > User: debian...@lists.debian.org
> > Usertags: flaky
> > 
> > Dear maintainer(s),
> 
> Hi Paul, thanks for the bug report.
>  
> > With a recent upload of gcc-10 to unstable, the autopkgtest of clazy
> > failed on arm64 in testing when that autopkgtest was run with the binary
> > packages of gcc-10 from unstable. I looked into the history of your
> > autopkgtest and it fails very often.
> 
> I'm not the direct uploader of this package, but I'll try to fix this.
> This seems to be a failure in llvm/clang though.
> 
> Team members: I'm currently looming at 
> 

I *think* the bug is related to the clang/llvm version used, as I think they
mismatched in the autopkgtests. Maybe both debian/control and
debian/test/control should depend upon a specific clang version, and we should
try to rebuild clazy with each new version as possible, as we do with
qt creator.

Bug#952464: clazy: flaky arm64 autopkgtest: unable to execute command: Segmentation fault

2020-02-25 Thread Lisandro Damián Nicanor Pérez Meyer 
On 20/02/24 09:10, Paul Gevers wrote:
> Source: clazy
> Version: 1.6-2
> Severity: serious
> Tags: sid bullseye
> X-Debbugs-CC: debian...@lists.debian.org
> User: debian...@lists.debian.org
> Usertags: flaky
> 
> Dear maintainer(s),

Hi Paul, thanks for the bug report.
 
> With a recent upload of gcc-10 to unstable, the autopkgtest of clazy
> failed on arm64 in testing when that autopkgtest was run with the binary
> packages of gcc-10 from unstable. I looked into the history of your
> autopkgtest and it fails very often.

I'm not the direct uploader of this package, but I'll try to fix this.
This seems to be a failure in llvm/clang though.

Team members: I'm currently looming at 


Cheers, Lisandro.

Bug#952464: clazy: flaky arm64 autopkgtest: unable to execute command: Segmentation fault

2020-02-24 Thread Paul Gevers 
Source: clazy
Version: 1.6-2
Severity: serious
Tags: sid bullseye
X-Debbugs-CC: debian...@lists.debian.org
User: debian...@lists.debian.org
Usertags: flaky

Dear maintainer(s),

With a recent upload of gcc-10 to unstable, the autopkgtest of clazy
failed on arm64 in testing when that autopkgtest was run with the binary
packages of gcc-10 from unstable. I looked into the history of your
autopkgtest and it fails very often.

Because the unstable-to-testing migration software now blocks on
regressions in testing, flaky tests, i.e. tests that flip between
passing and failing without changes to the list of installed packages,
are causing people unrelated to your package to spend time on these
tests. Please either fix the test to be more robust, or or use the
"flaky" restriction for the offending test until a solution has been found.

I copied the output at the bottom of this report. All the failing tests
that I inspected look like it, albeit the test that fails differs
between runs.

I'll have the migration software ignore the results of your autopkgtest
on arm64 until this bug is fixed.

Paul

https://ci.debian.net/data/autopkgtest/testing/arm64/c/clazy/4369731/log.gz

Qt version: 51205
Qt headers: /usr/include/aarch64-linux-gnu/qt5
clang -Xclang -load -Xclang ClazyPlugin.so -Xclang -add-plugin -Xclang
clazy  -Wno-unused-value -Qunused-arguments -std=c++14 -isystem
/usr/include/aarch64-linux-gnu/qt5 -fPIC -L /usr/lib/aarch64-linux-gnu
-Xclang -plugin-arg-clazy -Xclang qt4-compat  -c  -Xclang
-plugin-arg-clazy -Xclang old-style-connect clazy/qt4compat2.cpp
Qt version: 51205
Qt headers: /usr/include/aarch64-linux-gnu/qt5
clang -Xclang -load -Xclang ClazyPlugin.so -Xclang -add-plugin -Xclang
clazy  -Wno-unused-value -Qunused-arguments -std=c++14 -isystem
/usr/include/aarch64-linux-gnu/qt5 -fPIC -L /usr/lib/aarch64-linux-gnu
-c  -Xclang -plugin-arg-clazy -Xclang old-style-connect -Xclang
-plugin-arg-clazy -Xclang export-fixes clazy/onlyQt1.cpp
Qt version: 51205
Qt headers: /usr/include/aarch64-linux-gnu/qt5
clang -Xclang -load -Xclang ClazyPlugin.so -Xclang -add-plugin -Xclang
clazy  -Wno-unused-value -Qunused-arguments -std=c++14 -isystem
/usr/include/aarch64-linux-gnu/qt5 -fPIC -L /usr/lib/aarch64-linux-gnu
-Xclang -plugin-arg-clazy -Xclang only-qt  -c  -Xclang -plugin-arg-clazy
-Xclang old-style-connect clazy/onlyQt2.cpp
Qt version: 51205
Qt headers: /usr/include/aarch64-linux-gnu/qt5
clang -Xclang -load -Xclang ClazyPlugin.so -Xclang -add-plugin -Xclang
clazy  -Wno-unused-value -Qunused-arguments -std=c++14 -isystem
/usr/include/aarch64-linux-gnu/qt5 -fPIC -L /usr/lib/aarch64-linux-gnu
-c -Wno-deprecated-declarations -Xclang -plugin-arg-clazy -Xclang
raw-environment-function raw-environment-function/main.cpp
Qt version: 51205
Qt headers: /usr/include/aarch64-linux-gnu/qt5
clang -Xclang -load -Xclang ClazyPlugin.so -Xclang -add-plugin -Xclang
clazy  -Wno-unused-value -Qunused-arguments -std=c++14 -isystem
/usr/include/aarch64-linux-gnu/qt5 -fPIC -L /usr/lib/aarch64-linux-gnu
-c  -Xclang -plugin-arg-clazy -Xclang inefficient-qlist
inefficient-qlist/main.cpp
Qt version: 51205
Qt headers: /usr/include/aarch64-linux-gnu/qt5
clang -Xclang -load -Xclang ClazyPlugin.so -Xclang -add-plugin -Xclang
clazy  -Wno-unused-value -Qunused-arguments -std=c++14 -isystem
/usr/include/aarch64-linux-gnu/qt5 -fPIC -L /usr/lib/aarch64-linux-gnu
-c  -Xclang -plugin-arg-clazy -Xclang qenums qenums/main.cpp
Running: clang -Xclang -load -Xclang ClazyPlugin.so -Xclang -add-plugin
-Xclang clazy  -Wno-unused-value -Qunused-arguments -std=c++14 -isystem
/usr/include/aarch64-linux-gnu/qt5 -fPIC -L /usr/lib/aarch64-linux-gnu
-c  -Xclang -plugin-arg-clazy -Xclang rule-of-three
rule-of-three/bug403193.cpp
output_file=rule-of-three/bug403193.cpp.out
[FAIL] rule-of-three (Failed to build test. Check
rule-of-three/bug403193.cpp.out for details)
---
Contents of rule-of-three/bug403193.cpp.out:
Stack dump:
0.  Program arguments: /usr/lib/llvm-8/bin/clang -cc1 -triple
aarch64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free
-disable-llvm-verifier -discard-value-names -main-file-name
bug403193.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix
-mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases
-fuse-init-array -target-cpu generic -target-feature +neon -target-abi
aapcs -fallow-half-arguments-and-returns -dwarf-column-info
-debugger-tuning=gdb -coverage-notes-file
/tmp/autopkgtest-lxc.ji5ue40b/downtmp/build.xGK/src/tests/bug403193.gcno
-resource-dir /usr/lib/llvm-8/lib/clang/8.0.1 -isystem
/usr/include/aarch64-linux-gnu/qt5 -internal-isystem
/usr/bin/../lib/gcc/aarch64-linux-gnu/8/../../../../include/c++/8
-internal-isystem
/usr/bin/../lib/gcc/aarch64-linux-gnu/8/../../../../include/aarch64-linux-gnu/c++/8
-internal-isystem
/usr/bin/../lib/gcc/aarch64-linux-gnu/8/../../../../include/aarch64-linux-gnu/c++/8
-internal-isystem
/usr/bin/../lib/gcc/aarch64-linux-gnu/8/../../../../include/c++/8/backward