Bug#974107: Segfault when running isenkram-lookup

[Matthias Klumpp]

Am Fr., 13. Nov. 2020 um 01:28 Uhr schrieb Gunnar Hjalmarsson
:
>
> On 2020-11-12 23:20, Matthias Klumpp wrote:
> > You want this patch for AppStream:
> > https://github.com/ximion/appstream/commit/b52858bff55d358f925b8e64bab77b953067f248
>
> But only comments are changed there!?
>
> Still... I built AppStream with that commit applied:
>
> https://launchpad.net/~gunnarhj/+archive/ubuntu/appstream
>
> and that miraculously fixes it. Thanks!
>
> (I'm getting old.)
>
> > In any case, it'll be fixed in Debian soon-ish with the new AppStream
> > release, but Ubuntu will need a backported fix.
>
> Right, we'll need to backport to Ubuntu 20.10. For some reason isenkram
> is not present in the archive for Ubuntu 20.04. Is there any other
> reason for such an appstream backport to 20.04?
>
> On 2020-11-12 23:54, Petter Reinholdtsen wrote:
> > Should this BTS report be reassigned somewhere, given that the
> > code issue is not in isenkram?
>
> Yeah, looks like it should be reassigned to the appstream package.

Looks like I was speaking too soon and this isn't actually an issue in
AppStream - my fix resolves the issue with Python, but breaks the Vala
bindings instead: https://github.com/ximion/appstream/issues/289
So, this has to be fixed in some other tool...
Also, please don't backport that patch.

Cheers,
Matthias

-- 
I welcome VSRE emails. See http://vsre.info/

Bug#974107: Segfault when running isenkram-lookup

[Gunnar Hjalmarsson]

On 2020-11-12 23:20, Matthias Klumpp wrote:

You want this patch for AppStream:
https://github.com/ximion/appstream/commit/b52858bff55d358f925b8e64bab77b953067f248


But only comments are changed there!?

Still... I built AppStream with that commit applied:

https://launchpad.net/~gunnarhj/+archive/ubuntu/appstream

and that miraculously fixes it. Thanks!

(I'm getting old.)


In any case, it'll be fixed in Debian soon-ish with the new AppStream
release, but Ubuntu will need a backported fix.


Right, we'll need to backport to Ubuntu 20.10. For some reason isenkram 
is not present in the archive for Ubuntu 20.04. Is there any other 
reason for such an appstream backport to 20.04?


On 2020-11-12 23:54, Petter Reinholdtsen wrote:

Should this BTS report be reassigned somewhere, given that the
code issue is not in isenkram?


Yeah, looks like it should be reassigned to the appstream package.

--
Gunnar Hjalmarsson
https://launchpad.net/~gunnarhj

Bug#974107: Segfault when running isenkram-lookup

[Petter Reinholdtsen]

[Matthias Klumpp]
> Oh, btw: Thank you all for the last messages, knowing which commands
> to run to reproduce this easily helped quite a bit with finding the
> culprit quite quickly (and I also looked in the code for other
> instances of this, but found only the ones in AsPool).

Well, thank you for looking into this.  Should this BTS report be
reassigned somewhere, given that the code issue is not in isenkram?

-- 
Happy hacking
Petter Reinholdtsen

Bug#974107: Segfault when running isenkram-lookup

[Matthias Klumpp]

Am Do., 12. Nov. 2020 um 23:20 Uhr schrieb Matthias Klumpp :
> [...]
> You want this patch for AppStream:
> https://github.com/ximion/appstream/commit/b52858bff55d358f925b8e64bab77b953067f248
> Basically, Python thought it could free more objects than it should
> actually have freed - not sure why this has worked before, this
> behavior was apparently broken for quite a while. Maybe it just worked
> by accident, or another bug was fixed in Python which triggered this
> issue.
>
> In any case, it'll be fixed in Debian soon-ish with the new AppStream
> release, but Ubuntu will need a backported fix.
> [...]

Oh, btw: Thank you all for the last messages, knowing which commands
to run to reproduce this easily helped quite a bit with finding the
culprit quite quickly (and I also looked in the code for other
instances of this, but found only the ones in AsPool).

Cheers,
Matthias

-- 
I welcome VSRE emails. See http://vsre.info/

Bug#974107: Segfault when running isenkram-lookup

[Matthias Klumpp]

Am Do., 12. Nov. 2020 um 22:21 Uhr schrieb Gunnar Hjalmarsson
:
>
> On 2020-11-12 21:28, Petter Reinholdtsen wrote:
> > [Gunnar Hjalmarsson]
> >> Additional observations:
> >>
> >> * The testing failure in Ubuntu started around October 31.
> >>
> >> * The test-command-line script fails on amd64 and i386 but succeeds on
> >> other architectures.
> >
> > Is it possible to 'diff' the logs from a successful and a failing test,
> > to see what changed?  Perhaps a new version of some package or
> > something?
> >
> > I get the following when running in my Sid chroot:
> >
> >isenkram-lookup
> >
> >(process:10498): GLib-GObject-CRITICAL (recursed) **:
> >g_object_get_qdata: assertion 'G_IS_OBJECT (object)' failedAborted
> >
> > No idea what it mean. :)
>
> Unfortunately I don't have the knowledge to help with the analysis. But
> yes, since isenkram hasn't changed recently, the explanation ought to be
> changes in some other package(s).
>
> Anyway, I attached two logs from test runs in Ubuntu's autopkgtest which
> passed. One is on amd64 from October 29 and one is on arm64 from
> yesterday. They both show a bunch of warning messages, but lack the
> "KeyError" from a recent amd64 test run (attached in my last message).

You want this patch for AppStream:
https://github.com/ximion/appstream/commit/b52858bff55d358f925b8e64bab77b953067f248
Basically, Python thought it could free more objects than it should
actually have freed - not sure why this has worked before, this
behavior was apparently broken for quite a while. Maybe it just worked
by accident, or another bug was fixed in Python which triggered this
issue.

In any case, it'll be fixed in Debian soon-ish with the new AppStream
release, but Ubuntu will need a backported fix.

Cheers,
Matthias

-- 
I welcome VSRE emails. See http://vsre.info/

Bug#974107: Segfault when running isenkram-lookup

[Gunnar Hjalmarsson]

On 2020-11-12 21:28, Petter Reinholdtsen wrote:

[Gunnar Hjalmarsson]

Additional observations:

* The testing failure in Ubuntu started around October 31.

* The test-command-line script fails on amd64 and i386 but succeeds on
other architectures.


Is it possible to 'diff' the logs from a successful and a failing test,
to see what changed?  Perhaps a new version of some package or
something?

I get the following when running in my Sid chroot:

   isenkram-lookup

   (process:10498): GLib-GObject-CRITICAL (recursed) **:
   g_object_get_qdata: assertion 'G_IS_OBJECT (object)' failedAborted

No idea what it mean. :)


Unfortunately I don't have the knowledge to help with the analysis. But 
yes, since isenkram hasn't changed recently, the explanation ought to be 
changes in some other package(s).


Anyway, I attached two logs from test runs in Ubuntu's autopkgtest which 
passed. One is on amd64 from October 29 and one is on arm64 from 
yesterday. They both show a bunch of warning messages, but lack the 
"KeyError" from a recent amd64 test run (attached in my last message).


--
Gunnar Hjalmarsson
https://launchpad.net/~gunnarhj


test-command-line-stdout_success_amd64_20201029.gz
Description: application/gzip


test-command-line-stdout_success_arm64_2020.gz
Description: application/gzip

Bug#974107: Segfault when running isenkram-lookup

[Petter Reinholdtsen]

[Gunnar Hjalmarsson]
> Additional observations:
>
> * The testing failure in Ubuntu started around October 31.
>
> * The test-command-line script fails on amd64 and i386 but succeeds on
>other architectures.

Is it possible to 'diff' the logs from a successful and a failing test,
to see what changed?  Perhaps a new version of some package or
something?

I get the following when running in my Sid chroot:

  isenkram-lookup 

  (process:10498): GLib-GObject-CRITICAL (recursed) **:
  g_object_get_qdata: assertion 'G_IS_OBJECT (object)' failedAborted

No idea what it mean. :)

-- 
Happy hacking
Petter Reinholdtsen

Bug#974107: Segfault when running isenkram-lookup

[Gunnar Hjalmarsson]

Attached please find the stdout from Ubuntu's autopkgtest when running 
the test-command-line script in amd64.


--
Gunnar Hjalmarsson
https://launchpad.net/~gunnarhj


test-command-line-stdout.gz
Description: application/gzip

Bug#974107: Segfault when running isenkram-lookup

[Gunnar Hjalmarsson]

Additional observations:

* The testing failure in Ubuntu started around October 31.

* The test-command-line script fails on amd64 and i386 but succeeds on
  other architectures.

--
Gunnar Hjalmarsson
https://launchpad.net/~gunnarhj

Bug#974107: Segfault when running isenkram-lookup

[Petter Reinholdtsen]

[Matthias Klumpp]
> Can you give me some context for this code? If this is a warning
> coming from libappstream, it means that it somehow got fed a null
> pointer instead of a valid AsProvided object into its list, which
> shouldn't happen - is your code doing that? Are you feeding it some
> special metadata? What is the component name where this is failing?

Not quite sure what you are looking for.  Just running isenkram-lookup
from isenkram-cli seem to trigger the error.  The code in question is in
/usr/lib/python3/dist-packages/isenkram/lookup.py, method
pkgs_handling_appstream_modaliases().

-- 
Happy hacking
Petter Reinholdtsen

Bug#974107: Segfault when running isenkram-lookup

[Matthias Klumpp]

Am Di., 10. Nov. 2020 um 01:08 Uhr schrieb Petter Reinholdtsen
:
>
> [Gunnar Hjalmarsson]
> > On an updated Debian testing:
> >
> > $ isenkram-lookup
> > /usr/lib/python3/dist-packages/isenkram/lookup.py:85: Warning invalid
> > uninstantiatable type '(null)' in cast to 'AsProvided'
> >provided = cpt.get_provided_for_kind(AppStream.ProvidedKind.MODALIAS)
> > Segmentation fault
> >
> > I'm not really an isenkram user, but stumbled upon this issue in
> > connection with Ubuntu's testing of package uploads. The isenkram-lookup
> > command is included in the debian/tests/test-command-line script, which
> > causes various tests to fail.
>
> Thank you for the heads up.  I'm involving the appstream maintainer,
> perhaps he got an idea how to fix this.  Look like that part has changed
> since the code was written.

Can you give me some context for this code? If this is a warning
coming from libappstream, it means that it somehow got fed a null
pointer instead of a valid AsProvided object into its list, which
shouldn't happen - is your code doing that? Are you feeding it some
special metadata? What is the component name where this is failing?

Cheers,
Matthias

-- 
I welcome VSRE emails. See http://vsre.info/

Bug#974107: Segfault when running isenkram-lookup

[Petter Reinholdtsen]

[Gunnar Hjalmarsson]
> On an updated Debian testing:
>
> $ isenkram-lookup
> /usr/lib/python3/dist-packages/isenkram/lookup.py:85: Warning invalid 
> uninstantiatable type '(null)' in cast to 'AsProvided'
>provided = cpt.get_provided_for_kind(AppStream.ProvidedKind.MODALIAS)
> Segmentation fault
>
> I'm not really an isenkram user, but stumbled upon this issue in 
> connection with Ubuntu's testing of package uploads. The isenkram-lookup 
> command is included in the debian/tests/test-command-line script, which 
> causes various tests to fail.

Thank you for the heads up.  I'm involving the appstream maintainer,
perhaps he got an idea how to fix this.  Look like that part has changed
since the code was written.

-- 
Happy hacking
Petter Reinholdtsen

Bug#974107: Segfault when running isenkram-lookup

[Gunnar Hjalmarsson]

Package: isenkram-cli
Version: 0.44
Severity: serious

On an updated Debian testing:

$ isenkram-lookup
/usr/lib/python3/dist-packages/isenkram/lookup.py:85: Warning invalid 
uninstantiatable type '(null)' in cast to 'AsProvided'

  provided = cpt.get_provided_for_kind(AppStream.ProvidedKind.MODALIAS)
Segmentation fault

I'm not really an isenkram user, but stumbled upon this issue in 
connection with Ubuntu's testing of package uploads. The isenkram-lookup 
command is included in the debian/tests/test-command-line script, which 
causes various tests to fail.


I filed the Ubuntu bug  too, which 
includes some details about the crash. OTOH it seems to very easy to 
reproduce it...


Thanks!

--
Gunnar Hjalmarsson
https://launchpad.net/~gunnarhj