Re: Python binding SIGABRT/SIGSEGV
On Thu 10 Feb 2022 at 13:16 +0100, Michael J Gruber wrote:
> Austin Lund venit, vidit, dixit 2022-02-10 06:56:12:
>> I'm clearly doing this python code wrong by not using the iterator correctly:
>>
>> > import notmuch2
>> >
>> > d = notmuch2.Database()
>> > m = list(d.messages("since:today"))
>> > p = m[0].path
>> > print(p)
>>
>> But I seem to be getting a SIGABRT instead of a python stack trace. Is
>> this the expected behaviour?
>
> You didn't expect it :)
>
> And this can be confusing. d.messages() returns an iterator through
> Message objects whose lifetime depends on the iterator. In contrast,
> thread.get_messages() returns on iterator through OwnedMessage objects
> whose lifetime depends on the thread.
>
> As soon as the iterator is depleted, the returned objects are (possibly)
> gone. (Well, because it's return by reference in Python, and ...)
>
> If you're interested in m[0] only you can "cheat" by not depleting the
> iterator:
>
> mm = next(d.messages("since:today"))
>
> p = mm.path
>
> This never frees the object (I think).
>
> My attempts with notmuch2._message.OwnedMessage (and db as parent)
> failed. There must be a better way, possibly using a context manager or
> search.
>
> I guess usually people just use the iterator in a for loop and do
> something with the message inside the loop (while the iterator is not
> depleted), such as converting it into a proper email.Message object
> (i.e. instantiating a new object from it).
Hum, to be fair I consider this a serious bug in the notmuch2 bindings.
You should not be able to crash whatever you do and not need to
understand the internal notmuch memory model. I tried to build the
bindings so they would keep alive what they need, but it seems it fails
here. It would be good to figure out if this can be fixed.
Cheers,
Floris
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Python binding SIGABRT/SIGSEGV
On Fri, Feb 11 2022, Michael J. Gruber wrote:
> Austin Lund venit, vidit, dixit 2022-02-10 23:21:58:
>> On Thu, Feb 10, 2022 at 01:12:47PM +0100, Michael J Gruber wrote:
>> > Austin Lund venit, vidit, dixit 2022-02-10 06:56:12:
>> > > I'm clearly doing this python code wrong by not using the iterator
>> > > correctly:
>> > >
>> > > > import notmuch2
>> > > >
>> > > > d = notmuch2.Database()
>> > > > m = list(d.messages("since:today"))
>> > > > p = m[0].path
>> > > > print(p)
>> > >
>> > > But I seem to be getting a SIGABRT instead of a python stack trace. Is
>> > > this the expected behaviour?
>> >
>> > You didn't expect it :)
>> >
>> > And this can be confusing. d.messages() returns an iterator through
>> > Message objects whose lifetime depends on the iterator. In contrast,
>> > thread.get_messages() returns on iterator through OwnedMessage objects
>> > whose lifetime depends on the thread.
>>
>> I guess I didn't say it explicitly, but I would 'expect' the python
>> interpreter to raise an exception rather than having an unhandled
>> exception terminate the program. Perhaps raising a MemoryError or
>> ReferenceError or some other exception would be better than an unhandled
>> SIGABRT.
>>
>
> In fact you did. Sorry for overlooking this. (I still find db.messages()
> vs thread.get_messages() confusing.)
>
> The way memory handling works, one could even expect a more specific
> notmuch2.ObjectDestroyedError.
>
> As for the SIGABRT: I guess this is what "python -X faulthandler" is
> for, especially in the context of C extension modules:
>
> LANG=C python -X faulthandler t.py
> Fatal Python error: Aborted
>
> Current thread 0x7f5bf667e740 (most recent call first):
> File "/usr/lib64/python3.10/site-packages/notmuch2/_message.py", line
> 131 in path
> File "/tmp/t.py", line 5 in
>
> Extension modules: _cffi_backend (total: 1)
> Abgebrochen (Speicherabzug geschrieben)
>
> (Yes, it disrespects LANG)
You'd probably need LC_ALL=C for "overwriting" whatever in other LC_*
variables has been set. IIRC LANG just set default in case particular
LC_* is not set
Unfortunately this does not help solving SIGABRT case...
Tomi
>
> Line 131 is
>
> ret = capi.lib.notmuch_message_get_filename(self._msg_p)
>
> in the path method, and since _msg_p is a base.MemoryPointer() I would
> have hoped (now) to get a notmuch2.ObjectDestroyedError. Maybe _msg_p is
> destroyed but the Message object is not? Fishing in the dark here ...
>
> Michael
>
> P.S.: Sorry for the repeated message. I had problems with list bounces
> in the past, then remembered the original alias wrong, then replied to
> the reply to the wrong (only off-list-delivered) post. g...@grubix.eu is
> the one subscribed to the list. I'll remember now.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Python binding SIGABRT/SIGSEGV
Austin Lund venit, vidit, dixit 2022-02-10 23:21:58:
> On Thu, Feb 10, 2022 at 01:12:47PM +0100, Michael J Gruber wrote:
> > Austin Lund venit, vidit, dixit 2022-02-10 06:56:12:
> > > I'm clearly doing this python code wrong by not using the iterator
> > > correctly:
> > >
> > > > import notmuch2
> > > >
> > > > d = notmuch2.Database()
> > > > m = list(d.messages("since:today"))
> > > > p = m[0].path
> > > > print(p)
> > >
> > > But I seem to be getting a SIGABRT instead of a python stack trace. Is
> > > this the expected behaviour?
> >
> > You didn't expect it :)
> >
> > And this can be confusing. d.messages() returns an iterator through
> > Message objects whose lifetime depends on the iterator. In contrast,
> > thread.get_messages() returns on iterator through OwnedMessage objects
> > whose lifetime depends on the thread.
>
> I guess I didn't say it explicitly, but I would 'expect' the python
> interpreter to raise an exception rather than having an unhandled
> exception terminate the program. Perhaps raising a MemoryError or
> ReferenceError or some other exception would be better than an unhandled
> SIGABRT.
>
In fact you did. Sorry for overlooking this. (I still find db.messages()
vs thread.get_messages() confusing.)
The way memory handling works, one could even expect a more specific
notmuch2.ObjectDestroyedError.
As for the SIGABRT: I guess this is what "python -X faulthandler" is
for, especially in the context of C extension modules:
LANG=C python -X faulthandler t.py
Fatal Python error: Aborted
Current thread 0x7f5bf667e740 (most recent call first):
File "/usr/lib64/python3.10/site-packages/notmuch2/_message.py", line
131 in path
File "/tmp/t.py", line 5 in
Extension modules: _cffi_backend (total: 1)
Abgebrochen (Speicherabzug geschrieben)
(Yes, it disrespects LANG)
Line 131 is
ret = capi.lib.notmuch_message_get_filename(self._msg_p)
in the path method, and since _msg_p is a base.MemoryPointer() I would
have hoped (now) to get a notmuch2.ObjectDestroyedError. Maybe _msg_p is
destroyed but the Message object is not? Fishing in the dark here ...
Michael
P.S.: Sorry for the repeated message. I had problems with list bounces
in the past, then remembered the original alias wrong, then replied to
the reply to the wrong (only off-list-delivered) post. g...@grubix.eu is
the one subscribed to the list. I'll remember now.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Python binding SIGABRT/SIGSEGV
On Thu, Feb 10, 2022 at 01:12:47PM +0100, Michael J Gruber wrote:
> Austin Lund venit, vidit, dixit 2022-02-10 06:56:12:
> > I'm clearly doing this python code wrong by not using the iterator
> > correctly:
> >
> > > import notmuch2
> > >
> > > d = notmuch2.Database()
> > > m = list(d.messages("since:today"))
> > > p = m[0].path
> > > print(p)
> >
> > But I seem to be getting a SIGABRT instead of a python stack trace. Is
> > this the expected behaviour?
>
> You didn't expect it :)
>
> And this can be confusing. d.messages() returns an iterator through
> Message objects whose lifetime depends on the iterator. In contrast,
> thread.get_messages() returns on iterator through OwnedMessage objects
> whose lifetime depends on the thread.
I guess I didn't say it explicitly, but I would 'expect' the python
interpreter to raise an exception rather than having an unhandled
exception terminate the program. Perhaps raising a MemoryError or
ReferenceError or some other exception would be better than an unhandled
SIGABRT.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Python binding SIGABRT/SIGSEGV
Austin Lund venit, vidit, dixit 2022-02-10 06:56:12:
> I'm clearly doing this python code wrong by not using the iterator correctly:
>
> > import notmuch2
> >
> > d = notmuch2.Database()
> > m = list(d.messages("since:today"))
> > p = m[0].path
> > print(p)
>
> But I seem to be getting a SIGABRT instead of a python stack trace. Is
> this the expected behaviour?
You didn't expect it :)
And this can be confusing. d.messages() returns an iterator through
Message objects whose lifetime depends on the iterator. In contrast,
thread.get_messages() returns on iterator through OwnedMessage objects
whose lifetime depends on the thread.
As soon as the iterator is depleted, the returned objects are (possibly)
gone. (Well, because it's return by reference in Python, and ...)
If you're interested in m[0] only you can "cheat" by not depleting the
iterator:
mm = next(d.messages("since:today"))
p = mm.path
This never frees the object (I think).
My attempts with notmuch2._message.OwnedMessage (and db as parent)
failed. There must be a better way, possibly using a context manager or
search.
I guess usually people just use the iterator in a for loop and do
something with the message inside the loop (while the iterator is not
depleted), such as converting it into a proper email.Message object
(i.e. instantiating a new object from it).
Michael
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Python binding SIGABRT/SIGSEGV
I'm clearly doing this python code wrong by not using the iterator correctly:
> import notmuch2
>
> d = notmuch2.Database()
> m = list(d.messages("since:today"))
> p = m[0].path
> print(p)
But I seem to be getting a SIGABRT instead of a python stack trace. Is
this the expected behaviour?
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
