> On 2 Mar 2022, at 14:22, t...@tomforb.es wrote:
>
> The faulthandler module is invaluable for tracking down segfaults in native
> code, however it is really lacking the ability to add some kind of useful
> breadcumb to aide debugging. Imagine you are running a large-scale
> distributed job over tens of millions of images and a single one causes
> opencv to segfault reliably. This can be very difficult to track down.
>
> It would be very useful to be able to add a string (limited by size) via the
> faulthandler module that is outputted along with the traceback. For example:
>
> ```
> for image in millions_of_images():
> faulthandler.context(f'{image.id=}')
> segfaulty_function(image)
>
> # Or maybe:
> for image in millions_of_images():
> with faulthandler.context(f'{image.id=}'):
> segfaulty_function(image)
> ```
>
> The traceback might be something like:
>
> ```
> Fatal Python error: Segmentation fault
>
> Current thread 0x00007fb899f39700 (most recent call first):
> .... (traceback)
> Segmentation fault
> Context: image_id=foo
> ```
>
> You could of course add logging to this function to print out the image ID
> before you run it through `segfaulty_function`, but if you're running this at
> high volume on a huge number of machines, this becomes a bit of an overhead,
> and it doesn't handle multiple threads running the function well.
This does sound useful. I use this pattern of setting a context that is only
logged if there is an error in my own code.
Until you have this have you considered turning on core dumps? Then you will be
able to see the image id in the dump file.
Alternatively you could write the context to a file. Then log the contents of
the file in a wrapper script that handlers python exiting on SEGV.
I understand what you are getting at about not wanting to log every attempt.
But you could have another program that was sent a message (over UDS) saying
"about to process {context}" and
after completion sent a message "processing of {context} completed. You would
log only the {context}
that are not completed.
Barry
> _______________________________________________
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/BG36W24KEEX4YJA7CZ3PGW4ISZGVXMH6/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/KD4YD7VRDCJPCHCN6PQ3JFAL2MDTLX3H/
Code of Conduct: http://python.org/psf/codeofconduct/
