Thank you Sam, this additional detail really helps me understand your proposal.
-Barry > On Oct 11, 2021, at 12:06, Sam Gross <colesb...@gmail.com> wrote: > > I’m unclear what is actually retried. You use this note throughout the > document, so I think it would help to clarify exactly what is retried and why > that solves the particular problem. I’m confused because, is it the refcount > increment that’s retried or the entire sequence of steps (i.e. do you go back > and reload the address of the item)? Is there some kind of waiting period > before the retry? I would infer that if you’re retrying the refcount > incrementing, it’s because you expect subsequent retries to transition from > zero to non-zero, but is that guaranteed? Are there possibilities of > deadlocks or race conditions? > > The entire operation is retried (not just the refcount). For "dict", this > means going back to step 1 and reloading the version tag and > PyDictKeysObject. The operation can fail (and need to be retried) only when > some other thread is concurrently modifying the dict. The reader needs to > perform the checks (and retry) to avoid returning inconsistent data, such as > an object that was never in the dict. With the checks and retry, returning > inconsistent or garbage data is not possible. > > The retry is performed after locking the dict, so the operation is retried at > most once -- the read operation can't fail when it holds the dict's lock > because the lock prevents concurrent modifications. It would have also been > possible to retry the operation in a loop without locking the dict, but I was > concerned about reader starvation. (In the doc I wrote "livelock", but > "reader starvation" is more accurate.) In particular, I was concerned that a > thread repeatedly modifying a dict might prevent other threads reading the > dict from making progress. I hadn't seen this in practice, but I'm aware that > reader starvation can be an issue for similar designs like Linux's seqlock. > Acquiring the dict's lock when retrying avoids the reader starvation issue. > > Deadlock isn't possible because the code does not acquire any other locks > while holding the dict's lock. For example, the code releases the dict's lock > before calling Py_DECREF or PyObject_RichCompareBool. > > The race condition question is a bit harder to answer precisely. Concurrent > reads and modifications of a dict won't cause the program to segfault, return > garbage data, or items that were never in the dict. > > Regards, > Sam > >
signature.asc
Description: Message signed with OpenPGP
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/65OZPNMIZ2INHEIKTW7H65SORRYDK4D4/ Code of Conduct: http://python.org/psf/codeofconduct/
