It's not quite as simple. Concurrency is actually not the main issue here.
The issue is that self._populate() only populates the app_dict, namespace_dict
and reverse_dict for the currently active language. By short-circuiting if
the resolver is populating, you have the chance to short-circuit while the
populating thread has a different active language. The short-circuiting
thread's language won't have been populated, and that will result in the
above KeyError.
The issue that self._populating is solving is that a RegexURLResolver can
recursively include itself if a namespace is used. Namespaces are loaded
lazily on first access, so this would generally not result in infinite
recursion. But, to include all namespaced views in self._callback_strs, you
need to load them immediately. self._populating prevents infinite recursion
in that specific case.
On a side note: recursively including the same resolver is possible under
some limited circumstances, but so far I've only seen it happen in the
Django test suite, and it doesn't even work if you don't include at least
one namespace between each recursive include. It's an edge-case scenario
that can be solved better by using a repeating pattern in your regex. I
don't think that it provides any value, but it does significantly
complicate the code. Removing this accidental "feature" would solve the
issue as well, without complicating the code any further.
Now, to solve the issue within the constraints of the current test suite,
you only need to prevent that self._populate() is called twice on the same
object within the same thread. A simple thread-local object should do the
trick:
class RegexURLResolver(LocaleRegexProvider):
def __init__(self, regex, urlconf_name, default_kwargs=None,
app_name=None, namespace=None):
...
self._local = threading.local()
self._local.populating = False
...
def _populate(self):
if self._local.populating:
return
self._local.populating = True
...
self._local.populating = False
This will work concurrently, because all lists (lookups, namespaces, apps)
are built up in local variables, and then set in
self._namespace_dict[language_code] etc. as an atomic operation. The worst
that can happen is that the list is overwritten atomically with an
identical list. self._callback_strs is a set, so updating it with values
that are already present is not a problem either.
Marten
On Wednesday, July 13, 2016 at 12:22:36 AM UTC+2, Cristiano Coelho wrote:
>
> Keep in mind your code guys is semantically different from the one of the
> first post.
>
> In the first post, the _populate method can be called more than once, and
> if the time window is long enough, the two or more calls will eventually
> run the # ... populate code again, but on your code, the populate code will
> only be called once doesn't matter how many times you call _populate,
> unless the populated variable is restarted somewhere else. So I don't know
> how is this exactly used but make sure to check it
>
> El martes, 12 de julio de 2016, 14:58:12 (UTC-3), Aymeric Augustin
> escribió:
>>
>> On 12 Jul 2016, at 19:46, Florian Apolloner <[email protected]> wrote:
>>
>>
>> On Tuesday, July 12, 2016 at 9:25:37 AM UTC+2, Aymeric Augustin wrote:
>>>
>>> Can you check the condition inside the lock? The following pattern seems
>>> simpler to me:
>>>
>>
>> The standard pattern in such cases is to check inside and outside.
>> Outside to avoid the lock if you already populated (the majority of
>> requests) and inside to see if another thread populated it in the time you
>> waited to get the lock…
>>
>>
>> Yes, actually that’s what I did the last time I implemented this pattern,
>> in Apps.populate.
>>
>> --
>> Aymeric.
>>
>>
>>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/dfa519d1-cee3-4ed5-b4ff-a5e902b5310e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
