If you are using locals it means each thread will always end up calling the actual populate code? Is there any point for the RegexURLResolver class to be a singleton then?
El jueves, 14 de julio de 2016, 9:12:54 (UTC-3), Marten Kenbeek escribió: > > It's not quite as simple. Concurrency is actually not the main issue here. > The issue is that self._populate() only populates the app_dict, > namespace_dict > and reverse_dict for the currently active language. By short-circuiting > if the resolver is populating, you have the chance to short-circuit while > the populating thread has a different active language. The short-circuiting > thread's language won't have been populated, and that will result in the > above KeyError. > > The issue that self._populating is solving is that a RegexURLResolver can > recursively include itself if a namespace is used. Namespaces are loaded > lazily on first access, so this would generally not result in infinite > recursion. But, to include all namespaced views in self._callback_strs, > you need to load them immediately. self._populating prevents infinite > recursion in that specific case. > > On a side note: recursively including the same resolver is possible under > some limited circumstances, but so far I've only seen it happen in the > Django test suite, and it doesn't even work if you don't include at least > one namespace between each recursive include. It's an edge-case scenario > that can be solved better by using a repeating pattern in your regex. I > don't think that it provides any value, but it does significantly > complicate the code. Removing this accidental "feature" would solve the > issue as well, without complicating the code any further. > > Now, to solve the issue within the constraints of the current test suite, > you only need to prevent that self._populate() is called twice on the > same object within the same thread. A simple thread-local object should do > the trick: > > class RegexURLResolver(LocaleRegexProvider): > def __init__(self, regex, urlconf_name, default_kwargs=None, > app_name=None, namespace=None): > > ... > > self._local = threading.local() > self._local.populating = False > ... > > def _populate(self): > if self._local.populating: > return > self._local.populating = True > ... > self._local.populating = False > > > This will work concurrently, because all lists (lookups, namespaces, apps) > are built up in local variables, and then set in > self._namespace_dict[language_code] etc. as an atomic operation. The > worst that can happen is that the list is overwritten atomically with an > identical list. self._callback_strs is a set, so updating it with values > that are already present is not a problem either. > > > Marten > > > On Wednesday, July 13, 2016 at 12:22:36 AM UTC+2, Cristiano Coelho wrote: >> >> Keep in mind your code guys is semantically different from the one of the >> first post. >> >> In the first post, the _populate method can be called more than once, and >> if the time window is long enough, the two or more calls will eventually >> run the # ... populate code again, but on your code, the populate code will >> only be called once doesn't matter how many times you call _populate, >> unless the populated variable is restarted somewhere else. So I don't know >> how is this exactly used but make sure to check it >> >> El martes, 12 de julio de 2016, 14:58:12 (UTC-3), Aymeric Augustin >> escribió: >>> >>> On 12 Jul 2016, at 19:46, Florian Apolloner <f.apo...@gmail.com> wrote: >>> >>> >>> On Tuesday, July 12, 2016 at 9:25:37 AM UTC+2, Aymeric Augustin wrote: >>>> >>>> Can you check the condition inside the lock? The following pattern >>>> seems simpler to me: >>>> >>> >>> The standard pattern in such cases is to check inside and outside. >>> Outside to avoid the lock if you already populated (the majority of >>> requests) and inside to see if another thread populated it in the time you >>> waited to get the lock… >>> >>> >>> Yes, actually that’s what I did the last time I implemented this >>> pattern, in Apps.populate. >>> >>> -- >>> Aymeric. >>> >>> >>> -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a9788779-8a04-44f1-b18b-e8048f4c6a4f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
