Re: Database pooling vs. persistent connections
Hi, On Sunday 17 February 2013, Aymeric Augustin wrote: > **tl;dr** I believe that persistent database connections are a good idea. > Please prove me wrong :) > > [...] > > So -- did I miss something? I think you haven't -- but several points came to mind, which weren't discussed in this thread. As far as I can tell, none of them are problems, but *I* may be missing something, so I thought I should raise them here. First -- the very mention of this topic reminded me of https://code.djangoproject.com/ticket/9964, which was my pet bug for about two years. The point of that bug, though, was to make sure that transactions are properly closed before (and regardless if) the connection is closed, so while related, it should be unaffected. Second -- Tasks out of requests. Core developers participating in this discussion already have this in mind, Aymeric has commented about it in https://code.djangoproject.com/ticket/17887, but it was left out of the discussion on the thread. The suggested changes, AFAICT, modify the behavior around the end of requests only -- for tasks, nobody closed the connection before, and nobody is going to do anything different now; so that's not "something missed" either. Third -- different use patterns of multi-db. AFAICT the change treats all connections as equals, so no new problems should arise, but I can't remove a gnawing suspicion that some interaction may pop up. In particular, I'm worried because not all such patterns are automatically tested; I know my pet pattern (two aliases for one db, for different transaction behavior -- for logging into db, which shouldn't be reverted when the main transaction is rolled back) has had problems unnoticed by others (see e.g. https://code.djangoproject.com/ticket/14415). And last (and probably least) -- coordinated distributed transactions and two- phase-commit are not really supported by Django, and if you want them you need to write backend-specific code (probably a custom backend -- I haven't had the [mis]fortune to need to do this yet). I suspect such code would be affected, I'm not sure if it becomes easier or harder to write. I know the issue is mostly theoretical, and even if there is a problem with it, that's not a blocker; still, I thought it was worth a mention. Thanks, Shai. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Ticket #17093 -- quarantine global state in django.template
Hello. I would like to report progress of my work (not too much new things, rather cleaning and polishing), discuss some issues and propose an idea for next Google Summer Of Code. Progres from last time: - Solved problem with TemplateResponse -- it receives optional keyword argument called `engine` (if omitted, then it calls get_default_engine). During pickling the engine is lost. During repickling get_default_engine() is called. - Dropped idea of invalidating caches: base.invalid_var_format_string and context._standard_context_processors. These settings are used in about a dozen places in tests and it's not really necessary now. - Renamed _Template back to Template. The Template class accepts optional keyword argument called `engine`. If omitted, then it calls get_default_engine(). I've also rebased commits so the branch forks from updated master. I've reordered and squashed commits and rewritten description of commits so all work ended in less then twenty well-described commits. I tried to make all tests green before next weekend (Django Sprint is coming!), but I failed. There are some failing tests (exactly 450 xD); they are heisenbugs -- they appear only when you run all tests; if you run only one test suit, then they disappear. I guess that the problem is that the default_engine should be invalidated between some tests. I also bisected commits and I found out that the commit introducing failures is [1]. I will try to fix these problems before the sprint on Saturday, because the current state is almost ready to review (and to merge, I hope) and I would like to have something that can be reviewed at the weekend. I will sum up what I've done. I've not introduced new features. I've gathered all pieces of global state into one (the default_engine). I've introduced TemplateEngine class and I've rewritten most tests so they create their own TemplateEngine instances instead of using the global one. I tried not to introduce backward incompatibilities. I've rewritten history so in order not to break existing links I won't use ticket_17093 branch any more. Instead, see ticket_17093v2 branch [1]. [1] https://github.com/chrismedrela/django/commit/ed578d64fff8c8eb58898548d5ef1c0815c25f24 [2] https://github.com/chrismedrela/django/tree/ticket_17093v2 > I agree that it doesn't mean less global state. But there is good global > state and bad global state. For example, we could make DjangoApplication > class to avoid global state but there is no need. Global settings is OK > since we don't need the ability to create more than one django > application instance. Even tests doesn't require more than one instance > at the same time, so we can just reinitialize the application when there > is need for that. That's done by change_settings decorator and it works > well. And there is bad global state, like current template system. ... > I didn't know about Jinja2 and that it's more suitable for use outside > Django. So it seems like the only goal is refactoring -- we need to > quarantize global state, but not dependencies on settings. We also don't > need to split the template system into django-dependent and > django-independent parts. > I'm not sure the distinction between "good global state" and "bad global state" is so clear, or even exists. A major part of the motivation for #17093 was that it could be a step along the path towards a global-state-free Django (using something like a DjangoApplication or ApplicationConfig object), which would allow running multiple Django sites in a single process (which is indeed something that some people, though not most, need). This is mentioned in the ticket description. > If we are not working towards that larger goal (and I'd love to hear other opinions on whether we should be or not), and external library use of Django templates is not a strong motivator, and the testing issues are manageable via override_settings, then I think the rationale for #17093 begins to look rather weak. No-global-state would be a better from-scratch design, certainly, but will it bring enough concrete benefit now to be worth the transition? OK, I've changed my mind. I agree that the goal of refactoring is to introduce something like DjangoApplication class whose instances will own components like default_engine. So everything will be global-state-free. However, it would be annoying if you had to pass the application instance everywhere and type "trains" like "request.application.query(MyModel).objects.all()". It would break backward compability in almost every line of code. So there must be one global instance of DjangoApplication. OK, let mi introduce the idea for Google Some Of Code of this year. The idea is stolen from Pyramid [3]. The main new concept is a renderer. A renderer may be current TemplateEngine as well as JSON serializer or mako renderer. When I'm typing a view I would like to
