Re: Template Caching - Ticket #6262
On Wed, Nov 11, 2009 at 8:15 PM, Mike Malonewrote: > Sup, > > I've been working on template caching (ticket #6262) with a mister > Alex Gaynor. The patch is getting pretty stable and I think it's close > to merge-able, so if anyone wants to take a look at what we've got and > provide feedback: go! > > Interesting background reading for people who haven't been part of > this conversation: > http://groups.google.com/group/django-developers/browse_thread/thread/b289b871285b86f5/b97ba9e2e9b9ad86 > > Ticket: http://code.djangoproject.com/ticket/6262 > > Latest patch: > http://code.djangoproject.com/attachment/ticket/6262/cache_templates.5.diff > > The more code reviewers the better, but if you don't have time to read > through the nitty gritty here's a high level overview of the changes: > > 1. The workhorse django.template.loader.get_template() function will > now return compiled Template objects directly if a template loader > returns one (where a Template object is defined as something with a > render() method). > 2. A RenderContext has been added to Context instances. This is > necessary so we can give template.Node instances a thread-safe place > to store state between calls to Node.render(). > 3. The built-in template tags were updated to use the render context > (specifically, CycleNode, BlockNode, and ExtendsNode). > 4. A caching loader has been added to the set of default loaders. To > use it, you instantiate the loader, passing a list of other loaders > that it should wrap. The first time you ask the loader for a template > it'll go through the wrapped loaders and find it, then cache it in > memory. Subsequent requests for the same template are served from > cache. Hazzah! > > The patch is pretty complete, it includes tests and docs, so please > take a look! I stuck the updated docs up on my website temporarily, so > if you want to take a look at them in a slightly more readable format > you can check out these URLs: > http://immike.net/django-docs/ref/templates/api.html#loader-types > http://immike.net/django-docs/howto/custom-template-tags.html#template-tag-thread-safety > http://immike.net/django-docs/ref/templates/api.html#using-an-alternative-template-language > > Thanks, > > Mike > > -- > > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-develop...@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=. > > > As Mike said I've been reviewing this patch and I'm basically happy with it's current state. My one thought was whether to add a CACHE_TEMPLATE_LOADERS setting, and any loaders in that are cached by the CachedLoader (assuming it's instantiated with no arguments, it'll still be possible to explicitly instantiate it with loaders). Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=.
Template Caching - Ticket #6262
Sup, I've been working on template caching (ticket #6262) with a mister Alex Gaynor. The patch is getting pretty stable and I think it's close to merge-able, so if anyone wants to take a look at what we've got and provide feedback: go! Interesting background reading for people who haven't been part of this conversation: http://groups.google.com/group/django-developers/browse_thread/thread/b289b871285b86f5/b97ba9e2e9b9ad86 Ticket: http://code.djangoproject.com/ticket/6262 Latest patch: http://code.djangoproject.com/attachment/ticket/6262/cache_templates.5.diff The more code reviewers the better, but if you don't have time to read through the nitty gritty here's a high level overview of the changes: 1. The workhorse django.template.loader.get_template() function will now return compiled Template objects directly if a template loader returns one (where a Template object is defined as something with a render() method). 2. A RenderContext has been added to Context instances. This is necessary so we can give template.Node instances a thread-safe place to store state between calls to Node.render(). 3. The built-in template tags were updated to use the render context (specifically, CycleNode, BlockNode, and ExtendsNode). 4. A caching loader has been added to the set of default loaders. To use it, you instantiate the loader, passing a list of other loaders that it should wrap. The first time you ask the loader for a template it'll go through the wrapped loaders and find it, then cache it in memory. Subsequent requests for the same template are served from cache. Hazzah! The patch is pretty complete, it includes tests and docs, so please take a look! I stuck the updated docs up on my website temporarily, so if you want to take a look at them in a slightly more readable format you can check out these URLs: http://immike.net/django-docs/ref/templates/api.html#loader-types http://immike.net/django-docs/howto/custom-template-tags.html#template-tag-thread-safety http://immike.net/django-docs/ref/templates/api.html#using-an-alternative-template-language Thanks, Mike -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=.
Re: Regularly-scheduled Django sprints, first December 11 - 13
On Thu, Nov 12, 2009 at 5:57 AM, mrtswrote: > > On Nov 11, 1:57 pm, Russell Keith-Magee > wrote: >> Also - I know you're very enthused about Git and GitHub, but Django >> uses SVN as the canonical store for trunk and branches, and this isn't >> likely to change in the near future. > > Thank you, but no, I'm not enthused about git and GitHub per > se :) -- I'm enthused that it's possible to maintain patches > and collaborate remotely with less agony by utilizing them. > > Please let's be friendly and relaxed -- I was just asking > how do people plan to manage things. SVN is fine as well, > but it puts the burden of integrating work on core. It's > not the tool, but the goal that matters. SVN is a sucky tool, but it really isn't the bottleneck in the process. The bottleneck is deciding which one of a thousand patches I (or any other member of core) should merge into trunk next. Git (or any other DVCS for that matter) doesn't solve that problem. No tool will - it's a social problem, and there's no such thing as a technical solution to a social problem. >> As I have said to you in the past - if you want to make your >> contribution to Django to be a DVCS repository that acts as a staging >> area for trunk-ready patches, that would be a very helpful >> contribution. This is doubly true during a sprint - sprints generate a >> lot of activity, so having good lieutenants to triage contributions is >> extremely useful. > > Done, http://github.com/django-mq/django-mq . Awesome. I look forward to seeing your work. > I'm willing to regularly review pull requests and add > collaborators who are willing to do the same. Thats fine, but keep in mind that the value of this branch goes down if the patches that are added to the branch aren't good. The risk of adding lots of contributors is that you end up with a branch that is just every ticket in Trac applied to trunk. What we need is a genuine lieutenant - someone who exercises personal taste and rejects patches when appropriate. A good lieutenant should also give feedback to the original developer. What we don't need is someone who just applies every patch they can. Applying a patch to a tree is easy. Working out whether a patch is a good contribution is hard. > But there's the problem that I'm not Andy Morton -- > considering my karma, I'm in no position in regard > to my "social capital" to lieutenant this. But neither was Andy Morton when he started. That's the point. You don't do this sort of thing *because* you have the karma - you do it to *gain* the sort of karma that Andy Morton has. Through a lot of hard work, Andy built a reputation as being someone with good design taste that Linus can rely on as a rich vein of probably-trunk-ready patches, and as a result, he is now a trusted lieutenant. Yours, Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multi-credential ORM configuration support
On Thu, Nov 12, 2009 at 6:20 AM, Warren Smithwrote: > > If this question would be more appropriate for django-users, please > let me know. I ask it here because I suspect that the answer to my > question may require someone with a fairly deep understanding of the > ORM internals. > > I have a legacy oracle database that I would like to use with the > Django ORM. Unfortunately, the security for this database is > implemented using oracle users and roles, making it necessary for each > user to have their connection to the database use their own > credentials, which translates into a connection for each user. > Although I can retrieve the credentials from the user and keep them in > session, it is not apparent to me how to configure the ORM to > appropriately use them. Unfortunately, changing the database security > model is not an option. > > The research I have done seems to indicate that this mode of operation > is not supported. > > Am I correct or am I missing something? You're pretty much correct. Django assumes that it has a single trusted login to the database. > If it is not officially supported, is there a way it could be made to > work safely? We are already using a custom version of django 1.1.1, > so I'm not opposed to making some additional custom changes, though I > prefer to avoid them. I can't think of an easy way - connection is essentially a singleton in Django 1.1. However... > Alex and Russell, would the work you have done in the multi-db branch > have any bearing on this use case? Multi-db allows you to set up multiple database credentials; the usual reason for this is to split DB load across multiple database servers, but I don't see any reason you couldn't also use it to provide multiple connections to a single server, each with different credentials. So - you should be able to use the using() query operator to force a particular query to use a particular set of credentials (and thus a particular database connection). I haven't given this particular use case enough thought to make any comment on how the UI could be made cleaner. So far, we've been concentrating on the 'user accounts are on db X', sharding model X based on PK, and supporting master/slave replication as the end-user use cases that we want to satisfy. We'd certainly be interested in any input on ways to improve usability for the multi-credential case. Yours Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ORM roadmap
> I find an orm usefull for 3 scenarios: > 1. - simple object retrieval posts.objects.all() > 2. - performance optimized object retrieval, your raw approach would > suffice here > 3. - generating complex queries, and reusing sql components > When you get to complex data models, Django currently fails on 2 and > 3. I'd agree and elaborate on #1, that ORMs are good for simple object retrieval, and also for eliminating some of the most grievous cross-platform issues. However, I must disagree on #2 and #3: I'd never consider the *addition* of a generalization-layer (an ORM in this case, but the idea is far more broadly applicable) a step towards optimization. Likewise, for my more complex queries, there's no good way to express them efficiently in *any* ORM I've used -- many end up either inexpressible, or they involve bringing back gobs of data client-side and post-processing it there instead of at the server. I use an ORM for the same reason I use Python: they make common tasks easy. Using Python/ORM, I can bang out most of what I need. But when I hit a wall (performance or complexity), both need to allow be to go *down* a layer (to C or SQL). So I'd say Django has done a fine job of making the ORM what it should be -- an approachable, cross-platform, cross-database data-access layer that covers most use-cases and gets out of your way when you need it to. -tkc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Multi-credential ORM configuration support
If this question would be more appropriate for django-users, please let me know. I ask it here because I suspect that the answer to my question may require someone with a fairly deep understanding of the ORM internals. I have a legacy oracle database that I would like to use with the Django ORM. Unfortunately, the security for this database is implemented using oracle users and roles, making it necessary for each user to have their connection to the database use their own credentials, which translates into a connection for each user. Although I can retrieve the credentials from the user and keep them in session, it is not apparent to me how to configure the ORM to appropriately use them. Unfortunately, changing the database security model is not an option. The research I have done seems to indicate that this mode of operation is not supported. Am I correct or am I missing something? If it is not officially supported, is there a way it could be made to work safely? We are already using a custom version of django 1.1.1, so I'm not opposed to making some additional custom changes, though I prefer to avoid them. Alex and Russell, would the work you have done in the multi-db branch have any bearing on this use case? -- Warren Smith --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Regularly-scheduled Django sprints, first December 11 - 13
On Wed, Nov 11, 2009 at 3:57 PM, mrtswrote: > Done, http://github.com/django-mq/django-mq . > > I'm willing to regularly review pull requests and add > collaborators who are willing to do the same. I'll keep an eye on your queue; I can easily push good patches from there upstream to SVN. If you stick patches into separate branches -- I suggest naming 'em something like "trac/1234-short-ticket-desc" (where 1234 is the ticket number) -- that'll make it easy for us Githeads to merge in ticket fixes you post there. Please, though, stress to people who want to use your queue that they should *also* post patches to Trac. It's easier to review that way, and it's more friendly to those not using Git. Thanks Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Regularly-scheduled Django sprints, first December 11 - 13
On Nov 11, 1:57 pm, Russell Keith-Mageewrote: > Also - I know you're very enthused about Git and GitHub, but Django > uses SVN as the canonical store for trunk and branches, and this isn't > likely to change in the near future. Thank you, but no, I'm not enthused about git and GitHub per se :) -- I'm enthused that it's possible to maintain patches and collaborate remotely with less agony by utilizing them. Please let's be friendly and relaxed -- I was just asking how do people plan to manage things. SVN is fine as well, but it puts the burden of integrating work on core. It's not the tool, but the goal that matters. As far as I understand, that's one of the issues we are trying to solve -- integrating contributors' work faster than core is able to. Good for core and good for the contributors. > As I have said to you in the past - if you want to make your > contribution to Django to be a DVCS repository that acts as a staging > area for trunk-ready patches, that would be a very helpful > contribution. This is doubly true during a sprint - sprints generate a > lot of activity, so having good lieutenants to triage contributions is > extremely useful. Done, http://github.com/django-mq/django-mq . I'm willing to regularly review pull requests and add collaborators who are willing to do the same. But there's the problem that I'm not Andy Morton -- considering my karma, I'm in no position in regard to my "social capital" to lieutenant this. > The important part is *not* the documentation of this process. The > important part is *doing the work*. > > Key phrase: Do it. Not write an essay about how you would do it, and > then enter into a debate about whether the workflow that you have > proposed is optimal. Do the work, then tell a core dev that you have > done the work. Fix the patch if required. Rinse. Repeat. Wasn't the fact that it isn't and can't be that easy the main problem we discussed in the angst-thread? That core is too overburdened and small in numbers to oversee >1800 open tickets? Doing the work is usually easy, integration and avoiding staleness is not. As for "don't write essays" -- some traction is needed behind the initiative and good coordination with core so that the contributions start eventually flowing from the repo back to the trunk. Few or no people will use the repo unless there's a general consensus that it's a good thing. That consensus can only come from clear understanding of it's purpose and workflow (the "essays") -- only then can people start contributing. Otherwise they will feel that it's a waste of their time on an obscure fringe thing. All in all, I think that we are not really on different positions regarding this, it's just the karma issue that spoils the soup :) So be it, I'll just try to keep my mouth shut. Best, Mart Sõmermaa --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: #7539 (ON DELETE support) aka ORM-09 - A call for feedback
Johannes Dollinger wrote: > > Am 10.11.2009 um 17:22 schrieb Michael Glassford: > >> I haven't had a chance to look at the patch yet, but what you describe >> here sounds good. I don't have any problem with you "hijacking" the >> work. >> >> Did your patch deal at all with the unit tests in my patch that >> intentionally failed to expose things that weren't working right yet? > > Only the first of your patches contains tests. Oops, right you are. The unit tests were unintentionally omitted. I've uploaded a new patch which is the pretty much the same as my previous patch but with my unit tests included (although comparing the two now I see a few other differences, probably because I also updated to the latest code in SVN). > But the ON DELETE > related tests in this patch that do not test database-level support > should be covered by my tests in modeltests/on_delete/. > Which of your tests failed? The tests that fail are in tests/modeltests/on_delete_django/tests.py. They have names like test_issue_1a, test_issue_1b, test_issue_2a, test_issue_2b, etc. The comments on the tests indicate the expected result. Once the issues they test for are fixed, they should all pass. Mike --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Regularly-scheduled Django sprints, first December 11 - 13
On Wed, Nov 11, 2009 at 12:00 AM, Jeremy Dunckwrote: > > Djangonauts, > To help speed along development on Django, I'd like us to start > holding regularly-scheduled sprints. On behalf of the core team, I'd like to thank Jeremy for taking the lead on this. Organizing an event like a sprint isn't glamourous work, but it really helps the community, and it's an area where the core team has been letting the community down. Thanks, Jeremy! > If you're willing to organize a sprint gathering during this time > span, please let me know and I'll help to ensure committers are > available and to promote the sprint gathering. (There's actually > pretty good guidance on the Sprints wiki page [3].) If there is any interest in an IRL sprint based in Perth, Western Australia, or Brisbane, Queensland, let me know - I may be able to call in some favors and organize a venue. Failing that, I'll be floating around the tubes on the day (and evening). Yours, Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Regularly-scheduled Django sprints, first December 11 - 13
On Wed, Nov 11, 2009 at 1:06 AM, mrtswrote: > > Great news! Have you discussed the workflow yet? > I.e. will a DVCS be involved, and if yes, will there be > a central repo for coordinating the effort? > > A special repo on Github would otherwise be perfect, > but as mentioned before, we have a problem with re-forking: > http://support.github.com/discussions/feature-requests/560-two-forks-of-a-single-repo Why do we need a central repository to coordinate a sprint effort? The whole *point* using a DVCS is that you *don't* need a central repository to coordinate activity. Also - I know you're very enthused about Git and GitHub, but Django uses SVN as the canonical store for trunk and branches, and this isn't likely to change in the near future. That said, many of the core developers use DVCS tools to manage their day-to-day Django contributions. If you comment on a ticket that your patch is also available as a pull from , some core developers will be able to exploit this. As I have said to you in the past - if you want to make your contribution to Django to be a DVCS repository that acts as a staging area for trunk-ready patches, that would be a very helpful contribution. This is doubly true during a sprint - sprints generate a lot of activity, so having good lieutenants to triage contributions is extremely useful. The important part is *not* the documentation of this process. The important part is *doing the work*. > Whatever workflow and tools have been or will be > picked, may I suggest that clear instructions be written > for participants -- e.g. something in the lines of [1]? Django's workflow *is* documented: http://docs.djangoproject.com/en/dev/internals/contributing/ The wiki page on Sprints clarify this process for sprints: http://code.djangoproject.com/wiki/Sprints#Howtocontribute Quoting: """ How to contribute It's easy: Pick something to work on and do it. """ Key phrase: Do it. Not write an essay about how you would do it, and then enter into a debate about whether the workflow that you have proposed is optimal. Do the work, then tell a core dev that you have done the work. Fix the patch if required. Rinse. Repeat. Yours, Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ORM roadmap
Thanks for your feedback. I appreciate it. I find an orm usefull for 3 scenarios: 1. - simple object retrieval posts.objects.all() 2. - performance optimized object retrieval, your raw approach would suffice here 3. - generating complex queries, and reusing sql components When you get to complex data models, Django currently fails on 2 and 3. Agreed though, I guess only a handful of apps encounter these problems. On Oct 4, 6:27 am, Russell Keith-Mageewrote: > On Sun, Oct 4, 2009 at 5:00 AM, James Bennett wrote: > > > On Sat, Oct 3, 2009 at 1:58 PM, Thierry > > wrote: > >> I know this is not a problem for most users. But when working with > >> complex projects or projects requiring some performance optimization > >> django orm doesn't suffice. Its a pity to see strong development on > >> django orm, while at the same time the sqlalchemy project has huge > >> traction. I currently run both orm's. The gap between sqlalchemy and > >> django orm is very large. Are there any plans to integrate sql > >> alchemy? > > > In a word, no. > > To put in one more voice of authority - speaking as a core developer, > I'm strongly opposed to modifying Django to use SQLAlchemy (or any > other external project for that matter) as the ORM. > > On top of the many valid reasons that James mentioned, there is one > more that I consider to be very important - one that stems from the > task that _any_ ORM is trying to perform. > > The goal of any SQL-backed ORM is to provide a syntax that makes it > easy to express SQL queries. The problem is, we already have a very > powerful, 100% feature complete syntax for expressing SQL queries - > it's called SQL. By very definition, _every_ SQL-backed ORM is > reinventing the wheel. > > ORMs have an important role to play in making the simple cases very > simple - and this is a sweet spot that Django's ORM, SQLAlchemy, and > any number of other ORM projects manage quite well. It is much easier > to write "Author.objects.all()" than to write "SELECT id, firstname, > lastname, birthdate, address1, address2, FROM author". > > However, this argument isn't about the simple cases - it is about the > complex cases. I will certainly grant that SQLAlchemy is certainly > able to cover more of the world of possible SQL queries than Django's > ORM. However, there are queries that even SQLAlchemy can't express (or > can't express elegantly). At this point, you can either continue > making modifications to your non-SQL language in an attempt to provide > 100% coverage of the SQL spec, or you can step back and say "No - we > already have a language that does this", and embrace it. > > Django has decided to take the latter approach. The recent proposals > for a raw() operator to make it easier to return Django model objects > as the result of a raw SQL query is an expression of this underlying > philosophical direction. > > So, no - for this reason, and many many others, we're not going to > adopt SQLAlchemy as Django's ORM. > > That said, we are committed to making it easier to use non-Django > components inside a Django project. If there is anything that we can > do to make it easier to use SQLAlchemy (or any other non-Django ORM) > inside a Django project, we're open to those suggestions. > > Yours, > Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---