Re: Multiple database support
Another couple weeks have slipped by and I continue to be crazy-busy. (But each week I'm busy for a different reason--so I continue to be foolishly optimistic that I'll soon get a week with some free time.) Anyway, I don't have time to read this thread through with the care it deserves, but I thought I shouldn't let that stop me from finally writing a description of the API I proposed at the PyCon sprint. Each app would have a databases.py file that contains classes used to define databases connections (in the same manner as classes are used to define models). Here's an example: from django.db import connections class LegacyDatabase(connections.DatabaseConnection): engine = 'sqlite3' name = '/foo/bar/legacy_db.sqlite3' (And the other DATABASE_* settings (from settings.py) could certainly be defined as attributes of a DatabaseConnection class.) JUST FOR TESTING, I propose we allow a database connection to be specified in a model with a Meta attribute, like this: from django.db import models from legacy.databases import LegacyDatabase class LegacyStuff(models.Model): ... class Meta: db_connection = LegacyDatabase Jacob expressed his extreme distaste for this at PyCon--for good reason. (We don't want to encourage coupling models to databases.) But just so we can get a working patch and start testing, I propose we go with this for now. Adrian suggested we allow the specification of database connections per-app using the new app() function being proposed for settings.py. I haven't seen a description of this since PyCon, but I think it would look something like: app(name='legacy', db_connection='LegacyDatabase') (I'm sure I'm leaving several important arguments out of this example.) Perhaps one could implement sharding by defining multiple DatabaseConnection classes in a databases.py file (we could support these files at the project level in addition to the app level) and putting them in a list. Then one could write a function to return the appropriate database to use and specify that callable in the argument to the app function (or perhaps as an argument to the url function in urls.py). I haven't given any thought to replication. Perhaps someone who needs this could think about whether this proposal could somehow make supporting replication easier (or if it might get in the way), or if it's simply orthogonal to this. -- Daryl On Wed, May 28, 2008 at 12:25 AM, koenb <[EMAIL PROTECTED]> wrote: > > On 22 mei, 18:28, "Ben Ford" <[EMAIL PROTECTED]> wrote: >> Hi all, >> >> I've now re-applied Daryls patch (which was against qsrf) to a clone of >> django trunk in a mercurial repo. It's available >> athttp://hg.woe-beti.deandthere's a trac set up for it >> athttp://trac.woe-beti.de. Feel free to make use of both of these. Although >> I've disabled to ability to create tickets perhaps the wiki might be a good >> place to discuss the API? Anyone can clone from the hg repo, give me a shout >> if you would like push access and I'll sort it out. >> >> Cheers, >> Ben >> >> -- >> Regards, >> Ben Ford >> [EMAIL PROTECTED] >> +447792598685 > > I have been adding some code to Ben's mercurial repo on [http://hg.woe- > beti.de], see also [http://trac.woe-beti.de]. > > What has been realised (more or less): > - connection and model registration > - validation that related objects use the same connection > - database_engine specific SQL generation (needs more checking) > - some management commands accept connection parameter, others can > generate output for multiple connections > - syncdb can sync different connections > - transaction management over connections > - some tests (needs a lot more work) > > This means point 3 of the discussion at [http://trac.woe-beti.de/wiki/ > Discuss] is somewhat working, but definitely needs a lot more testing. > > I do need some help with creating tests for all this though. > I have not figured out yet how to create tests that check that the > right SQL is being generated for the backend used. (Nor how to test > the right database was touched by an action, this is quite obvious > manually, but I do not know how to automate this.) > > I put some ideas on using multiple databases for copying (backup or > migration) of objects (point 4 of the discussion note) in [http:// > trac.woe-beti.de/wiki/APIModelDuplication]. > > Please comment, add, shoot etc. Any help will be much appreciated. > > Koen > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multiple database support
> Is there a ticket in django we could use to track progress on this? We could > use 4747, but if we do decide on a new API that might be a bit confusing... > We could of course just use the mailing list and trac project, thoughts? There's also http://code.djangoproject.com/ticket/1142. With the mailing list and trac project, do we need a ticket for more than just a place to attach patches to invite others to test? > I'll sort out the hg repo (it now needs to point at trunk - not qsrf) and > trac project if I get time this evening and make it public readable for > everyone who's interested. Thanks Ben. -- Daryl On Wed, May 21, 2008 at 3:33 AM, Ben Ford <[EMAIL PROTECTED]> wrote: > Hi all, > > I'll sort out the hg repo (it now needs to point at trunk - not qsrf) and > trac project if I get time this evening and make it public readable for > everyone who's interested. > > Is there a ticket in django we could use to track progress on this? We could > use 4747, but if we do decide on a new API that might be a bit confusing... > We could of course just use the mailing list and trac project, thoughts? > > It's great to see some interest in multiple db support again :-) > > Ben > > 2008/5/20 Nicola Larosa (tekNico) <[EMAIL PROTECTED]>: >> >> Daryl Spitzer wrote: >> > If I don't, I see if I can at least make enough time to write up the API >> > I came up with at PyCon. >> >> Please do, that would be great. >> >> -- >> Nicola Larosa - http://www.teknico.net/ >> >> > > > > -- > Regards, > Ben Ford > [EMAIL PROTECTED] > +447792598685 > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: document based database
Perhaps bedros meant to ask if anyone is working on support in Django for any "document based" databases. strokeDB looks (to my untrained eye) similar to CouchDB. You'll find plenty to read if you do a search for "couchdb django". -- Daryl On Tue, May 20, 2008 at 4:06 PM, Collin Grady <[EMAIL PROTECTED]> wrote: > > bedros said the following: >> are you guys aware of any document based database open source >> implementation in Python such as strokeDB for Ruby > > This question does not belong on this list - this list is for the discussion > of > the development of django itself, not for other questions. > > -- > Collin Grady > > The nation that controls magnetism controls the universe. >-- Chester Gould/Dick Tracy > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multiple database support
I've unfortunately been too busy to make time to work on this since PyCon. The last thing I've done (after writing some code on the flight home) is to send a patch to Ben Ford. Not long after that Ben created a Mercurial repository (with my patch) and a Trac project. You'll want to contact him. I would still like to get my patch working so others (and myself) can start testing it. I won't have time this week, but so far it looks like I may be able to make some time next week. If I don't, I see if I can at least make enough time to write up the API I came up with at PyCon. -- Daryl On Tue, May 20, 2008 at 8:05 AM, Casper Jensen <[EMAIL PROTECTED]> wrote: > > On Tue, May 20, 2008 at 4:56 PM, Nicola Larosa (tekNico) > <[EMAIL PROTECTED]> wrote: >> It would be nice to coordinate each one's efforts, to avoid wasting >> time. Ben, Daryl, any news? > Currently, I have not worked on the project, since the proposal, > because of job and university commitments. I plan to track the > development at begin to help with the development when I get more time > (over the summer). > > - Casper > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re: Porting Django to Python 3.0 as a GSoC project
Ah. I'm glad I brought it up. When the time comes to port my code, I'll try skipping step 1 first. -- Daryl On Fri, Mar 28, 2008 at 5:48 PM, Martin v. Löwis <[EMAIL PROTECTED]> wrote: > > > I think you misunderstand the role of 2.6. See the seven steps under > > "The recommended development model for a project that needs to support > > Python 2.6 and 3.0 simultaneously..." in > > http://www.python.org/dev/peps/pep-3000/#compatibility-and-transition. > > Step 1 reads "Port your project to Python 2.6." > > I believe this recommendation is flawed. It assumes that it is acceptable > that you break 2.5-and-earlier compatibility in the process of porting to > 2.6. > This, in turn, is assumed because it was considered impractical to have > code that runs on 2.3, yet 2to3 could still fix it correctly. Again in turn, > this assumption results from the actual lack of opportunity to try out any > other strategy. > > At the PyCon sprint, I tried a different strategy for Django, and it seems > to work fairly well. > > The authors of the PEP certainly did not mean to say that 2to3 can't possibly > create correct output if applied to source that also runs on 2.5. > > Regards, > Martin > > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re: Porting Django to Python 3.0 as a GSoC project
I think you misunderstand the role of 2.6. See the seven steps under "The recommended development model for a project that needs to support Python 2.6 and 3.0 simultaneously..." in http://www.python.org/dev/peps/pep-3000/#compatibility-and-transition. Step 1 reads "Port your project to Python 2.6." And step 6 is "If problems are found, make corrections to the 2.6 version of the source code and go back to step 3." I take this to mean that one likely needs to make 2.6-specific changes in order to get the code working on 3.0 after running it through 2to3. -- Daryl On Fri, Mar 28, 2008 at 12:32 PM, Patryk Zawadzki <[EMAIL PROTECTED]> wrote: > > On Fri, Mar 28, 2008 at 8:07 PM, Daryl Spitzer <[EMAIL PROTECTED]> wrote: > > On Fri, Mar 28, 2008 at 8:57 AM, Jacob Kaplan-Moss > > > > Ahh -- this was the part I was missing; my apologies for being dense. > > > I've been thinking of 2to3 as a one-time tool -- run it to move to > > > 3.0, and never look back -- not as part of a distribution process. > > Unless I'm missing something, wouldn't Django likely have to include > > code that would only work on 2.6 for this to work? > > As far as 2to3 is involved, you are supposed to modify the python 2.5 > code until 2to3 gives perfect conversion, not play with the results. > The goal is to prepare the code so the future move involves running > one tool, not to convert now and drop 2.5 support. So the only way > this could work is to alter possibly ambiguous 2.5 code and watch the > diffs generated by 2to3 without actually switching to python 2.6/3.0. > > -- > Patryk Zawadzki > PLD Linux Distribution > > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re: Porting Django to Python 3.0 as a GSoC project
On Fri, Mar 28, 2008 at 8:57 AM, Jacob Kaplan-Moss <[EMAIL PROTECTED]> wrote: > On Thu, Mar 27, 2008 at 5:10 PM, Martin v. Löwis > <[EMAIL PROTECTED]> wrote: > > So leave the code as-is, and have 2to3 fix it at installation > > time (whenever setup.py is invoked by 3.x; setup.py itself > > runs without changes on 3.x) > > Ahh -- this was the part I was missing; my apologies for being dense. > I've been thinking of 2to3 as a one-time tool -- run it to move to > 3.0, and never look back -- not as part of a distribution process. Unless I'm missing something, wouldn't Django likely have to include code that would only work on 2.6 for this to work? -- Daryl On Fri, Mar 28, 2008 at 8:57 AM, Jacob Kaplan-Moss <[EMAIL PROTECTED]> wrote: > > On Thu, Mar 27, 2008 at 5:10 PM, Martin v. Löwis > <[EMAIL PROTECTED]> wrote: > > So leave the code as-is, and have 2to3 fix it at installation > > time (whenever setup.py is invoked by 3.x; setup.py itself > > runs without changes on 3.x) > > Ahh -- this was the part I was missing; my apologies for being dense. > I've been thinking of 2to3 as a one-time tool -- run it to move to > 3.0, and never look back -- not as part of a distribution process. > Neat trick -- wish I'd thought of it :) > > So this means, though, that folks running from SVN will still need to > run setup.py every time they update, right? Not that that's a > dealbreaker -- I think Django-on-Py3k'ers will be on the cutting edge > anyway -- just wanna check. > > I'd still be happier thinking about this process post-Django-1.0 and > post-Py3k-release (or at least rc status), but I've got no objections > to other folks working on it until then. > > As for SoC, I can only see it working under these conditions: > > (1) A student with a fair deal of both Django and Python knowledge. > The Python knowledge is more important; the places where bugs in > Django are going to crop up are going to be internal, undocumented > areas. This student also should probably be prepared to work with 2to3 > fixers, though I've got a lot of knowledge here and can help out. > (2) A mentor ditto. I suspect finding a mentor interested in this > project could be difficult. > (3) Willingness from both parties to work from Martin's plan/patches > forward. Now that I understand how he's going about this, I'm > convinced it's the correct path. > (4) A *realistic*, *solid* project plan -- with dates -- that clearly > lay out what "success" means in this context. This plan should *not* > define "success" as "all code merged into Django in August" since that > probably will depend on lots of factors out of your control. > (5) A commitment from mentor and student to stick around after the SoC > and help idiots like me get the code merged once the time comes. > > Jacob > > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re: Porting Django to Python 3.0 as a GSoC project
Jacob writes: > It's hard enough maintaining 2.3, 2.4, and 2.5 side-by-side... Do you maintain them side-by-side, or do you just reject patches that require new features introduced in 2.4 and 2.5? (I just assumed that you maintain 2.3 compatibility by testing on 2.3.) -- Daryl On Thu, Mar 27, 2008 at 12:43 PM, Jacob Kaplan-Moss <[EMAIL PROTECTED]> wrote: > > On Thu, Mar 27, 2008 at 2:30 PM, Rodrigo Bernardo Pimentel > <[EMAIL PROTECTED]> wrote: > > Thanks again for your feedback, James! I'd love to hear more from other > > developers on this matter. > > I have to say I agree with James on this one. SoC projects out to be > stuff that can *finished* in a couple of months; Django on 3.0 is a > long process, and a sorta-kinda version just isn't worth anyone's > efforts. > > > > I have a different view of GSoC projects. I don't think it should be an > > isolated, "end of summer, end of story" project. > > I'm really glad you see it that way -- my dream for SoC is that > students stick around and become contributors. However, that's > something that's impossible to predict so we have to be pragmatic in > what projects we accept. I have no reason to doubt your commitment, > but something as big and open-ended as this project just isn't a good > match. > > The biggest problem is maintainance. I don't think we've got the > collective time and energy to maintain two parallel versions of Django > -- even if one version is automatically generated with 2to3. It's hard > enough maintaining 2.3, 2.4, and 2.5 side-by-side; 3.0 isn't yet worth > the effort. > > We've committed to 2.3 compatibility in Django 1.0, and that's what > we'll do. Once that's behind us, we can start dropping older versions > until we're ready to make the big jump. > > Jacob > > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
