On Wed, Dec 18, 2013 at 4:41 AM, Aymeric Augustin < aymeric.augus...@polytechnique.org> wrote:
> On 17 déc. 2013, at 20:38, Karen Tracey <kmtra...@gmail.com> wrote: > > https://groups.google.com/d/msg/django-developers/0IuJssTt8tc/TxdXQ2D0thcJ > > > Interesting. Now I remember reading that message last year. I have three > reactions. > > First, Russell and I agree that the interest has faded. Currently, the > best option is (probably) django-nonrel. We're open to suggestions that > help django-nonrel but we aren't working towards a merge. > > Second, Russell talks about the ORM API, not about the ORM code. He > implies is that little code, if any, would be shared between the relational > and the non-relational backends. We agree again. > > Hey - we're two for three already :-) Third, his examples conveniently ignore relations, which is where both > relational databases and Django’s ORM shine. That’s where I disagree. > Here’s an interesting example (extracted from a real project): > > (Subscription.objects > .valid_for_reporting() > .filter(inception_date__gte=self.start_dt, > inception_date__lt=self.end_dt) > .annotate(rentals_count=Count('subscriber__rentals')) > .annotate(rentals_duration=Sum('subscriber__rentals__dn_duration')) > .annotate(rentals_distance=Sum('subscriber__rentals__dn_distance')) > > .annotate(rentals_net_amount=Sum('subscriber__rentals__pricing__net_amount')) > > .annotate(rentals_vouchers_net_amount=Sum('subscriber__rentals__pricing__voucher_details__net_amount')) > .order_by('inception_date') > .values(...) > ) > > I have some doubts about the practicality of running such a query on an > NoSQL datastore or translating it automatically to a map-reduce job. > Nope - we agree here as well. Yes - my examples ignore relations - that's the point. Deep joined queries are *never* going to be API abstracted in a NoSQL store (at least, not in an efficient way, and I don't see the point in building a complex implementation of something inefficient). If you're doing complex queries, you're always going to need to be aware of the underlying store, and lean on those capabilities. My claim is that complete abstraction of the data store shouldn't be the goal. What we should be aiming for is sufficient API compatibility to allow for two things: * ModelForms wrapping a model from a NoSQL data store * An admin representation of a NoSQL data model. These two functions don't require complex relations. They require relatively simple CRUD operations on a single object; they require an implementation of filter, but not an implementation that allows joins of any kind inside a filter clause. Yes, this will cut off some functionality -- some custom filters, for example, won't be possible. It might be necessary to remove total object counts in admin ListViews -- but then, that would be a good performance boost for PostgreSQL as well. But broadly speaking, I see no reason why it shouldn't be possible to put store a data model in NoSQL, and visualise the contents of that model in admin. And you should be able to throw up a Form wrapping that data model. I see two ways to achieve this goal. The first is to put an alternate backend into Django's existing ORM. This is the approach attempted by Alex in his GSoC project. It's possible, and actually doesn't require that much work; but there are a couple of unresolved issues, mostly around handling of non-integer automatic primary keys. These are problems that we might be worth addressing anyway, because UUID-based auto primary keys would be a worthwhile extension to SQL data stores. The second approach is to produce a duck type compatible model API. There aren't (or shouldn't be) any instance checks tied to django.db.models.Model; as long as your NoSQL layer implements the same API, you should be able to drop it into the admin or a ModelForm. Taking this approach would have the side effect that we'd have to clean up the formal definition of _meta, which in itself would be a nice goal. Added benefit -- this approach doesn't actually require any changes to core itself -- the ducks can be completely external to Django. The only changes to core would be whatever cleanups and documentation the project identifies. The other, unrelated problem isn't technical at all -- its that that enthusiasm for this problem has pretty much dried up in the core team AFAICT. I certainly don't have any call for using a NoSQL store in my life at the moment. However, if a ready-to-use implementation of either of these approaches were to land on our doorstep, I wouldn't resist them being added to core. Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAJxq849UXnT20uNiOGcGy39VteE871mqPwiLgdzgZdcLnLk9mg%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
