On Tuesday, December 15, 2015 at 5:43:55 PM UTC+2, Samuel Bishop wrote: > > Having worked through the code of several Django nosql/alternative > database backend libraries, forks, etc... > > I've noticed that that one of the biggest challenges they run into, is > 'conforming' to many of the things Django expects these lowest layers to do. > > I opened this ticket https://code.djangoproject.com/ticket/25265 to begin > getting feedback on an initial idea for how to 'fix' the problem. > Since then I've had further time to ponder the problem and while it still > seems to me that the best mechanism is to draw a line between the 'upper' > and 'lower' layers of Django, > I'm no longer 100% sure the correct place to enable this is the queryset > via an additional method, because I've realized that this is not just an > opportunity to get NoSQL databases into Django, but also an opportunity to > finally provide support for alternative Python ORMs, such as SQLAlchemy. > > I've been digging around the code for this so I dont mind writing up the > code for this, but there is the big question of 'where to decouple' things. > Initial feedback in the thread > https://code.djangoproject.com/ticket/25265#comment:4 > <https://www.google.com/url?q=https%3A%2F%2Fcode.djangoproject.com%2Fticket%2F25265%23comment%3A4&sa=D&sntz=1&usg=AFQjCNEeTJV5U_vgQBRIQIppH6F1Hf991Q> > > has raised the suggestion that moving one layer further up may be the right > place to go. It would be very helpful for me to get extra input from Django > developers familiar with the QuerySet and Query, before I start writing, so > I would love to hear feedback on the idea. >
Assume the goal is perfect admin integration with a MongoDB backend. The approach can be either: 1) Use Django's standard models, create a QuerySet compatible MongoDBQuerySet. 2) Use completely different models, which respond to the APIs needed by Admin. This includes implementing a QuerySet compatible MongoDBQuerySet. There is a lot more work to 2), but the benefit is that you get to use models actually meant to be used with a non-relational backend. For example, Django's User, Permission and Group models are implemented in a way that makes sense for a relational backend. If you use relational schema on non-relational database you are going to face big problems if you try to run the site with any non-trivial amount of data. For this reason I believe 2) to be the right approach. But, to get there, a QuerySet compatible MongoDBQuerySet is needed anyways. Here the choices are those mentioned in https://code.djangoproject.com/ticket/25265#comment:4 <https://www.google.com/url?q=https%3A%2F%2Fcode.djangoproject.com%2Fticket%2F25265%23comment%3A4&sa=D&sntz=1&usg=AFQjCNEeTJV5U_vgQBRIQIppH6F1Hf991Q>. That is, you can go with Django's QuerySet and Query, and just implement a MongoDBCompiler. Or, you can use QuerySet with MongoDBQuery class. Or, finally, you can implement MongoDBQuerySet directly from scratch. If you implement Compiler or Query, you are targeting internal APIs which we *will* change in the future, maybe even in dramatic ways. If you target QuerySet, you are targeting a public API that doesn't change often. And, even if it changes, you will get a nice deprecation period for the changes. It might seem a lot of work to start from QuerySet, but for a non-relational backend there isn't actually *that* much work involved. Most of Django's Query and Compiler classes deal with joins, SQL's NULL peculiarities or SQL's way of doing aggregations. All of these are non-issues for non-relational backends. So, I think you should start with implementing a custom QuerySet for your wanted backend. You can also try to make it work with all Django models, but that approach is very likely to fail. For starters, Django's models use an autoincrementing integer field for primary key, whereas most (if not all) nonrelational databases use something different. Another interesting case is ManyToManyFields, which assumes a relational data model. It is very tempting to go with an approach where you just implement a custom Compiler class for your nonrelational backend. This would, in theory, allow users to run any existing Django application on non-relational database by just using a non-relational backend. The problem with this approach is that it doesn't work well enough in practice, and the maintenance overhead in the long run is huge. - Anssi -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4e40d965-37dc-428b-b9e8-508664db6b91%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
