Hello, The first item in Django's design philosophies is Loose coupling <https://docs.djangoproject.com/en/stable/misc/design-philosophies/#loose-coupling>. Per this principle, the database layer shouldn't know about the HTTP layer. This is a strong reason for keeping the HTTP request object away from the database layer (e.g. database routers or the transactions API). This is why "a hook which can be invoked with the request object as its param to make a decision" is extremely unlikely to be accepted.
If you disagree with the consequences of this philosophy, you can store the current HTTP request as a thread-local variable in your application and access it from anywhere, for example from a database router. Since I'm the original author of the transactions API, I've been thinking about your suggestion for the last few days. However, I'm not seeing what context could meaningfully be passed to `transaction.atomic()` besides the existing `using` parameter. Passing the HTTP request object is not an option. I would recommend running without persistent database connections (CONN_MAX_AGE = 0) and switching settings.DATABASE["default"] in a middleware at the beginning of every request. Modifying settings at runtime isn't officially supported, but this looks like the closest option to the intent you expressed. Best regards, -- Aymeric. > On 1 Jun 2021, at 12:09, N Aditya <gojeta.adi...@gmail.com> wrote: > > Hey all, > > I believe there's a major misunderstanding of what I've been trying to > convey. Let me rephrase: > > Problem: Include the request object as a decision param in the routers > framework/transaction APIs based on which a DB can be chosen i.e all > queries(reads/writes) and transactions are directed to a specific database > without having to explicitly mention it. This could be location based, IP > based, domain based or based on any other metadata available in the request. > > Explanation: > 1. This use-case is an extension of the examples provided in Django docs for > multi-database support wherein based on the model (or) based on the type of > operation(read/write) the database is chosen via the routers framework. I > need to achieve the same but this time it wouldn't be based on the model or > type of operation, rather it would be based on some metadata in the request. > 2. The multi-tenancy library which I've built just adds solidity to this > use-case. Simply put, it achieves routing based on request headers. > > The gap: I believe the essence of the routers framework is to provide a hook > by which ORM queries can be directed to a DB based on some condition without > having to explicitly specify it in every query. A similar hook is missing for > all of the transaction APIs. This is the gap I'm trying to bridge. To add > further context, in the library that I built, though the routers framework > currently doesn't receive the request object, I was able to work around using > contextvars but the essence is that the hooks were there. I'm unable to do > the same for the transaction APIs without monkey patching since such a hook > is unavailable. > > > I don't think this is a strong argument by any mean. To me this is > > basically saying "it might be useful in the future so lets add it now" > > without providing any details about why it might be useful in the first > > place. > > I thought I made it clear but reinstating anyway. I'd like the routers > framework to include the request object as part of the arguments list to each > of its methods so that decisions made can be based on it as well. If so, > there could be a separate method(hook) for transaction APIs as well, like > db_for_transaction which can take in the request object to make a decision. > It could include something else as well in the future. (Mentioned in > message-1 of this thread. Probably I forgot to mention the request object > part). > If the above is difficult to achieve, then we could at the least expose a > hook which can be invoked with the request object as its param to make a > decision. I believe this should be fairly simple to implement and maintain. > (Mentioned in message-3 of this thread). > > If you guys feel that this use-case is still invalid, I'd like to know a > logical explanation apart from maintenance overhead issues and waiting for > more people to come up with the same problem. > > ---------------------------------------------------------------- > > Simon, regd the multi-tenancy thing, I've done some extensive research on the > libraries available currently to achieve multi-tenancy (including yours) in > Django and none of them have adopted the isolated DB approach and declared it > production ready. There are multiple libraries using the schema per tenant > approach but the downsides are: > > 1. It is DB implementation specific i.e only DBs' which are using schemas > (like PostgreSQL) can take advantage of it. If they're using MongoDB or other > data stores, they're out of luck. > 2. In a microservices architecture, one may be using a combination of DBs' > for various purposes. For eg: PostgreSQL for OLTP, ElasticSearch for > real-time search etc., In such cases, they're out of luck again. > 3. A lot of B2B SAAS companies have a mandate that their data cannot be > collocated in the same physical space so as to avoid data leaks and breaches, > especially in healthcare systems (HIPAA). > > Also, there are a few libraries using the shared table FK approach to inject > relationship filters dynamically but after reading code, I found that it > doesn't cover all intricate/edge cases like: > 1. m2m relations between two tenant independent models. > 2. Not all methods of the QuerySet have been overridden properly. > > Though I agree with your point that all these libraries (including mine) > might not scale well for tenants in the range 50 - 100K, I don't see why it > wouldn't work for tenants in the order of a few 1000s. Also, I'd genuinely > like to know why the DATABASES dictionary or the CACHES dictionary would run > into connection leaks/KeyErrors with such an approach. > > Regards, > Aditya N > > On Tue, Jun 1, 2021 at 12:04 PM Florian Apolloner <f.apollo...@gmail.com > <mailto:f.apollo...@gmail.com>> wrote: > > > On Monday, May 31, 2021 at 12:13:58 PM UTC+2 Adam Johnson wrote: > I'm also -1 on changing anything in Django right now. > > -1 as well, I simply see no way how something like: > > ``` > with transaction.atomic(): > Model1.objects.create() > Model2.objects.create() > ``` > > will allow for any useful behavior via the router. Unless transaction.atomic > could inspect the contents and figure out the involved dependencies, but this > is surely way more than what could go into core for now. > > -- > You received this message because you are subscribed to a topic in the Google > Groups "Django developers (Contributions to Django itself)" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/django-developers/clzg6MiixFc/unsubscribe > <https://groups.google.com/d/topic/django-developers/clzg6MiixFc/unsubscribe>. > To unsubscribe from this group and all its topics, send an email to > django-developers+unsubscr...@googlegroups.com > <mailto:django-developers+unsubscr...@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/6292ccbb-bc83-48d0-b08e-aa765a29ce32n%40googlegroups.com > > <https://groups.google.com/d/msgid/django-developers/6292ccbb-bc83-48d0-b08e-aa765a29ce32n%40googlegroups.com?utm_medium=email&utm_source=footer>. > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com > <mailto:django-developers+unsubscr...@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/CABP%2BMjQNWQ2AK8%3DmSQKH0%3DdNnHHeV3kSrT13R--9%3DKz%3DvbfX%2Bw%40mail.gmail.com > > <https://groups.google.com/d/msgid/django-developers/CABP%2BMjQNWQ2AK8%3DmSQKH0%3DdNnHHeV3kSrT13R--9%3DKz%3DvbfX%2Bw%40mail.gmail.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/E07501AF-CAB2-41A3-9EF3-5DB41D8A0789%40polytechnique.org.
