Re: Improvements to the startproject template

[Ian Foote]

I want to add that I think either proposed change here would be an
improvement and that I'd prefer not to see this idea die because of
bikeshedding over the best option.

Regards,
Ian

On Thu, 21 Apr 2022 at 19:21, ome chukwuemeka 
wrote:

> I think this is a good suggestion!
>
> On Thu, Apr 21, 2022, 7:08 PM Albert  wrote:
>
>> It is possible to do in current version of Django with two lines of code:
>>
>> mkdir my_project
>>
>> django-admin startproject config my_project
>>
>> I have been working for many companies that use Django and I have seen
>> that each ot them has their own structure of project.
>>
>> And usually project is created once per couple of months, so I don't see
>> advantages of changing project structure.
>>
>> Regards,
>>
>> Albert
>>
>> Temat: Re: Improvements to the startproject template
>>
>> As there are different preferences and some see the change as worse than
>> the default and there is already a way to change the template for
>> startproject, wouldn't it be easier to provide different templates and list
>> them in the documentation, so that they can be used with the --template
>> argument? With a short section about each template and the pros and the
>> cons, and for what kind of project one is better suited.
>>
>> Maybe the three top templates could be shipped directly with django, so
>> that users could choose one with --template=simple, --template=nested or
>> --template=config or something in that direction.
>> On Thursday, April 21, 2022 at 11:16:23 AM UTC+2 ator...@redhat.com
>> wrote:
>>
>>> I personally dislike Adam's suggestion and feel like it makes it worse
>>> than the current default, but to each their own.
>>>
>>> I do prefer the proposed solution of the config directory, I am working
>>> on two django projects in parallel and one follows the proposed config
>>> scheme and the other doesn't (uses the default) and I find the proposed
>>> config scheme more natural to use and navigate, so that's a +1 from me too
>>>
>>> On Thursday, April 21, 2022 at 5:10:06 AM UTC+2 pem...@gmail.com wrote:
>>>
 For what it's worth, this is the (general) layout I've used for the
 past 8+ years of my professional Django development.

 Arthur

 On Wed, 20 Apr 2022 at 15:22, Olivier Dalang 
 wrote:

> +1 for Adam's suggestion, I use it as well and like it very much.
>
> > root folder
> - manage.py
> - ...
> > myproject
> - __init__.py
> - settings.py
> - urls.py
> - ...
> > myapp
> - __init__.py
> - models.py
> - ...
>
> Pros:
> - everything is well namespaced under myproject (`myproject.settings`,
> quite straightforward)
> - makes it clear that `settings.py`, `urls.py`, etc. concern the
> project as a whole, and not just an app.
> - also nice in settings.INSTALLED_APPS (`myproject.myapp` makes it
> clear that myapp is part of this project)
> - it leaves the root level for stuff like .gitignore, db.sqlite, etc,
> so the actual source stays clean from such files.
> - having a parent package allows (does not require) relative imports
> across modules of the same project, which more clearly conveys that such
> apps are tightly coupled
> - with manage.py still in the root folder, you don't need to cd into
> any folder to start working
>
> I use it all the time.
>
> Cheers,
>
> Olivier
>
>
>
>
> Le mer. 20 avr. 2022 à 18:50, Tom Carrick  a écrit :
>
>> I prefer Adam's suggestion in the forum post as it lets you namespace
>> everything under your project name nicely and avoids package name
>> collisions, although it doesn't solve the problem of having two 
>> directories
>> with the same name by default.
>>
>> That said, either would be an improvement on what we have so I'm in
>> favour of either approach over doing nothing.
>>
>> Cheers,
>> Tom
>>
>> On Wed, 20 Apr 2022 at 16:49, John M 
>> wrote:
>>
>>> I do exactly this for every new Django project, so it's +1 from me
>>> as well.
>>>
>>> John
>>> On 20/04/2022 12:01, da...@springbourne-tech.com wrote:
>>>
>>> +1 for me - this would be really useful.
>>>
>>> On Monday, April 18, 2022 at 9:02:02 PM UTC+1 pyt...@ian.feete.org
>>> wrote:
>>>
 Hi Tim,

 This feels like a good idea to me.

 Regards,
 Ian

 On Mon, 18 Apr 2022 at 18:17, Tim Allen 
 wrote:

> Greetings, friends!
>
> I've issued a PR that makes two changes to the
> `startproject` template:
>
>- instead of putting configuration files such
>as `settings.py`, `wsgi.py`, and the
>root `urls.py` in `my_project/my_project`, they are created

Re: Stop QuerySet repr from executing queries

[Ian Foote]

I've been working on the Kolo debugging tool and as part of that I've also
run into this issue. Generating unexpected queries when monitoring a django
project was a nasty surprise.

In Kolo's case I was also able to work around it with a monkeypatch, but
not needing this would be a nice performance win.

I also want to point to a similar case in Django where I think the default
behaviour is more helpful - that of forms. Calling `str` on a form can
trigger form validation, but calling `repr` avoids this. I'd be in favour
of moving the current `__repr__` implementation for `QuerySet` into
`__str__` and having an implementation of `__repr__` guaranteed not to
generate extra queries.

Regards,
Ian

On Fri, 15 Apr 2022 at 16:03, 'Kevin Weaver' via Django developers
(Contributions to Django itself)  wrote:

> I know this thread is more than a year old, but I was just bitten by this
> as well. We use the Elastic APM Python agent
>  which, like Sentry, calls
> `repr()` on local variables that it sends as part of stack traces. We use 
> Django
> REST Framework  and had a
> serializer that looked something like this (fake) example:
>
> ```
> class AccountSerializer(ModelSerializer):
>   owner = PrimaryKeyRelatedField(queryset=User.objects.all())
>   days_since_joined = serializers.SerializerMethodField()
>
>   def get_days_since_joined(self, obj):
> try:
>   ...
> except Exception:
>   logger.exception(...)
> ```
>
> What happened was:
>
>1. We have several places in the application that return a paginated
>list of, say, 50 accounts, using this serializer.
>2. Someone introduced the log line to catch exceptions when
>calculating `days_since_joined`, which is called for each of the 50
>accounts returned on the page. Due to a bug, the exception was almost
>always raised.
>3. Each time the exception is raised, the Elastic APM Python agent
>generates a stack trace and calls `repr()` on local variables in each stack
>frame. One of those variables in the `get_days_since_joined` frame is the
>`AccountSerializer` object `self`, which includes the `owner` property.
>Calling `repr()` on the `owner` property evaluates the `User.objects.all()`
>QuerySet, which fetches the first 21 User objects from the database.
>4. We now had `SELECT ... FROM user LIMIT 21` being called in a loop
>50 times each time someone tries to load a page of accounts, suddenly
>consuming significantly more database CPU and slowing down the application
>considerably until queries start timing out.
>
> To prevent this from happening again, we ended up applying something very
> similar to the suggestions above as a monkey patch:
>
> ```
> orig = QuerySet.__repr__
>
> def __repr__(self):
>   if settings.DEBUG or self._result_cache:
> return orig(self)
>
>   return f"<{self.__class__.__name__} [not evaluated]>"
>
> QuerySet.__repr__ = __repr__
> ```
>
> If folks are willing to reconsider their positions on this issue, I am
> more than willing to do what is necessary to contribute this change
> upstream, so that others are not impacted by this in the future.
>
> Thank you,
>
> Kevin
>
> On Tuesday, March 23, 2021 at 9:02:39 AM UTC-4 Adam Johnson wrote:
>
>> Would James' suggestion of reusing the result cache in __repr__ have
>> solved your issue? I would like to see that first.
>>
>> I'm not against the DEBUG-only fetching but there hasn't been any
>> suggestion what to show instead that could be of use.
>>
>> One can also mitigate against bad queries particularly well by setting a
>> statement timeout at the database level (at least MySQL and PostgreSQL
>> support this), which most users will generally want at a scale where repr()
>> becomes a problem.
>>
>> On Tue, 23 Mar 2021 at 12:40, Joachim Jablon  wrote:
>>
>>> Just been bitten by that same bug (combination of Sentry, using a
>>> Queryset.as_manager() that creates an unfiltered queryset as a local
>>> variable in the stack, a create signal that errored provoking a stacktrace
>>> that contained the queryset, a table that is always filtered by a field,
>>> and sorted by another field and an index that behaves poorly without the
>>> aformentionned filter).
>>>
>>> So would a contribution that would avoid evaluating the uncached
>>> queryset on repr when DEBUG is False likely to be accepted ? If so, I'm
>>> ready to get my hands dirty :)
>>>
>>> Cheers !
>>>
>>> Le dimanche 7 mars 2021 à 00:54:56 UTC+1, mic...@turbosys.com.br a
>>> écrit :
>>>
 I agree with Matt on that. Avoiding executing the queries on production
 would be helpful!

 Let me share my case. I use django-rest-framework in a specific project
 and DRF has a feature: a serializer has a fancier string representation
 when printed. I was using a serializer with a queryset in a view that had
 an exception further, which caused the serializer to be 

Re: Improvements to the startproject template

[Ian Foote]

Hi Tim,

This feels like a good idea to me.

Regards,
Ian

On Mon, 18 Apr 2022 at 18:17, Tim Allen  wrote:

> Greetings, friends!
>
> I've issued a PR that makes two changes to the `startproject` template:
>
>- instead of putting configuration files such
>as `settings.py`, `wsgi.py`, and the
>root `urls.py` in `my_project/my_project`, they are created
>in `my_project/config`
>- start the project with a custom User model app, `users`
>
> Over the years, I've taught or tutored over 100 Djangonauts starting their
> first project. Having to distinguish between two directories with the same
> name is a constant pain point in the teaching process - "cd into my_project
> ... no, the other one!"
>
>
>- The `config` option seemed to be the consensus from this thread in
>the forum: Django New Project Structure/Name - Using Django - Django
>Forum (djangoproject.com)
>
>- Ticket: https://github.com/django/django/pull/15609
>
> It is sometimes better to show rather than tell, so following our own
> documentation and including a custom User model with the initial project
> template reinforces the best practice that we explicitly point out in the
> documentation.
>
>
>- Ticket:  #27909 (Use AUTH_USER_MODEL in startproject template) –
>Django (djangoproject.com)
>
>- Avoids ever having this come up again:
>
> https://www.caktusgroup.com/blog/2019/04/26/how-switch-custom-django-user-model-mid-project/
>
> Here's a link to the PR: https://github.com/django/django/pull/15609
>
> My apologies for not starting with a discussion first. I'm an infrequent
> contributor to the Django codebase!
>
> Regards,
>
> Tim
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/33cb49d0-2469-47c0-920e-9501245c5a27n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFv-zfLccq7Dep7Esux5XYwEDpPKQVf9B3h3tuR0xxdQVt_e%2BQ%40mail.gmail.com.

Re: Proposed change in ORM model save logic

[Ian Foote]

Hi Barry,

Based on this I'm cautiously in favour. You set out a good case, it's much
easier to understand the "why" here than in the original ticket.

I'd avoid the extra optimisation of accessing __dict__ directly though - if
__set__ gains extra functionality, I'd prefer not to accidentally miss it
here.

Ian

On Fri, 15 Oct 2021 at 07:49, Barry Johnson  wrote:

> Trac ticket #33191 was recently closed as a “wontfix”, but Carlton
> encouraged bringing the matter to this group.  The actual issue is a small
> one, but it seems that Django could do better than it is doing today with
> one small change, and avoid the need for a counter-intuitive "x = x"
> statement added to application code.
>
> The claim is that Django is unnecessarily clearing the cached entry for an
> in-memory related object at a time that cache does not need to be cleared,
> and that this cache clearing can result in unwanted lazy reads down the
> line.  A one-line fix for the problem is suggested.
>
> Many apologies in advance for the extreme length of this post.  The
> requested change is small and subtle (although quite beneficial in some
> cases); we must dive deep into code to understand WHY it is important.
>
>
>
> Background
> ———
>
> My team is developing a major retail point of sale application; it’s
> currently live in nearly 1,000 stores and we’re negotiating contracts that
> could bring it to another 10,000 stores across the US over the next three
> to five years before going world-wide.  The backend is Django and
> PostgreSQL.
>
> One of the typical problems we face has to do with importing data from
> existing systems.  This data usually comes to us in large sets of flat
> ASCII files, perhaps a gigabyte or few at a time.  We have to parse this
> incoming data and fill our tables as quickly as possible.  It’s not usual
> for one such data import to create tens of millions of rows across a more
> than a hundred tables.  Our processes to do this are working reasonably
> well.
>
> In many cases, the incoming stream of data generates related records in
> several tables at a time.  For example, our product import populates
> related data across a dozen tables organized into parent/child hierarchies
> that are up to four levels deep.  The incoming data is grouped by product
> (not by functional data type); the rows and columns of the records for a
> single product will create ORM objects scattered across those dozen ORM
> models.  The top-level model, Product, has four child models with
> references to Product; and each of those child models may have other child
> tables referring back to THOSE models, etc.
>
> If the database schema sounds surprisingly complex, it is.  Big box retail
> is more complex than most people realize.  Each store typically carries
> 30,000 to 70,000 individual products; some are even larger.  Some of the
> retail chains to which we market our systems have more than $1 billion USD
> per year in sales.
>
> To be efficient, we process this incoming data in chunks:  We may
> instantiate ORM objects representing, say, 5,000 products and all of their
> related child objects.  For example, we’ll create a new instance of a
> Product model, then instantiate the various children that reference that
> product.  So a very typical pattern is something like
>
>  product = Product(values)
>  pv = ProductVariant(product=product, **more_values)
>  upc = UPC(product_variant=product_variant, **upc_values)
>
> It’s not that simple, of course; we’re reading in sequential data that
> generates multiple instances of the children in loops, etc., but
> essentially building a list of products and the multi-level hierarchy of
> child objects that dependent upon each of those products.  We then use
> bulk_create() to create all of the top-level products in one operation,
> then bulk_create() the various lists of first-level children, then
> bulk_create the children of that second level, etc.  LOTS of bulk creates.
>
>
>
> Prior to Django 3.2, we had to explicitly set the associated “_id” field
> for each child’s reference to a parent after the list of parents were
> bulk_created.  Using our examples above, we would see things like:
>
> bulk_create(list_of_products)
>
> for each variant in list_of_product_variants:
> variant.product_id = variant.product.id
> bulk_create(list_of_product_variants)
>
> for each upc in list_of_upc_entries:
>upc.product_variant_id = upc.product_variant.id
> bulk_create(list_of_upc_entries)
>
> […]
>
> Again, this is somewhat simplifying the code, but the key takeaway is that
> older versions of Django required us to manually pick up the primary key
> value from recently created instances and set the associated “_id” value in
> each instance pointing at the recently created objects.
>
> *As expected, setting the “_id” value of a foreign key field clears the
> internal cache entry containing the reference to the parent object.*  We
> would fully expect that if we said 

Re: Difference between AdminSite.admin_view and staff_member_required?

[Ian Foote]

I think deprecating it is a good idea. I think it's actually a small
footgun - I've seen it used to restrict access to non-admin functionality,
when `user_passes_test` would be appropriate. In the case I found it, it
made adding tests trickier than it should have been.

Ian

On Tue, 9 Mar 2021 at 21:51, Timothy McCurrach 
wrote:

> > It seems like a good idea. However, it leads us to another question. Do
> we really need the is_staff field on the User Model if it is deprecated?
>
> User.is_staff is still used, just from Adminsite.has_permission instead:
>
> https://github.com/django/django/blob/main/django/contrib/admin/sites.py#L196
>
> On Tue, Mar 9, 2021 at 9:45 PM Matthew Pava  wrote:
>
>> > Thoughts on deprecating it?
>>
>> It seems like a good idea. However, it leads us to another question. Do
>> we really need the is_staff field on the User Model if it is deprecated?
>>
>>
>>
>> *From:* django-developers@googlegroups.com <
>> django-developers@googlegroups.com> *On Behalf Of *Timothy McCurrach
>> *Sent:* Tuesday, March 9, 2021 3:22 PM
>> *To:* django-developers@googlegroups.com
>> *Subject:* Re: Difference between AdminSite.admin_view and
>> staff_member_required?
>>
>>
>>
>> staff_member_required feels like a bit of a relic from the past:
>>
>> - It looks like before a massive refactor of the admin that happened 12
>> years ago (
>> https://github.com/django/django/commit/a19ed8aea395e8e07164ff7d85bd7dff2f24edca
>> )
>> staff_member_required played a similar role to what Adminsite.admin_view
>> does now (checks permissions and redirects to login if appropriate). Before
>> the refactor, all of the relevant views were wrapped with
>> staff_member_required.
>> - The above commit introduced AdminSite which has its own
>> 'has_permissions' method. Adminsite.root (which was the old get_urls)
>> called self.has_permissions before calling any other views, so
>> staff_member_required was now no longer really required.
>> - staff_member_required did however continue to be used in the admindocs
>> app (which was now split off into its own app).
>> - Adminsite.root was eventually replaced with get_urls and the call to
>> has_permissions  was moved into `Adminsite.admin_view` (which also does a
>> few other things: adds csrf protection, and deals with caching)
>> - Over time staff_member_required became simpler and simpler as things
>> like user_passes_test were introduced, and there was no need to repeat
>> logic.
>> - It's now just a very specific version of `user_passes_test` and is only
>> used internally one place - the admindocs app.
>>
>>
>> Tobias - In terms of permissions and redirecting; the behaviour of
>> admin_view and `staff_member_required` are very similar. The differences
>> are that:
>>  i) admin_view defers to Adminsite.has_permission to do the actual
>> checking of permissions (so if that method is overwritten the behaviour
>> will change)
>>  ii) admin_view does a whole load of other things that all admin views
>> are going to need (mentioned above).
>>  iii) They have slightly different arguments you can pass in to modify
>> the behaviour
>>
>> If you are decorating a view of ModelAdmin, or AdminSite you should use
>> admin_view - as described in the docs under get_urls.
>>
>> Since staff_member_required is more or less a very specific version of
>> user_passes_test designed for admin views, it's use-case seems pretty
>> niche. If you were making an app like admindocs which behaves like an
>> extension of the admin app, without actually being part of the admin app
>> you could use it then, but I'm struggling to think of any other time you
>> would want to use it though.
>>
>> I think there's an argument to be made for deprecating it completely:
>>  - It's only used in one place internally, and the wider use case seems
>> pretty niche.
>>  - user_passes_test can do the same thing and seems to be the standard
>> way of doing this kind of thing anyway.
>>  - user_passes_test with a suitably named test argument would also be
>> more explicit, since the login_url and redirect_field_name would need to be
>> explicitly stated.
>>
>>  Definitely the documentation should be updated though. Thoughts on
>> deprecating it?
>>
>> Tim
>>
>>
>>
>> On Tue, Mar 9, 2021 at 7:08 PM Tobias Bengfort 
>> wrote:
>>
>> On 09/03/2021 18.03, Carlton Gibson wrote:
>> > Do you want to open a PR with suggested changes as a focus for
>> discussion?
>>
>> I would open a PR that improves the docs, but I still don't understand
>> the difference, so I wouldn't know what to write.
>>
>> The other alternative would be remove one of them, but at this point
>> that is probably too drastic.
>>
>> tobias
>>
>> --
>> You received this message 

Re: Feature Request: Enabling hooking into the autodetector from a third-party app.

[Ian Foote]

Hi Emil,

This is a very interesting idea and I find your use-case compelling. Adding
support for custom triggers to Django itself would resolve your main use
case this time but I also see value in a general solution that enables
Django's third-party ecosystem.

I think the next step is to open a new ticket (or find an existing one).

Ian

On Sun, 14 Apr 2019 at 15:20, Emil Madsen  wrote:

> Hello follow Djangonauts.
>
> I'm writing this post in an attempt to ignite a discussion about the
> autodetector, and how to make it expandable, such that third-party apps can
> generate their custom migrations automatically.
>
> This has previously been discussed here:
> * https://groups.google.com/d/msg/django
> -developers/qRNkReCZiCk/I8dIxxhoBwAJ
> * https://groups.google.com/d/msg/django
> -developers/pbj7U7FBL6Q/eJtRV7QwDwAJ
>
>
> *Motivation:*
>
> My motivation for writing this post is two fold:
>
>1. First, I recently attended DjangoCon Europe 2019, where Carlton
>Gibson had a keynote about contributing back to Django, I want to do
>so, and thus this is my first post on the mailing list.
>2. Secondly, I believe there are various cases where being able to
>expand the automigrator enables libraries to be a lot cleaner. For 
> instance:
>
>
>- Enforcing constriants on data-models, such as it's done with
>unique_together, and CheckConstraints. Currently these are implemented
>directly in Django, but given an expandable autodector they could have
>been implemented as libraries. In the long section at the end of this
>post, I mention a similar case to this, that was my original
>motivation for writing this post.
>- Enabling rebust auditing models via database triggers, by simply
>adding 'audit_trigger = True' to the Meta class of models that should be
>audited, as done by:
>   - https://github.com/carta/postgres_audit_triggers
>- Automatically enabling database extensions, when fields which
>require these extensions are utilized. As an example, utilizing PostGIS or
>HStore currently requires activating their corresponding extensions
>manually using CreateExtension('extension'), which could be automated.
>- Really any case, in which one would like to automatically generate
>migrations, and which cannot be achieved by custom fields.
>
>
> *The proposal:*
>
> Create a plugin system where plugins can register handlers / callbacks to
> be run from within the automigrator. Potentially this could even be
> implemented using the signals mechanism currently in Django, as presented
> in 'Building plugin ecosystems with Django', by Raphael Michel at
> DjangoCon Europe 2019.
>
> I imagine the signal should be fired here:
>
>- https://github.com/django/django/blob/master/django
>/db/migrations/autodetector.py#L192
>
> And that signals handlers should act as combination between the
> 'create_altered_X' and 'generate_altered_X' methods, resulting either to
> calls of 'self.add_operation', or returning a list of operations to add
> to the migrationsfile.
>
> There is an open question with regards to how dependencies and shared
> state should be handled.
> - I do not have enough insight to answer this.
>
> I am very open to other solutions, this is merely what I came up with,
> without having any throughout insight into how Django works internally.
> - I am willing to implement the plugin system, and create a PR for this,
> if a consensus is found.
>
>
> *My personal motivation (long section):*
>
> I have a database design, which includes several data invariants, which I
> want to uphold. I've been able to implement most of these data invariants
> using uniqueness (on fields, unique_together, UniqueConstraint), and
> CheckConstraint.
>
> CheckConstraint is excellent, and I'm very happy that made it to Django
> with 2.2, but for enforcing data invariants it has one major shortcoming. -
> It only operates on a single table.
>
> Currently to implement data invariants across multiple tables, I believe
> the suggestion is to override save methods, or to use save signals.
> - However these signals are not emitted for queryset operations, such as
> bulk_inserts or update s, and as such cannot be used to maintain data
> invariants, without relying on manual procedures that are error-prone, and
> take up review time. On another note, save methods and save signals, also
> do not help if the databsae is accseed outside of the Django ORM.
>
> Thus if you need reliable constraints that work across multiple tables, I
> don't believe that a robust solution is currently in place, thus I've been
> working on a prototype for a library to fulfill this role. The library
> enables very strong data invariant checking by allowing one to write
> database constraint triggers via Querysets.
>
> The envisioned library is very simple, and works as follows (based upon
> the pizza toppings example):
> class Topping(models.Model):
> name = 

Re: Proposal to format Django using black

[Ian Foote]

I'm in favour. Not needing to think much about code style when writing code
or when reviewing is very nice.

Ian

On Sat, 13 Apr 2019 at 13:52, Herman S  wrote:

> Hi.
>
> I propose that Django starts using 'black' [0] to auto-format all Python
> code.
> For those unfamiliar with 'black' I recommend reading the the projects
> README.
> The short version: it aims to reduce bike-shedding and non value-adding
> discussions; saving time reviewing code; and making the barrier to entry
> lower
> by taking some uncompromissing choices with regards to formatting.  This is
> similar to tools such as 'gofmt' for Go and 'prettier' for Javascript.
>
> Personally I first got involved contributing to Django couple of weeks
> back,
> and from anecdotal experience I can testify to how 'formatting of code'
> creates
> a huge barrier for entry. My PR at the time went multiple times back and
> forth
> tweaking formatting. Before this, I had to research the style used by
> exploring
> the docs at length and reading at least 10-20 different source – and even
> those
> were not always consistent. At the end of the day I felt like almost 50%
> of the
> time I used on the patch was not used on actually solving the issue at
> hand.
> Thinking about code formatting in 2019 is a mental energy better used for
> other
> things, and it feels unnecessary that core developers on Django spend
> their time
> "nit-picking" on these things.
>
> I recently led the efforts to make this change where I work. We have a
> 200K+
> LOC Django code-base with more than 30K commits. Some key take-aways: it
> has
> drastically changed the way we work with code across teams, new engineers
> are
> easier on-boarded, PR are more focused on architectural choices and "naming
> things", existing PRs before migration had surprisingly few conflicts and
> were
> easy to fix, hot code paths are already "blameable" and it's easy to blame
> a
> line of code and go past the "black-commit", and lastly the migration went
> without any issues or down-time.
>
> I had some really fruitful discussions at DjangoCon Europe this week on
> this
> very topic, and it seems we are not alone in these experiences. I would
> love to
> hear from all of you and hope that we can land on something that will
> enable
> *more* people to easier contribute back to this project.
>
> I've set up how this _could_ look depending on some configurables in Black:
>
> * Default config: https://github.com/hermansc/django/pull/1
> * Line length kept at 119: https://github.com/hermansc/django/pull/3
> * Line length kept at 119, no string normalization:
> https://github.com/hermansc/django/pull/2
>
> Please have a look at the Black documentation. It explains the benefits
> better
> than I possibly could do here.
>
> With kind regards,
> Herman Schistad
>
> [0]: https://github.com/ambv/black
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAN%3DnMTx0EE5WfXuccv_e3MBuCxp9u_pAV_ow5MxNST6MptTDBw%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFv-zf%2BAo57rc4k9%2B0Ndu%3DEVprv5JLGu0tM2ihdwWcddCr%3DcXg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Validation of m2m

[Ian Foote]

I don't think a new setting is the way to go. I'd prefer to add an
attribute (validate_m2m = False?) to the CreateView and UpdateView classes
that allows a developer to opt-in to the new behaviour when they need it.
This is more flexible and still maintains backwards compatibility.

Regards,
Ian

On Thu, 21 Feb 2019 at 11:03, Bernd Wechner  wrote:

> Most once sided discussion I've seen on a developers group in a while ;-),
> but I'll go so far as to suggest an API for the Django Core on this one. I
> mean I have a way to advance myself now, but methinks this need is so
> common that perhaps it's best sitting in the Django Core, albeit with a
> setting as it demands an atomic transaction for create and update views.
> Now this rests on a few premises, namely:
>
>
>1. That there is nothing overly broken with my suggestion that I am
>not seeing as yet (I am the noob after all, and while pretty Python and
>Django savvy, not diving into the core code very often at all, as little as
>I've needed to, the very benefit of a cool framewor, keeping us isolated
>from the dirty details ;-).
>2. That the observations I tabled are valid across all three database
>backends that Django supports. if it's only  1 (Postgres) or 2 then it
>shoudl definitely stay an enable-able option and not a default behaviour.
>
> A suggested API is as follows:
>
>
>1. Implement a  setting to enable this new behaviour. Looking over
>exiting settings they can get quite long in name (like
>DATA_UPLOAD_MAX_NUMBER_FIELDS for example) and so perhaps a name like:
>RELATIONSHIP_VALIDATORS. I take inspiration from this setting:
>
>AUTH_PASSWORD_VALIDATORS¶
>
> 
>
>Default: [] (Empty list)
>
>The list of validators that are used to check the strength of user’s
>passwords. See Password validation
>
> 
>for more details. By default, no validation is performed and all passwords
>are accepted.
>
>And propose:
>
>TOMANY_RELATIONSHIP_VALIDATORS¶
>
> 
>
>Default: False
>
>A boolean that specifies whether to use atomic transactions on in
>CreateView and UpdateView POST requests in order to support relationship
>(OneToMany and ManyToMany) validation in the Model itself. If true,
>Model.clean() is called as usual, then inside an atomic transaction all
>submitted forms and formsets are saved and Model.clean_relations() is
>called. It is important to note that when Model.clean() is saved ToMany
>relationship (creation or changes) are not visible to Model.clean() but are
>visible in Model.clean_relations() where they can be validated.
>
>2.
>
>Implement the code in CreateView.post(...) and UpdateView.post(...)
>that does this, that is a little savvier than my example in previous mail,
>but if form.is_valid() pass opens a transaction, saves the form and then
>for each other form or formset in the POSTed data (not sure off hand how to
>find those, more learning for me), check each with their .is_valid() and if
>passing save it, then call Model.clean_relations() (for each model involved
>(associated with any of the submitted forms or formsets). Of course has to
>work if the model fails to implement clean_relations(), either by checking
>it's there first of defining a default implementation which pay just pass.
>
>3.
>
>If Model.clean_relations() fails (throws an exception), roll back, if
>not commit.
>
> It's a rough outline that I think can work. And if it's good (which I
> await feedback on from the silent developer community) then there may even
> be a case for defaulting this setting to True, on new projects anyhow as it
> may be sensible to default to False on any existing site (not introduce a
> new transaction layer without warning on existing projects that upgrade
> Django - where there may well be a transaction system in place already).
>
> I mean if it's a good plan, and it actually gets some feedback to that
> efffect or that help it evolve into a good plan, thus far this looks
> eminently like something I could implement and submit as a PR (Pull
> Request), but I'm just as likely to code mine up and run as I am way way
> busy on other stuff ;-).
>
> Regards,
>
> Bernd.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> 

Re: Python string formatting

[Ian Foote]

I think {} formatting has some minor readability benefits. The symmetry of
the braces makes it easier (imo) to see at a glance what is going to be
interpolated. With % formatting the alphabetical characters can be obscured
by other text in the string, especially if there's not a space after it.
I've often wished string templates in Django form error messages could be
overwritten with a {} formatted string. On the other hand making a change
would be a lot of churn and I'm not sure the benefit would be worth it.

Ian

On Wed, 31 Oct 2018 at 22:54, Tim Graham  wrote:

> Another discussion is
> https://groups.google.com/d/topic/django-developers/J9CfWIgrgbY/discussion
> - % vs {} string formatting for public APIs
>
> I haven't seen any practical benefits to prefer format() over %s. I find
> the latter less verbose.
>
> I'm not certain what this argument is (things like
> http://lucumr.pocoo.org/2016/12/29/careful-with-str-format/ I imagine)
> but here's a previous concern that Florian raised regarding f-strings:
> "Knowing what certain members of the core team think about those
> f-strings, I think there will be first a big discussion if we will allow
> them at all in Django's codebase. Then there are further things to consider
> like gettext support etc…"
> https://groups.google.com/d/msg/django-developers/4rbVKJYm8DI/y9VNRBw0AgAJ
>
> On Wednesday, October 31, 2018 at 6:28:08 PM UTC-4, Tom Forbes wrote:
>>
>> In my experience f strings are vastly more expressive than .format, but
>> not much better than % based formatting.
>>
>> If we can make this change after we drop 3.5 I think it will be a lot
>> easier.
>>
>> On Wed, 31 Oct 2018, 21:09 Adrian Turjak >
>>> There was a push to deprecated % formatting but too many people
>>> complained and that never happened.
>>>
>>> While .format and g-strings are superior, % is here to stay for the
>>> foreseeable future. Too many people still use it (including myself
>>> sometimes).
>>>
>>> On 1 Nov. 2018 08:14, Carlton Gibson  wrote:
>>>
>>> We had a bit of a discussion of this on
>>>
>>>
>>> Python 2 docs have got this re format():
>>>
>>> > This method of string formatting is the new standard in Python 3, and
>>> should be preferred to the % formatting described in String Formatting
>>> Operations in new code.
>>>
>>> ​https://docs.python.org/2/library/stdtypes.html#str.format
>>>
>>> But it's not there for Python 3 (
>>> ​https://docs.python.org/3/library/stdtypes.html#str.format) so I'm not
>>> clear what we're meant to think.
>>> (I had thought %-formatting deprecated...)
>>>
>>> Anyone know?
>>>
>>> Regardless, waiting a bit and moving straight to f-strings (as Simon
>>> suggests) seems sensible. (If we're convinced we do want to change.)
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/e7cf5b0d-cff4-4c6d-a9ec-1af4a82d5a56%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/6b09193b-a53f-4eb4-9f36-2c7aa11f47cb%40email.android.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/65527cfe-d970-47b4-828c-7f55312d2120%40googlegroups.com
> 

Re: Idea: Allow queryset.get() and queryset.filter() to accept a positional argument for implicit primary key filtering

[Ian Foote]

I'm not in favour of this, it's too implicit for my liking and I don't
think any gains outweigh the increased complexity of implementation.

Ian

On Wed, 31 Oct 2018 at 19:04, charettes  wrote:

> My main concern with allowing extra args and kwargs to be mixed with the
> proposed
> pk arg notation is that it would allow lookups such as
>
> get(Q(is_active=True), pk)
>
> Simon
>
> Le mercredi 31 octobre 2018 14:53:35 UTC-4, Antwan a écrit :
>>
>> *I'm not convinced this would be as useful for `filter()` though as I
>> don't*
>> *recall wanting to retrieve a set of objects matching a value I know will*
>> *be unique.*
>>
>> The .filter() could be used for update
>>
>> e.g.:
>> Mymodel.objects.filter(1).update(key=value)
>>
>> This is also bringing consistency in the code, but I agree this could be
>> misleading.
>> The .get() is the main purpose.
>>
>> *Something to keep in mind as well is whether or not we want to allow*
>> *this to be coupled with extra args and kwargs.*
>>
>> Currently only kwargs are supposed to be used with get/filter. The only
>> args that can be used currently are Q() objects (and couples but I think
>> it's an issue).
>> This proposed design suggestion is to also allow scalar values (strings,
>> int, etc) too.
>>
>>
>> On Wednesday, 31 October 2018 18:12:53 UTC, charettes wrote:
>>>
>>> As I've mentioned on the ticket I've been wishing get(pk) could translate
>>> to get(pk=pk) for a while but I never got to suggest it. Probably
>>> because I
>>> didn't feel like adding (pk=...) was really that long. I'd note that
>>> most of the
>>> times I wished this existed was when doing some object manipulations
>>> through a Django shell.
>>>
>>> I'm not convinced this would be as useful for `filter()` though as I
>>> don't
>>> recall wanting to retrieve a set of objects matching a value I know will
>>> be unique.
>>>
>>> Something to keep in mind as well is whether or not we want to allow
>>> this to be coupled with extra args and kwargs.
>>>
>>> e.g.
>>>
>>> Book.objects.get(isbn, author__name='Daniel Roy Greenfeld')
>>>
>>> I'd be in favor of preventing pk and kwarg or Q args mixing.
>>>
>>> Count me +1 for the get() case and -1 for the filter() one.
>>>
>>> Simon
>>>
>>> Le mercredi 31 octobre 2018 13:13:34 UTC-4, Antwan a écrit :

 Hi,
 I'm creating this topic to see if there is interest to implement
 positional arguments in queryset filtering.

 Current situation

 Currently the only way to use positional arguments to filter can be
 either:

- Passing a single or multiple Q objects:

MyClass.objects.filter(Q(key=value))
MyClass.objects.filter(Q(key=value), Q(other_key=value))



- Passing a couple is also working (not sure if this is a happy
accident, should it be removed?)

MyClass.objects.filter((key, value))



- Combination of both is also proven to work

MyClass.objects.filter((key, value), Q(other_key=value))



 Suggestion

 This feature suggestion is to leverage the case when a non-Q / non
 couple object is passed, so it implicitly interpreted as the value for the
 model's pk.

 This could ease/simplify code by omitting pk when this is the only
 filter used:


 MyClass.objects.get(value)
 # Translates into: MyClass.objects.get(pk=value)


 or


 MyClass.objects.filter(value)
 # Translates into: MyClass.objects.filter(pk=value)


 or


 MyClass.objects.filter(Q(value))
 # Translates into: MyClass.objects.filter(Q(pk=value))


 Do you think it's worth it? It could be leveraged to simplify many
 situations.
 I'd be happy to proceed to the development myself if this is something
 gathering interest.

>>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/a29eed0a-687a-4964-aec3-a25eeeb6f441%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to 

Re: Widening participation (Thoughts from DjangoCon)

[Ian Foote]

Hi Carlton,

I've had similar thoughts sitting in the back of my mind for at least a
couple of months, so thank you for sharing this. I agree that finding
tickets is one of the big problems here, both for new contributors and for
sprint leaders. At Pycon UK I took on the role of sprint leader along with
Adam Johnson and directing people to appropriate tickets was a definite
difficulty. I was also unaware of the django core mentorship list and will
be joining that soon. I'm willing to spend some time mentoring a small
number of people, life permitting.

Ian

On Fri, 26 Oct 2018 at 14:44, Carlton Gibson 
wrote:

> Hi All.
>
> OK, so last week I was at DjangoCon US in San Diego. (Thank you if you
> organised that! Hi! if we met and chatted.)
> I gave a talk ("Your web framework needs you!") inspired by the
> discussion on the
> 
> DSF list and the proposal to dissolve Django Core
> . (Can’t see the DSF list? Join
> the DSF
> 
> .)
> I was asking for more participation in general, and participation that is
> more representative of the wider Django community in particular.
>
> There was lots of good input from many people, including (but not, at all,
> limited to) representatives of groups such Pyladies, DjangoGirls, and so
> on.
>
>
> The recurring themes seem to me to fit into three categories:
>
>1. The importance of *mentoring*.
>2. The difficulty of *finding tickets*.
>3. The importance of *sprints*.
>
> The rest here is a summary of that. Hopefully it’s useful.
>
> Mentoring
>
> For whatever reasons, the exiting *Contributing How-To*
>  doesn’t
> lead to contributions from a demographic that matches the wider Django
> Community.
>
> The point that came up again and again about this was that *mentoring* is
> one of the best (perhaps the best?) tool in helping to change this.
>
> Django Core Mentorship
>
> We don’t have an official mentoring programme but we do have the 
> django-core-mentorship
> list .
>
> This must be about the best-kept secret in the Django world: it’s gets ≈0
> traffic, but I told everybody at DjangoCon about it, and that they should
> use it.
>
> If you are not on django-core-mentorship, and you’re willing to help
> prospective contributors, please sign-up. I’m hoping we can drive some
> traffic to it.
>
> Maybe there’s call for something more formal, but at least until DCM is
> actually being used, that seems (to me) like something we can postpone.
>
> Finding Tickets
>
> The next thing was that there’s not enough guidance on what to work on.
>
> The guidance is to look for *Easy Pickings*. There are ≈1300 accepted
> open tickets in TRAC. 13 of these are marked *Easy Pickings*.
>
> That’s not enough. I think we’re too tight with it (or need another
> grade).
>
> There are *many* tickets which aren’t super hard: I put it that, most of
> our community solve harder problems every day *using Django* than most
> tickets require.
>
> Yes, they still require time, love, energy, etc — and maybe some mentoring
> — but it’s not primary research, in the main.
>
> I talked to people who had (at the conference) got the test suite running
> and such, but been overawed by the (for want of a better phrase) *sheer
> face* of issue tracker.
>
> We would do well to invite people better here. (I don’t have instant
> solutions.)
>
> Sprints
>
> I’m not historically a Django-sprinter. (I have too many children for that
> TBH, but they’re getting older…)
>
> I always thought it was for a hard-core to work on hard issues.
>
> Shows what I know… 
>
> It was strongly impressed upon me that the real benefit of sprints is
> being able to give new contributors the support they need to make their
> first (or second or…) contribution.
>
> In particular, groups such as Pyladies can organise a sprint event with
> the specific goal of helping members of the community get across the
> barriers to contributing. This can reach numbers that otherwise simply
> aren’t possible. (So wow. Basically.)
>
> Sprints & Mentoring
>
> Obviously having mentors at sprints is a key component.
>
> But even if you (or I) can’t attend a sprint, maybe we can (sometimes) be
> available at the right time to contribute remotely. Maybe, in fact, remote
> mentors are a key resource in more remote parts of the Django-sphere.
>
> It turns out just being on e.g. Twitter can be enough here.
>
> If we’re all on django-core-mentorship, maybe sprint organisers could post
> notice of an upcoming sprint.
>
> Sprints & Finding Tickets
>
> It turns out it’s equally hard for a sprint organiser to work out what
> tasks to give sprinters.
>
> At DjangoCon (some) people have a topic and asks others to join them. 

Requiring sqlparse for sqlite introspection

[Ian Foote]

Hi all,

On my pull request (https://github.com/django/django/pull/10406)
refactoring how Django creates database constraints I introduced a
dependency on sqlparse in the sqlite introspection code. This allows Django
to correctly read information about constraints on sqlite, particularly the
name.

When reviewing (
https://github.com/django/django/pull/10406#issuecomment-424542217) Tim
Graham raised the question of whether we should make sqlparse a mandatory
requirement for the sqlite tests or if we should skip those tests that
require it. In later discussion (
https://github.com/django/django/pull/10406#issuecomment-427362983), Tim
raised the point that the introspection code is used by migrations.

I'm not clear myself what the best route forwards here is, so I'm bringing
it to Django Developers for discussion (as Tim suggested).

Thanks,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFv-zfKadOeWit8M6GMmx4H2ChUCU6u%3DscHX8F7oBKJkHRbuVg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: OneToOneField not updating relationship with reverse relation after refresh_from_db()

[Ian Foote]

On 02/09/18 19:59, Shivam Jindal wrote:
> Hi Lan,
> 
> *m.user.refresh_from_db()* is sam as 
> 
> *u = m.user*
> *u.refresh_from_db()*
> 
> So If I am refreshing u, it should update all relationship and property
> of *u*(including reverse relation). Should It not?
> 

Hi Shivam,

As far as I can tell, u.refresh_from_db() only cares about making sure
u's state is in sync with the database. It doesn't care about the state
of any related objects, like m.

I suspect that cascading the sync to related objects would be difficult
to get right and might be unexpected behaviour to other users. I'm still
not convinced this is a bug and even if it is, I think it would require
a deprecation cycle to change, so I don't think it would be worth it.
Just calling m.refresh_from_db() instead seems sufficient to me.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/cdd0e68f-64e8-ef1c-5022-9ecab4ba604e%40feete.org.
For more options, visit https://groups.google.com/d/optout.

Re: OneToOneField not updating relationship with reverse relation after refresh_from_db()

[Ian Foote]

On 02/09/18 17:58, Shivam Jindal wrote:
> I have the following model
> 
> class Member(models.Model):
>     user = models.OneToOneField(User, null=True, blank=True)
> 
> *Operation sequence*
> 
> m = Member.objects.get(id=4) # This objects has attached user
> m.user # Print user objects successfully
> m1 = Member.objects.get(id=4) # Fetch same object from db 
> m1.user = None
> m1.save() # Remove user object from above member
> 
> *m.user.refresh_from_db()*
> *m.user.member* # It is printing the member object which was linked
> previously, but the user object should not have attached member object
> because the relationship should had been updated by calling
> *refresh_from_db()*  
> 

Hi Shivam,

I don't believe this is actually a bug. You're refreshing m.user, not m
itself, so as far as m is concerned m.user hasn't changed. If you use
m.refresh_from_db() instead you should see m.user updated to None.

Hope this helps,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8df15e03-4d44-225d-7116-6954d0821362%40feete.org.
For more options, visit https://groups.google.com/d/optout.

Re: Prevent DateRangeField overlap in model?

[Ian Foote]

Hi Brylie, Laurence,

I think it should be possible to do this using my Check Constraints work
(https://github.com/django/django/pull/7615) or something built on top
of it.

I plan to get this work into shape so it can land in Django 2.1.

I hope this helps,
Ian

On 03/10/17 10:40, Laurence Sonnenberg wrote:
> A pythonic way to do this would be extremely useful if it existed. As
> you suggested, I did this using a sql exclusion constraint statement on
> the table and a migration.
>  
> For example:
> 
> 000x_your_migration_name.py
> 
>  # -*- coding: utf-8 -*-
> from __future__ import unicode_literals
> 
> from django.db import migrations
> 
> SQL = """ALTER TABLE your_table DROP CONSTRAINT IF EXISTS
> range_overlap_exclusion;
>                ALTER TABLE your_table ADD CONSTRAINT range_overlap_exclusion
>                EXCLUDE USING gist (resource WITH =, date_range WITH &&);"""
> 
> 
> class Migration(migrations.Migration):
> 
>     dependencies = [
>         ('module_name', '000x_previous_migration'),
>     ]
> 
>     operations = [migrations.RunSQL(sql=SQL,
> reverse_sql=migrations.RunSQL.noop), ]
> 
> 
> On Sunday, October 1, 2017 at 11:53:53 PM UTC+2, Brylie Christopher
> Oxley wrote:
> 
> Now that Django supports the DateRangeField, is there a 'Pythonic'
> way to prevent records from having overlapping date ranges?
> 
> # Hypothetical use case
> One hypothetical use case would be a booking system, where you don't
> want people to book the same resource at the same time.
> 
> # Hypothetical example code
> 
> 
> class Booking(models.model):
> # The resource to be reserved
> resource = models.ForeignKey('Resource')
> # When to reserve the resource
> date_range = models.DateRangeField()
> 
> class Meta:
> unique_together = ('resource', 'date_range',)
> 
> This might have the effect of adding an exclusion constraint on the
> underlying table:
> 
> https://www.postgresql.org/docs/9.3/static/sql-createtable.html#SQL-CREATETABLE-EXCLUDE
> 
> 
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-developers+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-developers@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/f595965d-8646-40f8-8637-f5011ba8f535%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/837bbc77-dd7f-b3d5-d447-45d1613d10db%40feete.org.
For more options, visit https://groups.google.com/d/optout.

Re: More eyes on Check Constraints

[Ian Foote]

Hi David,

That's an interesting idea. I haven't thought very deeply about
composite foreign keys myself, so I can't say for certain how easy it
would be to build on top of my work.

I did a quick search for prior discussion about this and found
https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys and
https://code.djangoproject.com/ticket/373. These are probably a good
place to start - I plan to have a good read over the next week.

Your library also looks interesting. I also plan to take a close look to
see if there's anything I can learn from there. I wonder if it's worth
trying to converge our implementations - if my patch lands in Django it
would be great to have a compatible implementation that people can use
on earlier versions.

Thanks,
Ian

On 13/08/17 17:44, David Sanders wrote:
> Hi Ian,
> 
> Interestingly enough I'd just recently
> written https://github.com/rapilabs/django-db-constraints based on Matt
> Schinckel's idea.  I wrote that because although check constraints are
> great, I really wanted a way to supply any type of constraint -
> primarily composite foreign keys.  I don't suppose that this would be a
> possibility as a future extension of your pull request?
> 
> Cheers,
> David
> 
> 
> On Friday, August 11, 2017 at 4:15:26 AM UTC+10, Ian Foote wrote:
> 
> Hi all,
> 
> I've been working on https://code.djangoproject.com/ticket/11964
> <https://code.djangoproject.com/ticket/11964> to add
> support for Check Constraints to the Django ORM and migrations
> framework. My work can be found at
> https://github.com/django/django/pull/7615
> <https://github.com/django/django/pull/7615>.
> 
> I think the functionality is basically there, so what it really
> needs is
> some testing on real-world use-cases and more eyes to suggest any
> further simplifications or improvements that may be possible.
> 
> Thanks,
> 
> Ian
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-developers+unsubscr...@googlegroups.com
> <mailto:django-developers+unsubscr...@googlegroups.com>.
> To post to this group, send email to django-developers@googlegroups.com
> <mailto:django-developers@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/537b284f-400c-45c9-8f3b-91842c83f6d8%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/537b284f-400c-45c9-8f3b-91842c83f6d8%40googlegroups.com?utm_medium=email_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/999b4720-dfa3-4442-a406-0d577d4339b1%40feete.org.
For more options, visit https://groups.google.com/d/optout.

More eyes on Check Constraints

[Ian Foote]

Hi all,

I've been working on https://code.djangoproject.com/ticket/11964 to add 
support for Check Constraints to the Django ORM and migrations 
framework. My work can be found at 
https://github.com/django/django/pull/7615.


I think the functionality is basically there, so what it really needs is 
some testing on real-world use-cases and more eyes to suggest any 
further simplifications or improvements that may be possible.


Thanks,

Ian

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/58b62be6-7eb1-2b20-3940-1394aa9e39dd%40feete.org.
For more options, visit https://groups.google.com/d/optout.

Re: Ticket 28087: Allow filtering and ordering with RawQuerySet

[Ian Foote]

Hi Dmitriv,

I think you're running into https://code.djangoproject.com/ticket/27332.
Unfortunately this isn't fixed yet, but there's a pull request, so it
might land in 2.0: https://github.com/django/django/pull/7560

Ian

On 21/04/17 19:41, Dmitriy Sintsov wrote:
> If I understand correctly, LEFT JOIN is automatically chosen when
> ForeignKey(null=True) otherwise it will be INNER JOIN?
> 
> My ProfileRequisites.profile ForeignKey is not nullable yet I need to
> use LEFT JOIN in this query so I have profiles and ProfileRequisites in
> the same list. Profiles are always retrieved while ProfileRequisites
> only if these are available (LEFT):
> 
> 
> class ProfileRequisites(models.Model):
> 
> profile = models.ForeignKey(Profile, related_name='requisites')
> requisites_type = models.ForeignKey(
> ContentType, null=True, related_name='related_requisites', 
> verbose_name='Тип реквизитов'
> )
> requisites_id = models.PositiveIntegerField(verbose_name='Ссылка на 
> реквизиты')
> requisites_object = GenericForeignKey('requisites_type', 'requisites_id')
> is_preferred_withdrawal = models.NullBooleanField(default=None, 
> verbose_name='Предпочтительный способ вывода')
> 
> 


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/bda61696-baaa-4640-8523-a7db199951c3%40feete.org.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: [Feature Request] Shorthand syntax for filtering aggregations

[Ian Foote]

Hi Tom,

I really like the look of Count('emails').filter(unread=False) in the
annotate call.

Ian

On 12/04/17 21:35, Tom Forbes wrote:
> Hello,
> At the Djangocon sprints I wanted to add support for a postgres specific
> syntax for filtering aggregations, which is quite simple: MAX(something)
> FILTER (WHERE x=1).
> 
> During this the sprints I was told that it would be good to support this
> for all databases, and it can be done using the CASE syntax: MAX(CASE
> WHEN x=1 THEN something END).
> 
> Doing this with the ORM is quite verbose, it would be really nice to be
> able to have a shorthand syntax for filtering aggregates that can use
> database-specific syntax where available (the postgres syntax is faster
> than the CASE syntax). I've made a proof of concept merge request that
> implements this here:
> 
> https://github.com/django/django/pull/8352
> 
> I'd really like some feedback and to maybe have a discussion about what
> the API could look like. Currently I went for adding `.filter()` and
> `.exclude()` methods to the base Aggregate class. I like this approach
> as it's close to the familiar queryset syntax but I understand there are
> subtleties with combining them (I just AND them together currently).
> It's also better to be consistent - if we can't support all of the
> queryset filter() syntax then we shouldn't confuse people by having a
> filter method that acts differently.
> 
> An example of the API is this:
> 
> Mailboxes.objects.annotate(
>read_emails=Count('emails').filter(unread=False),
>unread_emails=Count('emails').filter(unread=True),
>recent_emails=Count('emails').filter(received_date__lt=one_week_ago)
> )
> 
> 
> I'd really like some feedback on what the API should look like. Is
> filter a good idea? Any feedback on the current merge request
> implementation is appreciated as well, I'm fairly new to the Django
> expression internals. I think I'm going the right way with having a
> separate FilteredAggregate expression but I'm not sure.
> 
> Many thanks,
> Tom
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-developers+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-developers@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFNZOJM0HDNCeLta6L9u7uJY7usJWn5u8ZYO4S8cDFOeOLYgGQ%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/736674a7-334c-2527-b84e-0e2f949f8685%40feete.org.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Add an optional parameter to values() that returns a nested dictionary for foreign keys

[Ian Foote]

I think the consensus here is to not add the extra commit, so I've 
closed the ticket as wontfix. 
(https://code.djangoproject.com/ticket/16735#comment:26)


Ian

On 20/08/16 15:24, Josh Smeaton wrote:
Just as an additional data point - most expressions that accept 
strings internally convert them to F(), so it's not entirely 
inconsistent with other behaviour. I don't really have a preference 
here though, so happy to go with what the majority prefer.


On Saturday, 20 August 2016 04:59:10 UTC+10, Loïc Bistuer wrote:

I prefer enforcing .values(alias=F(’something’)), to me
.values(alias=‘something’) reads as the equivalent of
.values(alias=Value(‘something’)).

-- 
Loïc


> On Aug 20, 2016, at 1:04 AM, Tim Graham  wrote:
>
> We now have support for expressions in values()/values_list() --
thanks Ian! With the new commit [0], aliases can be created like
this: .values(alias=F('field'))
>
> Ian has offered an additional commit in the pull request [1] to
allow .values(alias='field') (without the F() expression) to
automatically wrap the string in an F() expression to create an
alias. I'm not sure whether or not to accept that patch as I think
I prefer the look of the explicit F() rather than magically
treating strings as F() expressions. What do you think?
>
> [0]

https://github.com/django/django/commit/39f35d4b9de223b72c67bb1d12e65669b4e1355b



> [1] https://github.com/django/django/pull/7088

>
> On Wednesday, November 25, 2015 at 7:24:22 PM UTC-5, Josh
Smeaton wrote:
> I would really like two things for values to support.
>
> 1. Aliases .values(alias='field');
> 2. Expressions .values(alias=F('field'))
>
> I think these two features are absolute must haves, and the
syntaxes above are already standard in other parts of the ORM.
>
> If someone can come up with a way to support nested relations
while supporting the above syntax, then I'd be OK with that. But
at the moment, I'm firmly in the "this is the responsibility of a
serialiser" camp. I'm not convinced Django needs to support nested
objects at all. Is this something you could implement with your
own queryset method on a manager? Is this maybe something we could
look at creating a new queryset method called .values_dict() ?
>
> If it weren't for backwards compatibility, I'd suggest that
referencing the related object would automatically nest that
object. That would differentiate between the id and the field
values('related_id', 'related') -> '{"related_id": 1, "related":
{"id": 1, ..}}'.
>
> If there's (rough) consensus on having nested objects, then we
could allow something like: .values(..., ..., nested=('related',
'related__otherrelated')). If the value of nested is an iterable
then assume we're nesting, otherwise nested is an alias for the
field. I don't particularly like overloaded kwargs, but we're just
guarding against someone wanting to alias as "nested" which we
could call out in docs anyway.
>
> The more I think about this the more I think nesting and aliases
within a nest should probably be done in a different queryset
method. Or just handled by a serialiser. If you want more requests
per second, then add some more backends.
>
> --
> You received this message because you are subscribed to the
Google Groups "Django developers (Contributions to Django itself)"
group.
> To unsubscribe from this group and stop receiving emails from
it, send an email to django-develop...@googlegroups.com
.
> To post to this group, send email to
django-d...@googlegroups.com .
> Visit this group at
https://groups.google.com/group/django-developers
.
> To view this discussion on the web visit

https://groups.google.com/d/msgid/django-developers/00d4305c-175e-415c-b446-a53c7d15c00d%40googlegroups.com

.

> For more options, visit https://groups.google.com/d/optout
.

--
You received this message because you are subscribed to the Google 
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-developers+unsubscr...@googlegroups.com 
.
To post to this group, send email to 
django-developers@googlegroups.com 
.

Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit