Re: [Django] #30374: Paginator UnorderedObjectListWarning on union(all=True) of two sorted queries

[Django]

#30374: Paginator UnorderedObjectListWarning on union(all=True) of two sorted
queries
-+-
 Reporter:  Rich Rauenzahn   |Owner:  nobody
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  invalid
 Keywords:   | Triage Stage:
 |  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Rich Rauenzahn):

 ...Actually.  I could have sworn I read somewhere that union(all=True)
 preserves order because it doesn't remove duplicates, but I can't find it
 again.

 Postgres says:

 > UNION effectively appends the result of query2 to the result of query1
 (although there is no guarantee that this is the order in which the rows
 are actually returned).
 > Furthermore, it eliminates duplicate rows from its result, in the same
 way as DISTINCT, unless UNION ALL is used.

 In practice it appears to be true, but I don't think it's strictly defined
 in the spec.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.302aefa87e4bf23a012c0bca38f8d726%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #30374: Paginator UnorderedObjectListWarning on union(all=True) of two sorted queries

[Django]

#30374: Paginator UnorderedObjectListWarning on union(all=True) of two sorted
queries
-+-
 Reporter:  Rich Rauenzahn   |Owner:  nobody
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  invalid
 Keywords:   | Triage Stage:
 |  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Rich Rauenzahn):

 I *can't* add an `order_by()` because the key I need to sort by isn't in
 columns union'ed -- and sorting the union by `common_key` would *unorder*
 the sort by `foreign_relationship__value`

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.8476d857b9cdd0c7f58c6a5fdfdc9c5a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #30374: Paginator UnorderedObjectListWarning on union(all=True) of two sorted queries

[Django]

#30374: Paginator UnorderedObjectListWarning on union(all=True) of two sorted
queries
-+-
 Reporter:  Rich Rauenzahn   |Owner:  nobody
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  invalid
 Keywords:   | Triage Stage:
 |  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by felixxm):

 * status:  new => closed
 * component:  Uncategorized => Database layer (models, ORM)
 * version:  1.11 => master
 * resolution:   => invalid


Comment:

 I agree that in your case all composed queries are ordered but the final
 query is not. To avoid this warning you can add `order_by()` to the final
 queryset, e.g.
 {{{#!python
 haves =
 
MyModel.objects.filter(foreign_relationship=4).order_by('foreign_relationship__value',
 'common_key')
 havenots = MyModel.objects.exclude(id__in=haves).order_by('common_key')
 query = haves.union(havenots, all=True).order_by('common_key')
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.678c2c180674564eba12b755f85a9580%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #30374: Paginator UnorderedObjectListWarning on union(all=True) of two sorted queries

[Django]

#30374: Paginator UnorderedObjectListWarning on union(all=True) of two sorted
queries
+--
 Reporter:  Rich Rauenzahn  |Owner:  nobody
 Type:  Bug |   Status:  new
Component:  Uncategorized   |  Version:  1.11
 Severity:  Normal  |   Resolution:
 Keywords:  | Triage Stage:  Unreviewed
Has patch:  0   |  Needs documentation:  0
  Needs tests:  0   |  Patch needs improvement:  0
Easy pickings:  0   |UI/UX:  0
+--
Description changed by Rich Rauenzahn:

Old description:

> I wonder if this is a case you want to catch and *not* warn about.
>
> In my case, I'm doing this:
>
> {{{
> haves =
> MyModel.objects.filter(foreign_relationship=4).order_by('foreign_relationship__value',
> 'common_key')
> havenots = MyModel.objects.exclude(id__in=haves).order_by('common_key')
> query = haves.union(havenots, all=True)
> }}}
>
> I'm using this with a `Paginator`.  The `Paginator` complains the queries
> are not ordered, but they actually are (right?) due to the `all=True` in
> the `union`.
>
> Is this a case the warning ought to handle and ignore?
>
> The Django 1.11 source is:
>
> {{{
> def _check_object_list_is_ordered(self):
> """
> Warn if self.object_list is unordered (typically a QuerySet).
> """
> ordered = getattr(self.object_list, 'ordered', None)
> if ordered is not None and not ordered:
> obj_list_repr = (
> '{} {}'.format(self.object_list.model,
> self.object_list.__class__.__name__)
> if hasattr(self.object_list, 'model')
> else '{!r}'.format(self.object_list)
> )
> warnings.warn(
> 'Pagination may yield inconsistent results with an
> unordered '
> 'object_list: {}.'.format(obj_list_repr),
> UnorderedObjectListWarning,
> stacklevel=3
> )
> }}}
>
> Should `union(..., all=True)` set ordered in the `QuerySet`?  Or maybe
> should it look and see if all the source queries are ordered and
> propagate that to the `QuerySet` `union` returns?

New description:

 I wonder if this is a case you want to catch and *not* warn about.

 In my case, I'm doing this:

 {{{
 haves =
 
MyModel.objects.filter(foreign_relationship=4).order_by('foreign_relationship__value',
 'common_key')
 havenots = MyModel.objects.exclude(id__in=haves).order_by('common_key')
 query = haves.union(havenots, all=True)
 }}}

 I'm using this with a `Paginator`.  The `Paginator` complains the queries
 are not ordered, but they actually are (right?) due to the `all=True` in
 the `union`.

 Is this a case the warning ought to handle and ignore?

 The Django 1.11 source is:

 {{{
 def _check_object_list_is_ordered(self):
 """
 Warn if self.object_list is unordered (typically a QuerySet).
 """
 ordered = getattr(self.object_list, 'ordered', None)
 if ordered is not None and not ordered:
 obj_list_repr = (
 '{} {}'.format(self.object_list.model,
 self.object_list.__class__.__name__)
 if hasattr(self.object_list, 'model')
 else '{!r}'.format(self.object_list)
 )
 warnings.warn(
 'Pagination may yield inconsistent results with an
 unordered '
 'object_list: {}.'.format(obj_list_repr),
 UnorderedObjectListWarning,
 stacklevel=3
 )
 }}}

 Should `union(..., all=True)` set `ordered` in the `QuerySet`?  Or maybe
 should it look and see if all the source queries are ordered and propagate
 that to the `QuerySet` `union` returns?

--

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.4c5e22ea63dd2b59922d0191244d8833%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.