Re: [Django] #27123: prefetch_related return mistaken result

[Django]

#27123: prefetch_related return mistaken result
-+-
 Reporter:  mostafa  |Owner:  nobody
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  1.9
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  needsinfo
 Keywords:  prefetch_related,| Triage Stage:  Accepted
  ORM, postgresql|
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham):

 * status:  new => closed
 * resolution:   => needsinfo


Comment:

 Agreed, it seems we need additional information (i.e. a complete test
 case) from the reporter.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.d3dcdfc8322148d597636c5487dbe22d%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27123: prefetch_related return mistaken result

[Django]

#27123: prefetch_related return mistaken result
-+-
 Reporter:  mostafa  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.9
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch_related,| Triage Stage:  Accepted
  ORM, postgresql|
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by François Freitag):

 * cc: mail@… (added)


Comment:

 Hi,

 I fail to reproduce this issue. The code given by Baptiste leads to a test
 fail, but it's only a matter of ordering.
 The test below passes (I've commented the changes):

 {{{#!python
 # tests.py
 def test_reproduction(self):
 my_teams =  GlobalTeam.objects.prefetch_related(
 'user_profiles',
 ).annotate(
 user_cnt=(
 Count('user_profiles', distinct=True) + 1  # Added the "+
 1"
 ),
 ).distinct()

 self.assertEqual(len(my_teams), 2)

 self.assertQuerysetEqual(
 my_teams[1].user_profiles.all(),  # Changed my_teams[0] to
 my_teams[1]
 ['', '', ''],
 ordered=False,
 )
 self.assertQuerysetEqual(
 my_teams[0].user_profiles.all(),   # Change my_teams[1] to
 my_teams[0]
 [''],
 ordered=False,
 )

 evaluated = list(my_teams)
 self.assertQuerysetEqual(
 evaluated[1].user_profiles.all(),   # Changed evaluated[0] to
 evaluated[1]
 ['', '', ''],
 ordered=False,
 )
 self.assertQuerysetEqual(
 evaluated[0].user_profiles.all(),  # Changed evaluated[1] to
 evaluated[0]
 [''],
 ordered=False,
 )
 }}}

 I cannot reproduce the original issue:
 {{{#!python
 my_teams =  GlobalTeam.objects.filter(Q(team_admin__user=user) |
 Q(user_profiles__user=user)).select_related(
 'team_admin',
 'team_admin__user'
 ).prefetch_related(
 'user_profiles',
 'user_profiles__user',
 ).annotate(
 user_cnt=(
 Count('user_profiles', distinct=True) +
 1
 ),
 ).distinct()
 # Breakpoint...
 (Pdb) print(my_teams[0].user_profiles.all())
 []
 (Pdb) print(my_teams[1].user_profiles.all())
 [, , ]
 (Pdb) for team in my_teams: print(team.user_profiles.all())
 []
 [, , ]
 (Pdb) my_teams[0].user_profiles.all()
 []
 }}}

 I'm using django 1.9.9 and postgresql 9.3.14.
 I could not reproduce it on master as well (with postgresql 9.5.4).

 Have I missed something?

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.f5d80cd0ff793f2a8a16bc2f5783d059%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27123: prefetch_related return mistaken result

[Django]

#27123: prefetch_related return mistaken result
-+-
 Reporter:  mkarimim |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.9
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch_related,| Triage Stage:  Accepted
  ORM, postgresql|
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by bmispelon):

 * stage:  Unreviewed => Accepted


Comment:

 Hi,

 I can reproduce the issue on master (with postgres, not with sqlite) with
 the attached models and test:
 {{{#!py
 # models.py
 from django.db import models


 class User(models.Model):
 name = models.CharField(max_length=20, unique=True)


 class UserProfile(models.Model):
 user = models.OneToOneField('User', related_name='profile',
 on_delete=models.CASCADE)

 def __str__(self):
 return self.user.name


 class GlobalTeam(models.Model):
 team_name = models.CharField(max_length=100)
 user_profiles = models.ManyToManyField('UserProfile',
 related_name='user_teams')
 team_admin = models.ForeignKey('UserProfile',
 related_name='head_teams', on_delete=models.CASCADE)


 # tests.py
 from django.db.models import Count, Q
 from django.template import Template, Context
 from django.test import TestCase

 from .models import User, UserProfile, GlobalTeam


 class ReproTestCase(TestCase):
 @classmethod
 def setUpTestData(cls):
 users = [User.objects.create(name=c) for c in 'abcd']
 profiles = [UserProfile.objects.create(user=user) for user in
 users]

 team1 = GlobalTeam.objects.create(team_name='Team 1',
 team_admin=profiles[0])
 team2 = GlobalTeam.objects.create(team_name='Team 2',
 team_admin=profiles[0])

 team1.user_profiles.add(*profiles[:3])
 team2.user_profiles.add(profiles[3])


 def test_reproduction(self):
 my_teams =  GlobalTeam.objects.prefetch_related(
 'user_profiles',
 ).annotate(
 user_cnt=(
 Count('user_profiles', distinct=True)
 ),
 ).distinct()

 self.assertEqual(len(my_teams), 2)


 self.assertQuerysetEqual(
 my_teams[0].user_profiles.all(),
 ['', '', ''],
 ordered=False,
 )
 self.assertQuerysetEqual(
 my_teams[1].user_profiles.all(),
 [''],
 ordered=False,
 )

 evaluated = list(my_teams)
 self.assertQuerysetEqual(
 evaluated[0].user_profiles.all(),
 ['', '', ''],
 ordered=False,
 )
 self.assertQuerysetEqual(
 evaluated[1].user_profiles.all(),
 [''],
 ordered=False,
 )
 }}}

 Interestingly, removing the `+1` after the `Count(...)` makes the test
 pass. The same happens when removing the `.distinct()` call at the end or
 the `.annotate(...)`.

 I don't really understand what is happening and I'm not sure if this is a
 bug in Django or a misuse of the ORM but I'll assume the former and move
 the ticket forward.


 Thanks.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.f95bd44a91f8d9bb05d5afa619169f3b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27123: prefetch_related return mistaken result

[Django]

#27123: prefetch_related return mistaken result
-+-
 Reporter:  mkarimim |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.9
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch_related,| Triage Stage:
  ORM, postgresql|  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by mkarimim):

 If you see outputs, for first item the output is:
 {{{
 >>> my_teams[0].user_profiles.all()
 [, , ]
 }}}
 that's true output for this item and second item output is:
 {{{
 >>> my_teams[1].user_profiles.all()
 []
 }}}
 that's true output for this item, now I run a for loop on the list the
 output is:

 {{{
 >>> for team in my_teams:
 ...  print(team.user_profiles.all())
 ...
 [, , , ]
 []
 }}}

 as you see the output is wrong and  must be in the second
 item but after for loop it's in the first item!

 {{{
 >>> my_teams[0].user_profiles.all()
 [, , , ]
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.2b93cf5a260e242f33809fbdc91ab493%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27123: prefetch_related return mistaken result

[Django]

#27123: prefetch_related return mistaken result
-+-
 Reporter:  mkarimim |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.9
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch_related,| Triage Stage:
  ORM, postgresql|  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by timgraham):

 * needs_better_patch:   => 0
 * severity:  Release blocker => Normal
 * needs_tests:   => 0
 * needs_docs:   => 0


Comment:

 What's the expected result? Can you provide a failing test case including
 the data?

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.7ae4b6cb4cbebc11c85781acbf9bd01f%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.