Re: Does using the IPython shell add "LIMIT" to Django ORM discussion (using SQLite)

[Ramiro Morales]

On Thu, Jan 8, 2009 at 7:58 AM, Christopher Mutel  wrote:
>
> Hello all-
>
> I recently filed a bug about incorrect SQL generation, and Malcom
> Tredinnick said that the example SQL I provided couldn't be correct,
> because there was an extra LIMIT clause that shouldn't be there. After
> poking around for a bit, I realized that everytime I was executing my
> query in the IPython shell, it was appending a LIMIT 21 to the SQL,
> but this limit clause wasn't being appended if the query was part of a
> regular python process. Both queries are run against SQLite 3.
>
> I uploaded a small test case here that demonstrates this behaviour:
>
> http://www.bitbucket.org/cmutel/extra_limit_21/
>
> It is possible that I have poor python-fu, or is this known behaviour?
> Is it possible that Django is truncating the result list because
> IPython won't show all results anyway?
>
> I have tried searching the Django codebase, and the mailing list, but
> to no avail.

Django [1]generates a LIMIT 21 in the __repr__ method of a queryset,
because it limits such representation to 20 items when it's in the
context of (for example) showing it in a traceback, this was added in
[2]r9202. Maybe you are getting the SQL clause generated by you ROM
query with .query().as_sql() in the interactive Python interpreter and
triggering this?

See if r9717 solves you duplicate column name problem, the problem
in the query isn't caused by that LIMIT part as it isn't sent to the
DB in other 'normal' conditions.

Regards,

-- 
 Ramiro Morales

1. 
http://code.djangoproject.com/browser/django/trunk/django/db/models/query.py#L146
2. http://code.djangoproject.com/changeset/9202

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Does using the IPython shell add "LIMIT" to Django ORM discussion (using SQLite)

[Malcolm Tredinnick]

On Thu, 2009-01-08 at 10:58 +0100, Christopher Mutel wrote:
> Hello all-
> 
> I recently filed a bug about incorrect SQL generation, and Malcom
> Tredinnick said that the example SQL I provided couldn't be correct,
> because there was an extra LIMIT clause that shouldn't be there. After
> poking around for a bit, I realized that everytime I was executing my
> query in the IPython shell, it was appending a LIMIT 21 to the SQL,
> but this limit clause wasn't being appended if the query was part of a
> regular python process. Both queries are run against SQLite 3.

Aah .. I understand what's going on now! :-)

You were printing (or, at least, trying to print) the repr() of the
queryset. To avoid people accidentally trying to retrieve and print a
million results, we (well, I) changed that to only retrieve and print
the first 20 results and print "remainder truncated" if there were more.
This is achieved by limiting the query to 21 results (if there are 21
results there are more than 20, so we print the "truncated" message).
That only happens in the  __repr__() -- i.e. it's only for diagnostic
printing. No normal user code has this limit included automatically, so
you happily create a queryset that iterates over a million results.

It hadn't occurred to me when reading that ticket that you were pulling
the query out of django.db.connection.queries, but that makes sense now
that I see your code. I, almost invariably, use
"queryset.query.as_sql()" to view the query that the queryset is
creating and that wouldn't show the limit.

So, you aren't losing your mind (and, luckily, neither am I). The SQL
limit you're seeing isn't the direct representation of the queryset.
It's a result of trying to print the repr() of the queryset. Not related
to IPython or anything like that. Just a safeguard against accidental
repr() printing (after hearing people like Google, NASA and even a
semi-well-known media organisation in Kansas complain about making that
blunder, I broke down and added it a couple of months ago).

Thanks for following this up a bit more. Hopefully next time I see a
strange "LIMIT 21" on something, I'll remember how it might occur.

In any case, your example was clear enough that I could repeat the
problem and, as you have no doubt seen, it was actually exposing a
different and more significant problem which has now been fixed.

Regards,
Malcolm




--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Does using the IPython shell add "LIMIT" to Django ORM discussion (using SQLite)

[Christopher Mutel]

Hello all-

I recently filed a bug about incorrect SQL generation, and Malcom
Tredinnick said that the example SQL I provided couldn't be correct,
because there was an extra LIMIT clause that shouldn't be there. After
poking around for a bit, I realized that everytime I was executing my
query in the IPython shell, it was appending a LIMIT 21 to the SQL,
but this limit clause wasn't being appended if the query was part of a
regular python process. Both queries are run against SQLite 3.

I uploaded a small test case here that demonstrates this behaviour:

http://www.bitbucket.org/cmutel/extra_limit_21/

It is possible that I have poor python-fu, or is this known behaviour?
Is it possible that Django is truncating the result list because
IPython won't show all results anyway?

I have tried searching the Django codebase, and the mailing list, but
to no avail.

-Chris

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---