On Aug 6, 12:09 am, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
> If you're benchmarking this against a small dataset and an in-memory
> database like SQLite I'd fully expect to see the defer/only benchmark
> to be slower. That's because every time a QS is chained it needs to be
> copied, which is a relatively expensive operation. In a setup with
> small data, the time spent in Python is going to outweigh the time
> spent running the query in the database and sending the data over the
> wire.
A retest with:
for pk in xrange(1,50000):
user = User.objects.only("power_level").get(pk = pk)
d = user.power_level
replaced with:
qs = User.objects.only("power_level")
for pk in xrange(1, 50000):
user = qs.get(pk=pk)
d = user.power_level
# repeat for all tests.
Should remove the effect of query building. (btw why not "inplace()
function for query set if you know you won't be using the middle steps
in chained query building?)
I have been looking into object creation speed when loading many
objects simultaneously, for my particular case running the query as
values_list took something like 5ms (sorry, I don't remember the
values and I am not at work at the moment), 50ms when constructing
instances (1500 objects in my case). The problem seems to be that for
every object created we go and check which fields the model has, which
of them come from args, which are deferred and which have default
values. It would probably be much faster to check the field
classifications once, and the bulk create the objects using the
classifications.
I did a small test for building 1500 objects in this style (only
object building, not from query set), and the result was something
like 50% faster object building. And yes, signals were sent for every
object created + default() was called if it was callable for the
field. The iterator in query makes it hard to use bulk loading style
however...
The same problem is present in row iteration when constructing objects
from the rows returned from the DB. For each row we check which select
related things we have etc. This could be checked once before starting
to iterate through the rows and then use the calculated values each
time when building the object from the row.
- Anssi
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
