(Sorry, little mistake on the To: field and this didn't go to the list. My
apologies for the one who received this :P)
Hi everybody again, sorry for the delay in answering.
I've been doing some more testing, with both ORMs (SQLO and SQLA) in debug
mode, and seeing the SQL that gets sent to the db. In the meantime I also
added somewhat of a counter for my app, which counts in 10ms multiples, the
time that it takes to scan the directory. Pretty rough but it serves its
purpose.
So I started comparing both ORMs on different directories. An since SQLA
almost halved the time it took SQLO to scan a directory full of big
(1920x1200) wallpapers, I decided to debug it by scanning a directory with
only one image. First of all, the structure of the class that maps the
filesystem images onto my app, looks something like this:
class Photo(File):
hasthumb = BoolCol(default = None)
author = StringCol(default = None)
res = StringCol(default = None)
date_taken = StringCol(default = None)
soft = StringCol(default = None)
_thumb = PickleCol(default = None)
_icon = PickleCol(default = None)
def get_thumb(self):
return gtk.gdk.pixbuf_new_from_array(self._thumb,
gtk.gdk.COLORSPACE_RGB, 8)
def set_thumb(self, value):
if isinstance(value, gtk.gdk.Pixbuf):
self._thumb = value.get_pixels_array()
def get_icon(self):
return gtk.gdk.pixbuf_new_from_array(self._icon,
gtk.gdk.COLORSPACE_RGB, 8)
def set_icon(self, value):
if isinstance(value, gtk.gdk.Pixbuf):
self._icon = value.get_pixels_array()
thumb = property(get_thumb, set_thumb)
icon = property(get_icon, set_icon)
Those properties are there because in order to pickle an image and be able to
successfully unpickle it, you have to convert it to its pixels' array.
Otherwise, you can't unpickle it. Here is the code that loads both thumb and
icon into the object:
photo.icon = \
gtk.gdk.pixbuf_new_from_file_at_size(photo.strabs,
SETTINGS.thumblistsize,
SETTINGS.thumblistsize)
photo.thumb = \
gtk.gdk.pixbuf_new_from_file_at_size(photo.strabs,
SETTINGS.thumbpanesize,
SETTINGS.thumbpanesize)
photo.hasthumb = True
So, in my app, the thumb and icon for the Photo class are only loaded once, on
the original scan. Then, by looking at SQLO's output I found out this:
http://pastebin.com/raw.php?i=Xp1TjmZ0
Basically, if you look in the previous link for Setting icon or Setting
thumb you will see that both the icon and the thumb are being set TWICE! with
the same data.
Now I honestly don't know where this comes from. SQLA apparently doesn't do
this (hence, its better scanning time), and indeed it is an odd thing to do. I
debugged a little more and found that indeed the properties for both icon and
thumb are being called twice, though I don't know why.
Is there maybe something in the code of the PickleCol that triggers this
behaviour?
Thanks for your help and time :)
Cheers
Juan Manuel Santos
From: Oleg Broytman p...@phd.pp.ru
To: sqlobject-discuss@lists.sourceforge.net
Date: Tuesday 23 March 2010
On Tue, Mar 23, 2010 at 12:57:19AM -0300, Juan Manuel Santos wrote:
Are there any ideas on why would SQLO
be a bit slower
When you declare a table
class MyTable(SQLObject):
...columns...
and do an INSERT by calling MyTable(columns) SQLObject immediately does a
SELECT to get back autogenerated fields (timestamps and the like) This is
slow. It's ok to create rows like this occasionally but it is certainly bad
for mass-insertion.
For mass-insertion use SQLBuilder. Alas, it's underdocumented. Go by
example:
record = {'column1': value1, 'column2': value2, ...}
connection.query(connection.sqlrepr(
sqlbuilder.Insert(MyTable.sqlmeta.table, values=record)))
These are simple straightforward INSERTs without any additional
high-level burden - no SELECT, no caching, nothing. Fire and forget.
It is not as high-level as calling MyTable() but it is still high
enough - sqlrepr() does value conversion and quoting, e.g.
Oleg.
--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
sqlobject-discuss mailing list
sqlobject-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
