Hi, I would like to talk about an idea I had when writing some specific code and listening to the Guido's talk about async call in GAE storage.
Most of the time async call is not necessary in Tryton because we already use
a lot of "grouping" calls like for the BrowseRecord where we read a bunch of
ids at once.
But there is some cases where this pattern can not be used. For example, the
code that I was writing looks like that:
for record in self.browse(ids):
ids = self.search([
...
], order=[('x', 'ASC')], limit=1)
if ids:
...
I can not make this search working for a bunch of ids. So I'm facing a O(n)
method and this kind of method doesn't scale well.
So after re-reading the psycopg2 doc [1], I could create a thread per query
and read the result after which will allow parallelisation of the queries and
will improve the performences.
The idea will be to have something like:
queries = {}
for record in self.browse(ids):
queries[record.id] = Async(self.search, [...], order=[('x', 'ASC')],
limit=1)
for id, query in queries.iteritems():
ids = query.result()
if ids:
...
The Async method will be in charge of setting a new cursor for the thread,
creating a thread and keep the result until the main thread will read it.
Of course, for backend that doesn't support multi-threading, the method will
do the query directly.
What do you think about this API?
[1] http://initd.org/psycopg/docs/usage.html#thread-safety
--
Cédric Krier
B2CK SPRL
Rue de Rotterdam, 4
4000 Liège
Belgium
Tel: +32 472 54 46 59
Email/Jabber: [email protected]
Website: http://www.b2ck.com/
pgpvzqsFupxin.pgp
Description: PGP signature
