Le Mon, 16 May 2011 13:26:06 +0200,
Cédric Krier <[email protected]> a écrit :
> Hi,
>
> I would like to talk about an idea I had when writing some specific
> code and listening to the Guido's talk about async call in GAE
> storage.
>
> Most of the time async call is not necessary in Tryton because we
> already use a lot of "grouping" calls like for the BrowseRecord where
> we read a bunch of ids at once.
>
> But there is some cases where this pattern can not be used. For
> example, the code that I was writing looks like that:
>
> for record in self.browse(ids):
> ids = self.search([
> ...
> ], order=[('x', 'ASC')], limit=1)
> if ids:
> ...
>
> I can not make this search working for a bunch of ids. So I'm facing
> a O(n) method and this kind of method doesn't scale well.
>
> So after re-reading the psycopg2 doc [1], I could create a thread per
> query and read the result after which will allow parallelisation of
> the queries and will improve the performences.
It seems that it's not necessary to create new threads (at least for
postgresql). If I understand the doc[1] correctly you can use select[2]
on DB connection.
Maybe it would possible to use async call for _all_ the queries, and
then offer concurrent access using only one process and no thread :D
[1]
http://initd.org/psycopg/docs/advanced.html#asynchronous-notifications
&
http://initd.org/psycopg/docs/advanced.html#support-to-coroutine-libraries
[2] http://docs.python.org/library/select.html
> The idea will be to have something like:
>
> queries = {}
> for record in self.browse(ids):
> queries[record.id] = Async(self.search, [...], order=[('x',
> 'ASC')], limit=1)
>
> for id, query in queries.iteritems():
> ids = query.result()
> if ids:
> ...
>
> The Async method will be in charge of setting a new cursor for the
> thread, creating a thread and keep the result until the main thread
> will read it. Of course, for backend that doesn't support
> multi-threading, the method will do the query directly.
>
> What do you think about this API?
>
>
> [1] http://initd.org/psycopg/docs/usage.html#thread-safety
>
--
Bertrand Chenal
B2CK SPRL
Rue de Rotterdam, 4
4000 Liège
Belgium
Tel: +32 472 54 46 59
Email: [email protected]
Website: http://www.b2ck.com/
--
[email protected] mailing list
