How you are thing to show this to user ? Case 1: User request a search => you (web2py) dispatch all the 5 (up to 20) sub-external-searches, *only* after finishing all 5 - 20 sub-external-searches, send the response for the user.
Case 2: User request a search => you (web2py) put all the 5 - 20 sub-external-searches in a background tasks, send a response to the user like "Your search is being performed " (you can use javascript to polling the server. And show the final result after the background tasks have finshed. In case 1 in any language you cannot have a response in a reasonable time, because external access always have chance to have problem, in your network in other 5 to 20 networks. The case 2, I think, you can use the new web2py scheduler. In the case 2 and the scheduler seems to be a good idea to use threads, even with Python and its GIL. :-) [], On Tue, Feb 28, 2012 at 6:05 PM, Marcelo Zamperetti <[email protected]>wrote: > I know a bit of programming but still new to python and web2py (and to > web programming in general, actually), so sorry if I make a gaffe and > for my poor english. > I have the intention of developing a meta-search website. It queries a > bunch of other sites for results, processes them and show to the > user. > I need to query all the other sites at the same time (I'll start with > 5 but intend to go up to around 15 or 20), if I do one after another I > won't be able to respond to the user in a reasonable time. I found on > the book about background tasks, but none of the approaches seemed to > fit my problem. They are about running tasks at pre-determined times > or for running other applications. I just need something as simple as > threads, but I've read somewhere that I can't invoke threads like a > normal python application on the controller, and from what I've > understood it is in the controller that my code will run. > I'm probably missing something very simple, but I would be grateful if > someone could point me in the right way (even if the right is another > web framework or even another language). > Thank you very much. -- Luciano Pacheco blog.lucmult.com.br
