@Arndt, I can run multiple instances of my application but how exactly can I then "route the slow running tasks to the one and the rest to the other"? Could you also elaborate on the session cache?
@Jonathan, the response or fetched information returning from the "slow running task" (the connection to the SOAP-XML server) is not actually needed to complete the flow that the user is going through. It's actually an online contest (prize question): - page 1 loads: user is asked for some data (name, address, ...) -> user clicks submit: data is not yet persisted but stored in the session - page 2 loads: user is asked to answer a question and a tiebreaker (how many people will have participated by the end ...) -> user clicks submit: user's emailaddress is used to check if user already exists in the remote CRM (via SOAP), if not he is added, if yes, his CRM-id is fetched. Now, the user data along with his CRM-id and answers to both questions is persisted in an SQLite database. Now, in my idea: wouldn't it be better, in the second step (page 2) to just immediately persist the user submitted data to the SQLite database to allow the user to continue the flow and to hand off the connection to the remote CRM to another app/process/Celery/whatever after which I can (if needed) update the user's data in de database? Thanks already for all the responses! Maarten On Tuesday, 29 January 2013 03:17:25 UTC+1, Jonathan Vanasco wrote: > > I don't think you'd necessarily need to use celery/redis/etc for this. > > No matter what you use to fetch the remote system data , you'll need > to implement a user pattern like this: > > 1. User loads the web page , which is 'blank' with a loading message > 2. The web-page fires a javascript call to an API, which then returns > the relevant data and drops it on the page. > > If you don't use an ajax solution like that... no matter what you do, > you'll still have people clicking 'reload' before the 10 seconds are > up. > > My suggestion would be to add an animation like a "loading %" gif to > the screen. it can be complete BS -- you're not really showing the % > loaded, just that you're in a loading process. make it a 30s gif to > reach 100%. > > Then have your system call an API endpoint, which returns your data in > a json packet. it can have the entire HTML you want to render via > mako/whatever templates. just return it within a json wrapper for > easy placement. > > then you can look into: > - integrating a local cache to check if your 3rd party request was > asked / filled yet. this way , in case someone clicks 4 times, only > the first one generates a 3rd party call. the second and third say > 'busy' and the 4th reload has the data from the 1st call. > - offloading onto other systems. > > i'm a huge fan of using redis / celery / even custom twisted daemons > to do background processing and tasks that are essentially queued for > 'later'. in your case though, your web content generation will 'hang' > on the third party no matter what you do. the best way to meet your > goals is to change the user experience, not your platforms. > -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/pylons-discuss?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
