Hello John, We are doing OCR, so we need the pictures to be there :) But besides the initial API call (receipt creation) and the upload of the pictures, there is no user interaction at this point in the process.
I've given some thought about what you suggested, but I still see some flaws : - If we have Ruote poll our database for the status of the upload, we may hit some performance problems, as we would have 10-20 threads polling our db during peak time. - If we implement a timeout in the participant, the master process will fail, and then if a user reopens the app and the upload restarts the master process will not pick it up (correct me if I'm wrong regarding the timeout implementation). This regularly happens, especially with our iOS app in which we can't upload in the background. I guess the issue here is that we are trying to be as fast as possible with the OCR so we need to treat each picture as soon as they are uploaded and in parallel. On the other hand, if a single thread is polling the db then it's possible, what about a special kind of Receiver that knows "these are the waiting participants with their picture id, and these are the uploaded pictures in the last X seconds" ? >From another angle, we could also do the synchronization after the processing of each each picture, which could be easier to implement ? If each upload launches a separate ruote process, is there a way for these subprocesses to communicate with a master process inside Ruote ? They are still not launched by the master process, but this time everything happens inside Ruote I think that having a way to do inter-process communication inside ruote could be nice to have, what do you think ? Best regards, Adrien On Thursday, April 4, 2013 11:38:41 PM UTC+2, John Mettraux wrote: > > > On Thu, Apr 04, 2013 at 08:51:55AM -0700, Adrien Kohlbecker wrote: > > > > We are still slowly migrating our receipt processing to ruote and we hit > > the following roadblock : > > > > We have a master process, launched by an API call from the user > > Right after that call, the user uploads one or more pictures > > asynchronously. Each picture is processed independently. > > After each picture is processed, the master process should resume > > > > I was looking at the way you implemented AlertParticipant in ruote-amqp > but > > I don't think it works for our use case : > > > > - The pictures could finish uploading in two month, if the user closes > our > > app and does not reopen it right away, or in the worst case never, if > the > > user never reopens the app, so we can't have each AlertParticipant > polling > > a queue until uploaded > > > > - If the user is on a fast connection, the upload could finish before > our > > backend has had time to launch the master process, so we need a way to > > cache the notifications until they are ready to be processed. > > > > The issue here is that the subprocess is not launched by ruote so there > is > > no workitem to store and receive once the upload is done (especially > since > > the reply could happen before any workitem exists) > > > > How would you go about implementing something like that ? > > Hello Adrien, > > how do you correlate the pictures and the main process? I guess there is > some > kind of shared identifier. > > You could start the process after the picture upload. Ok, it doesn't work > if > you want the user to deal with the process while the pictures load... > > Back to the equivalent of an AlertParticipant. How about placing a timeout > attribute on it? > > http://ruote.rubyforge.org/common_attributes.html#timeout > > This could help solve the "it took 2 months" issue. That doesn't solve the > "it's already here" one. > > What about having a participant that looks like: > > ``` > class ImageGrabber < Ruote::Participant > > def on_workitem > > # somehow a blocking call... > # returns as soon as the pictures are done uploading > # returns immediately if the pictures are already uploaded > # > workitem.fields['images'] = ImageUploadService.fetch_uris(case_id) > > reply > end > > # probably called on_timeout > # > def on_cancel > > # kills the pending .fetch_uris if present > # > ImageUploadService.stop_fetching_uris(case_id) > end > > protected > > # use the ruote wfid as correlation id > # > def case_id > > workitem.wfid > end > end > ``` > ? > > The pictures are uploaded somewhere in some service. Let's communicate > with > that service. > > You could also bypass the "let's wait for the pictures to upload" > participant > and consider that the pictures are there (unless you do OCR, the pictures > are > probably not worth much for the application). You could complain later on, > when you notice that the pictures are not here yet. > > I guess the trick is not to wait for some notifications about the images, > but > to poll for them. Could that participant be summarized as "wait until the > pictures are uploaded"? > > <parentheses> > > Ruote is not meant to build wizards, you'd better gather all the info for > the > launch step and then launch. The same can be said for any "human" step, > Trying to piggyback a ruote process with a web wizard is hard. Having a > dedicated screen (set of screens) for a ruote "human" step is better. > > ``` > sequence do > fill_in_form_details > end > ``` > > is better than > > ``` > sequence do > fill_in_form_name > fill_in_form_address > fill_in_form_whatever > # ... > end > ``` > > Of course, that second flow makes sense if the human participant is > different > at each step... > > </parentheses> > > Ah, a warning, do not put the pictures themselves in the workitem or in > queue > messages, place the URIs or whatever id of the picture. Sorry, it may > already > be obvious to you. > > > I hope that this confusing mix will help anyway, best regards, > > -- > John Mettraux - http://lambda.io/jmettraux > > -- -- you received this message because you are subscribed to the "ruote users" group. to post : send email to [email protected] to unsubscribe : send email to [email protected] more options : http://groups.google.com/group/openwferu-users?hl=en --- You received this message because you are subscribed to the Google Groups "ruote" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
