On Thu, Apr 04, 2013 at 03:38:25PM -0700, Adrien Kohlbecker wrote:
>
> We are doing OCR, so we need the pictures to be there :) But besides the
> initial API call (receipt creation) and the upload of the pictures, there
> is no user interaction at this point in the process.
Hello,
so why not start the process when the pictures are uploaded?
> I've given some thought about what you suggested, but I still see some
> flaws :
>
> - If we have Ruote poll our database for the status of the upload, we may
> hit some performance problems, as we would have 10-20 threads polling our
> db during peak time.
>
> - If we implement a timeout in the participant, the master process will
> fail, and then if a user reopens the app and the upload restarts the master
> process will not pick it up (correct me if I'm wrong regarding the timeout
> implementation). This regularly happens, especially with our iOS app in
> which we can't upload in the background.
What do you currently have in place for the uploads?
Can it be wrapped in a service that ruote talks to via a participant (and/or
a receiver as you suggested below)?
> I guess the issue here is that we are trying to be as fast as possible with
> the OCR so we need to treat each picture as soon as they are uploaded and
> in parallel.
OK.
> On the other hand, if a single thread is polling the db then it's possible,
> what about a special kind of Receiver that knows "these are the waiting
> participants with their picture id, and these are the uploaded pictures in
> the last X seconds" ?
Yes, much better than my Participant + ImageUploadService suggestion. It
would have to survive a restart though. That knowledge in the receiver is a
cache somehow. When the participant "arrives" it may be allowed a db query
(cache miss, pass through) to see if the images are really not there (the
other case would be the images are there, but the receiver forgot about
them).
> From another angle, we could also do the synchronization after the
> processing of each each picture, which could be easier to implement ? If
> each upload launches a separate ruote process, is there a way for these
> subprocesses to communicate with a master process inside Ruote ? They are
> still not launched by the master process, but this time everything happens
> inside Ruote
>
> I think that having a way to do inter-process communication inside ruote
> could be nice to have, what do you think ?
There is the "listen" expression that brings some kind of inter-process
communication, although participants [+ receivers] are probably better fits,
they are sitting between ruote processes and external systems (other ruote
processes are external systems somehow).
```
RuoteEngine = Ruote::Dashboard.new(...)
main_flow =
Ruote.define do
wait_for_images
# ... the rest
end
upload_flow =
Ruote.define do
wait_for_upload
perform_ocr
notify_main_flow
end
main_wfid =
RuoteEngine.launch(
main_flow, 'images' => images)
images.each do |image_info|
RuoteEngine.launch(
upload_flow, 'main_wfid' => main_wfid, 'image_info' => image_info)
end
#
# OR
#
main_flow =
Ruote.define do
iterator :on => 'f:images', :to => 'f:image_info' do
wait_for_upload
perform_ocr
notify_main_flow
end
# ... the rest
end
# ...
```
I prefer the first version, smaller processes interacting.
So many ways to skin a cat.
Though I still prefer launching the main flow when the images are uploaded
and OCRized.
Not sure if it's a case of "hey ruote needs to be adapted to our needs",
sounds more like the classical case of "let's clearly define services so that
ruote and other components in our architecture can leverage them".
Please remember that I know nothing of your
architecture/intentions/requirements, just letting my imagination loose.
Thanks for reminding me of the receivers.
Best regards,
--
John Mettraux - http://lambda.io/jmettraux
--
--
you received this message because you are subscribed to the "ruote users" group.
to post : send email to [email protected]
to unsubscribe : send email to [email protected]
more options : http://groups.google.com/group/openwferu-users?hl=en
---
You received this message because you are subscribed to the Google Groups
"ruote" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
