On Fri, Apr 05, 2013 at 05:06:12PM +0200, Adrien Kohlbecker wrote:
>
> So I worked on this issue today and this is what I implemented :
>
> A synchronization mechanism between two processes (1-1 relation) that only
> let the processes continue when both processes have reached the same
> synchronization participant.
> It works by storing the synchronization key along with the first workitem
> to reach this point. Once a second participant is called with the same key,
> we fetch the workitem from the storage, receive it and thus resume the
> first process, and we reply immediately to the second one, continuing the
> second process.
>
> So there are no db hits when there is no activity, and it does not matter
> which process gets to the synchronization first :)
Hello Adrien,
> *master process *
>
> pdef = Ruote.process_definition :name => 'receipt_processing' do
>
> concurrent_iterator :on_field => 'receipt_subpicture_uuids', :to_var
> => 'uuid' do
> synchronize :key => "processed_image-${v:uuid}"
> do_something_with_the_ocr_results
> end
>
> rest of master process...
>
> end
>
> *child processes*
>
> pdef = Ruote.process_definition :name =>
> 'receipt_subpicture_processing' do
>
> synchronize :key =>"processed_image-${f:receipt_subpicture_uuid}"
>
> end
Which is equivalent to
```
# *master process *
pdef = Ruote.process_definition :name => 'receipt_processing' do
concurrent_iterator :on_field => 'receipt_subpicture_uuids', :to_var
=> 'uuid' do
process_image
do_something_with_the_ocr_results
end
# rest of master process...
end
```
can also be split into process and subprocess
```
# *master process *
pdef = Ruote.process_definition :name => 'process_receipts' do
define 'process_receipt_pictures' do
citerator :on_f => 'receipt_subpicture_uuids', :to => 'v:uuid' do
process_image
do_something_with_the_ocr_results
end
end
process_receipt_pictures
# rest of master process...
end
```
"So there are no db hits when there is no activity, and it does not matter
which process gets to the synchronization first"
And still, the master process doesn't really start until all the images got
OCRized. You could split into two processes but instead of synchronizing
them, you make the process_pictures process start the process_receipts
process once when all the pictures are ready. Maybe it's too simple...
> The code is available on GitHub :
>
> https://github.com/adrienkohlbecker/ruote-synchronize
That is nice, that deserves a README ;-)
Best regards,
--
John Mettraux - http://lambda.io/jmettraux
--
--
you received this message because you are subscribed to the "ruote users" group.
to post : send email to [email protected]
to unsubscribe : send email to [email protected]
more options : http://groups.google.com/group/openwferu-users?hl=en
---
You received this message because you are subscribed to the Google Groups
"ruote" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
