On Friday, February 28, 2014 10:04:59 PM UTC+1, Andrey K wrote:
>
> Thank Niphlod for your answer. it is already great if it is possible even
> theoretically!
>
it does even in the real world :-P
> Regarding implementation. Correct me please if I am wrong in following
> understanding - am I right that I need to:
> 1)copy the whole web2py and myapp with necessary libs to another server .
> 2)set up db connection in the 2nd box's web2py myapp to remote DB (first
> server) like:
> db= DAL('postgres://postgres:[email protected]/test', lazy_tables=False,
> migrate=True, fake_migrate=False)
> 3) run web2py scheduler on the second box as:
> web2py.py -K appname
>
exactly
>
> Is that all???It is sounds like magic!!!!
> How the second server job scheduler get understanding of calling task?
>
exactly as the one in the "original server". All needed informations are
right on the scheduler_* tables, nothing else is required.
> If it works I wonder how to control what server (not worker) get next task
> - like what should I put in the model to force job be assigned to the the
> box I want?
>
The workers will coordinate themselves and pick up a new task whenever it's
ready to be processed. The whole point of the scheduler is having a task
processed ASAP, no matter who processes it: the first available worker will
do the job.
Usually given that a task is queued and is ready to be processed, you don't
even want to know who processes it, as long as it is processed: that's how
scheduler works with default settings.
That being said, if you want to send a specific task to a worker that
exists only in a specific server, you can do so with "group_name"(s).
The concept is having different group of tasks to be processed from
schedulers being able to do so (i.e. one has all the libs, other one has
only a few, or first one is more "powerful", other one is less "powerful",
etc)
You can then specify that a worker has to process only one group of tasks
and the other one only another group of tasks.
As specified in the book, the syntax to launch schedulers with one or more
group_name is
python web2py.py -K myapp:group1:group2,myotherapp:group1
in that case, you'd launch
python web2py.py -K myapp:original
on the original server and
python web2py.py -K myapp:additional
on the additional one.
You can then queue tasks to be processed from the "original" box as
mysched.queue_task(......, group_name='original')
and tasks to be processed from the "additional" one as
mysched.queue_task(......, group_name='additional')
PS: all combinations of scheduler and group_names are allowed....let's say
the "additional" box will need to "help" the "original" one too , then
you'll have
- 1 worker from the original box processing "original"
- 1 worker from the additional box processing "additional" AND "original"
then if you launch
python web2py.py -K myapp:original
on the first box and
python web2py.py -K myapp:original:additional
on the additional
the workers will coordinate to let BOTH workers process tasks with
"original" group_name and only the additional box to process tasks with the
"additional" group.
Another example (more complicated): you have a set of tasks that needs
"precedence" (i.e. near real-time execution) and a set of tasks that can be
executed with more "relax", and 3 boxes....you can do some serious math :-P
box1) python web2py.py -K myapp:fast,myapp:fast,myapp:fast
box2) python web2py.py -K myapp:relax
box3) python web2py.py -K myapp:fast,myapp:fast,myapp:fast:relax
now you have
fast) 3 workers from box1, 2.5 workers on box1 --> 5.5 workers
relax) 1 worker from box2, 0.5 workers on box3 --> 1.5 workers
Now, for speech's sake, if all of your tasks take the same time to be
executed, and 100 "fast" and 100 "relax" tasks have been queued, in the
long run you'd observe that:
- ~ 54 "fast" tasks executed by "box1" (100 / 5.5 * 3)
- ~ 46 "fast" tasks executed by "box3" (100 / 5.5 * 2.5)
- ~ 66 "relax" tasks executed by "box2" (100 / 1.5 * 1)
- ~ 34 "relax" tasks executed by "box3" (100 / 1.5 * 0.5)
Also, you'd observe that to have the last "relax" task to be executed at
the same time of the last "fast" task, you **can** queue up to 366 "fast"
tasks to match 100 "relax" tasks (because 5.5 workers are "assigned" to
"fast" and "only" 1.5 are "assigned" to "relax" tasks)...
Hope the whole concept is clear: feel free to ask for more details if
needed.
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
