without sounding like a broken record, you are conflating 0mq routing with job
scheduling.
0mq does so much for you it is tempting to think it can do it all, but in this
case, it can't.
and a fair bit depends on the relative cost of doing a task.
the simplest way to get pretty much what you want (and close to your
architecture) is
1) task executers do a REQ-REP to ask for the next task from teh scheduler
(this is the best self-adaptive way to load balance); the executers can
be anonymous.
2) to fair share on the input side, just put a submitter id in the task and
have the scheduler round robin
on the id's.
andrew
On Feb 2, 2012, at 8:52 AM, Jesper Petersén wrote:
> Hi,
> I'm trying to create a parallel task system. Tasks are initiated by and owned
> by a user. The divide and conquer example from the guide doesn't quite do it
> because if we have one user pushing loads of tasks, anyone pushing a task
> after him would have to wait until all his jobs are finished.
>
> The goal is that tasks should be pulled fairly per user/client. I've created
> a diagram of what I think is a solution to this, but it doesn't feel optimal.
> I have roughly this diagram in code as well and it's working OK. Check the
> link below!
>
> https://docs.google.com/drawings/pub?id=1rQ0_CAyNs0nNYa6w5Hub5b73Afkv-mCDhl7JA2mdvYA&w=960&h=720
>
>
> The idea is that users push tasks to my client starter. The client starter
> then forwards these tasks to client threads (these are created if initially
> not existing and times out /stops if there's nothing to do for a while..) who
> then sends them to workers, one at a time by REQ. Actually, the only reason I
> do REQ here is because I don't want a client thread having more than one task
> being worked one. I'm not sure this is the best way to achieve fair client
> tasks..
>
> Any suggestions/guidelines highly appreciated!
>
> If you would like to view my diagram in Google docs, use this link:
> https://docs.google.com/drawings/d/1rQ0_CAyNs0nNYa6w5Hub5b73Afkv-mCDhl7JA2mdvYA/edit
> J
> _______________________________________________
> zeromq-dev mailing list
> [email protected]
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
------------------
Andrew Hume (best -> Telework) +1 623-551-2845
[email protected] (Work) +1 973-236-2014
AT&T Labs - Research; member of USENIX and LOPSA
_______________________________________________
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
