Thanks for the feedback. I'm probably going to modify quartz to work
with Zookeeper to start and launch jobs. Architecturally, I don't think
persisting Jobs or trigger history in ZK is a very good idea, it's
turning it into a persistent data store, which is not designed for. I
was thinking I cou
Todd Nine:
> [...]
> UC1: Synchronized Jobs
> 1. A job is fired across all nodes
> 2. The nodes wait until the barrier is entered by all participants
> 3. The nodes process the data and leave
> 4. On all nodes leaving the barrier, the Leader node marks the job as
> complete.
>
> UC2: Multiple Jobs
These are pretty easy to solve with ZK. Ephemerality, exclusive create,
atomic update and file versions allow you to implement most of the semantics
you need.
I don't know of any recipes available for this, but they would be worthy
additions to ZK.
On Mon, Aug 23, 2010 at 11:33 PM, Todd Nine wr
Solving UC1 and UC2 via zookeeper or some other framework if one is
recommended. We don't run Hadoop, just ZK and Cassandra as we don't have a
need for map/reduce. I'm searching for any existing framework that can
perform standard time based scheduling in a distributed environment. As I
said ear
Hi Todd,
Just to be clear, are you looking at solving UC1 and UC2 via zookeeper? Or is
this a broader question for scheduling on cassandra nodes? For the latter this
probably isnt the right mailing list.
Thanks
mahadev
On 8/23/10 4:02 PM, "Todd Nine" wrote:
Hi all,
We're using Zookeeper
Hi all,
We're using Zookeeper for Leader Election and system monitoring. We're
also using it for synchronizing our cluster wide jobs with barriers. We're
running into an issue where we now have a single job, but each node can fire
the job independently of others with different criteria in the
