problem is not a common one ?
> From: ha...@cloudera.com
> Date: Mon, 13 Aug 2012 17:33:02 +0530
> Subject: Re: Locks in M/R framework
> To: mapreduce-user@hadoop.apache.org
>
> David,
>
> While ZK can solve this, locking may only make you slower. Lets try to
> ke
David,
While ZK can solve this, locking may only make you slower. Lets try to
keep it simple?
Have you considered keeping two directories? One where the older data
is moved to (by the first job, instead of replacing files), for
consumption by the second job, which triggers by watching this
direct
How about introducing a distributed coordination and locking mechanism?
ZooKeeper would be a good candidate for that kind of thing.
On Mon, Aug 13, 2012 at 12:52 PM, David Ginzburg wrote:
> Hi,
>
> I have an HDFS folder and M/R job that periodically updates it by
> replacing the data with newly
