Re: Replication and failure

2011-07-28 Thread Harsh J
Mohit,

I believe Tom's book (Hadoop: The Definitive Guide) covers this
precisely well. Perhaps others too.

Replication is a best-effort sort of thing. If 2 nodes are all that is
available, then two replicas are written and one is left to the
replica monitor service to replicate later as possible (leading to an
underreplicated write for the moment). The scenario (with default
configs) would only fail if you have 0 DataNodes 'available' to write
to.

Or are you asking about what happens when a DN fails during a write operation?

On Thu, Jul 28, 2011 at 5:08 AM, Mohit Anchlia mohitanch...@gmail.com wrote:
 Just trying to understand what happens if there are 3 nodes with
 replication set to 3 and one node fails. Does it fail the writes too?

 If there is a link that I can look at will be great. I tried searching
 but didn't see any definitive answer.

 Thanks,
 Mohit




-- 
Harsh J


Re: Replication and failure

2011-07-28 Thread Mohit Anchlia
On Thu, Jul 28, 2011 at 12:17 AM, Harsh J ha...@cloudera.com wrote:
 Mohit,

 I believe Tom's book (Hadoop: The Definitive Guide) covers this
 precisely well. Perhaps others too.

 Replication is a best-effort sort of thing. If 2 nodes are all that is
 available, then two replicas are written and one is left to the
 replica monitor service to replicate later as possible (leading to an
 underreplicated write for the moment). The scenario (with default
 configs) would only fail if you have 0 DataNodes 'available' to write
 to.

Thanks Harsh. I think you answered my question. I thought that
replication of 3 is a must. And for that you really need atleast 4
nodes so that if one of the nodes die it can still write to 3 nodes. I
am assuming writes to replica nodes are always synchronous and not
eventually consistent.

 Or are you asking about what happens when a DN fails during a write operation?

I am assuming there will be some errors in this case.


 On Thu, Jul 28, 2011 at 5:08 AM, Mohit Anchlia mohitanch...@gmail.com wrote:
 Just trying to understand what happens if there are 3 nodes with
 replication set to 3 and one node fails. Does it fail the writes too?

 If there is a link that I can look at will be great. I tried searching
 but didn't see any definitive answer.

 Thanks,
 Mohit




 --
 Harsh J



Replication and failure

2011-07-27 Thread Mohit Anchlia
Just trying to understand what happens if there are 3 nodes with
replication set to 3 and one node fails. Does it fail the writes too?

If there is a link that I can look at will be great. I tried searching
but didn't see any definitive answer.

Thanks,
Mohit