That's definitely clearer (and it makes sense). Thanks a lot.
Bertrand On Fri, Sep 28, 2012 at 11:56 AM, Harsh J <[email protected]> wrote: > I don't know how much of this is 1.x compatible: > > - When a transaction is logged and sync'd, and a single edits storage > location fails during write, then only that storage location is > ejected out of the regular write list and skipped over (with states > being updated for that location in the UI, etc., immediately). The > rest remain active and NN lives on. The transaction is marked > successful and returns back to client with a similar code. Further > transactions too skip the ejected storage. > - If no edit streams remain after removal (i.e. last remaining disk is > removed due to a write error), then the transaction is failed and the > NN dies down, to prevent data loss due to lack of persistence. > - Hence, a transaction at the NN can be marked complete and return a > success, iff, at least one location successfully wrote the edit log > for it. > - If dfs.name.dir.restore is enabled, then the NN checks if its > ejected storages are healthy again and re adds them. The check, IIRC, > is done during checkpoint or checkpoint-like operations currently. > > I guess all of this is in 1.x too, but I haven't validated it > recently. It is certainly in the versions I've been using and > supporting at my org for quite a while now. The restore especially > comes in handy for customers/users with fairly common NFS mount > related issues, not requiring them to restart NN each time the NN > ejects the NFS out for a bad write. Although, for that to happen, a > soft mount is necessary and recommended, rather than a hard mount, > which would hang the NameNode and invalidate its whole "still > available despite a few volumes failing" feature. > > Does this help Bertrand? Happy to answer any further questions. > > On Fri, Sep 28, 2012 at 2:51 PM, Bertrand Dechoux <[email protected]> > wrote: > > Hi, > > > > I was wondering about the safety of multiples dfs.name.dir directories. > > On one hand, we want all copies to be synchronised but on the other hand > if > > a hard drive fail we would like the namenode still to be operational. > > How does that work? I know there is the source but I was hoping for a > higher > > level description. > > > > Regards > > > > Bertrand > > > > PS : I am interested about the behaviour of the last stable version ie > > 1.0.3. Not in the old issues that were solved. > > > > -- > Harsh J > -- Bertrand Dechoux
