Makes it two, countless enough. It's not that I disagree that the gasket between the keyboard and the chair (aka user) is a typical source of most of the troubles ;)
Cos On Sat, May 26, 2012 at 08:44AM, M. C. Srivas wrote: > On Fri, May 25, 2012 at 8:03 AM, Konstantin Boudnik <c...@apache.org> wrote: > > > BTW, Srivas, > > > > I could find a single countless example of horror story of 'hadoop fs > > -rmr' in > > a form of hypothetical question (and not on this list ;) > > http://is.gd/55KD1E > > > > > Hi Cos, accidentally deleting files is one of the most common user errors. > Here's a real one from just last month > > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201204.mbox/%3CCAPwpkBvEx4OTUbf6mf8t43oOjZM%2BExUths7XNn3UidqsN3Y8hA%40mail.gmail.com%3E > > > As Patrick says in the follow-up, the only way to recover in this situation > is to shutdown the cluster: > > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201204.mbox/%3CCANS822ga1ivAPi2C9PJsyz6nZgft4msKkH%3Dyj06-i_V%2Bu1B1AA%40mail.gmail.com%3E > > > > In fact, the above procedure is well-known and well-documented. Here's > even an excerpt from Jason's book ProHadoop where he says "it is not > uncommon for a user to accidentally delete large portions of the HDFS file > system due to a program error or a command-line error ... best bet is to > terminate the NN and 2-N immediately, and then shutdown the DNs as fast as > possible" > > http://books.google.com/books?id=8DV-EzeKigQC&pg=PA122&lpg=PA122&dq=how+to+recover+deleted+files+%2B+hadoop&source=bl&ots=prgSMk1SHL&sig=LPJ0j5MFwJ3zUAcOrvR6FbiWQuQ&hl=en&sa=X&ei=UfXAT76HJuabiALbkdn8Bw&ved=0CLQBEOgBMAQ#v=onepage&q=how%20to%20recover%20deleted%20files%20%2B%20hadoop&f=false > > > > Just for the sake of full disclosure, of course. > > > > > Enjoy, > > Cos > > > > On Tue, May 22, 2012 at 09:45PM, M. C. Srivas wrote: > > > On Tue, May 22, 2012 at 12:08 AM, Martinus Martinus > > > <martinus...@gmail.com>wrote: > > > > > > > Hi Todd, > > > > > > > > Thanks for your answer. Is that will have the same capability as the > > > > commercial M5 of MapR : http://www.mapr.com/products/why-mapr ? > > > > > > > > Thanks. > > > > > > > > > Hi Martinus, some major differences in HA between MapR's M5 and Apache > > > Hadoop > > > > > > 1. with M5, any node become master at any time. It is a fully > > active-active > > > system. You can get create a fully bomb-proof cluster, such that in a > > > 20-node cluster, you can configure to survive even if 19 of the 20 nodes > > > are lost. With Apache, it is a 1-1 active-passive system. > > > > > > 2. M5 does not require a NFS filer in the backend. Apache Hadoop > > requires a > > > Netapp or similar NFS filer to assist in saving the NN data, even in its > > HA > > > configuration. Note that for true HA, the Netapp or similar also will > > need > > > to be HA. > > > > > > 3. M5 has full HA for the Job-Tracker as well. > > > > > > Of course, HA is only a small part of the total business continuity > > story. > > > Full recovery in the face of any kind of failures is critical: > > > > > > With M5: > > > > > > - If there is a complete cluster crash and reboot (eg, a full > > > power-failure of the entire cluster), M5 will recover in 5-10 minutes, > > and > > > submitted jobs will resume from where they were. > > > > > > - with snapshots, if you upgrade your software and it corrupts data, M5 > > > provides snapshots to help you recover. The number of times I've seen > > > someone running "hadoop fs -rmr /" accidentally and asking for help on > > > this mailing list is beyond counting. With M5, it is completely > > recoverable > > > > > > - full disaster-recovery across clusters by mirroring. > > > > > > Hope that clarifies some of the differences. > > > > > > > > > > > > > > > > > > On Tue, May 22, 2012 at 2:26 PM, Todd Lipcon <t...@cloudera.com> > > wrote: > > > > > > > >> Hi Martinus, > > > >> > > > >> Hadoop HA is available in Hadoop 2.0.0. This release is currently > > > >> being voted on in the community. > > > >> > > > >> You can read more here: > > > >> > > > >> > > http://www.cloudera.com/blog/2012/03/high-availability-for-the-hadoop-distributed-file-system-hdfs/ > > > >> > > > >> -Todd > > > >> > > > >> On Mon, May 21, 2012 at 11:24 PM, Martinus Martinus > > > >> <martinus...@gmail.com> wrote: > > > >> > Hi, > > > >> > > > > >> > Is there any hadoop HA distribution out there? > > > >> > > > > >> > Thanks. > > > >> > > > >> > > > >> > > > >> -- > > > >> Todd Lipcon > > > >> Software Engineer, Cloudera > > > >> > > > > > > > > > >
signature.asc
Description: Digital signature
