Re: Hadoop HDFS + ZFS (RE: Some Doubts of hadoop functionality)

2008-01-07 Thread Allen Wittenauer
On 1/5/08 12:34 AM, "Greg Connor" <[EMAIL PROTECTED]> wrote: > Since you mentioned ZFS, I went and looked at it today, and it definitely is > all kinds of cool. ZFS is an excellent example of a robust, feature-rich > filesystem, at least if it does what it's documentation claims it does. It d

Hadoop HDFS + ZFS (RE: Some Doubts of hadoop functionality)

2008-01-05 Thread Greg Connor
Hi folks, I'm still a noob in the hadoop world, so I apologize if this is already asked and answered. This thread seems pretty recent, so hopefully it's OK if I jump in. I trust folks to politely correct me if I'm way off base. (This is not really a question per se, but more a request for com

Re: Some Doubts of hadoop functionality

2007-12-20 Thread Raghu Angadi
Joydeep Sen Sarma wrote: agreed - i think for anyone who is thinking of using hadoop as a place from where data is served - has to be distrubed by lack of data protection. replication in hadoop provides protection against hardware failures. not software failures. backups (and depending on how t

Re: Some Doubts of hadoop functionality

2007-12-20 Thread Ted Dunning
gt; tape - which is really what our filers are becoming) is worth discussing. >> for large data sets - the restore time would be so bad as to render these >> useless as a recovery path). >> >> >> >> From: Pat Ferrel [mailto:

Re: Some Doubts of hadoop functionality

2007-12-20 Thread Pat Ferrel
ion of backing up to tape (or even virtual > tape - which is really what our filers are becoming) is worth discussing. > for large data sets - the restore time would be so bad as to render these > useless as a recovery path). > > ____________ > > From: Pat Fer

Re: Some Doubts of hadoop functionality

2007-12-20 Thread Ted Dunning
Well, we are kind of a poster child for this kind of reliability calculus. We opted for Mogile for real-time serving because we could see how to split the master into shards and how to do HA on it. For batch oriented processes where a good processing model is important, we use hadoop. I would ha

Re: Some Doubts of hadoop functionality

2007-12-20 Thread Billy
me would be so bad as to render these useless as a recovery path). From: Pat Ferrel [mailto:[EMAIL PROTECTED] Sent: Thu 12/20/2007 7:25 AM To: hadoop-user@lucene.apache.org Subject: Re: Some Doubts of hadoop functionality > 2. If hadoop is configured in mul

RE: Some Doubts of hadoop functionality

2007-12-20 Thread Joydeep Sen Sarma
overy path). From: Pat Ferrel [mailto:[EMAIL PROTECTED] Sent: Thu 12/20/2007 7:25 AM To: hadoop-user@lucene.apache.org Subject: Re: Some Doubts of hadoop functionality > 2. If hadoop is configured in multinode cluster(with One machine as namenode >> > and jobtracker and

Re: Some Doubts of hadoop functionality

2007-12-20 Thread Pat Ferrel
> 2. If hadoop is configured in multinode cluster(with One machine as namenode >> > and jobtracker and other machine as slave. Namenode acts as a slave node >> > also) . How to handle the namenode failovers?. > > There are backup mechanisms that you can use to allow you rebuild the name > node. T

Re: Some Doubts of hadoop functionality

2007-12-20 Thread Ted Dunning
On 12/19/07 11:17 PM, "M.Shiva" <[EMAIL PROTECTED]> wrote: > 1.Did Separate machines/nodes needed for Namenode ,Jobtracker, Slavenodes No. I run my namenode and job-tracker on one of my storage/worker nodes. You can run everything on a single node and still get some interesting results becau