Re: Large-scale production installations, and another question

2008-07-01 Thread Kiren Sekar
We use Mogile at Sharpcast for our SugarSync service. We are currently storing about 220 million files, 120TB logical. On Jun 29, 2008, at 7:32 AM, Justin Huff wrote: We use Mogile at picnik.com About 14TB and 52 million files. --Justin dormando wrote: mike wrote: On 6/29/08, mike

Re: Large-scale production installations, and another question

2008-07-01 Thread Gabriel Ramuglia
Jared, I'm curious, how do you deal with files up to 1gb? Does mogilefs handle files this big better than it used to, or have you implemented separating large files into several chunks? I had read somewhere on the lists a while back that anything over 100mb often causes the storage daemon to

Re: Large-scale production installations, and another question

2008-06-29 Thread dormando
2) When the client issues the call Where is file $x? and it gets back a list of locations, does the tracker reply back in random order? Or is it always the same? Is it expected that the client take the list and randomize them too? Just wondering - if the server always replies with $x is on

Re: Large-scale production installations, and another question

2008-06-29 Thread mike
On 6/29/08, dormando [EMAIL PROTECTED] wrote: First off - thanks for the quick reply. On a Saturday night, too! It's weighted by a few things: - Nodes that are down are removed. - If IO monitoring is working, the list of paths are sorted, in order of least busy nodes. perfect. this is as

Re: Large-scale production installations, and another question

2008-06-29 Thread dormando
so if no_verify=0, then it issues a HEAD to see if the file exists, if so, it replies back yes, i have the file to the tracker, which combines those replies from the various storage nodes marked as the ones that have it, and then sends that to the client? couldn't there be a process which

Re: Large-scale production installations, and another question

2008-06-29 Thread mike
On 6/29/08, dormando [EMAIL PROTECTED] wrote: so if no_verify=0, then it issues a HEAD to see if the file exists, if so, it replies back yes, i have the file to the tracker, which combines those replies from the various storage nodes marked as the ones that have it, and then sends that to

Re: Large-scale production installations, and another question

2008-06-29 Thread dormando
mike wrote: On 6/29/08, dormando [EMAIL PROTECTED] wrote: so if no_verify=0, then it issues a HEAD to see if the file exists, if so, it replies back yes, i have the file to the tracker, which combines those replies from the various storage nodes marked as the ones that have it, and then sends

Re: Large-scale production installations, and another question

2008-06-29 Thread Justin Huff
We use Mogile at picnik.com About 14TB and 52 million files. --Justin dormando wrote: mike wrote: On 6/29/08, mike [EMAIL PROTECTED] wrote: Two parter: 1) Other than Danga properties like LJ, are there other sites using MogileFS in production? It'd be cool to see a list/stats (how many