Re: ChunkFS - measuring cross-chunk references

2007-05-19 Thread Karuna sagar K
On 4/25/07, Suparna Bhattacharya <[EMAIL PROTECTED]> wrote: On Wed, Apr 25, 2007 at 05:50:55AM +0530, Karuna sagar K wrote: One more set of numbers to calculate would be an estimate of cross-references across chunks of block groups -- 1 (=128MB), 2 (=256MB), 4 (=512MB), 8(=1GB) as suggested by

Re: ChunkFS - measuring cross-chunk references

2007-05-19 Thread Karuna sagar K
On 4/25/07, Suparna Bhattacharya [EMAIL PROTECTED] wrote: On Wed, Apr 25, 2007 at 05:50:55AM +0530, Karuna sagar K wrote: One more set of numbers to calculate would be an estimate of cross-references across chunks of block groups -- 1 (=128MB), 2 (=256MB), 4 (=512MB), 8(=1GB) as suggested by

Re: ChunkFS - measuring cross-chunk references

2007-05-06 Thread Valerie Henson
On Mon, Apr 23, 2007 at 02:05:47AM +0530, Karuna sagar K wrote: > Hi, > > The attached code contains program to estimate the cross-chunk > references for ChunkFS file system (idea from Valh). Below are the > results: Nice work! Thank you very much for doing this! -VAL - To unsubscribe from

Re: ChunkFS - measuring cross-chunk references

2007-05-06 Thread Valerie Henson
On Mon, Apr 23, 2007 at 02:53:33PM -0600, Andreas Dilger wrote: > > Also, is it considered a cross-chunk reference if a directory entry is > referencing an inode in another group? Should there be a continuation > inode in the local group, or is the directory entry itself enough? (Sorry for the

Re: ChunkFS - measuring cross-chunk references

2007-05-06 Thread Valerie Henson
On Mon, Apr 23, 2007 at 02:53:33PM -0600, Andreas Dilger wrote: Also, is it considered a cross-chunk reference if a directory entry is referencing an inode in another group? Should there be a continuation inode in the local group, or is the directory entry itself enough? (Sorry for the

Re: ChunkFS - measuring cross-chunk references

2007-05-06 Thread Valerie Henson
On Mon, Apr 23, 2007 at 02:05:47AM +0530, Karuna sagar K wrote: Hi, The attached code contains program to estimate the cross-chunk references for ChunkFS file system (idea from Valh). Below are the results: Nice work! Thank you very much for doing this! -VAL - To unsubscribe from this

Re: ChunkFS - measuring cross-chunk references

2007-04-25 Thread Suparna Bhattacharya
On Wed, Apr 25, 2007 at 05:50:55AM +0530, Karuna sagar K wrote: > On 4/24/07, Theodore Tso <[EMAIL PROTECTED]> wrote: > >On Mon, Apr 23, 2007 at 02:53:33PM -0600, Andreas Dilger wrote: > . > >It would also be good to distinguish between directories referencing > >files in another chunk,

Re: ChunkFS - measuring cross-chunk references

2007-04-25 Thread Suparna Bhattacharya
On Wed, Apr 25, 2007 at 05:50:55AM +0530, Karuna sagar K wrote: On 4/24/07, Theodore Tso [EMAIL PROTECTED] wrote: On Mon, Apr 23, 2007 at 02:53:33PM -0600, Andreas Dilger wrote: . It would also be good to distinguish between directories referencing files in another chunk, and

ChunkFS - measuring cross-chunk references

2007-04-24 Thread Karuna sagar K
On 4/24/07, Theodore Tso <[EMAIL PROTECTED]> wrote: On Mon, Apr 23, 2007 at 02:53:33PM -0600, Andreas Dilger wrote: . It would also be good to distinguish between directories referencing files in another chunk, and directories referencing subdirectories in another chunk (which would be

ChunkFS - measuring cross-chunk references

2007-04-24 Thread Karuna sagar K
On 4/24/07, Theodore Tso [EMAIL PROTECTED] wrote: On Mon, Apr 23, 2007 at 02:53:33PM -0600, Andreas Dilger wrote: . It would also be good to distinguish between directories referencing files in another chunk, and directories referencing subdirectories in another chunk (which would be

Re: ChunkFS - measuring cross-chunk references

2007-04-23 Thread Amit Gud
On Mon, 23 Apr 2007, Amit Gud wrote: On Mon, 23 Apr 2007, Arjan van de Ven wrote: > The other thing which we should consider is that chunkfs really > requires a 64-bit inode number space, which means either we only allow does it? I'd think it needs a "chunk space" number and a 32 bit

Re: ChunkFS - measuring cross-chunk references

2007-04-23 Thread Andreas Dilger
On Apr 23, 2007 15:04 +0530, Kalpak Shah wrote: > On Mon, 2007-04-23 at 12:49 +0530, Karuna sagar K wrote: > > The tool estimates the cross-chunk references from an extt2/3 file > > system. It considers a block group as one chunk and calcuates how many > > block groups does a file span across.

Re: ChunkFS - measuring cross-chunk references

2007-04-23 Thread Kalpak Shah
On Mon, 2007-04-23 at 12:49 +0530, Karuna sagar K wrote: > Hi, > > The tool estimates the cross-chunk references from an extt2/3 file > system. It considers a block group as one chunk and calcuates how many > block groups does a file span across. So, the block group size gives > the estimate of

Re: ChunkFS - measuring cross-chunk references

2007-04-23 Thread Karuna sagar K
Hi, The tool estimates the cross-chunk references from an extt2/3 file system. It considers a block group as one chunk and calcuates how many block groups does a file span across. So, the block group size gives the estimate of chunk size. The file systems were aged for about 3-4 months on a

Re: ChunkFS - measuring cross-chunk references

2007-04-23 Thread Karuna sagar K
Hi, The tool estimates the cross-chunk references from an extt2/3 file system. It considers a block group as one chunk and calcuates how many block groups does a file span across. So, the block group size gives the estimate of chunk size. The file systems were aged for about 3-4 months on a

Re: ChunkFS - measuring cross-chunk references

2007-04-23 Thread Kalpak Shah
On Mon, 2007-04-23 at 12:49 +0530, Karuna sagar K wrote: Hi, The tool estimates the cross-chunk references from an extt2/3 file system. It considers a block group as one chunk and calcuates how many block groups does a file span across. So, the block group size gives the estimate of chunk

Re: ChunkFS - measuring cross-chunk references

2007-04-23 Thread Andreas Dilger
On Apr 23, 2007 15:04 +0530, Kalpak Shah wrote: On Mon, 2007-04-23 at 12:49 +0530, Karuna sagar K wrote: The tool estimates the cross-chunk references from an extt2/3 file system. It considers a block group as one chunk and calcuates how many block groups does a file span across. So, the

Re: ChunkFS - measuring cross-chunk references

2007-04-23 Thread Amit Gud
On Mon, 23 Apr 2007, Amit Gud wrote: On Mon, 23 Apr 2007, Arjan van de Ven wrote: The other thing which we should consider is that chunkfs really requires a 64-bit inode number space, which means either we only allow does it? I'd think it needs a chunk space number and a 32 bit local

Re: ChunkFS - measuring cross-chunk references

2007-04-22 Thread Amit Gud
Karuna sagar K wrote: Hi, The attached code contains program to estimate the cross-chunk references for ChunkFS file system (idea from Valh). Below are the results: Nice to see some numbers! But would be really nice to know: - what the chunk size is - how the files were created or, more

ChunkFS - measuring cross-chunk references

2007-04-22 Thread Karuna sagar K
Hi, The attached code contains program to estimate the cross-chunk references for ChunkFS file system (idea from Valh). Below are the results: test on ext3, / partition-1 on 27 March 2007 Number of files = 217806 Number of directories = 24295 Total size = 8193116 KB Total data stored = 7557892

ChunkFS - measuring cross-chunk references

2007-04-22 Thread Karuna sagar K
Hi, The attached code contains program to estimate the cross-chunk references for ChunkFS file system (idea from Valh). Below are the results: test on ext3, / partition-1 on 27 March 2007 Number of files = 217806 Number of directories = 24295 Total size = 8193116 KB Total data stored = 7557892

Re: ChunkFS - measuring cross-chunk references

2007-04-22 Thread Amit Gud
Karuna sagar K wrote: Hi, The attached code contains program to estimate the cross-chunk references for ChunkFS file system (idea from Valh). Below are the results: Nice to see some numbers! But would be really nice to know: - what the chunk size is - how the files were created or, more