Re: OHSM benchmarking [Was Re: Copying Data Blocks]

2009-01-24 Thread Greg Freemyer
On Thu, Jan 22, 2009 at 11:18 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: On Thu, Jan 22, 2009 at 8:50 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Thu, Jan 22, 2009 at 3:49 AM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Help us with the benchmarking ? My first question

Re: OHSM benchmarking [Was Re: Copying Data Blocks]

2009-01-24 Thread Martin Møller Skarbiniks Pedersen
2009/1/25 Greg Freemyer greg.freem...@gmail.com: Then first do a dd test to see how fast it works from pure userspace. ie. dd if=/dev/zero of=testfile bs=1m count=2000// build a 2 GB test file time (dd if=testfile-on-tier1 of=testfile-on-tier2 bs=1m; sync) repeat at least a couple

Re: Copying Data Blocks

2009-01-22 Thread Sandeep K Sinha
Hi Greg, On Sat, Jan 17, 2009 at 4:34 AM, Greg Freemyer greg.freem...@gmail.com wrote: snip But, first I will see the performance metrics of dividing the copy operation in some chunks. Agreed. Somewhere I think I read you were doing 1 GB in less than a second or something like that. Am

OHSM benchmarking [Was Re: Copying Data Blocks]

2009-01-22 Thread Greg Freemyer
On Thu, Jan 22, 2009 at 3:49 AM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Help us with the benchmarking ? My first question would be: Why are you benchmarking at all? I can see a basic benchmark just to prove you are actually moving data in a reasonably efficient way. Disk drives are

Re: Copying Data Blocks

2009-01-19 Thread Peter Teoh
Firstly, I would like to congratulate everyone for pulling the conversation this long - this is the 100th element in this threadand my gmail is getting slower... Anywayway back 10 years agoit is common practise both in Unix/AIX/Oracle database worldthat harddisk

Re: Copying Data Blocks

2009-01-16 Thread Greg Freemyer
On Thu, Jan 15, 2009 at 6:24 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Thu, Jan 15, 2009 at 12:55 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Manish, On Thu, Jan 15, 2009 at 10:31 PM, Manish Katiyar mkati...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:27 PM, Greg Freemyer

Re: Copying Data Blocks

2009-01-16 Thread Sandeep K Sinha
Hi Greg, On Fri, Jan 16, 2009 at 5:50 AM, Greg Freemyer greg.freem...@gmail.com wrote: On Thu, Jan 15, 2009 at 12:47 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hey, On Thu, Jan 15, 2009 at 10:27 PM, Greg Freemyer snip I think I've said it before, but I would think the best real

Re: Copying Data Blocks

2009-01-16 Thread Sandeep K Sinha
Hi Again, On Fri, Jan 16, 2009 at 6:19 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Thu, Jan 15, 2009 at 6:24 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Thu, Jan 15, 2009 at 12:55 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Manish, On Thu, Jan 15, 2009 at 10:31

Re: Copying Data Blocks

2009-01-16 Thread Greg Freemyer
On Fri, Jan 16, 2009 at 8:36 AM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Again, big snip Ya surely. Thanks a million Greg. The source is ready and unit testing for most of the modules of the deisgn is done. We are integrating things, will be able to provide you a basic POC code by

Re: Copying Data Blocks

2009-01-16 Thread Greg Freemyer
On Fri, Jan 16, 2009 at 8:26 AM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Greg, On Fri, Jan 16, 2009 at 5:50 AM, Greg Freemyer greg.freem...@gmail.com wrote: On Thu, Jan 15, 2009 at 12:47 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hey, On Thu, Jan 15, 2009 at 10:27 PM,

Re: Copying Data Blocks

2009-01-16 Thread Rohit Sharma
Hi Greg, Thanks for such great insights. On Fri, Jan 16, 2009 at 11:41 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Fri, Jan 16, 2009 at 8:26 AM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Greg, On Fri, Jan 16, 2009 at 5:50 AM, Greg Freemyer greg.freem...@gmail.com wrote:

Re: Copying Data Blocks

2009-01-16 Thread Greg Freemyer
snip But, first I will see the performance metrics of dividing the copy operation in some chunks. Agreed. Somewhere I think I read you were doing 1 GB in less than a second or something like that. Am I remembering right? Yes that is true. The approx figure for 512M file was 230

Re: Why Ext3? [WAS: Re: Copying Data Blocks]

2009-01-15 Thread Sandeep K Sinha
Hi Greg, On Wed, Jan 14, 2009 at 8:53 PM, Greg Freemyer greg.freem...@gmail.com wrote: HSM team, Reading this thread got me taking a basic look at ext4 for the first time. I get the impression from what I read that ext3 is basically feature frozen for new features and that major

Re: Copying Data Blocks

2009-01-15 Thread Sandeep K Sinha
Hey Peter, On Thu, Jan 15, 2009 at 6:19 AM, Peter Teoh htmldevelo...@gmail.com wrote: Frankly, I am quite losts in the sea of argument :-).but let try sharing my points: I can see the inode locking for the entire duration of a file reorg would be unacceptable at some later point in the

Re: Copying Data Blocks

2009-01-15 Thread Greg Freemyer
On Wed, Jan 14, 2009 at 7:59 PM, Peter Teoh htmldevelo...@gmail.com wrote: Frankly, I am quite losts in the sea of argument :-).but let try sharing my points: I can see the inode locking for the entire duration of a file reorg would be unacceptable at some later point in the development

Re: Copying Data Blocks

2009-01-15 Thread Peter Teoh
On Thu, Jan 15, 2009 at 10:49 PM, Greg Freemyer greg.freem...@gmail.com wrote: I dont' think the above paragraph is an issue with re-org as currently designed. Neither for the ext4_defrag patchset that is under consideration for acceptance, nor the work the OHSM team is doing. well...it

Re: Copying Data Blocks

2009-01-15 Thread Greg Freemyer
On Thu, Jan 15, 2009 at 10:41 AM, Peter Teoh htmldevelo...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:49 PM, Greg Freemyer greg.freem...@gmail.com wrote: I dont' think the above paragraph is an issue with re-org as currently designed. Neither for the ext4_defrag patchset that is under

Re: Copying Data Blocks

2009-01-15 Thread Manish Katiyar
On Thu, Jan 15, 2009 at 10:27 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:41 AM, Peter Teoh htmldevelo...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:49 PM, Greg Freemyer greg.freem...@gmail.com wrote: I dont' think the above paragraph is an issue with re-org

Re: Copying Data Blocks

2009-01-15 Thread Sandeep K Sinha
Hi Manish, On Thu, Jan 15, 2009 at 10:31 PM, Manish Katiyar mkati...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:27 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:41 AM, Peter Teoh htmldevelo...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:49 PM, Greg Freemyer

Re: Copying Data Blocks

2009-01-15 Thread Sandeep K Sinha
One more thing, I take a lock on the inode allocate destination blocks copy and swap unlock inode Now, even if the buffers havent reached the disk, any new write will see that the buffer is dirty and hence we will not have any issues. I mean after releasing the lock on the inode, there will be

Re: Copying Data Blocks

2009-01-15 Thread Sandeep K Sinha
Hi Manish, On Thu, Jan 15, 2009 at 11:54 PM, Manish Katiyar mkati...@gmail.com wrote: On Thu, Jan 15, 2009 at 11:25 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Manish, On Thu, Jan 15, 2009 at 10:31 PM, Manish Katiyar mkati...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:27 PM, Greg

Re: Copying Data Blocks

2009-01-15 Thread Manish Katiyar
On Thu, Jan 15, 2009 at 11:25 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Manish, On Thu, Jan 15, 2009 at 10:31 PM, Manish Katiyar mkati...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:27 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:41 AM, Peter Teoh

Re: Copying Data Blocks

2009-01-15 Thread Manish Katiyar
On Fri, Jan 16, 2009 at 12:03 AM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Manish, On Thu, Jan 15, 2009 at 11:54 PM, Manish Katiyar mkati...@gmail.com wrote: On Thu, Jan 15, 2009 at 11:25 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Manish, On Thu, Jan 15, 2009 at 10:31

Re: Copying Data Blocks

2009-01-15 Thread Sandeep K Sinha
On Fri, Jan 16, 2009 at 12:13 AM, Manish Katiyar mkati...@gmail.com wrote: On Fri, Jan 16, 2009 at 12:03 AM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Manish, On Thu, Jan 15, 2009 at 11:54 PM, Manish Katiyar mkati...@gmail.com wrote: On Thu, Jan 15, 2009 at 11:25 PM, Sandeep K Sinha

Re: Copying Data Blocks

2009-01-15 Thread Greg Freemyer
On Thu, Jan 15, 2009 at 12:55 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Manish, On Thu, Jan 15, 2009 at 10:31 PM, Manish Katiyar mkati...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:27 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Thu, Jan 15, 2009 at 10:41 AM, Peter Teoh

Re: Copying Data Blocks

2009-01-15 Thread Greg Freemyer
On Thu, Jan 15, 2009 at 12:47 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hey, On Thu, Jan 15, 2009 at 10:27 PM, Greg Freemyer snip I think I've said it before, but I would think the best real world implementation would be: === pre-allocate destination data blocks For each block

Re: Copying Data Blocks

2009-01-14 Thread Sandeep K Sinha
-i_blocks; spin_unlock(inode-i_lock); up_write(EXT4_I(inode)-i_data_sem); But for data it should be i_data_sem. Is that correct? On Mon, Jan 12, 2009 at 2:18 PM, Rohit Sharma imreckl...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data

Re: Copying Data Blocks

2009-01-14 Thread Manish Katiyar
, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case what type of lock on inode should be used, semaphore, mutex or spinlock? On Sun, Jan 11, 2009

Re: Copying Data Blocks

2009-01-14 Thread Sandeep K Sinha
. Is that correct? On Mon, Jan 12, 2009 at 2:18 PM, Rohit Sharma imreckl...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case

Re: Copying Data Blocks

2009-01-14 Thread Sandeep K Sinha
(EXT4_I(inode)-i_data_sem); But for data it should be i_data_sem. Is that correct? On Mon, Jan 12, 2009 at 2:18 PM, Rohit Sharma imreckl...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation

Why Ext3? [WAS: Re: Copying Data Blocks]

2009-01-14 Thread Greg Freemyer
HSM team, Reading this thread got me taking a basic look at ext4 for the first time. I get the impression from what I read that ext3 is basically feature frozen for new features and that major improvements like a HSM would need to go into ext4 if they are to be accepted into the kernel. Why are

Re: Copying Data Blocks

2009-01-14 Thread Peter Teoh
Well, Well, Well Firstly, the source inode remians intact only the pointers in the source inodes are updated. And we don't freeze the whole FS, we just take a lock on the inode. So, you can operate on all other inodes. We intend to reduce the granularity of locking to per block from per

Re: Copying Data Blocks

2009-01-14 Thread Greg Freemyer
On Wed, Jan 14, 2009 at 10:42 AM, Peter Teoh htmldevelo...@gmail.com wrote: Well, Well, Well Firstly, the source inode remians intact only the pointers in the source inodes are updated. And we don't freeze the whole FS, we just take a lock on the inode. So, you can operate on all other

Re: Copying Data Blocks

2009-01-14 Thread Sandeep K Sinha
Hi, On Wed, Jan 14, 2009 at 10:43 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Wed, Jan 14, 2009 at 10:42 AM, Peter Teoh htmldevelo...@gmail.com wrote: Well, Well, Well Firstly, the source inode remians intact only the pointers in the source inodes are updated. And we don't freeze

Re: Copying Data Blocks

2009-01-14 Thread Greg Freemyer
snip I again wonder if the ext4_defrag logic uses it? If so, HSM could leverage their design and basically piggy back on top of it. But only for a ext4 implementation. Makes sense. Yes, We have started analysis this. We will update you on this too. I think, it should be helpful for us.

Re: Copying Data Blocks

2009-01-14 Thread Peter Teoh
Frankly, I am quite losts in the sea of argument :-).but let try sharing my points: I can see the inode locking for the entire duration of a file reorg would be unacceptable at some later point in the development / release cycle. That's exactly we are intending. We are planning to get

Re: Copying Data Blocks

2009-01-14 Thread Peter Teoh
Frankly, I am quite losts in the sea of argument :-).but let try sharing my points: I can see the inode locking for the entire duration of a file reorg would be unacceptable at some later point in the development / release cycle. That's exactly we are intending. We are planning to get

Re: Copying Data Blocks

2009-01-13 Thread Sandeep K Sinha
having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case what type of lock on inode should be used, semaphore, mutex or spinlock? On Sun, Jan 11, 2009 at 8:45 PM, Peter

Re: Copying Data Blocks

2009-01-13 Thread Manish Katiyar
, Jan 12, 2009 at 2:18 PM, Rohit Sharma imreckl...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case what type of lock on inode should

Re: Copying Data Blocks

2009-01-13 Thread Sandeep K Sinha
(EXT4_I(inode)-i_data_sem); But for data it should be i_data_sem. Is that correct? On Mon, Jan 12, 2009 at 2:18 PM, Rohit Sharma imreckl...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so

Re: Copying Data Blocks

2009-01-13 Thread Peter Teoh
this brings in a new dimension of analysis: On Wed, Jan 14, 2009 at 1:14 PM, Manish Katiyar mkati...@gmail.com wrote: Correct but that also raises few more questions. Sandeep , do you have any pre-requisites about the sizing of disks for OHSM to work ?? For example lets say I have 3 disks

Re: Copying Data Blocks

2009-01-13 Thread Sandeep K Sinha
issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case what type of lock on inode should be used, semaphore, mutex or spinlock? On Sun, Jan 11, 2009 at 8:45 PM, Peter Teoh

Re: Copying Data Blocks

2009-01-13 Thread Sandeep K Sinha
)-i_data_sem); But for data it should be i_data_sem. Is that correct? On Mon, Jan 12, 2009 at 2:18 PM, Rohit Sharma imreckl...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read

Re: Copying Data Blocks

2009-01-13 Thread Sandeep K Sinha
On Wed, Jan 14, 2009 at 11:13 AM, Peter Teoh htmldevelo...@gmail.com wrote: this brings in a new dimension of analysis: On Wed, Jan 14, 2009 at 1:14 PM, Manish Katiyar mkati...@gmail.com wrote: Correct but that also raises few more questions. Sandeep , do you have any pre-requisites

Re: Copying Data Blocks

2009-01-13 Thread Manish Katiyar
12, 2009 at 2:18 PM, Rohit Sharma imreckl...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case what type of lock on inode should

Re: Copying Data Blocks

2009-01-12 Thread Sandeep K Sinha
-i_blocks; spin_unlock(inode-i_lock); up_write(EXT4_I(inode)-i_data_sem); But for data it should be i_data_sem. Is that correct? On Mon, Jan 12, 2009 at 2:18 PM, Rohit Sharma imreckl...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data blocks. We

Re: Copying Data Blocks

2009-01-12 Thread Peter Teoh
some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case what type of lock on inode should be used, semaphore, mutex or spinlock? On Sun, Jan 11, 2009 at 8:45 PM, Peter Teoh

Re: Copying Data Blocks

2009-01-12 Thread Sandeep K Sinha
. Is that correct? On Mon, Jan 12, 2009 at 2:18 PM, Rohit Sharma imreckl...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case

Re: Copying Data Blocks

2009-01-11 Thread Peter Teoh
Here are some tips on the blockdevice API: http://lkml.org/lkml/2006/1/24/287 http://linux.derkeiler.com/Mailing-Lists/Kernel/2006-01/msg09388.html as indicated, documentation is rather sparse in this area. not sure if anyone else have a summary list of blockdevice API and its explanation?

Re: Copying Data Blocks

2009-01-11 Thread Peter Teoh
Sorry.some mistakes...a resent: Here are some tips on the blockdevice API: http://lkml.org/lkml/2006/1/24/287 http://linux.derkeiler.com/Mailing-Lists/Kernel/2006-01/msg09388.html as indicated, documentation is rather sparse in this area. not sure if anyone else have a summary list of

Re: Copying Data Blocks

2009-01-11 Thread Rohit Sharma
Hi, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case what type of lock on inode should be used, semaphore, mutex or spinlock? On Sun, Jan 11, 2009 at 8:45

Re: Copying Data Blocks

2009-01-11 Thread Sandeep K Sinha
Hi Peter, As our work veery tightly couples with the FS for our intial implementation and we are working around with the inodes, we will require to work at the filesystem level itself. Though it may not be very feasible, but we will try to push everything to the blockdev level once our base

Re: Copying Data Blocks

2009-01-11 Thread Sandeep K Sinha
Hi Rohit, On Mon, Jan 12, 2009 at 11:48 AM, Rohit Sharma imreckl...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case what type

Re: Copying Data Blocks

2009-01-11 Thread Peter Teoh
...@gmail.com wrote: Hi, I am having some issues in locking inode while copying data blocks. We are trying to keep file system live during this operation, so both read and write operations should work. In this case what type of lock on inode should be used, semaphore, mutex or spinlock? On Sun

Re: Copying Data Blocks

2009-01-10 Thread Sandeep K Sinha
Well the way to avoid memcopy to copy the physical data from the source data block to destination data block can be following, I have tested this and it works perfectly fine. 1. get the buffer head of the source inode's data blocks in 'x'. 2. allocate a new destination block. 3. get the buffer

Re: Copying Data Blocks

2009-01-10 Thread Sandeep K Sinha
Hi Greg, On Fri, Jan 9, 2009 at 5:49 AM, Greg Freemyer greg.freem...@gmail.com wrote: Both a top post and bottom post. == Top Post Lost the context for this, but looking at your site: http://code.google.com/p/fscops/ I see a very valuable HSM goal, but I don't see the biggest user of

Re: Copying Data Blocks

2009-01-10 Thread Manish Katiyar
On Sat, Jan 10, 2009 at 1:31 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Well the way to avoid memcopy to copy the physical data from the source data block to destination data block can be following, I have tested this and it works perfectly fine. 1. get the buffer head of the source

Fwd: Copying Data Blocks

2009-01-10 Thread Manish Katiyar
[Forgot to cc the list] -- Forwarded message -- From: Manish Katiyar mkati...@gmail.com Date: Sat, Jan 10, 2009 at 8:31 PM Subject: Re: Copying Data Blocks To: Sandeep K Sinha sandeepksi...@gmail.com On Sat, Jan 10, 2009 at 1:38 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote

Re: Copying Data Blocks

2009-01-10 Thread Greg Freemyer
On Sat, Jan 10, 2009 at 3:08 AM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Greg, On Fri, Jan 9, 2009 at 5:49 AM, Greg Freemyer greg.freem...@gmail.com wrote: Both a top post and bottom post. == Top Post Lost the context for this, but looking at your site:

Re: Copying Data Blocks

2009-01-10 Thread Sandeep K Sinha
Hey, On Sat, Jan 10, 2009 at 8:40 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Sat, Jan 10, 2009 at 3:08 AM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Greg, On Fri, Jan 9, 2009 at 5:49 AM, Greg Freemyer greg.freem...@gmail.com wrote: Both a top post and bottom post. ==

Re: Copying Data Blocks

2009-01-10 Thread Manish Katiyar
On Sat, Jan 10, 2009 at 9:49 PM, Manish Katiyar mkati...@gmail.com wrote: On Sat, Jan 10, 2009 at 9:09 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: See the problem here would be that the inode number would change in case we allocate a new inode for our destination file after relocation.

Re: Copying Data Blocks

2009-01-10 Thread Sandeep K Sinha
On Sat, Jan 10, 2009 at 9:54 PM, Manish Katiyar mkati...@gmail.com wrote: On Sat, Jan 10, 2009 at 9:49 PM, Manish Katiyar mkati...@gmail.com wrote: On Sat, Jan 10, 2009 at 9:09 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: See the problem here would be that the inode number would change

Re: Copying Data Blocks

2009-01-10 Thread Manish Katiyar
On Sat, Jan 10, 2009 at 9:09 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: See the problem here would be that the inode number would change in case we allocate a new inode for our destination file after relocation. One basic question that should have been probably asked long back *unless

Re: Copying Data Blocks

2009-01-10 Thread Sandeep K Sinha
Hi Manish, On Sat, Jan 10, 2009 at 9:49 PM, Manish Katiyar mkati...@gmail.com wrote: On Sat, Jan 10, 2009 at 9:09 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: See the problem here would be that the inode number would change in case we allocate a new inode for our destination file after

Re: Copying Data Blocks

2009-01-10 Thread Manish Katiyar
On Sat, Jan 10, 2009 at 9:58 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Manish, On Sat, Jan 10, 2009 at 9:49 PM, Manish Katiyar mkati...@gmail.com wrote: On Sat, Jan 10, 2009 at 9:09 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: See the problem here would be that the inode

Re: Copying Data Blocks

2009-01-09 Thread Rohit Sharma
The idea given by Greg looks perfectly fine, i am going ahead to implement it, I expect to face many challenges while implementing Greg's algorithm because of the way ext2 is designed. I will update you guys soon. On Fri, Jan 9, 2009 at 6:22 AM, Greg Freemyer greg.freem...@gmail.com wrote: HSM

Re: Copying Data Blocks

2009-01-08 Thread Sandeep K Sinha
Hi Greg, On Wed, Jan 7, 2009 at 9:03 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Wed, Jan 7, 2009 at 5:48 AM, Rohit Sharma imreckl...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:44 PM, Manish Katiyar mkati...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:17 PM, Sandeep K Sinha

Re: Copying Data Blocks

2009-01-08 Thread Greg Freemyer
On Thu, Jan 8, 2009 at 12:55 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Greg, So if I understand your high level desire, you want to write a filesystem re-org (or defrag or something) that works one file at a time. You have to do it in the kernel because you want to control the

Re: Copying Data Blocks

2009-01-08 Thread Greg Freemyer
Both a top post and bottom post. == Top Post Lost the context for this, but looking at your site: http://code.google.com/p/fscops/ I see a very valuable HSM goal, but I don't see the biggest user of future HSM implementations. Namely servers that add SSD drives as even faster / more expensive

Re: Copying Data Blocks

2009-01-08 Thread Greg Freemyer
HSM guys, Looking at your website, I don't see a way to join a mailing list. http://code.google.com/p/fscops/ Is there one? I added in fsc...@gmail.com under the assumption is was a list. Anyway, if you have a list I'd like to join it. Greg -- Greg Freemyer Litigation Triage Solutions

Re: Copying Data Blocks

2009-01-07 Thread Manish Katiyar
On Wed, Jan 7, 2009 at 12:17 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Ok, Let me rephrase what rohit is exactly trying to question. There is an inode X which has say some N number of data blocks. Now, through his own kernel module and some changes to the file system, he wants to

Re: Copying Data Blocks

2009-01-07 Thread Rohit Sharma
On Wed, Jan 7, 2009 at 12:44 PM, Manish Katiyar mkati...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:17 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Ok, Let me rephrase what rohit is exactly trying to question. There is an inode X which has say some N number of data blocks. Now, through

Re: Copying Data Blocks

2009-01-07 Thread Greg Freemyer
On Wed, Jan 7, 2009 at 5:48 AM, Rohit Sharma imreckl...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:44 PM, Manish Katiyar mkati...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:17 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Ok, Let me rephrase what rohit is exactly trying to question.

Re: Copying Data Blocks

2009-01-07 Thread Peter Teoh
On Wed, Jan 7, 2009 at 11:33 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Wed, Jan 7, 2009 at 5:48 AM, Rohit Sharma imreckl...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:44 PM, Manish Katiyar mkati...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:17 PM, Sandeep K Sinha

Re: Copying Data Blocks

2009-01-07 Thread Manish Katiyar
On Wed, Jan 7, 2009 at 12:44 PM, Manish Katiyar mkati...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:17 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Ok, Let me rephrase what rohit is exactly trying to question. There is an inode X which has say some N number of data blocks. Now, through

Re: Copying Data Blocks

2009-01-07 Thread Sandeep K Sinha
Hi Greg, Just to give you a context of the problem : refer: http://code.google.com/p/fscops/ reply inline. On Wed, Jan 7, 2009 at 9:03 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Wed, Jan 7, 2009 at 5:48 AM, Rohit Sharma imreckl...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:44 PM,

Re: Copying Data Blocks

2009-01-07 Thread Sandeep K Sinha
Hey Manish, On Wed, Jan 7, 2009 at 10:29 PM, Manish Katiyar mkati...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:44 PM, Manish Katiyar mkati...@gmail.com wrote: On Wed, Jan 7, 2009 at 12:17 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Ok, Let me rephrase what rohit is exactly trying to

Re: Copying Data Blocks

2009-01-07 Thread Greg Freemyer
On Wed, Jan 7, 2009 at 11:05 PM, Sandeep K Sinha sandeepksi...@gmail.com wrote: Hi Greg, Just to give you a context of the problem : refer: http://code.google.com/p/fscops/ reply inline. On Wed, Jan 7, 2009 at 9:03 PM, Greg Freemyer greg.freem...@gmail.com wrote: On Wed, Jan 7, 2009 at

Copying Data Blocks

2009-01-06 Thread Rohit Sharma
I want to read data blocks from one inode and copy it to other inode. I mean to copy data from data blocks associated with one inode to the data blocks associated with other inode. Is that possible in kernel space.? -- To unsubscribe from this list: send an email with unsubscribe kernelnewbies

Re: Copying Data Blocks

2009-01-06 Thread kanishk rastogi
@nl.linux.org Sent: Tuesday, 6 January, 2009 4:06:59 PM Subject: Copying Data Blocks I want to read data blocks from one inode and copy it to other inode. I mean to copy data from data blocks associated with one inode to the data blocks associated with other inode. Is that possible in kernel

Re: Copying Data Blocks

2009-01-06 Thread Manish Katiyar
On Tue, Jan 6, 2009 at 4:06 PM, Rohit Sharma imreckl...@gmail.com wrote: I want to read data blocks from one inode and copy it to other inode. I mean to copy data from data blocks associated with one inode to the data blocks associated with other inode. Copying 4K chunk of data itself is a

Re: Copying Data Blocks

2009-01-06 Thread rishi agrawal
yes i suppose the donor inode is known moreover the receiver inode's number is also known On Tue, Jan 6, 2009 at 6:54 PM, Manish Katiyar mkati...@gmail.com wrote: On Tue, Jan 6, 2009 at 4:06 PM, Rohit Sharma imreckl...@gmail.com wrote: I want to read data blocks from one inode and copy it

Re: Copying Data Blocks

2009-01-06 Thread Rohit Sharma
On Tue, Jan 6, 2009 at 7:42 PM, rishi agrawal postri...@gmail.com wrote: yes i suppose the donor inode is known moreover the receiver inode's number is also known On Tue, Jan 6, 2009 at 6:54 PM, Manish Katiyar mkati...@gmail.com wrote: On Tue, Jan 6, 2009 at 4:06 PM, Rohit Sharma

Re: Copying Data Blocks

2009-01-06 Thread Manish Katiyar
On Tue, Jan 6, 2009 at 10:15 PM, Rohit Sharma imreckl...@gmail.com wrote: On Tue, Jan 6, 2009 at 7:42 PM, rishi agrawal postri...@gmail.com wrote: yes i suppose the donor inode is known moreover the receiver inode's number is also known On Tue, Jan 6, 2009 at 6:54 PM, Manish Katiyar

Re: Copying Data Blocks

2009-01-06 Thread Rohit Sharma
On Tue, Jan 6, 2009 at 10:43 PM, Manish Katiyar mkati...@gmail.com wrote: On Tue, Jan 6, 2009 at 10:15 PM, Rohit Sharma imreckl...@gmail.com wrote: On Tue, Jan 6, 2009 at 7:42 PM, rishi agrawal postri...@gmail.com wrote: yes i suppose the donor inode is known moreover the receiver inode's

Re: Copying Data Blocks

2009-01-06 Thread Rohit Sharma
We can find out no. of block currently being used by the donor inode, The data we read from donor inode has to be in some buffer or page, is there a way we can associate this buffer with the reciever inode or write the contents of the buffer to the new inode. On Tue, Jan 6, 2009 at 10:46 PM,

Re: Copying Data Blocks

2009-01-06 Thread Manish Katiyar
On Tue, Jan 6, 2009 at 10:55 PM, Rohit Sharma imreckl...@gmail.com wrote: We can find out no. of block currently being used by the donor inode, The data we read from donor inode has to be in some buffer or page, Since we know the blocknumber of donor inode, it should be possible to do a raw

Re: Copying Data Blocks

2009-01-06 Thread Rohit Sharma
On Tue, Jan 6, 2009 at 10:58 PM, Manish Katiyar mkati...@gmail.com wrote: On Tue, Jan 6, 2009 at 10:55 PM, Rohit Sharma imreckl...@gmail.com wrote: We can find out no. of block currently being used by the donor inode, The data we read from donor inode has to be in some buffer or page, Since

Re: Copying Data Blocks

2009-01-06 Thread Rohit Sharma
On Tue, Jan 6, 2009 at 11:09 PM, Manish Katiyar mkati...@gmail.com wrote: On Tue, Jan 6, 2009 at 11:06 PM, Rohit Sharma imreckl...@gmail.com wrote: On Tue, Jan 6, 2009 at 10:58 PM, Manish Katiyar mkati...@gmail.com wrote: On Tue, Jan 6, 2009 at 10:55 PM, Rohit Sharma imreckl...@gmail.com wrote:

Re: Copying Data Blocks

2009-01-06 Thread Greg Freemyer
On Tue, Jan 6, 2009 at 12:46 PM, Rohit Sharma imreckl...@gmail.com wrote: On Tue, Jan 6, 2009 at 11:09 PM, Manish Katiyar mkati...@gmail.com wrote: On Tue, Jan 6, 2009 at 11:06 PM, Rohit Sharma imreckl...@gmail.com wrote: On Tue, Jan 6, 2009 at 10:58 PM, Manish Katiyar mkati...@gmail.com wrote:

Re: Copying Data Blocks

2009-01-06 Thread Manish Katiyar
On Tue, Jan 6, 2009 at 10:58 PM, Manish Katiyar mkati...@gmail.com wrote: On Tue, Jan 6, 2009 at 10:55 PM, Rohit Sharma imreckl...@gmail.com wrote: We can find out no. of block currently being used by the donor inode, The data we read from donor inode has to be in some buffer or page, Since

Re: Copying Data Blocks

2009-01-06 Thread Erik Mouw
On Tue, 6 Jan 2009 23:16:14 +0530 Rohit Sharma imreckl...@gmail.com wrote: On Tue, Jan 6, 2009 at 11:09 PM, Manish Katiyar mkati...@gmail.com wrote: Apart from performance, is there anything else you are worried about ? Performance is only a bottleneck, this can be done in user land but

Re: Copying Data Blocks

2009-01-06 Thread Om
Erik Mouw wrote: On Tue, 6 Jan 2009 23:16:14 +0530 Rohit Sharma imreckl...@gmail.com wrote: On Tue, Jan 6, 2009 at 11:09 PM, Manish Katiyar mkati...@gmail.com wrote: Apart from performance, is there anything else you are worried about ? Performance is only a bottleneck, this can be done in

Re: Copying Data Blocks

2009-01-06 Thread Sandeep K Sinha
Ok, Let me rephrase what rohit is exactly trying to question. There is an inode X which has say some N number of data blocks. Now, through his own kernel module and some changes to the file system, he wants to create a new inode Y in the FS and physically copy all the data from the old inode to