Re: [RFC][PATCH 0/3] ext4 online defrag (ver 0.2)

2007-01-17 Thread Takashi Sato
Hi, On Jan 16, 2007 21:03 +0900, [EMAIL PROTECTED] wrote: 1. Add new ioctl(EXT4_IOC_DEFRAG) which returns the first physical block number of the specified file. With this ioctl, a command gets the specified directory's. Maybe I don't understand, but how is this different from the

Re: [RFC][PATCH 0/3] ext4 online defrag (ver 0.2)

2007-01-18 Thread Takashi Sato
Hi, On Jan 16, 2007 21:03 +0900, [EMAIL PROTECTED] wrote: 1. Add new ioctl(EXT4_IOC_DEFRAG) which returns the first physical block number of the specified file. With this ioctl, a command gets the specified directory's. Maybe I don't understand, but how is this different from the

Re: [RFC][PATCH 0/3] ext4 online defrag (ver 0.2)

2007-01-19 Thread Takashi Sato
Hi, Thank you for your comment. 1. Add new ioctl(EXT4_IOC_DEFRAG) which returns the first physical block number of the specified file. With this ioctl, a command gets the specified directory's. Maybe I don't understand, but how is this different from the long-time FIBMAP ioctl? I can

Re: [RFC][PATCH 2/3] Move the file data to the new blocks

2007-02-07 Thread Takashi Sato
Hi, +ext4_ext_replace_branches(struct inode *org_inode, struct inode *dest_inode, + pgoff_t from_page, pgoff_t dest_from_page, + pgoff_t count_page, unsigned long *delete_start) +{ + struct ext4_ext_path *org_path = NULL; + struct ext4_ext_path *dest_path = NULL; + struct ext4_extent *oext,

[RFC][PATCH 0/3] ext4 online defrag (ver 0.3)

2007-02-08 Thread Takashi Sato
Hi all, I've updated my online defrag patches to support leaf node which is filled with extents and has no space for new extent any more. Any comments from reviews or tests are very welcome. Implementation: 1. When the leaf node is filled with extents and there is no space for additional

[RFC][PATCH 1/3] Allocate new contiguous blocks

2007-02-08 Thread Takashi Sato
Search contiguous free blocks with Alex's mutil-block allocation and allocate them for the temporary inode. This patch applies on top of Alex's patches. [RFC] delayed allocation, mballoc, etc http://marc.theaimsgroup.com/?l=linux-ext4m=116493228301966w=2 Signed-off-by: Takashi Sato [EMAIL

[RFC][PATCH 2/3] Move the file data to the new blocks

2007-02-08 Thread Takashi Sato
Move the blocks on the temporary inode to the original inode by a page. 1. Read the file data from the old blocks to the page 2. Move the block on the temporary inode to the original inode 3. Write the file data on the page into the new blocks Signed-off-by: Takashi Sato [EMAIL PROTECTED

[RFC][PATCH 3/3] Online defrag command

2007-02-08 Thread Takashi Sato
The defrag command. Usage is as follows: o Put the multiple files closer together. # e4defrag -r directory-name o Defrag for a single file. # e4defrag file-name o Defrag for all files on ext4. # e4defrag device-name Signed-off-by: Takashi Sato [EMAIL PROTECTED] --- /* * e4defrag, ext4

Re: [RFC][PATCH 3/3] Online defrag command

2007-02-09 Thread Takashi Sato
Hi, On Thu, Feb 08 2007, Takashi Sato wrote: The defrag command. Usage is as follows: o Put the multiple files closer together. # e4defrag -r directory-name o Defrag for a single file. # e4defrag file-name o Defrag for all files on ext4. # e4defrag device-name Would it be possible

[RFC][PATCH 1/4] Allocate new contiguous blocks

2007-04-26 Thread Takashi Sato
Search contiguous free blocks with Alex's mutil-block allocation and allocate them for the temporary inode. This patch applies on top of Alex's patches. [RFC] delayed allocation, mballoc, etc http://marc.theaimsgroup.com/?l=linux-ext4m=116493228301966w=2 Signed-off-by: Takashi Sato [EMAIL

[RFC][PATCH 4/4] ext4_locality_group bug fix

2007-04-26 Thread Takashi Sato
Move lg_list to s_locality_dirty and mark lg as dirty if nr_to_write(total page count which has not written in disk yet) is 0 or less and lg_io is not empty in ext4_lg_sync_single_group(). This makes sure that inode is written to disk. Signed-off-by: Takashi Sato [EMAIL PROTECTED] --- diff -Nrup

ext4 online defrag (ver 0.4)

2007-04-26 Thread Takashi Sato
Hi all, I have made following changes to the previous online defrag patchset to improve it. Note that there is no functional change. 1. Change the handling of temporary inode. Now ext4_ext_defrag() calls ext4_new_inode()/iput() pair instead of new_inode()/delete_ext_defrag_inode(). Because

[RFC][PATCH 2/4] Move the file data to the new blocks

2007-04-26 Thread Takashi Sato
Move the blocks on the temporary inode to the original inode by a page. 1. Read the file data from the old blocks to the page 2. Move the block on the temporary inode to the original inode 3. Write the file data on the page into the new blocks Signed-off-by: Takashi Sato [EMAIL PROTECTED

[RFC][PATCH 3/4] Online defrag command

2007-04-26 Thread Takashi Sato
The defrag command. Usage is as follows: o Put the multiple files closer together. # e4defrag -r directory-name o Defrag for a single file. # e4defrag file-name o Defrag for all files on ext4. # e4defrag device-name Signed-off-by: Takashi Sato [EMAIL PROTECTED] --- /* * e4defrag, ext4

[RFC][PATCH 0/10] ext4 online defrag (ver 0.5)

2007-06-20 Thread Takashi Sato
Hi all, I have updated my online defrag patchset for addition of a new function. This function is defragmentation for free space. If filesytem has insufficient contiguous free blocks, defrag tries to move other files to make sufficient space and reallocates the contiguous blocks for the target

[RFC][PATCH 1/10] Allocate new contiguous blocks

2007-06-20 Thread Takashi Sato
Search contiguous free blocks with Alex's mutil-block allocation and allocate them for the temporary inode. This patch applies on top of Alex's patches. [RFC] delayed allocation, mballoc, etc http://marc.theaimsgroup.com/?l=linux-ext4m=116493228301966w=2 Signed-off-by: Takashi Sato [EMAIL

[RFC][PATCH 2/10] Move the file data to the new blocks

2007-06-20 Thread Takashi Sato
Move the blocks on the temporary inode to the original inode by a page. 1. Read the file data from the old blocks to the page 2. Move the block on the temporary inode to the original inode 3. Write the file data on the page into the new blocks Signed-off-by: Takashi Sato [EMAIL PROTECTED] Signed

[RFC][PATCH 3/10] Get block group information

2007-06-20 Thread Takashi Sato
- Get s_blocks_per_group and s_inodes_per_group of target filesystem. Signed-off-by: Takashi Sato [EMAIL PROTECTED] Signed-off-by: Akira Fujita [EMAIL PROTECTED] --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation/dontdiff -upNr linux-2.6.19-rc6-test1/fs/ext4/balloc.c Online

[RFC][PATCH 4/10] Get free blocks distribution of the target block group

2007-06-20 Thread Takashi Sato
- Get free blocks distribution of the target block group to know how many free blocks it has. Signed-off-by: Takashi Sato [EMAIL PROTECTED] Signed-off-by: Akira Fujita [EMAIL PROTECTED] --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation/dontdiff -upNr Online-Defrag_linux-2.6.19-rc6

[RFC][PATCH 5/10] Get all extents information of specified inode number

2007-06-20 Thread Takashi Sato
- Get all extents information of specified inode number to calculate the combination of extents which should be moved to other block group. Signed-off-by: Takashi Sato [EMAIL PROTECTED] Signed-off-by: Akira Fujita [EMAIL PROTECTED] --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation

[RFC][PATCH 6/10] Move files from target block group to other block group

2007-06-20 Thread Takashi Sato
- To make contiguous free blocks, move files from the target block group to other block group. Signed-off-by: Takashi Sato [EMAIL PROTECTED] Signed-off-by: Akira Fujita [EMAIL PROTECTED] --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation/dontdiff -upNr Online-Defrag_linux-2.6.19-rc6

[RFC][PATCH 7/10] Reserve freed blocks

2007-06-20 Thread Takashi Sato
- Reserve the free blocks in the target area, not to be used by other process. Signed-off-by: Takashi Sato [EMAIL PROTECTED] Signed-off-by: Akira Fujita [EMAIL PROTECTED] --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation/dontdiff -upNr Online-Defrag_linux-2.6.19-rc6-git

[RFC][PATCH 8/10] Release reserved block

2007-06-20 Thread Takashi Sato
- Release reserved blocks if defrag failed. Signed-off-by: Takashi Sato [EMAIL PROTECTED] Signed-off-by: Akira Fujita [EMAIL PROTECTED] --- diff -X Online-Defrag_linux-2.6.19-rc6-git/Documentation/dontdiff -upNr Online-Defrag_linux-2.6.19-rc6-git-BLOCK_RELEASE/fs/ext4/extents.c Online

[RFC][PATCH 9/10] Fix bugs in multi-block allocation and locality-group

2007-06-20 Thread Takashi Sato
- Move lg_list to s_locality_dirty in ext4_lg_sync_single_group() to flush all of dirty inodes. - Fix ext4_mb_new_blocks() to return err value when defrag failed. Signed-off-by: Takashi Sato [EMAIL PROTECTED] Signed-off-by: Akira Fujita [EMAIL PROTECTED] --- diff -X Online-Defrag_linux-2.6.19

[RFC][PATCH 10/10] Online defrag command

2007-06-20 Thread Takashi Sato
device-name Signed-off-by: Takashi Sato [EMAIL PROTECTED] Signed-off-by: Akira Fujita [EMAIL PROTECTED] --- /* * e4defrag.c - ext4 filesystem defragmenter */ #ifndef _LARGEFILE_SOURCE #define _LARGEFILE_SOURCE #endif #ifndef _LARGEFILE64_SOURCE #define _LARGEFILE64_SOURCE #endif #define

[RFC] ext3 freeze feature

2008-01-25 Thread Takashi Sato
to unfreeze the filesystem automatically after the lapse of the specified time. Any comments are very welcome. Signed-off-by: Takashi Sato [EMAIL PROTECTED] --- diff -uprN -X linux-2.6.24-rc8/Documentation/dontdiff linux-2.6.24-rc8/fs/ext3/ioctl.c linux-2.6.24-rc8-freeze/fs/ext3/ioctl.c --- linux

Re: [RFC] ext3 freeze feature

2008-01-25 Thread Takashi Sato
Hi, I am also wondering whether we should have system call(s) for these: On Jan 25, 2008 12:59 PM, Takashi Sato [EMAIL PROTECTED] wrote: + case EXT3_IOC_FREEZE: { + case EXT3_IOC_THAW: { And just convert XFS to use them too? I think it is reasonable to implement

Re: [RFC] ext3 freeze feature

2008-01-28 Thread Takashi Sato
Hi, What you *could* do is to start putting processes to sleep if they attempt to write to the frozen filesystem, and then detect the deadlock case where the process holding the file descriptor used to freeze the filesystem gets frozen because it attempted to write to the filesystem --- at

Re: [RFC] ext3 freeze feature

2008-01-28 Thread Takashi Sato
Hi, Thank you for your comments. That's inherently unsafe - you can have multiple unfreezes running in parallel which seriously screws with the bdev semaphore count that is used to lock the device due to doing multiple up()s for every down. Your timeout thingy guarantee that at some point you

Re: [RFC] ext3 freeze feature

2008-02-06 Thread Takashi Sato
Hi, What you *could* do is to start putting processes to sleep if they attempt to write to the frozen filesystem, and then detect the deadlock case where the process holding the file descriptor used to freeze the filesystem gets frozen because it attempted to write to the filesystem --- at

Re: [RFC] ext3 freeze feature

2008-02-08 Thread Takashi Sato
Hi, Ted wrote: And I do agree that we probably should just implement this in filesystem independent way, in which case all of the filesystems that support this already have super_operations functions write_super_lockfs() and unlockfs(). So if this is done using a new system call, there should

Re: [RFC] ext3 freeze feature

2008-02-13 Thread Takashi Sato
Hi, P.S. Oh yeah, it should be noted that freezing at the filesystem layer does *not* guarantee that changes to the block device aren't happening via mmap()'ed files. The LVM needs to freeze writes the block device level if it wants to guarantee a completely stable snapshot image. So the

Re: [RFC] ext3 freeze feature

2008-02-15 Thread Takashi Sato
Hi, Christoph Hellwig wrote: On Fri, Feb 08, 2008 at 08:26:57AM -0500, Andreas Dilger wrote: You may as well make the common ioctl the same as the XFS version, both by number and parameters, so that applications which already understand the XFS ioctl will work on other filesystems. Yes. In