RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-06-03 Thread Namjae Jeon
> On Mon, Jun 02, 2014 at 03:06:13PM +0200, Lukáš Czerner wrote:
> > > > So what will happen when there is not enough space when "inserting a
> > > > range" ? And how should user proceed from there ?
> > > If insert range fails with an ENOSPC error, user could use collapse
> > > range on the same range to remove the hole.
> > > And after freeing more space, he can again try inserting range.
> > > Ofcourse, this type of guidance should be properly documented in
> > > manpage. When updating fallocate(2) manpage, I will keep  in mind to
> > > describe ENOSPC handling.
> >
> > Why collapse ? The hole is already there right ? Why not just use
> > fallocate to allocate the space for the hole. And that's my point
> > actually. Why not do it this way in the first place, because this is
> > really counterintuitive.
> 
> It's worse than that.  It's possible that the reason why you got the
> ENOSPC warning was because the operation to move the extents down
> required allocating a block, and it was *that* block allocation which
> failed.  So it's not deterministic whether or not the file's extent
> mappings were modified after a ENOSPC error, and so it's not clear
> whether or not a collapse_range function will undo the range that had
> been inserted --- or whether it ends up deleting existing data blocks.
> 
> In generally, you really want system calls to have all-or-nothing
> effects, where if the system call returns an error, the state of the
> file has not been changed.  And for that reason, I agree with Lukáš
> that it is really a good idea to decouple moving the blocks down, and
> allocating space --- and to make sure that if there is any failure
> while inserting the range, the state of the file is not modified at all.
Okay, I will remove allocating space part in insert range patch.
But renaming flags as FALLOC_FL_INSERT_HOLE is needed to concent with
XFS people. Because Dave prefered to call it FALLOC_FL_INSERT_RANGE
so that it looks like it is related to collapse range.

Hi Dave.
Do you have any objection about renaming as insert hole ?

Thanks for opinions!

> 
> Cheers,
> 
>   - Ted

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-06-03 Thread Namjae Jeon
 On Mon, Jun 02, 2014 at 03:06:13PM +0200, Lukáš Czerner wrote:
So what will happen when there is not enough space when inserting a
range ? And how should user proceed from there ?
   If insert range fails with an ENOSPC error, user could use collapse
   range on the same range to remove the hole.
   And after freeing more space, he can again try inserting range.
   Ofcourse, this type of guidance should be properly documented in
   manpage. When updating fallocate(2) manpage, I will keep  in mind to
   describe ENOSPC handling.
 
  Why collapse ? The hole is already there right ? Why not just use
  fallocate to allocate the space for the hole. And that's my point
  actually. Why not do it this way in the first place, because this is
  really counterintuitive.
 
 It's worse than that.  It's possible that the reason why you got the
 ENOSPC warning was because the operation to move the extents down
 required allocating a block, and it was *that* block allocation which
 failed.  So it's not deterministic whether or not the file's extent
 mappings were modified after a ENOSPC error, and so it's not clear
 whether or not a collapse_range function will undo the range that had
 been inserted --- or whether it ends up deleting existing data blocks.
 
 In generally, you really want system calls to have all-or-nothing
 effects, where if the system call returns an error, the state of the
 file has not been changed.  And for that reason, I agree with Lukáš
 that it is really a good idea to decouple moving the blocks down, and
 allocating space --- and to make sure that if there is any failure
 while inserting the range, the state of the file is not modified at all.
Okay, I will remove allocating space part in insert range patch.
But renaming flags as FALLOC_FL_INSERT_HOLE is needed to concent with
XFS people. Because Dave prefered to call it FALLOC_FL_INSERT_RANGE
so that it looks like it is related to collapse range.

Hi Dave.
Do you have any objection about renaming as insert hole ?

Thanks for opinions!

 
 Cheers,
 
   - Ted

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-06-02 Thread Theodore Ts'o
On Mon, Jun 02, 2014 at 03:06:13PM +0200, Lukáš Czerner wrote:
> > > So what will happen when there is not enough space when "inserting a
> > > range" ? And how should user proceed from there ?
> > If insert range fails with an ENOSPC error, user could use collapse
> > range on the same range to remove the hole.
> > And after freeing more space, he can again try inserting range.
> > Ofcourse, this type of guidance should be properly documented in
> > manpage. When updating fallocate(2) manpage, I will keep  in mind to
> > describe ENOSPC handling.
> 
> Why collapse ? The hole is already there right ? Why not just use
> fallocate to allocate the space for the hole. And that's my point
> actually. Why not do it this way in the first place, because this is
> really counterintuitive.

It's worse than that.  It's possible that the reason why you got the
ENOSPC warning was because the operation to move the extents down
required allocating a block, and it was *that* block allocation which
failed.  So it's not deterministic whether or not the file's extent
mappings were modified after a ENOSPC error, and so it's not clear
whether or not a collapse_range function will undo the range that had
been inserted --- or whether it ends up deleting existing data blocks.

In generally, you really want system calls to have all-or-nothing
effects, where if the system call returns an error, the state of the
file has not been changed.  And for that reason, I agree with Lukáš
that it is really a good idea to decouple moving the blocks down, and
allocating space --- and to make sure that if there is any failure
while inserting the range, the state of the file is not modified at all.

Cheers,

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-06-02 Thread Lukáš Czerner
On Mon, 2 Jun 2014, Namjae Jeon wrote:

> Date: Mon, 02 Jun 2014 20:52:51 +0900
> From: Namjae Jeon 
> To: 'Lukáš Czerner' 
> Cc: 'Dave Chinner' , 'Theodore Ts'o' ,
> 'linux-ext4' , x...@oss.sgi.com,
> linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
> 'Ashish Sangwan' 
> Subject: RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for
> fallocate
> 
> > 
> > > Date: Sat, 31 May 2014 16:40:29 +0900
> > > From: Namjae Jeon 
> > > To: 'Lukáš Czerner' 
> > > Cc: 'Dave Chinner' , 'Theodore Ts'o' ,
> > > 'linux-ext4' , x...@oss.sgi.com,
> > > linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
> > > 'Ashish Sangwan' 
> > > Subject: RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for
> > > fallocate
> > >
> > >
> > > > > Date: Thu, 08 May 2014 19:23:19 +0900
> > > > > From: Namjae Jeon 
> > > > > To: Dave Chinner , Theodore Ts'o 
> > > > > Cc: linux-ext4 , x...@oss.sgi.com,
> > > > > linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
> > > > > Ashish Sangwan 
> > > > > Subject: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for 
> > > > > fallocate
> > > > >
> > > > > In continuation of the work of making the process of non linear 
> > > > > editing of
> > > > > media files faster, we introduce here the new flag 
> > > > > FALLOC_FL_INSERT_RANGE
> > > > > for fallocate.
> > > > >
> > > > > This flag will work opposite to the newly added 
> > > > > FALLOC_FL_COLLAPSE_RANGE flag.
> > > > > As such, specifying FALLOC_FL_INSERT_RANGE flag will insert 
> > > > > zeroed-out space
> > > > > in between the file within the range specified by offset and len. 
> > > > > User can
> > > > > write new data in this space. e.g. ads.
> > > > > Like collapse range, currently we have the limitation that offset and 
> > > > > len should
> > > > > be block size aligned for both XFS and Ext4.
> > > > >
> > > > > The semantics of the flag are :
> > > > > 1) It allocates new zeroed out on disk space of len bytes starting
> > > > >at offset byte without overwriting any existing data. All the data 
> > > > > blocks
> > > > >from offset to EOF are shifted towards right to make space for 
> > > > > inserting
> > > > >new blocks
> > > >
> > > > Hi,
> > > >
> > > > this sounds a little bit weird to me. I understand the reason for
> > > > this, but this is effectively two operations masking as one. We
> > > > shift the existing data and then we allocate unwritten extents for
> > > > the hole we've created.
> > > >
> > > > What would make more sense to me is to implement only the first
> > > > operation - the shift. And then let the user to allocate unwritten
> > > > extents for the hole using simple fallocate.
> > > >
> > > > The reason is that if you succeed the first part and then fail the
> > > > second due to ENOSPC or any other reason the file will end up in
> > > > undefined state unnecessarily. Yes in your current implementation
> > > > it seems that you'll always end up with the hole in the file and the
> > > > rest is properly shifted, but that may vary from file system to file
> > > > system. Some might choose to roll back the shift, some might not.
> > > >
> > > > If FALLOC_FL_INSERT_RANGE (or any name you wish to choose) would
> > > > just simply shift the extents then you'll get rid of this and the
> > > > only thing that user needs to do (if he chooses to) is to use
> > > > fallocate for the hole created by the shift. If it fails, then
> > > > well, he can try again without any consequences. As a bonus you get
> > > > the possibility to leave the hole in the file which might be useful
> > > > as well.
> > > >
> > > > With current behaviour this might get very confusing very quickly.
> > > >
> > > > What do you and others think ?
> > > Hi Lukas.
> > > Insert range inherently means inserting a real range of space into
> > > the file to provide guaranteed space with user in the inserted area
> > > so that further writes sho

RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-06-02 Thread Namjae Jeon
> 
> > Date: Sat, 31 May 2014 16:40:29 +0900
> > From: Namjae Jeon 
> > To: 'Lukáš Czerner' 
> > Cc: 'Dave Chinner' , 'Theodore Ts'o' ,
> > 'linux-ext4' , x...@oss.sgi.com,
> > linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
> >     'Ashish Sangwan' 
> > Subject: RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for
> > fallocate
> >
> >
> > > > Date: Thu, 08 May 2014 19:23:19 +0900
> > > > From: Namjae Jeon 
> > > > To: Dave Chinner , Theodore Ts'o 
> > > > Cc: linux-ext4 , x...@oss.sgi.com,
> > > >     linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
> > > > Ashish Sangwan 
> > > > Subject: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for 
> > > > fallocate
> > > >
> > > > In continuation of the work of making the process of non linear editing 
> > > > of
> > > > media files faster, we introduce here the new flag 
> > > > FALLOC_FL_INSERT_RANGE
> > > > for fallocate.
> > > >
> > > > This flag will work opposite to the newly added 
> > > > FALLOC_FL_COLLAPSE_RANGE flag.
> > > > As such, specifying FALLOC_FL_INSERT_RANGE flag will insert zeroed-out 
> > > > space
> > > > in between the file within the range specified by offset and len. User 
> > > > can
> > > > write new data in this space. e.g. ads.
> > > > Like collapse range, currently we have the limitation that offset and 
> > > > len should
> > > > be block size aligned for both XFS and Ext4.
> > > >
> > > > The semantics of the flag are :
> > > > 1) It allocates new zeroed out on disk space of len bytes starting
> > > >at offset byte without overwriting any existing data. All the data 
> > > > blocks
> > > >from offset to EOF are shifted towards right to make space for 
> > > > inserting
> > > >new blocks
> > >
> > > Hi,
> > >
> > > this sounds a little bit weird to me. I understand the reason for
> > > this, but this is effectively two operations masking as one. We
> > > shift the existing data and then we allocate unwritten extents for
> > > the hole we've created.
> > >
> > > What would make more sense to me is to implement only the first
> > > operation - the shift. And then let the user to allocate unwritten
> > > extents for the hole using simple fallocate.
> > >
> > > The reason is that if you succeed the first part and then fail the
> > > second due to ENOSPC or any other reason the file will end up in
> > > undefined state unnecessarily. Yes in your current implementation
> > > it seems that you'll always end up with the hole in the file and the
> > > rest is properly shifted, but that may vary from file system to file
> > > system. Some might choose to roll back the shift, some might not.
> > >
> > > If FALLOC_FL_INSERT_RANGE (or any name you wish to choose) would
> > > just simply shift the extents then you'll get rid of this and the
> > > only thing that user needs to do (if he chooses to) is to use
> > > fallocate for the hole created by the shift. If it fails, then
> > > well, he can try again without any consequences. As a bonus you get
> > > the possibility to leave the hole in the file which might be useful
> > > as well.
> > >
> > > With current behaviour this might get very confusing very quickly.
> > >
> > > What do you and others think ?
> > Hi Lukas.
> > Insert range inherently means inserting a real range of space into
> > the file to provide guaranteed space with user in the inserted area
> > so that further writes should not fail with an -ENOSPC at least.
> > If insert range cannot gurantees the above semantics, It should
> > return error to user space.
> 
> So what will happen when there is not enough space when "inserting a
> range" ? And how should user proceed from there ?
If insert range fails with an ENOSPC error, user could use collapse
range on the same range to remove the hole.
And after freeing more space, he can again try inserting range.
Ofcourse, this type of guidance should be properly documented in
manpage. When updating fallocate(2) manpage, I will keep  in mind to
describe ENOSPC handling.
> 
> >
> > If insert range has been performed on a file, It will reserve space
> > that write never fail in the inserted area,
> > In case of full parti

RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-06-02 Thread Lukáš Czerner
On Sat, 31 May 2014, Namjae Jeon wrote:

> Date: Sat, 31 May 2014 16:40:29 +0900
> From: Namjae Jeon 
> To: 'Lukáš Czerner' 
> Cc: 'Dave Chinner' , 'Theodore Ts'o' ,
> 'linux-ext4' , x...@oss.sgi.com,
> linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
> 'Ashish Sangwan' 
> Subject: RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for
> fallocate
> 
>  
> > > Date: Thu, 08 May 2014 19:23:19 +0900
> > > From: Namjae Jeon 
> > > To: Dave Chinner , Theodore Ts'o 
> > > Cc: linux-ext4 , x...@oss.sgi.com,
> > > linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
> > > Ashish Sangwan 
> > > Subject: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for 
> > > fallocate
> > >
> > > In continuation of the work of making the process of non linear editing of
> > > media files faster, we introduce here the new flag FALLOC_FL_INSERT_RANGE
> > > for fallocate.
> > >
> > > This flag will work opposite to the newly added FALLOC_FL_COLLAPSE_RANGE 
> > > flag.
> > > As such, specifying FALLOC_FL_INSERT_RANGE flag will insert zeroed-out 
> > > space
> > > in between the file within the range specified by offset and len. User can
> > > write new data in this space. e.g. ads.
> > > Like collapse range, currently we have the limitation that offset and len 
> > > should
> > > be block size aligned for both XFS and Ext4.
> > >
> > > The semantics of the flag are :
> > > 1) It allocates new zeroed out on disk space of len bytes starting
> > >at offset byte without overwriting any existing data. All the data 
> > > blocks
> > >from offset to EOF are shifted towards right to make space for 
> > > inserting
> > >new blocks
> > 
> > Hi,
> > 
> > this sounds a little bit weird to me. I understand the reason for
> > this, but this is effectively two operations masking as one. We
> > shift the existing data and then we allocate unwritten extents for
> > the hole we've created.
> > 
> > What would make more sense to me is to implement only the first
> > operation - the shift. And then let the user to allocate unwritten
> > extents for the hole using simple fallocate.
> > 
> > The reason is that if you succeed the first part and then fail the
> > second due to ENOSPC or any other reason the file will end up in
> > undefined state unnecessarily. Yes in your current implementation
> > it seems that you'll always end up with the hole in the file and the
> > rest is properly shifted, but that may vary from file system to file
> > system. Some might choose to roll back the shift, some might not.
> > 
> > If FALLOC_FL_INSERT_RANGE (or any name you wish to choose) would
> > just simply shift the extents then you'll get rid of this and the
> > only thing that user needs to do (if he chooses to) is to use
> > fallocate for the hole created by the shift. If it fails, then
> > well, he can try again without any consequences. As a bonus you get
> > the possibility to leave the hole in the file which might be useful
> > as well.
> > 
> > With current behaviour this might get very confusing very quickly.
> > 
> > What do you and others think ?
> Hi Lukas.
> Insert range inherently means inserting a real range of space into
> the file to provide guaranteed space with user in the inserted area
> so that further writes should not fail with an -ENOSPC at least.
> If insert range cannot gurantees the above semantics, It should
> return error to user space.

So what will happen when there is not enough space when "inserting a
range" ? And how should user proceed from there ?

> 
> If insert range has been performed on a file, It will reserve space
> that write never fail in the inserted area,
> In case of full partition or small available size than the range
> user want, It seems odd just only left inserting a hole in the middle
> of file and return success to user when no one can really write to
> this hole.

There is a fallocate for allocation, so as I already said you can
shift the extents to make a hole in the file and then use fallocate
to allocate space for it and you'll get the same result. You are
basically doing that now as well, but when the allocation fails the
whole "insert range" ioct fails, however the extents are already
shifter and there is already a holi in the file so freeing some
space and running this ioctl again will not help you at all. While
if you fail a fallocate, you can free some space and run it again
without any problems. The res

RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-06-02 Thread Lukáš Czerner
On Sat, 31 May 2014, Namjae Jeon wrote:

 Date: Sat, 31 May 2014 16:40:29 +0900
 From: Namjae Jeon namjae.j...@samsung.com
 To: 'Lukáš Czerner' lczer...@redhat.com
 Cc: 'Dave Chinner' da...@fromorbit.com, 'Theodore Ts'o' ty...@mit.edu,
 'linux-ext4' linux-e...@vger.kernel.org, x...@oss.sgi.com,
 linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
 'Ashish Sangwan' a.sang...@samsung.com
 Subject: RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for
 fallocate
 
  
   Date: Thu, 08 May 2014 19:23:19 +0900
   From: Namjae Jeon namjae.j...@samsung.com
   To: Dave Chinner da...@fromorbit.com, Theodore Ts'o ty...@mit.edu
   Cc: linux-ext4 linux-e...@vger.kernel.org, x...@oss.sgi.com,
   linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
   Ashish Sangwan a.sang...@samsung.com
   Subject: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for 
   fallocate
  
   In continuation of the work of making the process of non linear editing of
   media files faster, we introduce here the new flag FALLOC_FL_INSERT_RANGE
   for fallocate.
  
   This flag will work opposite to the newly added FALLOC_FL_COLLAPSE_RANGE 
   flag.
   As such, specifying FALLOC_FL_INSERT_RANGE flag will insert zeroed-out 
   space
   in between the file within the range specified by offset and len. User can
   write new data in this space. e.g. ads.
   Like collapse range, currently we have the limitation that offset and len 
   should
   be block size aligned for both XFS and Ext4.
  
   The semantics of the flag are :
   1) It allocates new zeroed out on disk space of len bytes starting
  at offset byte without overwriting any existing data. All the data 
   blocks
  from offset to EOF are shifted towards right to make space for 
   inserting
  new blocks
  
  Hi,
  
  this sounds a little bit weird to me. I understand the reason for
  this, but this is effectively two operations masking as one. We
  shift the existing data and then we allocate unwritten extents for
  the hole we've created.
  
  What would make more sense to me is to implement only the first
  operation - the shift. And then let the user to allocate unwritten
  extents for the hole using simple fallocate.
  
  The reason is that if you succeed the first part and then fail the
  second due to ENOSPC or any other reason the file will end up in
  undefined state unnecessarily. Yes in your current implementation
  it seems that you'll always end up with the hole in the file and the
  rest is properly shifted, but that may vary from file system to file
  system. Some might choose to roll back the shift, some might not.
  
  If FALLOC_FL_INSERT_RANGE (or any name you wish to choose) would
  just simply shift the extents then you'll get rid of this and the
  only thing that user needs to do (if he chooses to) is to use
  fallocate for the hole created by the shift. If it fails, then
  well, he can try again without any consequences. As a bonus you get
  the possibility to leave the hole in the file which might be useful
  as well.
  
  With current behaviour this might get very confusing very quickly.
  
  What do you and others think ?
 Hi Lukas.
 Insert range inherently means inserting a real range of space into
 the file to provide guaranteed space with user in the inserted area
 so that further writes should not fail with an -ENOSPC at least.
 If insert range cannot gurantees the above semantics, It should
 return error to user space.

So what will happen when there is not enough space when inserting a
range ? And how should user proceed from there ?

 
 If insert range has been performed on a file, It will reserve space
 that write never fail in the inserted area,
 In case of full partition or small available size than the range
 user want, It seems odd just only left inserting a hole in the middle
 of file and return success to user when no one can really write to
 this hole.

There is a fallocate for allocation, so as I already said you can
shift the extents to make a hole in the file and then use fallocate
to allocate space for it and you'll get the same result. You are
basically doing that now as well, but when the allocation fails the
whole insert range ioct fails, however the extents are already
shifter and there is already a holi in the file so freeing some
space and running this ioctl again will not help you at all. While
if you fail a fallocate, you can free some space and run it again
without any problems. The result will be as expected.

What I am arguing about is basically that your insert range ioct is
masking two operations as one. Why not to make it transparent and
split it into shift extents and fallocate ? Then there is a
question about the name because it's no longer insert range but
rather insert hole which I think is better and arguably more
useful semantic.

Thanks!
-Lukas

 
 Thanks!
  
  Thanks!
  -LUkas
  
  
   2) It should be used exclusively. No other fallocate flag in combination.
   3

RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-06-02 Thread Namjae Jeon
 
  Date: Sat, 31 May 2014 16:40:29 +0900
  From: Namjae Jeon namjae.j...@samsung.com
  To: 'Lukáš Czerner' lczer...@redhat.com
  Cc: 'Dave Chinner' da...@fromorbit.com, 'Theodore Ts'o' ty...@mit.edu,
  'linux-ext4' linux-e...@vger.kernel.org, x...@oss.sgi.com,
  linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
  'Ashish Sangwan' a.sang...@samsung.com
  Subject: RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for
  fallocate
 
 
Date: Thu, 08 May 2014 19:23:19 +0900
From: Namjae Jeon namjae.j...@samsung.com
To: Dave Chinner da...@fromorbit.com, Theodore Ts'o ty...@mit.edu
Cc: linux-ext4 linux-e...@vger.kernel.org, x...@oss.sgi.com,
linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
Ashish Sangwan a.sang...@samsung.com
Subject: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for 
fallocate
   
In continuation of the work of making the process of non linear editing 
of
media files faster, we introduce here the new flag 
FALLOC_FL_INSERT_RANGE
for fallocate.
   
This flag will work opposite to the newly added 
FALLOC_FL_COLLAPSE_RANGE flag.
As such, specifying FALLOC_FL_INSERT_RANGE flag will insert zeroed-out 
space
in between the file within the range specified by offset and len. User 
can
write new data in this space. e.g. ads.
Like collapse range, currently we have the limitation that offset and 
len should
be block size aligned for both XFS and Ext4.
   
The semantics of the flag are :
1) It allocates new zeroed out on disk space of len bytes starting
   at offset byte without overwriting any existing data. All the data 
blocks
   from offset to EOF are shifted towards right to make space for 
inserting
   new blocks
  
   Hi,
  
   this sounds a little bit weird to me. I understand the reason for
   this, but this is effectively two operations masking as one. We
   shift the existing data and then we allocate unwritten extents for
   the hole we've created.
  
   What would make more sense to me is to implement only the first
   operation - the shift. And then let the user to allocate unwritten
   extents for the hole using simple fallocate.
  
   The reason is that if you succeed the first part and then fail the
   second due to ENOSPC or any other reason the file will end up in
   undefined state unnecessarily. Yes in your current implementation
   it seems that you'll always end up with the hole in the file and the
   rest is properly shifted, but that may vary from file system to file
   system. Some might choose to roll back the shift, some might not.
  
   If FALLOC_FL_INSERT_RANGE (or any name you wish to choose) would
   just simply shift the extents then you'll get rid of this and the
   only thing that user needs to do (if he chooses to) is to use
   fallocate for the hole created by the shift. If it fails, then
   well, he can try again without any consequences. As a bonus you get
   the possibility to leave the hole in the file which might be useful
   as well.
  
   With current behaviour this might get very confusing very quickly.
  
   What do you and others think ?
  Hi Lukas.
  Insert range inherently means inserting a real range of space into
  the file to provide guaranteed space with user in the inserted area
  so that further writes should not fail with an -ENOSPC at least.
  If insert range cannot gurantees the above semantics, It should
  return error to user space.
 
 So what will happen when there is not enough space when inserting a
 range ? And how should user proceed from there ?
If insert range fails with an ENOSPC error, user could use collapse
range on the same range to remove the hole.
And after freeing more space, he can again try inserting range.
Ofcourse, this type of guidance should be properly documented in
manpage. When updating fallocate(2) manpage, I will keep  in mind to
describe ENOSPC handling.
 
 
  If insert range has been performed on a file, It will reserve space
  that write never fail in the inserted area,
  In case of full partition or small available size than the range
  user want, It seems odd just only left inserting a hole in the middle
  of file and return success to user when no one can really write to
  this hole.
 
 There is a fallocate for allocation, so as I already said you can
 shift the extents to make a hole in the file and then use fallocate
 to allocate space for it and you'll get the same result. You are
 basically doing that now as well, but when the allocation fails the
 whole insert range ioct fails, however the extents are already
 shifter and there is already a holi in the file so freeing some
 space and running this ioctl again will not help you at all. While
 if you fail a fallocate, you can free some space and run it again
 without any problems. The result will be as expected.
 
 What I am arguing about is basically that your insert range ioct

RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-06-02 Thread Lukáš Czerner
On Mon, 2 Jun 2014, Namjae Jeon wrote:

 Date: Mon, 02 Jun 2014 20:52:51 +0900
 From: Namjae Jeon namjae.j...@samsung.com
 To: 'Lukáš Czerner' lczer...@redhat.com
 Cc: 'Dave Chinner' da...@fromorbit.com, 'Theodore Ts'o' ty...@mit.edu,
 'linux-ext4' linux-e...@vger.kernel.org, x...@oss.sgi.com,
 linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
 'Ashish Sangwan' a.sang...@samsung.com
 Subject: RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for
 fallocate
 
  
   Date: Sat, 31 May 2014 16:40:29 +0900
   From: Namjae Jeon namjae.j...@samsung.com
   To: 'Lukáš Czerner' lczer...@redhat.com
   Cc: 'Dave Chinner' da...@fromorbit.com, 'Theodore Ts'o' ty...@mit.edu,
   'linux-ext4' linux-e...@vger.kernel.org, x...@oss.sgi.com,
   linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
   'Ashish Sangwan' a.sang...@samsung.com
   Subject: RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for
   fallocate
  
  
 Date: Thu, 08 May 2014 19:23:19 +0900
 From: Namjae Jeon namjae.j...@samsung.com
 To: Dave Chinner da...@fromorbit.com, Theodore Ts'o ty...@mit.edu
 Cc: linux-ext4 linux-e...@vger.kernel.org, x...@oss.sgi.com,
 linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
 Ashish Sangwan a.sang...@samsung.com
 Subject: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for 
 fallocate

 In continuation of the work of making the process of non linear 
 editing of
 media files faster, we introduce here the new flag 
 FALLOC_FL_INSERT_RANGE
 for fallocate.

 This flag will work opposite to the newly added 
 FALLOC_FL_COLLAPSE_RANGE flag.
 As such, specifying FALLOC_FL_INSERT_RANGE flag will insert 
 zeroed-out space
 in between the file within the range specified by offset and len. 
 User can
 write new data in this space. e.g. ads.
 Like collapse range, currently we have the limitation that offset and 
 len should
 be block size aligned for both XFS and Ext4.

 The semantics of the flag are :
 1) It allocates new zeroed out on disk space of len bytes starting
at offset byte without overwriting any existing data. All the data 
 blocks
from offset to EOF are shifted towards right to make space for 
 inserting
new blocks
   
Hi,
   
this sounds a little bit weird to me. I understand the reason for
this, but this is effectively two operations masking as one. We
shift the existing data and then we allocate unwritten extents for
the hole we've created.
   
What would make more sense to me is to implement only the first
operation - the shift. And then let the user to allocate unwritten
extents for the hole using simple fallocate.
   
The reason is that if you succeed the first part and then fail the
second due to ENOSPC or any other reason the file will end up in
undefined state unnecessarily. Yes in your current implementation
it seems that you'll always end up with the hole in the file and the
rest is properly shifted, but that may vary from file system to file
system. Some might choose to roll back the shift, some might not.
   
If FALLOC_FL_INSERT_RANGE (or any name you wish to choose) would
just simply shift the extents then you'll get rid of this and the
only thing that user needs to do (if he chooses to) is to use
fallocate for the hole created by the shift. If it fails, then
well, he can try again without any consequences. As a bonus you get
the possibility to leave the hole in the file which might be useful
as well.
   
With current behaviour this might get very confusing very quickly.
   
What do you and others think ?
   Hi Lukas.
   Insert range inherently means inserting a real range of space into
   the file to provide guaranteed space with user in the inserted area
   so that further writes should not fail with an -ENOSPC at least.
   If insert range cannot gurantees the above semantics, It should
   return error to user space.
  
  So what will happen when there is not enough space when inserting a
  range ? And how should user proceed from there ?
 If insert range fails with an ENOSPC error, user could use collapse
 range on the same range to remove the hole.
 And after freeing more space, he can again try inserting range.
 Ofcourse, this type of guidance should be properly documented in
 manpage. When updating fallocate(2) manpage, I will keep  in mind to
 describe ENOSPC handling.

Why collapse ? The hole is already there right ? Why not just use
fallocate to allocate the space for the hole. And that's my point
actually. Why not do it this way in the first place, because this is
really counterintuitive.

  
  
   If insert range has been performed on a file, It will reserve space
   that write never fail in the inserted area,
   In case of full partition or small available size than the range

Re: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-06-02 Thread Theodore Ts'o
On Mon, Jun 02, 2014 at 03:06:13PM +0200, Lukáš Czerner wrote:
   So what will happen when there is not enough space when inserting a
   range ? And how should user proceed from there ?
  If insert range fails with an ENOSPC error, user could use collapse
  range on the same range to remove the hole.
  And after freeing more space, he can again try inserting range.
  Ofcourse, this type of guidance should be properly documented in
  manpage. When updating fallocate(2) manpage, I will keep  in mind to
  describe ENOSPC handling.
 
 Why collapse ? The hole is already there right ? Why not just use
 fallocate to allocate the space for the hole. And that's my point
 actually. Why not do it this way in the first place, because this is
 really counterintuitive.

It's worse than that.  It's possible that the reason why you got the
ENOSPC warning was because the operation to move the extents down
required allocating a block, and it was *that* block allocation which
failed.  So it's not deterministic whether or not the file's extent
mappings were modified after a ENOSPC error, and so it's not clear
whether or not a collapse_range function will undo the range that had
been inserted --- or whether it ends up deleting existing data blocks.

In generally, you really want system calls to have all-or-nothing
effects, where if the system call returns an error, the state of the
file has not been changed.  And for that reason, I agree with Lukáš
that it is really a good idea to decouple moving the blocks down, and
allocating space --- and to make sure that if there is any failure
while inserting the range, the state of the file is not modified at all.

Cheers,

- Ted
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-05-31 Thread Namjae Jeon
 
> > Date: Thu, 08 May 2014 19:23:19 +0900
> > From: Namjae Jeon 
> > To: Dave Chinner , Theodore Ts'o 
> > Cc: linux-ext4 , x...@oss.sgi.com,
> > linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
> > Ashish Sangwan 
> > Subject: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate
> >
> > In continuation of the work of making the process of non linear editing of
> > media files faster, we introduce here the new flag FALLOC_FL_INSERT_RANGE
> > for fallocate.
> >
> > This flag will work opposite to the newly added FALLOC_FL_COLLAPSE_RANGE 
> > flag.
> > As such, specifying FALLOC_FL_INSERT_RANGE flag will insert zeroed-out space
> > in between the file within the range specified by offset and len. User can
> > write new data in this space. e.g. ads.
> > Like collapse range, currently we have the limitation that offset and len 
> > should
> > be block size aligned for both XFS and Ext4.
> >
> > The semantics of the flag are :
> > 1) It allocates new zeroed out on disk space of len bytes starting
> >at offset byte without overwriting any existing data. All the data blocks
> >from offset to EOF are shifted towards right to make space for inserting
> >new blocks
> 
> Hi,
> 
> this sounds a little bit weird to me. I understand the reason for
> this, but this is effectively two operations masking as one. We
> shift the existing data and then we allocate unwritten extents for
> the hole we've created.
> 
> What would make more sense to me is to implement only the first
> operation - the shift. And then let the user to allocate unwritten
> extents for the hole using simple fallocate.
> 
> The reason is that if you succeed the first part and then fail the
> second due to ENOSPC or any other reason the file will end up in
> undefined state unnecessarily. Yes in your current implementation
> it seems that you'll always end up with the hole in the file and the
> rest is properly shifted, but that may vary from file system to file
> system. Some might choose to roll back the shift, some might not.
> 
> If FALLOC_FL_INSERT_RANGE (or any name you wish to choose) would
> just simply shift the extents then you'll get rid of this and the
> only thing that user needs to do (if he chooses to) is to use
> fallocate for the hole created by the shift. If it fails, then
> well, he can try again without any consequences. As a bonus you get
> the possibility to leave the hole in the file which might be useful
> as well.
> 
> With current behaviour this might get very confusing very quickly.
> 
> What do you and others think ?
Hi Lukas.
Insert range inherently means inserting a real range of space into
the file to provide guaranteed space with user in the inserted area
so that further writes should not fail with an -ENOSPC at least.
If insert range cannot gurantees the above semantics, It should
return error to user space.

If insert range has been performed on a file, It will reserve space
that write never fail in the inserted area,
In case of full partition or small available size than the range
user want, It seems odd just only left inserting a hole in the middle
of file and return success to user when no one can really write to
this hole.

Thanks!
> 
> Thanks!
> -LUkas
> 
> 
> > 2) It should be used exclusively. No other fallocate flag in combination.
> > 3) Offset and length supplied to fallocate should be fs block size aligned
> >in case of xfs and ext4.
> > 4) Insert range does not work for the case when offset is overlapping/beyond
> >i_size. If the user wants to allocate space at the end of file they are
> >advised to use either ftruncate(2) or fallocate(2) with mode 0.
> > 5) It increses the size of file by len bytes.
> >
> >
> > Namjae Jeon (10):
> >  fs: Add support FALLOC_FL_INSERT_RANGE for fallocate
> >  xfs: Add support FALLOC_FL_INSERT_RANGE for fallocate
> >  ext4: Add support FALLOC_FL_INSERT_RANGE for fallocate
> >  xfsprogs: xfs_io: add finsert command for insert range via fallocate
> >  xfstests: generic/027: Standard insert range tests
> >  xfstests: generic/028: Delayed allocation insert range
> >  xfstests: generic/029: Multi insert range tests
> >  xfstests: generic/030: Delayed allocation multi insert
> >  xfstests: fsstress: Add fallocate insert range operation
> >  xfstests: fsx: Add fallocate insert range operation
> >
> >

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


RE: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-05-31 Thread Namjae Jeon
 
  Date: Thu, 08 May 2014 19:23:19 +0900
  From: Namjae Jeon namjae.j...@samsung.com
  To: Dave Chinner da...@fromorbit.com, Theodore Ts'o ty...@mit.edu
  Cc: linux-ext4 linux-e...@vger.kernel.org, x...@oss.sgi.com,
  linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
  Ashish Sangwan a.sang...@samsung.com
  Subject: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate
 
  In continuation of the work of making the process of non linear editing of
  media files faster, we introduce here the new flag FALLOC_FL_INSERT_RANGE
  for fallocate.
 
  This flag will work opposite to the newly added FALLOC_FL_COLLAPSE_RANGE 
  flag.
  As such, specifying FALLOC_FL_INSERT_RANGE flag will insert zeroed-out space
  in between the file within the range specified by offset and len. User can
  write new data in this space. e.g. ads.
  Like collapse range, currently we have the limitation that offset and len 
  should
  be block size aligned for both XFS and Ext4.
 
  The semantics of the flag are :
  1) It allocates new zeroed out on disk space of len bytes starting
 at offset byte without overwriting any existing data. All the data blocks
 from offset to EOF are shifted towards right to make space for inserting
 new blocks
 
 Hi,
 
 this sounds a little bit weird to me. I understand the reason for
 this, but this is effectively two operations masking as one. We
 shift the existing data and then we allocate unwritten extents for
 the hole we've created.
 
 What would make more sense to me is to implement only the first
 operation - the shift. And then let the user to allocate unwritten
 extents for the hole using simple fallocate.
 
 The reason is that if you succeed the first part and then fail the
 second due to ENOSPC or any other reason the file will end up in
 undefined state unnecessarily. Yes in your current implementation
 it seems that you'll always end up with the hole in the file and the
 rest is properly shifted, but that may vary from file system to file
 system. Some might choose to roll back the shift, some might not.
 
 If FALLOC_FL_INSERT_RANGE (or any name you wish to choose) would
 just simply shift the extents then you'll get rid of this and the
 only thing that user needs to do (if he chooses to) is to use
 fallocate for the hole created by the shift. If it fails, then
 well, he can try again without any consequences. As a bonus you get
 the possibility to leave the hole in the file which might be useful
 as well.
 
 With current behaviour this might get very confusing very quickly.
 
 What do you and others think ?
Hi Lukas.
Insert range inherently means inserting a real range of space into
the file to provide guaranteed space with user in the inserted area
so that further writes should not fail with an -ENOSPC at least.
If insert range cannot gurantees the above semantics, It should
return error to user space.

If insert range has been performed on a file, It will reserve space
that write never fail in the inserted area,
In case of full partition or small available size than the range
user want, It seems odd just only left inserting a hole in the middle
of file and return success to user when no one can really write to
this hole.

Thanks!
 
 Thanks!
 -LUkas
 
 
  2) It should be used exclusively. No other fallocate flag in combination.
  3) Offset and length supplied to fallocate should be fs block size aligned
 in case of xfs and ext4.
  4) Insert range does not work for the case when offset is overlapping/beyond
 i_size. If the user wants to allocate space at the end of file they are
 advised to use either ftruncate(2) or fallocate(2) with mode 0.
  5) It increses the size of file by len bytes.
 
 
  Namjae Jeon (10):
   fs: Add support FALLOC_FL_INSERT_RANGE for fallocate
   xfs: Add support FALLOC_FL_INSERT_RANGE for fallocate
   ext4: Add support FALLOC_FL_INSERT_RANGE for fallocate
   xfsprogs: xfs_io: add finsert command for insert range via fallocate
   xfstests: generic/027: Standard insert range tests
   xfstests: generic/028: Delayed allocation insert range
   xfstests: generic/029: Multi insert range tests
   xfstests: generic/030: Delayed allocation multi insert
   xfstests: fsstress: Add fallocate insert range operation
   xfstests: fsx: Add fallocate insert range operation
 
 

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-05-30 Thread Lukáš Czerner
On Thu, 8 May 2014, Namjae Jeon wrote:

> Date: Thu, 08 May 2014 19:23:19 +0900
> From: Namjae Jeon 
> To: Dave Chinner , Theodore Ts'o 
> Cc: linux-ext4 , x...@oss.sgi.com,
> linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
> Ashish Sangwan 
> Subject: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate
> 
> In continuation of the work of making the process of non linear editing of
> media files faster, we introduce here the new flag FALLOC_FL_INSERT_RANGE
> for fallocate.
> 
> This flag will work opposite to the newly added FALLOC_FL_COLLAPSE_RANGE flag.
> As such, specifying FALLOC_FL_INSERT_RANGE flag will insert zeroed-out space
> in between the file within the range specified by offset and len. User can
> write new data in this space. e.g. ads.
> Like collapse range, currently we have the limitation that offset and len 
> should
> be block size aligned for both XFS and Ext4.
> 
> The semantics of the flag are :
> 1) It allocates new zeroed out on disk space of len bytes starting
>at offset byte without overwriting any existing data. All the data blocks
>from offset to EOF are shifted towards right to make space for inserting
>new blocks

Hi,

this sounds a little bit weird to me. I understand the reason for
this, but this is effectively two operations masking as one. We
shift the existing data and then we allocate unwritten extents for
the hole we've created.

What would make more sense to me is to implement only the first
operation - the shift. And then let the user to allocate unwritten
extents for the hole using simple fallocate.

The reason is that if you succeed the first part and then fail the
second due to ENOSPC or any other reason the file will end up in
undefined state unnecessarily. Yes in your current implementation
it seems that you'll always end up with the hole in the file and the
rest is properly shifted, but that may vary from file system to file
system. Some might choose to roll back the shift, some might not.

If FALLOC_FL_INSERT_RANGE (or any name you wish to choose) would
just simply shift the extents then you'll get rid of this and the
only thing that user needs to do (if he chooses to) is to use
fallocate for the hole created by the shift. If it fails, then
well, he can try again without any consequences. As a bonus you get
the possibility to leave the hole in the file which might be useful
as well.

With current behaviour this might get very confusing very quickly.

What do you and others think ?

Thanks!
-LUkas


> 2) It should be used exclusively. No other fallocate flag in combination.
> 3) Offset and length supplied to fallocate should be fs block size aligned
>in case of xfs and ext4.
> 4) Insert range does not work for the case when offset is overlapping/beyond
>i_size. If the user wants to allocate space at the end of file they are
>advised to use either ftruncate(2) or fallocate(2) with mode 0.
> 5) It increses the size of file by len bytes.
> 
> 
> Namjae Jeon (10):
>  fs: Add support FALLOC_FL_INSERT_RANGE for fallocate
>  xfs: Add support FALLOC_FL_INSERT_RANGE for fallocate
>  ext4: Add support FALLOC_FL_INSERT_RANGE for fallocate
>  xfsprogs: xfs_io: add finsert command for insert range via fallocate
>  xfstests: generic/027: Standard insert range tests
>  xfstests: generic/028: Delayed allocation insert range
>  xfstests: generic/029: Multi insert range tests
>  xfstests: generic/030: Delayed allocation multi insert
>  xfstests: fsstress: Add fallocate insert range operation
>  xfstests: fsx: Add fallocate insert range operation
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-05-30 Thread Lukáš Czerner
On Thu, 8 May 2014, Namjae Jeon wrote:

 Date: Thu, 08 May 2014 19:23:19 +0900
 From: Namjae Jeon namjae.j...@samsung.com
 To: Dave Chinner da...@fromorbit.com, Theodore Ts'o ty...@mit.edu
 Cc: linux-ext4 linux-e...@vger.kernel.org, x...@oss.sgi.com,
 linux-fsde...@vger.kernel.org, linux-kernel@vger.kernel.org,
 Ashish Sangwan a.sang...@samsung.com
 Subject: [PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate
 
 In continuation of the work of making the process of non linear editing of
 media files faster, we introduce here the new flag FALLOC_FL_INSERT_RANGE
 for fallocate.
 
 This flag will work opposite to the newly added FALLOC_FL_COLLAPSE_RANGE flag.
 As such, specifying FALLOC_FL_INSERT_RANGE flag will insert zeroed-out space
 in between the file within the range specified by offset and len. User can
 write new data in this space. e.g. ads.
 Like collapse range, currently we have the limitation that offset and len 
 should
 be block size aligned for both XFS and Ext4.
 
 The semantics of the flag are :
 1) It allocates new zeroed out on disk space of len bytes starting
at offset byte without overwriting any existing data. All the data blocks
from offset to EOF are shifted towards right to make space for inserting
new blocks

Hi,

this sounds a little bit weird to me. I understand the reason for
this, but this is effectively two operations masking as one. We
shift the existing data and then we allocate unwritten extents for
the hole we've created.

What would make more sense to me is to implement only the first
operation - the shift. And then let the user to allocate unwritten
extents for the hole using simple fallocate.

The reason is that if you succeed the first part and then fail the
second due to ENOSPC or any other reason the file will end up in
undefined state unnecessarily. Yes in your current implementation
it seems that you'll always end up with the hole in the file and the
rest is properly shifted, but that may vary from file system to file
system. Some might choose to roll back the shift, some might not.

If FALLOC_FL_INSERT_RANGE (or any name you wish to choose) would
just simply shift the extents then you'll get rid of this and the
only thing that user needs to do (if he chooses to) is to use
fallocate for the hole created by the shift. If it fails, then
well, he can try again without any consequences. As a bonus you get
the possibility to leave the hole in the file which might be useful
as well.

With current behaviour this might get very confusing very quickly.

What do you and others think ?

Thanks!
-LUkas


 2) It should be used exclusively. No other fallocate flag in combination.
 3) Offset and length supplied to fallocate should be fs block size aligned
in case of xfs and ext4.
 4) Insert range does not work for the case when offset is overlapping/beyond
i_size. If the user wants to allocate space at the end of file they are
advised to use either ftruncate(2) or fallocate(2) with mode 0.
 5) It increses the size of file by len bytes.
 
 
 Namjae Jeon (10):
  fs: Add support FALLOC_FL_INSERT_RANGE for fallocate
  xfs: Add support FALLOC_FL_INSERT_RANGE for fallocate
  ext4: Add support FALLOC_FL_INSERT_RANGE for fallocate
  xfsprogs: xfs_io: add finsert command for insert range via fallocate
  xfstests: generic/027: Standard insert range tests
  xfstests: generic/028: Delayed allocation insert range
  xfstests: generic/029: Multi insert range tests
  xfstests: generic/030: Delayed allocation multi insert
  xfstests: fsstress: Add fallocate insert range operation
  xfstests: fsx: Add fallocate insert range operation
 
 
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-05-08 Thread Namjae Jeon
In continuation of the work of making the process of non linear editing of
media files faster, we introduce here the new flag FALLOC_FL_INSERT_RANGE
for fallocate.

This flag will work opposite to the newly added FALLOC_FL_COLLAPSE_RANGE flag.
As such, specifying FALLOC_FL_INSERT_RANGE flag will insert zeroed-out space
in between the file within the range specified by offset and len. User can
write new data in this space. e.g. ads.
Like collapse range, currently we have the limitation that offset and len should
be block size aligned for both XFS and Ext4.

The semantics of the flag are :
1) It allocates new zeroed out on disk space of len bytes starting
   at offset byte without overwriting any existing data. All the data blocks
   from offset to EOF are shifted towards right to make space for inserting
   new blocks
2) It should be used exclusively. No other fallocate flag in combination.
3) Offset and length supplied to fallocate should be fs block size aligned
   in case of xfs and ext4.
4) Insert range does not work for the case when offset is overlapping/beyond
   i_size. If the user wants to allocate space at the end of file they are
   advised to use either ftruncate(2) or fallocate(2) with mode 0.
5) It increses the size of file by len bytes.


Namjae Jeon (10):
 fs: Add support FALLOC_FL_INSERT_RANGE for fallocate
 xfs: Add support FALLOC_FL_INSERT_RANGE for fallocate
 ext4: Add support FALLOC_FL_INSERT_RANGE for fallocate
 xfsprogs: xfs_io: add finsert command for insert range via fallocate
 xfstests: generic/027: Standard insert range tests
 xfstests: generic/028: Delayed allocation insert range
 xfstests: generic/029: Multi insert range tests
 xfstests: generic/030: Delayed allocation multi insert
 xfstests: fsstress: Add fallocate insert range operation
 xfstests: fsx: Add fallocate insert range operation

-- 
1.7.11-rc0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[PATCH v2 0/10] fs: Introduce FALLOC_FL_INSERT_RANGE for fallocate

2014-05-08 Thread Namjae Jeon
In continuation of the work of making the process of non linear editing of
media files faster, we introduce here the new flag FALLOC_FL_INSERT_RANGE
for fallocate.

This flag will work opposite to the newly added FALLOC_FL_COLLAPSE_RANGE flag.
As such, specifying FALLOC_FL_INSERT_RANGE flag will insert zeroed-out space
in between the file within the range specified by offset and len. User can
write new data in this space. e.g. ads.
Like collapse range, currently we have the limitation that offset and len should
be block size aligned for both XFS and Ext4.

The semantics of the flag are :
1) It allocates new zeroed out on disk space of len bytes starting
   at offset byte without overwriting any existing data. All the data blocks
   from offset to EOF are shifted towards right to make space for inserting
   new blocks
2) It should be used exclusively. No other fallocate flag in combination.
3) Offset and length supplied to fallocate should be fs block size aligned
   in case of xfs and ext4.
4) Insert range does not work for the case when offset is overlapping/beyond
   i_size. If the user wants to allocate space at the end of file they are
   advised to use either ftruncate(2) or fallocate(2) with mode 0.
5) It increses the size of file by len bytes.


Namjae Jeon (10):
 fs: Add support FALLOC_FL_INSERT_RANGE for fallocate
 xfs: Add support FALLOC_FL_INSERT_RANGE for fallocate
 ext4: Add support FALLOC_FL_INSERT_RANGE for fallocate
 xfsprogs: xfs_io: add finsert command for insert range via fallocate
 xfstests: generic/027: Standard insert range tests
 xfstests: generic/028: Delayed allocation insert range
 xfstests: generic/029: Multi insert range tests
 xfstests: generic/030: Delayed allocation multi insert
 xfstests: fsstress: Add fallocate insert range operation
 xfstests: fsx: Add fallocate insert range operation

-- 
1.7.11-rc0

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/