Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-25 Thread Eric Blake
On 04/24/2011 11:49 AM, Jamie Lokier wrote: My problem with FIND_* is that we are messing with the well understood semantics of lseek(). fcntl() looks a better fit for FIND_HOLE/DATA anyway. With fcntl(), it would have to be something like: off_t offset = start; if (fcntl (fd, F_FIND_HOLE,

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-25 Thread Jamie Lokier
Eric Blake wrote: On 04/24/2011 11:49 AM, Jamie Lokier wrote: My problem with FIND_* is that we are messing with the well understood semantics of lseek(). fcntl() looks a better fit for FIND_HOLE/DATA anyway. With fcntl(), it would have to be something like: off_t offset = start;

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-25 Thread Nick Bowler
Hi Eric, On 2011-04-22 07:06 -0600, Eric Blake wrote: I've created a request to standardize SEEK_HOLE and SEEK_DATA in the next revision of POSIX; comments are welcome to make sure that everyone is happy with wording: http://austingroupbugs.net/view.php?id=415 Reading through your proposal,

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-25 Thread Eric Blake
On 04/25/2011 09:02 AM, Nick Bowler wrote: Hi Nick, * File A (sparse file created by lseek/write beyond end-of-file): data | hole 0 | data || hole 1 (virtual) * File B (sparse file created by truncate beyond end-of-file): data | hole 0 || hole 1 (virtual) Excluding the error

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-24 Thread Jamie Lokier
Sunil Mushran wrote: On 04/22/2011 04:50 AM, Eric Blake wrote: That blog also mentioned the useful idea of adding FIND_HOLE and FIND_DATA, not implemented in Solaris, but which could easily be provided as additional lseek constants in Linux to locate the start of the next chunk without

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-24 Thread Valdis . Kletnieks
On Thu, 21 Apr 2011 15:42:33 EDT, Josef Bacik said: -SEEK_HOLE: this moves the file pos to the nearest hole in the file from the given position. Do we want the semantic to be the nearest hole? Or did we really want the next hole? Loops like a bullet loaded in the chamber and pointed at the

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Markus Trippelsdorf
On 2011.04.22 at 00:50 -0400, Christoph Hellwig wrote: [Eric: please don't drop the Cc list, thanks!] On Thu, Apr 21, 2011 at 09:22:55PM -0400, Josef Bacik wrote: since all files have a virtual hole at the end, but leaves the position unchanged). ??I'd have to write a test program on

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Eric Blake
On 04/22/2011 05:28 AM, Markus Trippelsdorf wrote: On 2011.04.22 at 00:50 -0400, Christoph Hellwig wrote: [Eric: please don't drop the Cc list, thanks!] That's the fault of gmane. My apologies for not being subscribed in the first place, and for gmane's refusal to cc all. Now that I'm on the

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Eric Blake
On 04/22/2011 05:28 AM, Markus Trippelsdorf wrote: I would be surprised if the bug is around for such a long time, but otherwise I concur. It's a bug. Let me quote what Jeff Bonwick wrote on his blog: »I'm not sure where you got the impression that either SEEK_HOLE or SEEK_DATA doesn't

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Sunil Mushran
On 04/22/2011 04:50 AM, Eric Blake wrote: That blog also mentioned the useful idea of adding FIND_HOLE and FIND_DATA, not implemented in Solaris, but which could easily be provided as additional lseek constants in Linux to locate the start of the next chunk without repositioning and which could

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Eric Blake
On 04/22/2011 10:28 AM, Sunil Mushran wrote: On 04/22/2011 04:50 AM, Eric Blake wrote: That blog also mentioned the useful idea of adding FIND_HOLE and FIND_DATA, not implemented in Solaris, but which could easily be provided as additional lseek constants in Linux to locate the start of the

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Sunil Mushran
On 04/22/2011 09:40 AM, Eric Blake wrote: On 04/22/2011 10:28 AM, Sunil Mushran wrote: while(1) { read(block); if (block_all_zeroes) lseek(SEEK_DATA); } What's wrong with the above? If this is the case, even SEEK_HOLE is not needed but should be added as it is already in

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Eric Blake
On 04/22/2011 10:57 AM, Sunil Mushran wrote: On 04/22/2011 09:40 AM, Eric Blake wrote: On 04/22/2011 10:28 AM, Sunil Mushran wrote: while(1) { read(block); if (block_all_zeroes) lseek(SEEK_DATA); } What's wrong with the above? If this is the case, even SEEK_HOLE is not

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Sunil Mushran
On 04/22/2011 10:03 AM, Eric Blake wrote: cp can read whatever blocksize it chooses. If that block contains zero, it would signal cp that maybe it should SEEK_DATA and skip reading all those blocks. That's all. We are not trying to achieve perfection. We are just trying to reduce cpu waste. If

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Andreas Dilger
On 2011-04-22, at 11:08 AM, Sunil Mushran sunil.mush...@oracle.com wrote: On 04/22/2011 10:03 AM, Eric Blake wrote: cp can read whatever blocksize it chooses. If that block contains zero, it would signal cp that maybe it should SEEK_DATA and skip reading all those blocks. That's all. We are

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Jonathan Nieder
Hi, Josef Bacik wrote: This just gets us ready to support the SEEK_HOLE and SEEK_DATA flags. Turns out using fiemap in things like cp cause more problems than it solves, so lets try and give userspace an interface that doesn't suck. So we have That's easy to believe, but could you

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Sunil Mushran
On 04/22/2011 01:10 PM, Jonathan Nieder wrote: Hi, Josef Bacik wrote: This just gets us ready to support the SEEK_HOLE and SEEK_DATA flags. Turns out using fiemap in things like cp cause more problems than it solves, so lets try and give userspace an interface that doesn't suck. So we have

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-22 Thread Sunil Mushran
On 04/22/2011 11:06 AM, Andreas Dilger wrote: Sure, there are lots of wasted syscalls, but in this case the cost of doing extra SEEK_DATA and SEEK_HOLE operations may be fairly costly. This involves scanning the whole disk layout, possibly over a network, which might need tens or hundreds of

[PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-21 Thread Josef Bacik
This just gets us ready to support the SEEK_HOLE and SEEK_DATA flags. Turns out using fiemap in things like cp cause more problems than it solves, so lets try and give userspace an interface that doesn't suck. So we have -SEEK_HOLE: this moves the file pos to the nearest hole in the file from

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-21 Thread Theodore Tso
On Apr 21, 2011, at 3:42 PM, Josef Bacik wrote: + case SEEK_DATA: + case SEEK_HOLE: + return -EINVAL; } Maybe we should be returning EOPNOTSUPP? -- Ted -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-21 Thread Sunil Mushran
On 04/21/2011 01:45 PM, Theodore Tso wrote: On Apr 21, 2011, at 3:42 PM, Josef Bacik wrote: + case SEEK_DATA: + case SEEK_HOLE: + return -EINVAL; } Maybe we should be returning EOPNOTSUPP? -- Ted For SEEK_HOLE yes. But we could let the default SEEK_DATA be

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-21 Thread Matthew Wilcox
On Thu, Apr 21, 2011 at 04:45:54PM -0400, Theodore Tso wrote: On Apr 21, 2011, at 3:42 PM, Josef Bacik wrote: + case SEEK_DATA: + case SEEK_HOLE: + return -EINVAL; } Maybe we should be returning EOPNOTSUPP? That's definitely wrong ... -ENOSYS might be better! --

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-21 Thread Christoph Hellwig
We'll also need: - a patch to lseek(2) to document the new flags - some testcases for xfstests, specially dealing with things that were problematic in FIEMAP, e.g. data in delalloc extents, making sure stuff in unwrittent extents partially converted actually gets copied, etc. -- To

Re: [PATCH 1/2] fs: add SEEK_HOLE and SEEK_DATA flags

2011-04-21 Thread Christoph Hellwig
[Eric: please don't drop the Cc list, thanks!] On Thu, Apr 21, 2011 at 09:22:55PM -0400, Josef Bacik wrote: since all files have a virtual hole at the end, but leaves the position unchanged). ??I'd have to write a test program on Solaris to see whether that definition is actually true,