Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-31 Thread Jim Meyering
Pádraig Brady wrote: ... I'm going to use this 2.6.38 check to only enable FIEMAP_FLAG_SYNC before Linux kernel 2.6.38. It's always worth avoiding sync if possible. Proposed patch attached. I'll submit my 3 outstanding fiemap patches tomorrow. ... Subject: [PATCH] copy: with fiemap copy,

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-30 Thread Pádraig Brady
On 28/03/11 23:06, Pádraig Brady wrote: On 18/03/11 16:04, Pádraig Brady wrote: On 18/03/11 13:48, Chris Mason wrote: Excerpts from Pádraig Brady's message of 2011-03-18 09:19:44 -0400: On 18/03/11 12:04, Chris Mason wrote: Excerpts from Jim Meyering's message of 2011-03-18 07:52:43 -0400:

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-30 Thread Chris Mason
Excerpts from Pádraig Brady's message of 2011-03-30 07:02:44 -0400: On 28/03/11 23:06, Pádraig Brady wrote: On 18/03/11 16:04, Pádraig Brady wrote: On 18/03/11 13:48, Chris Mason wrote: Excerpts from Pádraig Brady's message of 2011-03-18 09:19:44 -0400: On 18/03/11 12:04, Chris Mason

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-30 Thread Pádraig Brady
On 17/03/11 07:24, Mike Frysinger wrote: On Wednesday, March 16, 2011 19:55:56 Pádraig Brady wrote: On 16/03/11 19:18, Mike Frysinger wrote: well, in the bug report i was working with, we were seeing data loss. i wonder if it'd be possible to detect the fs/kernel version and error out when

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-28 Thread Pádraig Brady
On 18/03/11 16:04, Pádraig Brady wrote: On 18/03/11 13:48, Chris Mason wrote: Excerpts from Pádraig Brady's message of 2011-03-18 09:19:44 -0400: On 18/03/11 12:04, Chris Mason wrote: Excerpts from Jim Meyering's message of 2011-03-18 07:52:43 -0400: Pádraig Brady wrote: So am I right in

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-22 Thread Pádraig Brady
On 18/03/11 16:32, Eric Sandeen wrote: On 3/18/11 8:48 AM, Chris Mason wrote: ... Btrfs before 2.6.38 may have real trouble though, even with the sync. We were returning overlapping ranges to you, so the destination would end up bigger than the original. This could be fixed in cp by

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-19 Thread Pádraig Brady
On 19/03/11 12:07, Jim Meyering wrote: Pádraig Brady wrote: On 18/03/11 13:19, Pádraig Brady wrote: Bah humbug. Looks like there is no such issue. This actually seems like an issue in a coreutils test script, which made it seem like the SYNC done by `filefrag -vs` was ineffective. Proposed

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Mike Frysinger
On Thursday, March 17, 2011 19:00:26 Pádraig Brady wrote: On 17/03/11 19:29, Mike Frysinger wrote: On Thursday, March 17, 2011 05:32:33 Pádraig Brady wrote: Also note the make_holes heuristic variable is no longer used in extent_copy due to this patch which came after the 8.10

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Pádraig Brady
On 17/03/11 23:00, Pádraig Brady wrote: On 17/03/11 19:29, Mike Frysinger wrote: On Thursday, March 17, 2011 05:32:33 Pádraig Brady wrote: This is the kind of patch appropriate for a distro, as they may or may not have a fix backported to their kernel. Gentoo, being a source distribution,

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Jim Meyering
Pádraig Brady wrote: On 17/03/11 23:00, Pádraig Brady wrote: On 17/03/11 19:29, Mike Frysinger wrote: On Thursday, March 17, 2011 05:32:33 Pádraig Brady wrote: This is the kind of patch appropriate for a distro, as they may or may not have a fix backported to their kernel. Gentoo, being a

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Chris Mason
Excerpts from Jim Meyering's message of 2011-03-18 07:52:43 -0400: Pádraig Brady wrote: On 17/03/11 23:00, Pádraig Brady wrote: On 17/03/11 19:29, Mike Frysinger wrote: On Thursday, March 17, 2011 05:32:33 Pádraig Brady wrote: This is the kind of patch appropriate for a distro, as they

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Chris Mason
Excerpts from Pádraig Brady's message of 2011-03-18 09:19:44 -0400: On 18/03/11 12:04, Chris Mason wrote: Excerpts from Jim Meyering's message of 2011-03-18 07:52:43 -0400: Pádraig Brady wrote: I also now remember this discussion: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=6131 where

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Jim Meyering
Pádraig Brady wrote: ... OK thanks for the info. So: XFS may miss some extents for sparse files before 2.6.36 BTRFS and EXT4 need a sync before fiemap() before 2.6.38 BTRFS can return overlapping extents before 2.6.38 It seems like we should at least detect overlapping extents in our:

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Eric Sandeen
On 3/18/11 8:48 AM, Chris Mason wrote: ... Btrfs before 2.6.38 may have real trouble though, even with the sync. We were returning overlapping ranges to you, so the destination would end up bigger than the original. This could be fixed in cp by making sure to never seek backwards based on

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Jim Meyering
Eric Sandeen wrote: ... The testcase in coreutils should definitely remove the fdatasync hack, as that just makes the test pass without worrying about things in the real world. :) A better test suite might be to generate random holey / delalloc files and copy those around. We should write

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Pádraig Brady
On 18/03/11 13:19, Pádraig Brady wrote: Bah humbug. Looks like there is no such issue. This actually seems like an issue in a coreutils test script, which made it seem like the SYNC done by `filefrag -vs` was ineffective. Proposed fix attached. My perl is still weak, so I won't apply until

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-17 Thread Mike Frysinger
On Wednesday, March 16, 2011 19:55:56 Pádraig Brady wrote: On 16/03/11 19:18, Mike Frysinger wrote: well, in the bug report i was working with, we were seeing data loss. i wonder if it'd be possible to detect the fs/kernel version and error out when versions are found that are known to be

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-17 Thread Pádraig Brady
On 17/03/11 07:24, Mike Frysinger wrote: On Wednesday, March 16, 2011 19:55:56 Pádraig Brady wrote: On 16/03/11 19:18, Mike Frysinger wrote: well, in the bug report i was working with, we were seeing data loss. i wonder if it'd be possible to detect the fs/kernel version and error out when

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-17 Thread Mike Frysinger
On Thursday, March 17, 2011 05:32:33 Pádraig Brady wrote: This is the kind of patch appropriate for a distro, as they may or may not have a fix backported to their kernel. Gentoo, being a source distribution, has no kernel requirement. people are free to run on pretty much anything. we dont

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-17 Thread Pádraig Brady
On 17/03/11 19:29, Mike Frysinger wrote: On Thursday, March 17, 2011 05:32:33 Pádraig Brady wrote: This is the kind of patch appropriate for a distro, as they may or may not have a fix backported to their kernel. Gentoo, being a source distribution, has no kernel requirement. people are

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-16 Thread Pádraig Brady
On 16/03/11 19:18, Mike Frysinger wrote: On Wednesday, March 16, 2011 11:26:40 Pádraig Brady wrote: On 14/02/11 06:05, Jim Meyering wrote: Pádraig Brady wrote: For my own reference, I can probably skip performance tests on (older) btrfs by checking `filefrag` output. Also in `cp`, if we see

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-02-13 Thread Pádraig Brady
Unfortunately, after checking BTRFS I see that fiemap behaves differently to EXT4. IMHO the EXT4 operation seems correct, and gives full info about the structure of a file, which cp for example can use to efficiently and accurately reproduce the structure at the destination. On EXT4 (on

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-02-13 Thread Jim Meyering
Pádraig Brady wrote: Unfortunately, after checking BTRFS I see that fiemap behaves differently to EXT4. IMHO the EXT4 operation seems correct, and gives full info about the structure of a file, which cp for example can use to efficiently and accurately reproduce the structure at the

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-02-10 Thread Pádraig Brady
On 09/02/11 23:35, Pádraig Brady wrote: Subject: [PATCH] copy: adjust fiemap handling of sparse files Don't depend on heuristics to detect sparse files if fiemap is available. Also don't introduce new holes unless --sparse=always has been specified. * src/copy.c (extent_copy): Pass

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-02-10 Thread Pádraig Brady
On 10/02/11 10:53, Jim Meyering wrote: While using --sparse=always is a lot less common, it looks like there's room for improvement there, too. I.e., we should be able to eliminate all of these lseek calls on the output descriptor there, too: $ strace -e lseek cp --sparse=always

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-02-09 Thread Jim Meyering
detection heuristic fail BTW? There are probably a few, but none that I know of. Anwyay, we don't need the heuristic with fiemap, so I changed accordingly in the attached. ... Subject: [PATCH] copy: adjust fiemap handling of sparse files Don't depend on heuristics to detect sparse files

[PATCH] copy: adjust fiemap handling of sparse files

2011-02-08 Thread Pádraig Brady
fiemap handling of sparse files Don't depend on heuristics to detect sparse files if fiemap is available. Also don't introduce new holes unless --sparse=always has been specified. * src/copy.c (extent_copy): Pass the user specified sparse mode, and handle as described above. Also a redundant lseek