[coreutils] [PATCH] maint: remove a redundant sort parameter from a test

2010-05-24 Thread Pádraig Brady
I just pushed this trivial change in case others used this test as a template for other scripts. diff --git a/tests/misc/sort-month b/tests/misc/sort-month index aee5215..8a8e4fa 100755 --- a/tests/misc/sort-month +++ b/tests/misc/sort-month @@ -30,7 +30,7 @@ locale --version /dev/null 21 || for

Re: [coreutils] considering uncrustify for automatic indentation: prepare

2010-05-31 Thread Pádraig Brady
On 31/05/10 10:12, Jim Meyering wrote: [PATCH 1/4] stat: use gnulib's alignof module [PATCH 2/4] maint: correct indentation of case_GETOPT_* macro uses [PATCH 3/4] maint: replace each for (;;) with while (true) [PATCH 4/4] maint: make spacing around = consistent, even in IF_LINT

Re: [coreutils] How to check uniqueness on one or multiple columns in a table and print the duplicated rows?

2010-06-17 Thread Pádraig Brady
On 17/06/10 21:53, Peng Yu wrote: Hi, Uniq doesn't have an option to check the uniqueness for a given column (although it can exclude the first a few columns). It will need 'cut' to check uniqueness on a give number of columns and print the duplicated rows. This is not convenient. I'm

bug#6627: Build failure when pthread.h isn’t available

2010-07-14 Thread Pádraig Brady
From: ludo at gnu.org (Ludovic Courtès) Date: Tue, 13 Jul 2010 18:16:30 +0200 Hello, Coreutils fails to build when pthread.h isn’t available: Thanks Ludo, I thought the fixed version would have built before you noticed. I'll CC you the next time.

Re: [coreutils] [PATCH] fadvise: new module providing a simpler interface to posix_fadvise

2010-07-20 Thread Pádraig Brady
To get the same benefits noticed with `sort` http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=dae35bac I'm applying the same hint to all appropriate utils in the attached. $ grep -l SEQUENTIAL *.c base64.c cat.c cksum.c comm.c cut.c expand.c fmt.c fold.c join.c md5sum.c nl.c paste.c

Re: [coreutils] [PATCH] sort: fix bug with EOF at buffer refill

2010-07-27 Thread Pádraig Brady
On 27/07/10 04:53, Paul Eggert wrote: * src/sort.c (fillbuf): Don't append eol unless the line is nonempty. This fixes a bug that was partly but not completely fixed by the aadc67dfdb47f28bb8d1fa5e0fe0f52e2a8c51bf commit (dated July 15). * tests/misc/sort (realloc-buf-2): New test, which

Re: [coreutils] [PATCH] sort: -h now handles comparisons such as 6000K vs 5M and 5MiB vs 5MB

2010-07-30 Thread Pádraig Brady
On 30/07/10 08:59, Paul Eggert wrote: I discovered a race condition in coreutils sort, when running multithreaded, in the check_mixed_SI_IEC function. Rather than add interlocks, I decided to address the underlying problem by fixing coreutils to do the right thing with mixed SI and IEC

Re: [coreutils] [PATCH] tests: silence 'make syntax-check'

2010-08-11 Thread Pádraig Brady
41fe1906319d008af5c5f1d8fbeeadf45cde5333 Author: Pádraig Brady p...@draigbrady.com Date: Wed Aug 11 10:49:22 2010 +0100 maint: exclude all tests from the set_program_name syntax-check * .x-sc_program_name: Exclude all current and future c files in gl/tests from this check * gl/tests/test-di-set.c

Re: [coreutils] [PATCH] sort: -R now uses less memory on long lines with internal NULs

2010-08-11 Thread Pádraig Brady
On 05/08/10 00:12, Paul Eggert wrote: @@ -2038,41 +2027,117 @@ static int compare_random (char *restrict texta, size_t lena, char *restrict textb, size_t lenb) { - int diff; + uint32_t dig[2][MD5_DIGEST_SIZE / sizeof (uint32_t)]; I know this is just moved code but it

Re: [coreutils] [PATCH] Fix typo in texinfo annotation.

2010-08-14 Thread Pádraig Brady
On 14/08/10 06:17, Ralf Wildenhues wrote: Hi coreutils maintainers, this trivial patch avoids this warning from CVS texinfo's makeinfo: coreutils.texi:3807: warning: `.' or `,' must follow @xref, not `)'. coreutils.texi:3807: warning: for cross-references in parentheses, use @pxref.

bug#7042: df --help does not show `-m' option

2010-09-16 Thread Pádraig Brady
On 16/09/10 12:59, Petr Pisar wrote: Hello, I found `df' utility from coreutils-8.5 does not describe `-m' option that is mentioned in info page and the program accepts it. -- Petr -m was deprecated in 2001 http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=d1772031 It is not

[coreutils] [PATCH] tests: fix a printf portability issue

2010-09-17 Thread Pádraig Brady
not all printf commands support \xhh diff --git a/tests/misc/sort-debug-keys b/tests/misc/sort-debug-keys index 57a52a6..4e8beff 100755 --- a/tests/misc/sort-debug-keys +++ b/tests/misc/sort-debug-keys @@ -275,7 +275,7 @@ printf 2.,,3\n2.4\n | sort -s -k1n --debug printf 2,,3\n2.4\n | sort -s

Re: [coreutils] [PATCH] tests: fix a printf portability issue

2010-09-19 Thread Pádraig Brady
On 19/09/10 07:50, Jim Meyering wrote: Pádraig Brady wrote: not all printf commands support \xhh diff --git a/tests/misc/sort-debug-keys b/tests/misc/sort-debug-keys index 57a52a6..4e8beff 100755 --- a/tests/misc/sort-debug-keys +++ b/tests/misc/sort-debug-keys @@ -275,7 +275,7 @@ printf 2

[coreutils] Re: [PATCH] join: support multi-byte character encodings

2010-09-20 Thread Pádraig Brady
On 20/09/10 01:55, Bruno Haible wrote: [CCing bug-libunistring. This is a reply to http://lists.gnu.org/archive/html/coreutils/2010-09/msg00032.html.] Hi Pádraig, I was doing some performance analysis of the above patch and noticed it performed very well usually but not when

[coreutils] tr: case mapping anomaly

2010-09-24 Thread Pádraig Brady
I was just looking at a bug reported to fedora there where this abort()s $ LC_ALL=en_US tr '[:upper:] ' '[:lower:]' It stems from the fact that there are 56 upper and 59 lower chars in iso-8859-1. But I also noticed an anomaly which would affect the fix, which is, that [:upper:] and [:lower:]

Re: [coreutils] tr: case mapping anomaly

2010-09-25 Thread Pádraig Brady
On 25/09/10 00:22, Eric Blake wrote: On 09/24/2010 04:47 PM, Pádraig Brady wrote: I was just looking at a bug reported to fedora there where this abort()s $ LC_ALL=en_US tr '[:upper:] ' '[:lower:]' Behavior is already unspecified by POSIX when string1 is longer than string2. But given

Re: [coreutils] tr: case mapping anomaly

2010-09-29 Thread Pádraig Brady
On 29/09/10 07:45, Jim Meyering wrote: Pádraig Brady wrote: +# characters in some locales, triggered abort()s and invalid behavior +if test $(LC_ALL=en_US.ISO-8859-1 locale charmap 2/dev/null) = ISO-8859-1; +then + ( No big deal, but why run this in a subshell? It doesn't seem

Re: [coreutils] [PATCH 1/2] stat: support printing birthtime

2010-09-30 Thread Pádraig Brady
On 01/10/10 00:32, Eric Blake wrote: * src/stat.c (print_stat): New %w and %W formats. (do_stat): Include %w in verbose format. (usage): Document them. * doc/coreutils.texi (stat invocation): Likewise. * NEWS: Likewise. Suggested by Andre Osku Schmidt. --- I've tested that this works on

Re: [coreutils] [PATCH 2/2] stat: print timestamps to full resolution

2010-09-30 Thread Pádraig Brady
On 01/10/10 00:32, Eric Blake wrote: * src/stat.c (epoch_time): New function. (print_stat): Use it for %[WXYZ]. * NEWS: Document this. * tests/touch/60-seconds: Adjust test to match. --- It bugs me that %x has more information than %X in 'stat --format', especially, since we don't support

Re: [coreutils] [PATCH] maint: suppress a bogus used-uninitialized warning in tr.c

2010-10-01 Thread Pádraig Brady
On 01/10/10 10:24, Jim Meyering wrote: Without this, I'd get cc1: warnings being treated as errors tr.c: In function 'main': tr.c:1400: warning: 'char_to_repeat' may be used uninitialized in this function There must have been some limitation with my compiler gcc (GCC) 4.4.1

Re: [coreutils] stat and SELinux context

2010-10-01 Thread Pádraig Brady
On 01/10/10 16:49, Eric Blake wrote: In working on the birthtime stat support, I noticed that we documented that -Z has been removed, on the grounds that you can get at it with --format=%C (although NEWS didn't mention the replacement). Would it make sense to patch the default stat output

Re: [coreutils] [PATCHv2] stat: print SELinux context when available

2010-10-05 Thread Pádraig Brady
The attached fixes a logic inversion issue, and doesn't print context in file system mode which is confusing to me at least. There is also the argument not to print context in terse mode at all, as we'll have issues with outputting other extended attributes like capabilities and ACLs etc?

Re: [coreutils] [PATCHv2] stat: print SELinux context when available

2010-10-05 Thread Pádraig Brady
On 05/10/10 10:03, Jim Meyering wrote: - if (0 is_selinux_enabled ()) + if (is_selinux_enabled ()) ... - if (0 is_selinux_enabled ()) + if (is_selinux_enabled ()) However, the changes to the use of is_selinux_enabled look wrong, since that function

Re: [coreutils] join feature: auto-format

2010-10-06 Thread Pádraig Brady
On 06/10/10 21:41, Assaf Gordon wrote: Hello, I'd like to (re)suggest a feature for the join program - the ability to automatically build an output format line (similar but easier than using -o). I've previously mentioned it here (but got no favorable responses):

Re: [coreutils] join feature: auto-format

2010-10-07 Thread Pádraig Brady
On 07/10/10 01:03, Pádraig Brady wrote: On 06/10/10 21:41, Assaf Gordon wrote: Hello, I'd like to (re)suggest a feature for the join program - the ability to automatically build an output format line (similar but easier than using -o). I've previously mentioned it here (but got

[coreutils] [PATCH] split: fix reporting of read errors

2010-10-07 Thread Pádraig Brady
Ok to push this for the imminent release? cheers, Pádraig. From ac6837d1d76e01126b98fc97df6226fc26ea365f Mon Sep 17 00:00:00 2001 From: =?utf-8?q?P=C3=A1draig=20Brady?= p...@draigbrady.com Date: Thu, 7 Oct 2010 13:12:36 +0100 Subject: [PATCH] split: fix reporting of read errors The bug was

Re: [coreutils] join feature: auto-format

2010-10-07 Thread Pádraig Brady
On 07/10/10 18:43, Assaf Gordon wrote: Pádraig Brady wrote, On 10/07/2010 06:22 AM: On 07/10/10 01:03, Pádraig Brady wrote: On 06/10/10 21:41, Assaf Gordon wrote: The --auto-format feature simply builds the -o format line automatically, based on the number of columns from both input files

Re: [coreutils] [PATCH] split: fix reporting of read errors

2010-10-07 Thread Pádraig Brady
On 07/10/10 13:38, Jim Meyering wrote: Pádraig Brady wrote: Ok to push this for the imminent release? Subject: [PATCH] split: fix reporting of read errors The bug was introduced with commit 23f6d41f, 19-02-2003. * src/split.c (bytes_split, lines_split, line_bytes_split): Correctly check

Re: [Coreutils] Sort enhancement request: 1 1K 1M 1G etc.

2010-10-08 Thread Pádraig Brady
On 08/10/10 04:16, Philip Ganchev wrote: It would be useful if sort could understand numeric abbreviations, for example 1k = 1,000 and 1K = 1024. This need arises for me very often when I want a sorted list of human-readable file sizes like the output of du -h. Currently, to use sort you have

Re: [coreutils] new snapshot available: coreutils-8.5.188-9af44

2010-10-11 Thread Pádraig Brady
On 10/10/10 20:57, Jim Meyering wrote: Here is a snapshot of the latest coreutils development sources. Please build it and run make check on any systems you can, and report any problems to bug-coreut...@gnu.org. I expect to make a stable release in two or three days. I just noticed

[coreutils] [PATCH] tail: fix checking of currently unavailable directories

2010-10-11 Thread Pádraig Brady
I'll apply the attached tomorrow at some stage. cheers, Pádraig. From 7730599b6984d670760dd8fcacae54d7ed0a1496 Mon Sep 17 00:00:00 2001 From: =?utf-8?q?P=C3=A1draig=20Brady?= p...@draigbrady.com Date: Tue, 12 Oct 2010 01:39:58 +0100 Subject: [PATCH] tail: fix checking of currently unavailable

Re: [coreutils] [PATCH] tail: fix checking of currently unavailable directories

2010-10-11 Thread Pádraig Brady
On 12/10/10 02:18, Pádraig Brady wrote: I'll apply the attached tomorrow at some stage. With this squashed in to replace the confusing no space left on device error, with the reverting to polling information. This has the side effect of tail-2/wait passing on effected systems. diff --git a/src

Re: [coreutils] [PATCH] tail: fix checking of currently unavailable directories

2010-10-12 Thread Pádraig Brady
On 12/10/10 02:44, Pádraig Brady wrote: On 12/10/10 02:18, Pádraig Brady wrote: I'll apply the attached tomorrow at some stage. With this squashed in to replace the confusing no space left on device error, with the reverting to polling information. This has the side effect of tail-2/wait

Re: [coreutils] Re: [PATCH 2/2] stat: print timestamps to full resolution

2010-10-21 Thread Pádraig Brady
On 21/10/10 15:01, Jim Meyering wrote: Andreas Schwab wrote: Jim Meyering j...@meyering.net writes: And besides, with coreutils-8.6 already released, reverting the change is no longer an option. Why? I'm pretty sure more breakage will pop up over time. Hmm... I see what you mean.

Re: [coreutils] Re: [PATCH 2/2] stat: print timestamps to full resolution

2010-10-21 Thread Pádraig Brady
On 21/10/10 15:18, Pádraig Brady wrote: On 21/10/10 15:01, Jim Meyering wrote: Andreas Schwab wrote: Jim Meyering j...@meyering.net writes: And besides, with coreutils-8.6 already released, reverting the change is no longer an option. Why? I'm pretty sure more breakage will pop up over

Re: [coreutils] Re: [PATCH 2/2] stat: print timestamps to full resolution

2010-10-21 Thread Pádraig Brady
On 21/10/10 15:49, Eric Blake wrote: On 10/21/2010 08:42 AM, Pádraig Brady wrote: Or use '.' rather than ':' which also has the advantage of being backward compat with older stats, To clarify, coreutils= 8.5 output these timestamps using an int format internally, and so ignored any specified

Re: [coreutils] stat: reverting recent %X %Y %Z change

2010-10-22 Thread Pádraig Brady
On 22/10/10 18:43, Jim Meyering wrote: Part of reverting this change: stat now outputs the full sub-second resolution for the atime, mtime, and ctime values since the Epoch, when using the %X, %Y, and %Z directives of the --format option. This matches the fact that %x, %y,

[coreutils] [PATCH] truncate: fix a very unlikely case for undiagnosed errors

2010-10-24 Thread Pádraig Brady
The attached is more to suppress any errors from possible future static analysis, than for any practical reason. From b1d246341525b9f3e32914bd29764439528620ea Mon Sep 17 00:00:00 2001 From: =?utf-8?q?P=C3=A1draig=20Brady?= p...@draigbrady.com Date: Sun, 24 Oct 2010 22:44:52 +0100 Subject: [PATCH]

Re: [coreutils] cp/reflink-perm fails on btrfs (and probably on ocfs2, too)

2010-10-26 Thread Pádraig Brady
On 26/10/10 14:54, Jim Meyering wrote: Pádraig Brady wrote: On 26/10/10 12:48, Pádraig Brady wrote: So in summary error if any of --link, --symbolic-link, --reflink or --attributes-only are combined. I.E. leave the docs alone and do: Thanks. That sounds good. Do you feel like writing

Re: [coreutils][patch] seq: User specified terminator

2010-10-27 Thread Pádraig Brady
On 27/10/10 18:27, William Plusnick wrote: I added a way to specify the string printed after the last number in the seq command, as suggested by a comment. I am not sure how helpful this option is, because I personally don't use the seq command much. I also added documentation to both usage

Re: [coreutils] [PATCH] Cater for extra strace output when building 32-on-64.

2010-11-01 Thread Pádraig Brady
On 30/10/10 14:20, Nix wrote: When building 32-bit coreutils on a 64-bit Linux platform, the stat-free-symlinks test fails because the strace output it diffs against contains an extra informative line emitted by strace of the general form [ Process PID=28429 runs in 32 bit mode. ] So

Re: [coreutils] [PATCH] Cater for extra strace output when building 32-on-64.

2010-11-01 Thread Pádraig Brady
On 01/11/10 23:05, Nix wrote: On 1 Nov 2010, Pádraig Brady told this: That looks like a buglet in strace, patch below. There doesn't look to be any more cases of incorrectly outputting to stdout: Ah, yeah, oops, it probably was meant to go to stderr, wasn't it. I didn't think

[coreutils] Re: tests/backref-multibyte-slow timeout 5s is too short for my imac 400

2010-11-04 Thread Pádraig Brady
is that it be installed by developers, and we can arrange for that. Pádraig Brady wrote timeout for coreutils, so I've Cc'd him and that list. Pádraig, what do you think? Now that I ask that, I think the better approach, for now at least, is to write a little perl script that does what I'm proposing. I

Re: [coreutils] [patch] Re: Install enhancement request: capabilities

2010-11-04 Thread Pádraig Brady
Thanks for the patch! I think the feature is worth it. Currently install does not preserve xattrs and so looses any previous capabilities associated with a file. In any case, capabilities don't need to be implemented using xattrs, and might not be on tmpfs on Linux for example when support is

Re: [coreutils] [patch] Re: Install enhancement request: capabilities

2010-11-04 Thread Pádraig Brady
On 04/11/10 11:08, Pádraig Brady wrote: Thanks for the patch! I think the feature is worth it. Currently install does not preserve xattrs and so looses any previous capabilities associated with a file. In any case, capabilities don't need to be implemented using xattrs, and might

[coreutils] test: fix a dash portability problem with redirected symlinked ttys

2010-11-09 Thread Pádraig Brady
Another bug I noticed with dash-0.5.6-2.fc11.i586 is that it doesn't redirect from symlinks correctly for background processes. $ dash -c tty /dev/stdin $ dash -c tty /dev/stdin /dev/pts/3 $ bash -c tty /dev/stdin /dev/pts/3 $ dash -c tty $(readlink -f /dev/stdin) /dev/pts/3 OK to apply the

[coreutils] [PATCH] maint: add a missed fadvise-tests module

2010-11-15 Thread Pádraig Brady
commit b19733bb42e1897afd93f40aeca83e76013f7075 Author: Pádraig Brady p...@draigbrady.com Date: Mon Nov 15 09:36:16 2010 + maint: add a missed fadvise-tests module * gl/modules/fadvise-tests: Add the module previously missed in commit 63b5e816, 2010-07-14, fadvise: new module

[coreutils] [PATCH] build: add `patch` as a bootstrap dependency

2010-11-16 Thread Pádraig Brady
FYI. commit 79adacc43178916392af3dd581ded87f3aea1655 Author: Pádraig Brady p...@draigbrady.com Date: Tue Nov 16 07:32:32 2010 + build: add `patch` as a bootstrap dependency * bootstrap.conf (buildreq): require `patch` as it's used by gnulib-tool to apply local diffs to gnulib

Re: [coreutils] [PATCH 1/5] tests: factor out VERBOSE-only --version-printing code

2010-11-17 Thread Pádraig Brady
On 17/11/10 21:31, Jim Meyering wrote: FYI, here are 5 change-sets to make a few hundred (mostly automated) changes like these: -test $VERBOSE = yes chown --version +print_ver_ chown -test $VERBOSE = yes { cp --version; mv --version; } +print_ver_ cp mv -test

[coreutils] over aggressive threads in sort

2010-11-22 Thread Pádraig Brady
FYI https://bugzilla.redhat.com/show_bug.cgi?id=655096

Re: [coreutils] [PATCH] head: optionally indicate underrun of set limit

2010-11-24 Thread Pádraig Brady
On 23/11/10 16:34, Pádraig Brady wrote: On 23/11/10 16:24, Stefan Tomanek wrote: Dies schrieb Stefan Tomanek (stefan.toma...@wertarbyte.de): It is often convinient to detect whether head has in fact printed the requested number of lines or if EOF was reached before reaching that point

Re: [coreutils] [PATCH] Cater for extra strace output when building 32-on-64.

2010-12-01 Thread Pádraig Brady
On 30/11/10 18:09, Dmitry V. Levin wrote: On Mon, Nov 01, 2010 at 02:53:29PM +, Pádraig Brady wrote: [...] --- syscall.c.orig 2010-11-01 14:46:41.292576453 + +++ syscall.c 2010-11-01 14:47:10.164576378 + @@ -953,7 +953,7 @@ call = ptrace(PTRACE_PEEKTEXT

[coreutils] [PATCH] split: fix a case where --elide-empty-files causes invalid chunking

2010-12-08 Thread Pádraig Brady
This is a fairly esoteric issue I noticed in passing. No need for a NEWS entry I think. I was wondering about adding ---io-blksize when writing the test originally, and now that it can be used to trigger this bug I think it's warranted? cheers, Pádraig. From

Re: [coreutils] [PATCH] sort: fix some --compress reaper bugs

2010-12-14 Thread Pádraig Brady
On 14/12/10 09:55, Jim Meyering wrote: Paul Eggert wrote: +# Give compression subprocesses time to react when 'sort' exits. +# Otherwise, under NFS, when 'sort' unlinks the temp files and they +# are renamed to .nfs instead of being removed, the parent cleanup +# of this directory will

Re: [coreutils] [PATCH] sort: fix some --compress reaper bugs

2010-12-14 Thread Pádraig Brady
On 14/12/10 18:20, Paul Eggert wrote: On 14/12/10 09:55, Jim Meyering wrote: It'd be a shame to make everyone sleep even one second for NFS. Well, it is marked as an _expensive_ test. :-) On 12/14/10 02:07, Pádraig Brady wrote: If the subprocesses were reparented to the shell, then one

[coreutils] Re: bug#7597: multi-threaded sort can segfault (unrelated to the sort -u segfault)

2010-12-16 Thread Pádraig Brady
On 16/12/10 22:06, Paul Eggert wrote: On 12/12/10 07:41, Jim Meyering wrote: Paul Eggert wrote: There are also some test cases I need to add for the (unrelated) sort-compression bug, which is next on my list of coreutils bugs to look at. It would be great to fix that for 8.8, too, but

[coreutils] cppcheck

2010-12-17 Thread Pádraig Brady
I just ran cppcheck 1.46 on the latest coreutils. It ran for a _long_ time and reported these false positives [coreutils/src/system.h:355]: (error) Resource leak: fd [coreutils/src/chown-core.c:212]: (error) Resource leak: fd [coreutils/src/ls.c:1963]: (error) Unusual pointer arithmetic

[coreutils] clang-analyzer

2010-12-17 Thread Pádraig Brady
I just ran this on the latest coreutils: scan-build -o clang ./configure scan-build -o clang make and it flagged a possible problem in wc where it could spin if it got a read error on a large file containing file names to process. I think the following may address this: diff --git a/src/wc.c

Re: [coreutils] RE: cp command performance

2010-12-24 Thread Pádraig Brady
Didn't anyone get my previous reply on this. I'm wondering about my email server now. Anyway in summary what I said was the 3 commands you presented would perform about the same. Things to consider. The deletion/truncation bit will proceed more quickly on some file system types. If both files

bug#7759: split(1) regression in v8.7-25-gbe10739

2010-12-30 Thread Pádraig Brady
On 30/12/10 11:15, Jim Meyering wrote: Hi Pádraig Thanks for the quick fix. Please include the bugzilla URL and credit Sergey and Dmitry in the commit log. It'd be good to list Sergey's name/email in THANKS, too. Done and pushed. thanks, Pádraig.

Re: [coreutils] join feature: auto-format

2011-01-07 Thread Pádraig Brady
On 06/01/11 12:05, Pádraig Brady wrote: On 07/10/10 19:25, Pádraig Brady wrote: On 07/10/10 18:43, Assaf Gordon wrote: Pádraig Brady wrote, On 10/07/2010 06:22 AM: On 07/10/10 01:03, Pádraig Brady wrote: On 06/10/10 21:41, Assaf Gordon wrote: The --auto-format feature simply builds the -o

Re: [coreutils] Re: Bug#609049: du: process_file: Assertion `level == prev_level - 1' failed

2011-01-09 Thread Pádraig Brady
On 09/01/11 22:05, Jim Meyering wrote: diff --git a/tests/du/move-dir-while-traversing b/tests/du/move-dir-while-traversing +# We use a python-inotify script, so... +python -m pyinotify -h /dev/null \ + || skip_ 'python-inotify package not installed' A small point is that error is a bit

[coreutils] [PATCH] maint: trivial system header file cleanups

2011-01-13 Thread Pádraig Brady
From e1aaf8903db97f3240b1551fd6936ccdc652dfc8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C3=A1draig=20Brady?= p...@draigbrady.com Date: Thu, 13 Jan 2011 09:36:38 + Subject: [PATCH] maint: trivial system header file cleanups * src/system.h: Note where it should be included, and make ordering

Re: [coreutils] basename/dirname can't handle stdin?

2011-01-13 Thread Pádraig Brady
On 13/01/11 23:15, Jeff Blaine wrote: So then, please review. One question at the bottom. # Most basic usage basename /foo/bar.txt = bar.txt # Old/current basename NAME SUFFIX compat- # ibility basename /foo/bar.txt .txt = bar

Re: [coreutils] basename/dirname can't handle stdin?

2011-01-13 Thread Pádraig Brady
On 14/01/11 00:02, Eric Blake wrote: On 01/13/2011 04:53 PM, Pádraig Brady wrote: So in summary, just implement -a and -s like BSD does? I think you've persuaded me that a filter mode doesn't buy us much (it's only useful if unambiguous, but xargs -0 works just as well at giving us

Re: [coreutils] [PATCH] uniq: don't continue field processing after end of line

2011-01-17 Thread Pádraig Brady
On 16/01/11 23:53, Sami Kerola wrote: Hi, I notice uniq -f 'insane_large_number' will make process to be busy long time without good reason. Attached patch should fix this symptom. I'd slightly amend that to the following, to match the other limit checks in the function. diff --git

Re: [coreutils] [PATCH] doc: show how to shred using a single zero-writing pass

2011-01-17 Thread Pádraig Brady
On 17/01/11 10:36, Jim Meyering wrote: From 7dc6335653afcdad9a3ffa327877571734644285 Mon Sep 17 00:00:00 2001 From: Jim Meyering meyer...@redhat.com Date: Mon, 17 Jan 2011 11:32:35 +0100 Subject: [PATCH] doc: show how to shred using a single zero-writing pass * doc/coreutils.texi (shred

[coreutils] [PATCH] don't warn about disorder against empty input

2011-01-26 Thread Pádraig Brady
I notice that join doesn't warn about disorder when there are no mismatches between the two files, which is a good thing. $ join -a1 (printf '1\n0\n') (printf '1\n0\n') 1 0 However it does warn if there is no input in one of the files. I'm inclined to think that the following should be treated

[PATCH] cp: use a bigger buffer size when writing zeros

2011-01-31 Thread Pádraig Brady
Another buglet: commit 4a64fd8efe00629b0424bb59c8ec8d9d3ef4542f Author: Pádraig Brady p...@draigbrady.com Date: Mon Jan 31 22:04:35 2011 + cp: use a bigger buffer size when writing zeros * src/copy.c (write_zeros): This bug caused 4 or 8 bytes to be written at a time which

Re: RFC: new cp option: --efficient-sparse=HOW

2011-01-31 Thread Pádraig Brady
On 31/01/11 21:46, Jim Meyering wrote: Now that we have can read sparse files efficiently, what if I want to copy a 20PiB sparse file, and yet I want to be sure that it does so efficiently. Few people can afford to wait around while a normal processor and storage system process that much raw

Re: RFC: new cp option: --efficient-sparse=HOW

2011-02-04 Thread Pádraig Brady
On 03/02/11 20:29, Jim Meyering wrote: Does anyone know how to determine if a file system (say the one with .) supports the FIEMAP ioctl, but without compiling/running a C program? I.e., via perl or python? I've written a tiny C program that works and a Perl one that is supposed to be

Re: coreutils-8.10 released [stable]

2011-02-07 Thread Pádraig Brady
On 05/02/11 13:59, Jim Meyering wrote: Pádraig Brady wrote: Yep, just did that, and got a couple of failures for sparse-fiemap on and ext3 and loopback ext4 file systems. On a very quick glance, I think cp is OK and that the filefrag matching is a bit brittle. Attached are filefrag outputs

Re: coreutils-8.10 released [stable]

2011-02-08 Thread Pádraig Brady
On 08/02/11 08:42, Jim Meyering wrote: Do you think it's worth resorting to the FS-name-based test when python is not available? I don't think so, because df can currently allow the test to proceed erroneously: http://lists.gnu.org/archive/html/coreutils/2011-02/msg00018.html Also the overlap

[PATCH] copy: adjust fiemap handling of sparse files

2011-02-08 Thread Pádraig Brady
I was looking at adding fallocate() to copy.c, now that the fiemap code has gone in and I noticed that if there was allocated space at the end of a file, not accounted for in st_size, then any holes would not be detected. In what other cases does the sparse detection heuristic fail BTW? Anwyay,

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-02-10 Thread Pádraig Brady
On 09/02/11 23:35, Pádraig Brady wrote: Subject: [PATCH] copy: adjust fiemap handling of sparse files Don't depend on heuristics to detect sparse files if fiemap is available. Also don't introduce new holes unless --sparse=always has been specified. * src/copy.c (extent_copy): Pass

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-02-10 Thread Pádraig Brady
On 10/02/11 10:53, Jim Meyering wrote: While using --sparse=always is a lot less common, it looks like there's room for improvement there, too. I.e., we should be able to eliminate all of these lseek calls on the output descriptor there, too: $ strace -e lseek cp --sparse=always

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-02-13 Thread Pádraig Brady
Unfortunately, after checking BTRFS I see that fiemap behaves differently to EXT4. IMHO the EXT4 operation seems correct, and gives full info about the structure of a file, which cp for example can use to efficiently and accurately reproduce the structure at the destination. On EXT4 (on

Re: coreutils-8.10 released [stable]

2011-02-13 Thread Pádraig Brady
On 08/02/11 10:14, Pádraig Brady wrote: On 08/02/11 08:42, Jim Meyering wrote: Do you think it's worth resorting to the FS-name-based test when python is not available? I don't think so, because df can currently allow the test to proceed erroneously: http://lists.gnu.org/archive/html

Re: [PATCH] Replace spaces with tab on tests/Makefile.am

2011-02-19 Thread Pádraig Brady
Pushed. thanks, Pádraig.

bug#8001: cp (8.10) sparse handling fails on compressed btrfs (cp/fiemap-2)

2011-02-19 Thread Pádraig Brady
On 19/02/11 18:28, Mike Frysinger wrote: based on other threads (which i havent been following too closely), did we settle on this being a btrfs bug ? -mike Nope, cp 8.10 is not absolved yet. It may be btrfs not honoring FIEMAP_FLAG_SYNC, and/or it may be cp needing to handle

Re: A new solution for sorting du -h

2011-02-22 Thread Pádraig Brady
On 22/02/11 19:51, Alan Evans wrote: There are many discussions about sorting the human readable form of du for output in reports/cron jobs and the like. sort got -h to operation on the suffixed numbers directly in release 7.5 (2009-08-20) cheers, Pádraig.

[PATCH] dd: add a flag to discard cached data

2011-02-22 Thread Pádraig Brady
I noticed fadvise(DONTNEED) getting some love in the kernel recently http://lkml.org/lkml/2011/2/17/169 Which prompted me to implement this. With the attached one can now: # Advise to drop cache for whole file dd if=ifile iflag=nocache count=0 # Ensure drop cache for whole file dd

Re: [PATCH] dd: add a flag to discard cached data

2011-02-24 Thread Pádraig Brady
On 24/02/11 07:52, Jim Meyering wrote: One quick question: does the test need something to make it skip (not fail) on systems that lack kernel support for the feature? Oops right. Attached is latest version. thanks for the review, Pádraig. From fe067f8f52defc54636ae862df03a2115ac6266f Mon Sep

[PATCH] doc: group dd conv= options that are actually flags

2011-02-25 Thread Pádraig Brady
This requires reorganizing translations a little, but I think it's worth it. From db8405e00f30d794dfa91761cd101278f10008dc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C3=A1draig=20Brady?= p...@draigbrady.com Date: Fri, 25 Feb 2011 08:16:23 + Subject: [PATCH] doc: group dd conv= options that are

bug#7362: dd strangeness

2011-02-25 Thread Pádraig Brady
On 02/02/11 13:23, Pádraig Brady wrote: This looks like another candidate to auto enable fullblock for. https://bugzilla.redhat.com/show_bug.cgi?id=614605 I.E. oflag=direct Attached is a proposed solution to this. I'm worried about the last condition though where we enable 'fullblock' when

[PATCH] dd: don't discard data with if=fullblock

2011-02-28 Thread Pádraig Brady
I just noticed that dd may discard read data in the presence of I/O errors. Would something like this suffice? cheers, Pádraig. diff --git a/src/dd.c b/src/dd.c index 1a0e177..a1d47ff 100644 --- a/src/dd.c +++ b/src/dd.c @@ -811,12 +811,28 @@ static ssize_t iread_fullblock (int fd, char *buf,

auto merging extents

2011-03-07 Thread Pádraig Brady
I was wondering about adding fallocate() to cp, to efficiently allocate the destination before writing. With that in mind, I think it would be beneficial to auto merge extents, so that fragments in the source were not propagated to the dest? This should also be more efficient even without

Re: auto merging extents

2011-03-10 Thread Pádraig Brady
On 09/03/11 18:23, Jim Meyering wrote: Pádraig Brady wrote: - i--; - if (scan-ext_info[i].ext_flags FIEMAP_EXTENT_LAST) + scan-ei_count = si; + + si--; + if (scan-ext_info[si].ext_flags FIEMAP_EXTENT_LAST) { scan-hit_final_extent = true; return true

Re: auto merging extents

2011-03-10 Thread Pádraig Brady
On 10/03/11 14:13, Jim Meyering wrote: Have you thought of adding a test that would check for the merged extents in the result, say using filefrag? That's a bit tricky, at least until I merge fallocate() support, and even then we're at the behest of the file system as to what actually happens

[PATCH] maint: use wcswidth from gnulib

2011-03-13 Thread Pádraig Brady
FYI, after a suggestion from Bruno Haible, I applied: * gl/lib/mbsalign.c (rpl_wcswidth): Remove this in favor of the equivalent wcswidth replacement in gnulib. * bootstrap.conf: Depend on the wcswidth module.

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-14 Thread Pádraig Brady
On 12/03/11 15:34, Jim Meyering wrote: Jim Meyering wrote: Jim Meyering wrote: Jim Meyering wrote: Running make -j25 check on a nominal-12-core F14 system would cause serious difficulty leading to an OOM kill -- and this is brand new. It worked fine yesterday. I tracked it down to all of

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-14 Thread Pádraig Brady
On 14/03/11 15:31, Pádraig Brady wrote: On 12/03/11 15:34, Jim Meyering wrote: Jim Meyering wrote: Jim Meyering wrote: Jim Meyering wrote: Running make -j25 check on a nominal-12-core F14 system would cause serious difficulty leading to an OOM kill -- and this is brand new. It worked fine

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-16 Thread Pádraig Brady
On 16/03/11 12:07, Jim Meyering wrote: Pádraig Brady wrote: I've not fully analyzed this yet, and I'm not saying it's wrong, but the above change seems to have a large effect on thread creation when smaller buffers are used (you hinted previously that being less aggressive with the amount

Re: parallel sort at fault? [Re: [PATCH] tests: avoid gross inefficiency...

2011-03-16 Thread Pádraig Brady
On 16/03/11 15:32, Jim Meyering wrote: Pádraig Brady wrote: With SUBTHREAD_LINES_HEURISTIC=128k and -S1M option to sort we get no threads as nlines never gets above 12787 (there looks to be around 80 bytes overhead per line?). Only when -S = 12M do we get nlines high enough to create

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-16 Thread Pádraig Brady
On 16/03/11 19:18, Mike Frysinger wrote: On Wednesday, March 16, 2011 11:26:40 Pádraig Brady wrote: On 14/02/11 06:05, Jim Meyering wrote: Pádraig Brady wrote: For my own reference, I can probably skip performance tests on (older) btrfs by checking `filefrag` output. Also in `cp`, if we see

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-17 Thread Pádraig Brady
On 17/03/11 07:24, Mike Frysinger wrote: On Wednesday, March 16, 2011 19:55:56 Pádraig Brady wrote: On 16/03/11 19:18, Mike Frysinger wrote: well, in the bug report i was working with, we were seeing data loss. i wonder if it'd be possible to detect the fs/kernel version and error out when

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-17 Thread Pádraig Brady
On 17/03/11 19:29, Mike Frysinger wrote: On Thursday, March 17, 2011 05:32:33 Pádraig Brady wrote: This is the kind of patch appropriate for a distro, as they may or may not have a fix backported to their kernel. Gentoo, being a source distribution, has no kernel requirement. people

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Pádraig Brady
On 17/03/11 23:00, Pádraig Brady wrote: On 17/03/11 19:29, Mike Frysinger wrote: On Thursday, March 17, 2011 05:32:33 Pádraig Brady wrote: This is the kind of patch appropriate for a distro, as they may or may not have a fix backported to their kernel. Gentoo, being a source distribution

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-18 Thread Pádraig Brady
On 18/03/11 13:19, Pádraig Brady wrote: Bah humbug. Looks like there is no such issue. This actually seems like an issue in a coreutils test script, which made it seem like the SYNC done by `filefrag -vs` was ineffective. Proposed fix attached. My perl is still weak, so I won't apply until

Re: [PATCH] copy: adjust fiemap handling of sparse files

2011-03-19 Thread Pádraig Brady
On 19/03/11 12:07, Jim Meyering wrote: Pádraig Brady wrote: On 18/03/11 13:19, Pádraig Brady wrote: Bah humbug. Looks like there is no such issue. This actually seems like an issue in a coreutils test script, which made it seem like the SYNC done by `filefrag -vs` was ineffective. Proposed

  1   2   3   4   5   6   7   8   9   10   >