Re: Make mv work better with SELinux.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/04/2012 10:38 AM, Pádraig Brady wrote: On 10/08/2012 09:24 PM, Daniel J Walsh wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 One of if not the most common problem people hit with SELinux is the mv command, which maintains the file context of the source destination. mv /home/dwalsh/index.html /var/www/html/ This blows up on everybody and then the users have no idea why. I was thinking about adding -Z (--restorecon) to mv and having it basically do a internal restorecon on the destination. Then we could suggest people who get burnt by this to: alias mv=mv -Z In Fedora 18 we have greatly enhanced matchpathcon, by pre-compiling the regex, so there should be very little slow down in doing this. A question on performance. So there was a large matchpathcon() performance issue in Fedora 11 time, where we had a 20x slow down if matchpathcon_init_prefix() wasn't called https://bugzilla.redhat.com/show_bug.cgi?id=479502#c24 That should be fixed: In Fedora 18 we have greatly enhanced matchpathcon, by pre-compiling the regex, so there should be very little slow down in doing this. We have seen a 40x speed up using the new version of matchpathcon. Does calling matchpathcon_init_prefix() still provide benefit on Fedora 18? More importantly, since the new selinux::restorecon_private() doesn't call matchpathcon_init_prefix(), will it have the large performance issues on Fedora = 17 and other SELinux supporting platforms? Yes, if you have a tight loop of mv -Z X Y, then you would take a hit on each invocation. Since this is optional, I do not see this as a problem. The install version was not optional, so a make install was exhibiting a huge overhead. Basically almost .33/sec slow down. As of Fedora 18, there should be very limited overhead so this should not be a problem. matchpathcon_init_prefix theoretically would still give you a speed up, but I don't believe it is measurable with the new libraries. Not a huge issue since install(1) enables setdefaultfilecon() by default, whereas the new proposal would only enable when -Z is specified. Yes I missed this, I though install had disabled this functionality altogether. That's an inconsistency in the patch in this thread actually. install -Z runs the new restorecon(), while also running the old setdefaultfilecon(). Seems like we may need to drop the new install -Z code for now, and possible in future merge restorecon() and setdefaultfilecon() Yes I agree. This was my mistake cheers, Pádraig. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlC/ob8ACgkQrlYvE4MpobNZRQCfa0hlo19AOY8IGnRARqpQfHZO gbcAnRX9S8YD+TPdXpKZK/kVct5HqUIm =HU0c -END PGP SIGNATURE-
Re: Make mv work better with SELinux.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/04/2012 10:56 AM, Pádraig Brady wrote: On 12/04/2012 03:38 PM, Pádraig Brady wrote: On 10/08/2012 09:24 PM, Daniel J Walsh wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 One of if not the most common problem people hit with SELinux is the mv command, which maintains the file context of the source destination. mv /home/dwalsh/index.html /var/www/html/ This blows up on everybody and then the users have no idea why. I was thinking about adding -Z (--restorecon) to mv and having it basically do a internal restorecon on the destination. Then we could suggest people who get burnt by this to: alias mv=mv -Z In Fedora 18 we have greatly enhanced matchpathcon, by pre-compiling the regex, so there should be very little slow down in doing this. A question on performance. So there was a large matchpathcon() performance issue in Fedora 11 time, where we had a 20x slow down if matchpathcon_init_prefix() wasn't called https://bugzilla.redhat.com/show_bug.cgi?id=479502#c24 Does calling matchpathcon_init_prefix() still provide benefit on Fedora 18? More importantly, since the new selinux::restorecon_private() doesn't call matchpathcon_init_prefix(), will it have the large performance issues on Fedora = 17 and other SELinux supporting platforms? Not a huge issue since install(1) enables setdefaultfilecon() by default, whereas the new proposal would only enable when -Z is specified. That's an inconsistency in the patch in this thread actually. install -Z runs the new restorecon(), while also running the old setdefaultfilecon(). Seems like we may need to drop the new install -Z code for now, and possible in future merge restorecon() and setdefaultfilecon() Also could you comment on the different schemes used by restorecon() and setdefaultfilecon(). The old setdefaultfilecon() sets the context of the dest files to that returned by matchpathcon directly, whereas the new restorecon() only uses the type portion of the context from matchpathcon() and inserts that into the exisiting context for the dest file. thanks, Pádraig. restorecon says set the file to the system default, and does not take into account the level of the currently running process. Lets imaging you have a process running as staff_u:sysadm_r:sysadm_t:s0:c1,c2 on an MLS box. cp ~dwalsh/resolv.conf /etc The file would be created as staff_u:object_r:etc_t:s0:c1,c2 If we went with the traditional restorecon command, cp -Z ~dwalsh/resolv.conf /etc The file would be created as system_u:object_r:resolv_conf_t:s0 With the new method it will be created as: staff_u:object_r:resolv_conf_t:s0:c1,c2 Which means it would honor the intent of MLS to preserve the MLS level of the data and preserve the user identity of the data. Which seems to be more correct. restorecon /etc/resolv.conf In F18 also does not change the MLS level or SELinux User by default, It only changes the type field. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlC/pAAACgkQrlYvE4MpobPuJwCgrOF4kv+KdXDknZXXlnGqDOkU 6d4AoLRi5SNQSrfsFflIy4PAJMxwpWs3 =odVY -END PGP SIGNATURE-
Re: New feature in mv
Raphael S Carvalho wrote: So what do you think? Which language should I use? I'm going to sleep in few minutes, so I would get started tomorrow. As Bernhard Voelker we could create a C program which would use those tools (mv, cp, ln). It would be hard to object to a C program. But this is such a simple task that I think anything other than a shell script would be tremendous overkill. Bob
Re: Command-line program to convert 'human' sizes?
Hello, Attached is a working version of numfmt. The following are implemented: === Usage: ./src/numfmt [OPTIONS] [NUMBER] Reformats NUMBER(s) to/from human-readable values. Numbers can be processed either from stdin or command arguments. --from=UNIT Auto-scale input numbers to UNITs. Default is 'none'. See UNIT below. --from-unit=N Specify the input unit size (instead of the default 1). --to=UNIT Auto-scale output numbers to UNITs. See UNIT below. --to-unit=N Specify the output unit size (instead of the default 1). --round=METHOD Round input numbers. METHOD can be: ceiling (the default), floor, nearest --suffix=SUFFIX Add SUFFIX to output numbers, and accept optional SUFFIX in input numbers. --padding=N Pad the output to N characters. Default is right-aligned. Negative N will left-align. Note: if N is too small, the output will be truncated, and a warning will be printed to stderr. --grouping Group digits together (e.g. 1,000,000). Uses the locale-defined grouping (i.e. have no effect in C/POSIX locales). --field N Replace the number in input field N (default is 1) -d, --delimiter=X use X instead of whitespace for field delimiter === Also included in the patch is a test file, testing all sorts of combination of the parameters (hopefully catches most of the corner cases). There's also an undocumented option --debug that will show what's going on: === $ /src/numfmt --debug --field 2 --suffix=Foo --from=SI --to=IEC Hello 70MFoo World Extracting Fields: input: 'Hello 70MFoo World' field: 2 prefix: 'Hello' number: '70MFoo' suffix: 'World' Trimming suffix 'Foo' Parsing number: input string: '70M' remaining characters: '' numeric value: 7000 Formatting output: value: 7000 humanized: '67M' Hello 67MFoo World === Comments are welcomed, -gordon numfmt3.patch.gz Description: GNU Zip compressed data
Re: Command-line program to convert 'human' sizes?
Assaf Gordon wrote, On 12/05/2012 06:13 PM: Attached is a working version of numfmt. Somewhat related: How do I add a rule to build the man page (I'm working on passing 'make syntax-check'). I've added the following line to 'man/local.mk': man/numfmt.1:src/numfmt But it doesn't get build (after bootstrap+configure+make). Thanks, -gordon
Re: Command-line program to convert 'human' sizes?
Assaf Gordon wrote: Assaf Gordon wrote, On 12/05/2012 06:13 PM: Attached is a working version of numfmt. Somewhat related: How do I add a rule to build the man page (I'm working on passing 'make syntax-check'). I've added the following line to 'man/local.mk': man/numfmt.1:src/numfmt But it doesn't get build (after bootstrap+configure+make). You'll have to add numfmt to the list of programs in build-aux/gen-lists-of-programs.sh Then, be sure to run make syntax-check, and it'll cross-check a few other things that have to be synced with that list.
Re: Command-line program to convert 'human' sizes?
On 12/05/12 19:58, Jim Meyering wrote: Assaf Gordon wrote: Somewhat related: How do I add a rule to build the man page (I'm working on passing 'make syntax-check'). I've added the following line to 'man/local.mk': man/numfmt.1:src/numfmt But it doesn't get build (after bootstrap+configure+make). You'll have to add numfmt to the list of programs in build-aux/gen-lists-of-programs.sh Then, be sure to run make syntax-check, and it'll cross-check a few other things that have to be synced with that list. I already added it to gen-lists-of-programs.sh (under normal_progs) - and the compilation works fine. I've also added a line to tests/local.mk and make checks works fine. But the man page part seems to be ignored. The strange thing is that make doesn't complain about the job, it simply ignores it: === $ grep numfmt man/local.mk man/numfmt.1:src/numfmt $ ls man/numfmt.1 ls: cannot access man/numfmt.1: No such file or directory $ make man/numfmt.1 gordon@tango:~/projects/coreutils$ ls man/numfmt.1 ls: cannot access man/numfmt.1: No such file or directory === When I add -d to make, it ends with these messages: === $ make -d man/numfmt.1 snip Prerequisite `src/numfmt.o' is older than target `src/numfmt'. Prerequisite `src/libver.a' is older than target `src/numfmt'. Prerequisite `lib/libcoreutils.a' is older than target `src/numfmt'. Prerequisite `lib/libcoreutils.a' is older than target `src/numfmt'. Prerequisite `src/.dirstamp' is older than target `src/numfmt'. No need to remake target `src/numfmt'. Finished prerequisites of target file `man/numfmt.1'. Must remake target `man/numfmt.1'. Successfully remade target file `man/numfmt.1'. === But the man file is not created. Thanks, -gordon
Re: Command-line program to convert 'human' sizes?
On 12/05/12 20:49, Assaf Gordon wrote: On 12/05/12 19:58, Jim Meyering wrote: Assaf Gordon wrote: Somewhat related: How do I add a rule to build the man page (I'm working on passing 'make syntax-check'). You'll have to add numfmt to the list of programs in build-aux/gen-lists-of-programs.sh I already added it to gen-lists-of-programs.sh (under normal_progs) - and the compilation works fine. I've also added a line to tests/local.mk and make checks works fine. But the man page part seems to be ignored. Nevermind - figured it out: A stub man/numfmt.x file is required.
Re: Command-line program to convert 'human' sizes?
Hello, With the attached patch, numfmt passes make syntax-check and almost passes make check and make distcheck. Regarding the checks: tests/misc/numfmt.pl passes all tests successfully. But: 1. When running make check, tests/df/total-verify.sh fails, so the check isn't complete. 2. When running make check TESTS=tests/misc/numfmt VERBOSE=yes, the tests script passes, but the process later fails with this error: $ make check TESTS=tests/misc/numfmt VERBOSE=yes GENpublic-submodule-commit make check-recursive make[1]: Entering directory `/home/gordon/projects/coreutils' Making check in po make[2]: Entering directory `/home/gordon/projects/coreutils/po' make[2]: Leaving directory `/home/gordon/projects/coreutils/po' Making check in . make[2]: Entering directory `/home/gordon/projects/coreutils' make check-TESTS check-local make[3]: Entering directory `/home/gordon/projects/coreutils' make[4]: Entering directory `/home/gordon/projects/coreutils' PASS: tests/misc/numfmt.pl = 1 test passed = make[4]: Leaving directory `/home/gordon/projects/coreutils' GENcheck-README GENcheck-duplicate-no-install GENsc-avoid-builtin GENsc-avoid-io GENsc-avoid-non-zero GENsc-avoid-path GENsc-avoid-timezone GENsc-avoid-zeroes GENsc-exponent-grouping GENsc-lower-case-var GENsc-use-small-caps-NUL GENcheck-texinfo make[3]: Leaving directory `/home/gordon/projects/coreutils' make[2]: Leaving directory `/home/gordon/projects/coreutils' Making check in gnulib-tests make[2]: Entering directory `/home/gordon/projects/coreutils/gnulib-tests' make check-recursive make[3]: Entering directory `/home/gordon/projects/coreutils/gnulib-tests' snip make[5]: Leaving directory `/home/gordon/projects/coreutils/gnulib-tests' make check-TESTS make[5]: Entering directory `/home/gordon/projects/coreutils/gnulib-tests' make[6]: Entering directory `/home/gordon/projects/coreutils/gnulib-tests' make[6]: *** No rule to make target `tests/misc/numfmt.log', needed by `test-suite.log'. Stop. make[6]: Leaving directory `/home/gordon/projects/coreutils/gnulib-tests' make[5]: *** [check-TESTS] Error 2 make[5]: Leaving directory `/home/gordon/projects/coreutils/gnulib-tests' make[4]: *** [check-am] Error 2 make[4]: Leaving directory `/home/gordon/projects/coreutils/gnulib-tests' make[3]: *** [check-recursive] Error 1 make[3]: Leaving directory `/home/gordon/projects/coreutils/gnulib-tests' make[2]: *** [check] Error 2 make[2]: Leaving directory `/home/gordon/projects/coreutils/gnulib-tests' make[1]: *** [check-recursive] Error 1 make[1]: Leaving directory `/home/gordon/projects/coreutils' make: *** [check] Error 2 ## Strangely, the log file does exist: $ ls -l tests/misc/numfmt.log -rw-r--r-- 1 gordon gordon 1069 Dec 5 21:51 tests/misc/numfmt.log Any advice is appreciated, -gordon numfmt4.patch.gz Description: GNU Zip compressed data
Suggestion: update README/HACKING regarding tests
As per: http://lists.gnu.org/archive/html/coreutils/2012-09/msg00144.html , Perhaps you'll agree to update README/HACKING about how to run individual tests: === diff --git a/HACKING b/HACKING index de8cd7b..01e7605 100644 --- a/HACKING +++ b/HACKING @@ -438,9 +438,11 @@ Nearly every significant change must be accompanied by a test suite addition that exercises it. If you fix a bug, add at least one test that fails without the patch, but that succeeds once your patch is applied. If you add a feature, add tests to exercise as much of the new code -as possible. Note to run tests/misc/new-test in isolation you can do: +as possible. If you add a new test file (as opposed to adding a test to an +existing test file) add the new test file to 'tests/local.mk'. +Note to run tests/misc/new-test in isolation you can do: - (cd tests make check TESTS=misc/new-test VERBOSE=yes) + make TESTS=tests/misc/new-test SUBDIRS=. VERBOSE=yes Variables that are significant for tests with their default values are: diff --git a/README b/README index 21c9b03..15ed29b 100644 --- a/README +++ b/README @@ -176,7 +176,7 @@ in verbose mode for each failing test. For example, if the test that fails is tests/misc/df, then you would run this command: - (cd tests make check TESTS=misc/df VERBOSE=yes) log 21 + make check TESTS=tests/misc/df SUBDIRS=. VERBOSE=yes log 21 For some tests, you can get even more detail by adding DEBUG=yes. Then include the contents of the file 'log' in your bug report. === Regards, -gordon
bug#13030: factor: infinite loop on Linux/powerpc
On 11/29/2012 08:34 PM, Pádraig Brady wrote: On 11/29/2012 02:28 PM, Colin Watson wrote: The fix looks good thanks. I'll commit [...] Now, make syntax-check fails: Colin Watson maint.mk: remove the above names from THANKS.in make: *** [sc_THANKS_in_duplicates] Error 1 Before removing that entry: Are you the same Colin Watson as in THANKS.in? Colin Watsoncj...@riva.ucam.org Have a nice day, Berny
bug#13030: factor: infinite loop on Linux/powerpc
On Wed, Dec 05, 2012 at 09:58:11AM +0100, Bernhard Voelker wrote: On 11/29/2012 08:34 PM, Pádraig Brady wrote: The fix looks good thanks. I'll commit [...] Now, make syntax-check fails: Colin Watson maint.mk: remove the above names from THANKS.in make: *** [sc_THANKS_in_duplicates] Error 1 Before removing that entry: Are you the same Colin Watson as in THANKS.in? Colin Watsoncj...@riva.ucam.org Yes, that's me. The new address is better. Cheers, -- Colin Watson [cjwat...@ubuntu.com]
bug#13030: factor: infinite loop on Linux/powerpc
On 12/05/2012 11:08 AM, Colin Watson wrote: On Wed, Dec 05, 2012 at 09:58:11AM +0100, Bernhard Voelker wrote: Now, make syntax-check fails: Colin Watson maint.mk: remove the above names from THANKS.in make: *** [sc_THANKS_in_duplicates] Error 1 Before removing that entry: Are you the same Colin Watson as in THANKS.in? Colin Watsoncj...@riva.ucam.org Yes, that's me. The new address is better. Thanks, I'll push this soon. Have a nice day, Berny From 7b4c56b4c65b2b58c185908ff1cbdf0b0399140d Mon Sep 17 00:00:00 2001 From: Bernhard Voelker m...@bernhard-voelker.de Date: Wed, 5 Dec 2012 11:47:18 +0100 Subject: [PATCH] maint: remove now auto-added entry from THANKS.in The syntax-check sc_THANKS_in_duplicates complained about that excess entry. * THANKS.in (Colin Watson): Remove entry, now that it will be automatically included in the generated THANKS file. --- THANKS.in |1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/THANKS.in b/THANKS.in index 6a79e04..f0ef9b8 100644 --- a/THANKS.in +++ b/THANKS.in @@ -128,7 +128,6 @@ Chusslove Illichcaslav.i...@gmx.net Clark Morgancmor...@aracnet.com Clement Wangclem.w...@overture.com Colin Plumb co...@nyx.net -Colin Watsoncj...@riva.ucam.org Collin Rogowski col...@rogowski.de Cray-Cyber Project http://www.cray-cyber.org Cristian Cadar cris...@stanford.edu -- 1.7.7
bug#13080: [PATHC] improve error reporting
On 12/05/2012 01:57 AM, Pádraig Brady wrote:
Keying on 0x80 doesn't seem right.
I'll apply this hunk instead in your name.
Please confirm.
diff --git a/src/cut.c b/src/cut.c
index 4219d24..f2e63dc 100644
--- a/src/cut.c
+++ b/src/cut.c
@@ -846,7 +846,7 @@ main (int argc, char **argv)
if (operating_mode == undefined_mode)
FATAL_ERROR (_(you must specify a list of bytes, characters, or
fields));
- if (delim != '\0' operating_mode != field_mode)
+ if (delim_specified operating_mode != field_mode)
FATAL_ERROR (_(an input delimiter may be specified only\
when operating on fields));
I got confirmation of the adjustment privately,
so I'm going to push the attached two patches,
which augment/adjust the cut test accordingly.
thanks,
Pádraig.
From 4cf194b4f1e425973f2d226905f49649e904a0e1 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= p...@draigbrady.com
Date: Wed, 5 Dec 2012 13:51:01 +
Subject: [PATCH] tests: cut.pl: adjust for changed diagnostic
* tests/misc/cut.pl: Since we now output the more
complete error message irrespective of running
in a multi-byte locale or not, adjust the test accordingly.
---
tests/misc/cut.pl | 14 +-
1 files changed, 1 insertions(+), 13 deletions(-)
diff --git a/tests/misc/cut.pl b/tests/misc/cut.pl
index aff0cbe..a402a75 100755
--- a/tests/misc/cut.pl
+++ b/tests/misc/cut.pl
@@ -30,7 +30,7 @@ my $mb_locale = $ENV{LOCALE_FR_UTF8};
my $prog = 'cut';
my $try = Try '$prog --help' for more information.\n;
my $from_1 = $prog: fields and positions are numbered from 1\n$try;
-my $inval = $prog: invalid byte or field list\n$try;
+my $inval = $prog: invalid byte, character or field list\n$try;
my $no_endpoint = $prog: invalid range with no endpoint: -\n$try;
my $nofield = $prog: an input delimiter may be specified only when .
operating on fields\n$try;
@@ -195,18 +195,6 @@ if ($mb_locale ne 'C')
my @new_t = @$t;
my $test_name = shift @new_t;
-# Depending on whether cut is multi-byte-patched,
-# it emits different diagnostics:
-# non-MB: invalid byte or field list
-# MB: invalid byte, character or field list
-# Adjust the expected error output accordingly.
-if (grep {ref $_ eq 'HASH' exists $_-{ERR} $_-{ERR} eq $inval}
-(@new_t))
- {
-my $sub = {ERR_SUBST = 's/, character//'};
-push @new_t, $sub;
-push @$t, $sub;
- }
push @new, [$test_name-mb, @new_t, {ENV = LC_ALL=$mb_locale}];
}
push @Tests, @new;
--
1.7.6.4
bug#13080: [PATHC] improve error reporting
On 12/05/2012 01:55 PM, Pádraig Brady wrote:
On 12/05/2012 01:57 AM, Pádraig Brady wrote:
Keying on 0x80 doesn't seem right.
I'll apply this hunk instead in your name.
Please confirm.
diff --git a/src/cut.c b/src/cut.c
index 4219d24..f2e63dc 100644
--- a/src/cut.c
+++ b/src/cut.c
@@ -846,7 +846,7 @@ main (int argc, char **argv)
if (operating_mode == undefined_mode)
FATAL_ERROR (_(you must specify a list of bytes, characters, or
fields));
- if (delim != '\0' operating_mode != field_mode)
+ if (delim_specified operating_mode != field_mode)
FATAL_ERROR (_(an input delimiter may be specified only\
when operating on fields));
I got confirmation of the adjustment privately,
so I'm going to push the attached two patches,
which augment/adjust the cut test accordingly.
The previous mail omitted the following patch.
sorry,
Pádraig.
From b95a20224e40dd5e2d9eb3f0611e1db51b6f4e04 Mon Sep 17 00:00:00 2001
From: Cojocaru Alexandru xo...@gmx.com
Date: Wed, 5 Dec 2012 13:13:51 +
Subject: [PATCH 1/2] cut: improve error reporting
* src/cut.c (main): Treat a NUL delimiter (-d '') consistently
with non NUL delimiters, and disallow such a delimiter option,
unless a field is also specified.
(set_fields): Provide a more accurate error message when
a given list is invalid.
* tests/misc/cut.pl: Add a test case.
---
src/cut.c |6 +++---
tests/misc/cut.pl |7 +++
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/src/cut.c b/src/cut.c
index 4219d24..dc83072 100644
--- a/src/cut.c
+++ b/src/cut.c
@@ -365,7 +365,7 @@ set_fields (const char *fieldstr)
in_digits = false;
/* Starting a range. */
if (dash_found)
-FATAL_ERROR (_(invalid byte or field list));
+FATAL_ERROR (_(invalid byte, character or field list));
dash_found = true;
fieldstr++;
@@ -491,7 +491,7 @@ set_fields (const char *fieldstr)
fieldstr++;
}
else
-FATAL_ERROR (_(invalid byte or field list));
+FATAL_ERROR (_(invalid byte, character or field list));
}
max_range_endpoint = 0;
@@ -846,7 +846,7 @@ main (int argc, char **argv)
if (operating_mode == undefined_mode)
FATAL_ERROR (_(you must specify a list of bytes, characters, or fields));
- if (delim != '\0' operating_mode != field_mode)
+ if (delim_specified operating_mode != field_mode)
FATAL_ERROR (_(an input delimiter may be specified only\
when operating on fields));
diff --git a/tests/misc/cut.pl b/tests/misc/cut.pl
index 27768ff..aff0cbe 100755
--- a/tests/misc/cut.pl
+++ b/tests/misc/cut.pl
@@ -32,6 +32,8 @@ my $try = Try '$prog --help' for more information.\n;
my $from_1 = $prog: fields and positions are numbered from 1\n$try;
my $inval = $prog: invalid byte or field list\n$try;
my $no_endpoint = $prog: invalid range with no endpoint: -\n$try;
+my $nofield = $prog: an input delimiter may be specified only when .
+ operating on fields\n$try;
my @Tests =
(
@@ -118,6 +120,11 @@ my @Tests =
['multichar-od', qw(-d: --out=_._), '-f2,3', {IN=a:b:c\n},
{OUT=b_._c\n}],
+ # Ensure delim is not allowed without a field
+ # Prior to 8.21, a NUL delim was allowed without a field
+ ['delim-no-field1', qw(-d ''), '-b1', {EXIT=1}, {ERR=$nofield}],
+ ['delim-no-field2', qw(-d:), '-b1', {EXIT=1}, {ERR=$nofield}],
+
# Prior to 1.22i, you couldn't use a delimiter that would sign-extend.
['8bit-delim', '-d', \255, '--out=_', '-f2,3', {IN=a\255b\255c\n},
{OUT=b_c\n}],
--
1.7.6.4
bug#13089: Wish: split every n'th into n pipes
I often have data that can be processed in parallel. It would be great if split --filter could look at every n'th line instead of chunking into n chunks: cat bigfile | split --every-nth -n 8 --filter grep foo The above should start 8 greps and give each a line in round robin manner. Ideally it should be possible to do so non-blocking so if some lines take longer for one instance of grep, then the rest of the greps are not blocked. /Ole
bug#13089: Wish: split every n'th into n pipes
tag 13089 + notabug close 13089 On 12/05/2012 02:11 PM, Ole Tange wrote: I often have data that can be processed in parallel. It would be great if split --filter could look at every n'th line instead of chunking into n chunks: cat bigfile | split --every-nth -n 8 --filter grep foo The above should start 8 greps and give each a line in round robin manner. Ideally it should be possible to do so non-blocking so if some lines take longer for one instance of grep, then the rest of the greps are not blocked. So that's mostly supported already (notice the r/ below): $ seq 8000 | split -n r/8 --filter='wc -l' | uniq -c 8 1000 The concurrency is achieved through standard I/O buffers between split and the filters (note also the -u split option). I'm not sure non blocking I/O would be of much benefit, since the filters will be the same, and if we did that, then we'd have to worry about internal buffering in split. We had a similar question about tee, yesterday, and I think the answer is the same here, that the complexity doesn't seem warranted for such edge cases. thanks, Pádraig.
bug#13099: possible test bug in tests/df/total-verify.sh
Hello,
(properly reporting a bug I mentioned in another email).
The test df/total-verify.sh fails on my system, due to a wrong(?) regex to verify the
output of df --total -i -P.
To make a long story short, my df produces the following line:
===
$ df --total -i -P
Filesystem Inodes IUsed IFree IUse% Mounted on
snip
//bluearctitan3/foobar0 0 0 - /media/foobar
===
The filesystem is mounted with fuse/sshfs (not sure if that makes a difference).
The test error is:
===
check-df: invalid input line
: //bluearctitan3/foobar0 0 0 - /media/foobar
===
The 'bug' is that check-df script (in 'total-verify.sh') assumes that there
are two (2) spaces following the dash, whereas on my system there's only one space.
A possible fix (not sure about the side-effects, though);
diff --git a/tests/df/total-verify.sh b/tests/df/total-verify.sh
index 18d215f..c36a7a7 100755
--- a/tests/df/total-verify.sh
+++ b/tests/df/total-verify.sh
@@ -32,7 +32,7 @@ while ()
# /dev/sdc1 0 0 0- /c
# tmpfs1536000 12965 15230351% /tmp
# total5285932 787409 4498523 15% -
-/^(.*?) +(-?\d+|-) +(-?\d+|-) +(-?\d+|-) +(?:- |[0-9]+%) (.*)$/
+/^(.*?) +(-?\d+|-) +(-?\d+|-) +(-?\d+|-) +(?:-|[0-9]+%) +(.*)$/
or die $0: invalid input line\n: $_;
if ($1 eq 'total' $5 eq '-')
{
Regards,
-gordon
bug#13099: possible test bug in tests/df/total-verify.sh
Correction: On 12/05/12 22:33, Assaf Gordon wrote: === $ df --total -i -P Filesystem Inodes IUsed IFree IUse% Mounted on snip //bluearctitan3/foobar0 0 0 - /media/foobar === The filesystem is mounted with fuse/sshfs (not sure if that makes a difference). The filesystem is mounted with CIFS, not sshfs.
bug#13098: [PATCH] cut.c: Fix memory leak
From 82f2b062c0e21d9a0d64f9ceab363d2a79f5a6eb Mon Sep 17 00:00:00 2001
From: Cojocaru Alexandru xo...@gmx.com
Date: Thu, 6 Dec 2012 03:03:41 +0100
Subject: [PATCH] cut: fix memory leak
* src/cut.c (set_fields): don't allocate memory for
`printable_field' if there are no finite ranges.
The bug was introduced on 2012-2-7 via commit 2e636af.
---
src/cut.c | 12 +++-
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/src/cut.c b/src/cut.c
index 4219d24..87abd15 100644
--- a/src/cut.c
+++ b/src/cut.c
@@ -500,14 +500,13 @@ set_fields (const char *fieldstr)
if (rp[i].hi max_range_endpoint)
max_range_endpoint = rp[i].hi;
}
- if (max_range_endpoint eol_range_start)
-max_range_endpoint = eol_range_start;
/* Allocate an array large enough so that it may be indexed by
the field numbers corresponding to all finite ranges
(i.e. '2-6' or '-4', but not '5-') in FIELDSTR. */
- printable_field = xzalloc (max_range_endpoint / CHAR_BIT + 1);
+ if (n_rp)
+printable_field = xzalloc (max_range_endpoint / CHAR_BIT + 1);
qsort (rp, n_rp, sizeof (rp[0]), compare_ranges);
@@ -531,8 +530,11 @@ set_fields (const char *fieldstr)
if (output_delimiter_specified
!complement
- eol_range_start !is_printable_field (eol_range_start))
-mark_range_start (eol_range_start);
+ eol_range_start
+ printable_field !is_printable_field (eol_range_start))
+{
+ mark_range_start (eol_range_start);
+}
free (rp);
--
1.8.0.1
--
Cojocaru Alexandru xo...@gmx.com
