Jim Meyering wrote: > Jim Meyering wrote: > >> Pádraig Brady wrote: >> ... >>> How about the attached? >> >>> From: Cojocaru Alexandru <[email protected]> >>> Date: Thu, 6 Dec 2012 03:03:41 +0100 >>> Subject: [PATCH] cut: avoid a redundant heap allocation >>> >>> * src/cut.c (set_fields): Don't allocate memory for >>> `printable_field' if there are no finite ranges. >>> The extra allocation was introduced via commit v8.10-3-g2e636af. >> ... >> >> Thanks to both of you. >> That's a fine bug fix, actually. >> Consider that before, this would fail on my 64-bit system: >> >> $ : | cut -b999999999999999999- >> cut: memory exhausted >> [Exit 1] >> $ >> >> Now, it no longer tries to allocate all that memory, so completes normally: >> >> $ : | cut -b999999999999999999- >> $ > > Note that on a 32-bit system it fails the same way in both cases: > > $ : | src/cut -b999999999999999999- > src/cut: byte offset '999999999999999999' is too large > [Exit 1] > > And even with 2^32-1, it'll probably just allocate the memory, > assuming 500MB doesn't cause trouble. > To differentiate portably, we probably have to use ulimit: > > Limit VM to 22000k, but make the old cut require about 10x that: > > $ (ulimit -v 22000; :| cut -b$(echo 2^31|bc)-) > cut: memory exhausted > [Exit 1] > > The new, just-built one passes: > > $ (ulimit -v 22000; :| ./cut -b$(echo 2^31|bc)-) > $ > > If someone feels like turning the above into a test > case, please do (and update NEWS). Otherwise, I'll > get to it over the weekend.
Here it is: >From a2833b8399f56bff6fe08eb212c01a5cb1b74606 Mon Sep 17 00:00:00 2001 From: Jim Meyering <[email protected]> Date: Sat, 8 Dec 2012 12:04:14 -0800 Subject: [PATCH] tests: add test case and note that last week's cut change is a bug fix * tests/misc/cut-huge-to-eol-range.sh: New test, showing that the change in v8.20-51-g7d03466 is a bug fix after all. * tests/local.mk (all_tests): Add it. * NEWS (Bug fixes): Mention it. --- NEWS | 4 ++++ tests/local.mk | 1 + tests/misc/cut-huge-to-eol-range.sh | 30 ++++++++++++++++++++++++++++++ 3 files changed, 35 insertions(+) create mode 100755 tests/misc/cut-huge-to-eol-range.sh diff --git a/NEWS b/NEWS index 7c17869..a7a5bc3 100644 --- a/NEWS +++ b/NEWS @@ -10,6 +10,10 @@ GNU coreutils NEWS -*- outline -*- ** Bug fixes + cut with a range like "N-" no longer allocates N/8 bytes. That buffer + would never be used, and allocation failure could cause cut to fail. + [bug introduced in coreutils-8.10] + cut no longer accepts the invalid range 0-, which made it print empty lines. Instead, cut now fails and emits an appropriate diagnostic. [This bug was present in "the beginning".] diff --git a/tests/local.mk b/tests/local.mk index d5bb6f7..5eeddd5 100644 --- a/tests/local.mk +++ b/tests/local.mk @@ -246,6 +246,7 @@ all_tests = \ tests/misc/pwd-option.sh \ tests/misc/chcon-fail.sh \ tests/misc/cut.pl \ + tests/misc/cut-huge-to-eol-range.sh \ tests/misc/wc.pl \ tests/misc/wc-files0-from.pl \ tests/misc/wc-files0.sh \ diff --git a/tests/misc/cut-huge-to-eol-range.sh b/tests/misc/cut-huge-to-eol-range.sh new file mode 100755 index 0000000..fea8505 --- /dev/null +++ b/tests/misc/cut-huge-to-eol-range.sh @@ -0,0 +1,30 @@ +#!/bin/sh +# Ensure that cut does not allocate mem for a range like -b9999999999999- + +# Copyright (C) 2012 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src +print_ver_ cut +require_ulimit_ + +# 2^31-1 +# From coreutils-8.10 through 8.20, this would make cut try to allocate +# a 256MiB bit vector. With a 20MB limit on VM, the following would fail. +(ulimit -v 20000; : | cut -b2147483647- > err 2>&1) || fail=1 + +compare /dev/null err || fail=1 + +Exit $fail -- 1.8.0.1.352.gfb4c622
