df command hangs system.
Hi, I'm facing the problem on my Linux machine with version info: Linux wren 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:56:28 EST 2006 x86_64 x86_64 x86_64 GNU/Linux While executing df -kP or df command, the system will go in wait state until forcefully kill by ctrl+c, [EMAIL PROTECTED] rsingh2]# df -kP Filesystem 1024-blocks Used Available Capacity Mounted on /dev/sda3140014124 13877564 119024184 11% / /dev/sda1 101086 13074 82793 14% /boot none 2022636 0 2022636 0% /dev/shm titan:/vol/vol_qa167772160 119488576 48283584 72% /qa titan:/vol/vol_users 109051904 44750720 64301184 42% /users titan:/vol/vol_road 483183840 338922336 144261504 71% /road Please let me know, what should be the workaround for this problem and cause of this problem. Thanks, Suresh Papneja DISCLAIMER: --- The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any mail and attachments please check them for viruses and defect. --- ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Alignment bug in ls with UTF-8 filenames under Mac OS X
Vincent Lefevre [EMAIL PROTECTED] wrote:
On 2007-01-19 01:23:44 +0100, Bruno Haible wrote:
Apple Terminal version 1.4.6, part of MacOS X 10.3.9, is affected.
I forgot to say. This is still not fixed in Terminal 1.5 (133) from
Mac OS X 10.4.8.
Thanks.
I've checked this in:
* coreutils.texi (ls: General output formatting): Mention the
workarounds to accommodate the Apple Terminal bug.
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 6fc6704..89e97d8 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -6419,6 +6419,13 @@ Assume that each tab stop is @var{cols} columns wide.
The default is 8.
@command{ls} uses tabs where possible in the output, for efficiency. If
@var{cols} is zero, do not use tabs at all.
[EMAIL PROTECTED] FIXME: remove in 2009, if Apple Terminal has been fixed for
long enough.
+Some terminal emulators (at least Apple Terminal 1.5 (133) from Mac OS X
10.4.8)
+do not properly align columns to the right of a TAB following a
[EMAIL PROTECTED] byte. If you use such a terminal emulator, use the
[EMAIL PROTECTED] option or put @code{TABSIZE=0} in your environment to tell
[EMAIL PROTECTED] to align using spaces, not tabs.
+
@item -w
@itemx [EMAIL PROTECTED]
@opindex -w
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: df command hangs system.
On Fri, 19 Jan 2007, Suresh Kumar Papneja, Noida wrote:
While executing df -kP or df command, the system will go in wait
state until forcefully kill by ctrl+c,
It's likely that an NFS mount is hanging in the statfs call. You should
be able to see this by running df under strace.
$ strace -e statfs,statfs64 df
statfs64(/, 84, {f_type=EXT2_SUPER_MAGIC, f_bsize=4096, f_blocks=4828858,
f_bfree=2920424, f_bavail=2675132, f_files=2452800, f_ffree=2200287, f_fsid={0, 0}, f_namelen=255,
f_frsize=4096}) = 0
etc.
Cheers,
Phil
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
'sort' race condition with atexit cleanup and signals
Here's a patch to fix a race condition that Dan Hipschman and I
thought of while walking across the courtyard to Boelter Hall at UCLA.
The problem is that a signal can come in while cleanup is running
during a premature exit (e.g., due to an I/O error), causing 'sort' to
unlink a file that has already been unlinked. This might in theory
cause 'sort' to unlink some other process's temp file.
Dan wrote the initial version of the patch and I tweaked it a bit. I
thought it a bit cleaner to have a single cleanup function than
multiple calls to atexit, since we should not invoke some of this
stuff until the signal mask is known.
2007-01-19 Dan Hipschman [EMAIL PROTECTED]
and Paul Eggert [EMAIL PROTECTED]
* src/sort.c (cleanup): Clear temphead at the end.
(exit_cleanup): New function.
(main): Don't invoke atexit until we're ready.
Invoke it with exit_cleanup, not with cleanup and close_stdout,
to avoid a race condition with cleanup and signal handling.
diff --git a/src/sort.c b/src/sort.c
index f03237c..326866f 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -417,6 +417,25 @@ cleanup (void)
for (node = temphead; node; node = node-next)
unlink (node-name);
+ temphead = NULL;
+}
+
+/* Cleanup actions to take when exiting. */
+
+static void
+exit_cleanup (void)
+{
+ if (temphead)
+{
+ /* Clean up any remaining temporary files in a critical section so
+that a signal handler does not try to clean them too. */
+ sigset_t oldset;
+ sigprocmask (SIG_BLOCK, caught_signals, oldset);
+ cleanup ();
+ sigprocmask (SIG_SETMASK, oldset, NULL);
+}
+
+ close_stdout ();
}
/* Create a new temporary file, returning its newly allocated name.
@@ -2302,10 +2321,7 @@ main (int argc, char **argv)
bindtextdomain (PACKAGE, LOCALEDIR);
textdomain (PACKAGE);
- atexit (cleanup);
-
initialize_exit_failure (SORT_FAILURE);
- atexit (close_stdout);
hard_LC_COLLATE = hard_locale (LC_COLLATE);
#if HAVE_NL_LANGINFO
@@ -2365,6 +2381,9 @@ main (int argc, char **argv)
#endif
}
+ /* The signal mask is known, so it is safe to invoke exit_cleanup. */
+ atexit (exit_cleanup);
+
gkey.sword = gkey.eword = SIZE_MAX;
gkey.ignore = NULL;
gkey.translate = NULL;
M ChangeLog
M src/sort.c
Committed as 600ef4a9207574c4459963a7948faa67de142e9c
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: 'sort' race condition with atexit cleanup and signals
Paul Eggert [EMAIL PROTECTED] wrote: Here's a patch to fix a race condition that Dan Hipschman and I thought of while walking across the courtyard to Boelter Hall at UCLA. The problem is that a signal can come in while cleanup is running during a premature exit (e.g., due to an I/O error), causing 'sort' to unlink a file that has already been unlinked. This might in theory cause 'sort' to unlink some other process's temp file. Dan wrote the initial version of the patch and I tweaked it a bit. I thought it a bit cleaner to have a single cleanup function than multiple calls to atexit, since we should not invoke some of this stuff until the signal mask is known. 2007-01-19 Dan Hipschman [EMAIL PROTECTED] and Paul Eggert [EMAIL PROTECTED] * src/sort.c (cleanup): Clear temphead at the end. (exit_cleanup): New function. (main): Don't invoke atexit until we're ready. Invoke it with exit_cleanup, not with cleanup and close_stdout, to avoid a race condition with cleanup and signal handling. Come on guys... Where's the test case? :-) 1/2 With a delay-inducing unlink wrapper, I'll bet it's feasible. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
sort etc. should catch SIGXFSZ etc.
If 'sort' runs out of disk space or CPU time, it doesn't clean up its
temporary files properly. 'ls' and 'csplit' have similar problems. I
noticed that 'sort', 'ls', and 'csplit' are not consistent about which
signals they catch; it seems to me that they should be. Which signals
to catch is a bit of a judgment call, e.g., I have a program alarm
such that alarm 20 sort runs 'sort' in an environment where it will
get SIGALRM in 20 seconds, which is a nice utility to have. So I was
fairly conservative here, and listed in The usual suspects. a
larger signal set than one might otherwise list.
I considered putting the 'sig' array into a separate module to avoid
duplication, but it didn't quite seem worth the hassle, as its
contents aren't the same in ls.c.
2007-01-19 Paul Eggert [EMAIL PROTECTED]
Standardize on list of signals when an app catches signals.
* src/csplit.c (main): Also catch SIGALRM, SIGPIPE, SIGPOLL,
SIGPROF, SIGVTALRM, SIGXCPU, SIGXFSZ.
* src/ls.c (main): Likewise (except SIGPIPE was already caught).
Note that ls.c is special, as it also catches SIGTSTP.
* src/sort.c (main): Likewise. Also catch SIGQUIT.
diff --git a/src/csplit.c b/src/csplit.c
index 382fd66..a26c80d 100644
--- a/src/csplit.c
+++ b/src/csplit.c
@@ -1402,7 +1402,26 @@ main (int argc, char **argv)
{
int i;
-static int const sig[] = { SIGHUP, SIGINT, SIGQUIT, SIGTERM };
+static int const sig[] =
+ {
+ /* The usual suspects. */
+ SIGALRM, SIGHUP, SIGINT, SIGPIPE, SIGQUIT, SIGTERM,
+#ifdef SIGPOLL
+ SIGPOLL,
+#endif
+#ifdef SIGPROF
+ SIGPROF,
+#endif
+#ifdef SIGVTALRM
+ SIGVTALRM,
+#endif
+#ifdef SIGXCPU
+ SIGXCPU,
+#endif
+#ifdef SIGXFSZ
+ SIGXFSZ,
+#endif
+ };
enum { nsigs = sizeof sig / sizeof sig[0] };
#if SA_NOCLDSTOP
diff --git a/src/ls.c b/src/ls.c
index feba591..3864ed0 100644
--- a/src/ls.c
+++ b/src/ls.c
@@ -1113,8 +1113,29 @@ main (int argc, char **argv)
int n_files;
/* The signals that are trapped, and the number of such signals. */
- static int const sig[] = { SIGHUP, SIGINT, SIGPIPE,
-SIGQUIT, SIGTERM, SIGTSTP };
+ static int const sig[] =
+{
+ /* This one is handled specially. */
+ SIGTSTP,
+
+ /* The usual suspects. */
+ SIGALRM, SIGHUP, SIGINT, SIGPIPE, SIGQUIT, SIGTERM,
+#ifdef SIGPOLL
+ SIGPOLL,
+#endif
+#ifdef SIGPROF
+ SIGPROF,
+#endif
+#ifdef SIGVTALRM
+ SIGVTALRM,
+#endif
+#ifdef SIGXCPU
+ SIGXCPU,
+#endif
+#ifdef SIGXFSZ
+ SIGXFSZ,
+#endif
+};
enum { nsigs = sizeof sig / sizeof sig[0] };
#if ! SA_NOCLDSTOP
diff --git a/src/sort.c b/src/sort.c
index 326866f..8a22796 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -2350,7 +2350,26 @@ main (int argc, char **argv)
{
size_t i;
-static int const sig[] = { SIGHUP, SIGINT, SIGPIPE, SIGTERM };
+static int const sig[] =
+ {
+ /* The usual suspects. */
+ SIGALRM, SIGHUP, SIGINT, SIGPIPE, SIGQUIT, SIGTERM,
+#ifdef SIGPOLL
+ SIGPOLL,
+#endif
+#ifdef SIGPROF
+ SIGPROF,
+#endif
+#ifdef SIGVTALRM
+ SIGVTALRM,
+#endif
+#ifdef SIGXCPU
+ SIGXCPU,
+#endif
+#ifdef SIGXFSZ
+ SIGXFSZ,
+#endif
+ };
enum { nsigs = sizeof sig / sizeof sig[0] };
#if SA_NOCLDSTOP
M ChangeLog
M src/csplit.c
M src/ls.c
M src/sort.c
Committed as 62efe8374443cffce754050f4d7f48bd043c31b7
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: 'sort' race condition with atexit cleanup and signals
Paul Eggert [EMAIL PROTECTED] wrote: Here's a patch to fix a race condition that Dan Hipschman and I thought of while walking across the courtyard to Boelter Hall at UCLA. The problem is that a signal can come in while cleanup is running during a premature exit (e.g., due to an I/O error), causing 'sort' to unlink a file that has already been unlinked. This might in theory cause 'sort' to unlink some other process's temp file. Dan wrote the initial version of the patch and I tweaked it a bit. I thought it a bit cleaner to have a single cleanup function than multiple calls to atexit, since we should not invoke some of this stuff until the signal mask is known. 2007-01-19 Dan Hipschman [EMAIL PROTECTED] and Paul Eggert [EMAIL PROTECTED] * src/sort.c (cleanup): Clear temphead at the end. (exit_cleanup): New function. (main): Don't invoke atexit until we're ready. Invoke it with exit_cleanup, not with cleanup and close_stdout, to avoid a race condition with cleanup and signal handling. Thank you. I've checked that in. http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commit;h=e175f0d5b ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: RFC: change chown *not* to look up numeric user/group names
Jim Meyering [EMAIL PROTECTED] wrote:
...
I propose to change GNU chown to perform that look-up of an all-numeric
user or group string only when the POSIXLY_CORRECT envvar is set.
Otherwise, (when POSIXLY_CORRECT is not set and a name is a valid user
ID or group ID), chown would use the value obtained from converting the
string with a function like strtoul.
For consistency, the same policy would apply to chgrp.
Here's a postscript to the discussion that started above, aka,
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/8682
Today I realized that there is a way to avoid the look-up
without resorting to a new option.
The trick is to realize that in commands like the following
chown +0:+0 file
chgrp +0 file
the getpwnam and getgrnam look-up operations must always
fail, since + is not a valid character in a user name.
Hence, chown and chgrp can skip those function calls when
the first byte is +. chown and chgrp have always accepted
a leading sign on those arguments.
So if you want to avoid the expense of what you deem to be a pointless
name-to-ID look-up of 0 (or any numeric value), just prepend a + --
if you're using the patch below.
Here's a demo:
The old way:
# strace -e open,chown ./chown 0:0 /t/Z
open(/etc/ld.so.cache, O_RDONLY) = 3
open(/lib/libc.so.6, O_RDONLY)= 3
open(/etc/nsswitch.conf, O_RDONLY)= 3
open(/etc/ld.so.cache, O_RDONLY) = 3
open(/lib/libnss_compat.so.2, O_RDONLY) = 3
open(/lib/libnsl.so.1, O_RDONLY) = 3
open(/etc/ld.so.cache, O_RDONLY) = 3
open(/lib/libnss_nis.so.2, O_RDONLY) = 3
open(/lib/libnss_files.so.2, O_RDONLY) = 3
open(/etc/passwd, O_RDONLY) = 3
open(/etc/group, O_RDONLY)= 3
open(., O_RDONLY) = 3
chown(/t/Z, 0, 0) = 0
Process 17271 detached
New way: no look-up:
# strace -e open,chown ./chown +0:+0 /t/Z
open(/etc/ld.so.cache, O_RDONLY) = 3
open(/lib/libc.so.6, O_RDONLY)= 3
open(., O_RDONLY) = 3
chown(/t/Z, 0, 0) = 0
Process 17234 detached
However, there is a drawback: this syntax is not portable.
For example, Solaris 10's chgrp and chown reject it, e.g.,
$ /bin/chgrp +0 k
chgrp: unknown group: +0
[Exit 2]
Well, seeing that, I'm less enthusiastic about this change.
I'll sleep on it. If I do apply it, I'll also document that
GNU chown and chgrp accept this syntax, and the semantics.
Here's the proposed gnulib patch:
2007-01-19 Jim Meyering [EMAIL PROTECTED]
* lib/userspec.c (parse_with_separator): If a user or group string
starts with +, skip the corresponding name-to-ID look-up, since
such a look-up must fail: user and group names may not include +.
Index: lib/userspec.c
===
RCS file: /sources/gnulib/gnulib/lib/userspec.c,v
retrieving revision 1.53
diff -u -p -r1.53 userspec.c
--- lib/userspec.c 13 Sep 2006 22:38:14 - 1.53
+++ lib/userspec.c 19 Jan 2007 22:38:12 -
@@ -1,5 +1,5 @@
/* userspec.c -- Parse a user and group string.
- Copyright (C) 1989-1992, 1997-1998, 2000, 2002-2006 Free Software
+ Copyright (C) 1989-1992, 1997-1998, 2000, 2002-2007 Free Software
Foundation, Inc.
This program is free software; you can redistribute it and/or modify
@@ -156,7 +156,8 @@ parse_with_separator (char const *spec,
if (u != NULL)
{
- pwd = getpwnam (u);
+ /* If it starts with +, skip the look-up. */
+ pwd = (*u == '+' ? NULL : getpwnam (u));
if (pwd == NULL)
{
bool use_login_group = (separator != NULL g == NULL);
@@ -196,7 +197,8 @@ parse_with_separator (char const *spec,
if (g != NULL error_msg == NULL)
{
/* Explicit group. */
- grp = getgrnam (g);
+ /* If it starts with +, skip the look-up. */
+ grp = (*g == '+' ? NULL : getgrnam (g));
if (grp == NULL)
{
unsigned long int tmp;
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: feature request: gzip/bzip support for sort
Hi,
I think this patch addresses everything Paul mentioned in his critique
of my last attempt. I did look at gnulib pipe module, but there were
some problems with using it out of the box. First, it takes a
filename as the stdout of the child process, when for temp files it's
better to pass a file descriptor. That's a rather small issue, though,
and would be pretty easy to add an additional function that fixed that.
Another problem is that since it does the fork/exec in one call, the
race condition where the child and parent both receive signals between
the fork and child's exec can't be solved the way I solve it in this
patch. Rather than change the way sort handles signals, I pleaded with
Paul (well, I asked him nicely :-) to be able to put off the portability
issues of fork until I had more time. Hence, I've just added a check
for fork() in configure.ac and any system that doesn't have it will have
to wait a while for this feature.
I also tried to address Jim's concern about fork() failing. I clean up
zombie children every so often in the main loops of sort() and
mergefps() so as not to bleed the system dry. This seems to keep the
number of defunct and running child processes close to a minimum. If we
can't fork, I retry some number of times, sleeping a bit in between and
killing off zombies (sounds like a good time :-). I was able to test
the case where fork() fails with a macro like this:
unsigned int foo;
#define fork() (foo++ % 3 ? fork() : (errno = EAGAIN, -1))
I'm not sure how to test the case where we fork a compression process,
it exits, we reap it, and we create another process with the same PID
before we run the decompression process on the file it created. I did
address this issue, and of course I *think* my solution is correct, but
like I said, I'm not sure how to test it.
Anyway, here's the patch.
Index: NEWS
===
RCS file: /sources/coreutils/coreutils/NEWS,v
retrieving revision 1.467
diff -p -u -r1.467 NEWS
--- NEWS18 Jan 2007 09:18:21 - 1.467
+++ NEWS20 Jan 2007 05:49:27 -
@@ -29,6 +29,11 @@ GNU coreutils NEWS
rm --interactive=never F no longer prompts for an unwritable F
+** New features
+
+ sort can now compresses temporary files to improve performance of
+ very large sorts.
+
* Noteworthy changes in release 6.7 (2006-12-08) [stable]
Index: configure.ac
===
RCS file: /sources/coreutils/coreutils/configure.ac,v
retrieving revision 1.102
diff -p -u -r1.102 configure.ac
--- configure.ac28 Dec 2006 08:18:17 - 1.102
+++ configure.ac20 Jan 2007 05:49:27 -
@@ -39,6 +39,8 @@ gl_EARLY
gl_INIT
coreutils_MACROS
+AC_FUNC_FORK
+
AC_CHECK_FUNCS(uname,
OPTIONAL_BIN_PROGS=$OPTIONAL_BIN_PROGS uname\$(EXEEXT)
MAN=$MAN uname.1)
Index: doc/coreutils.texi
===
RCS file: /sources/coreutils/coreutils/doc/coreutils.texi,v
retrieving revision 1.363
diff -p -u -r1.363 coreutils.texi
--- doc/coreutils.texi 4 Jan 2007 11:04:46 - 1.363
+++ doc/coreutils.texi 20 Jan 2007 05:49:27 -
@@ -3411,6 +3411,19 @@ value as the directory for temporary fil
@option{--temporary-directory} (@option{-T}) option in turn overrides
the environment variable.
[EMAIL PROTECTED] GNUSORT_COMPRESSOR
+To improve performance when sorting very large files, GNU sort will,
+by default, try to compress temporary files with the program
[EMAIL PROTECTED]/bin/gzip}. The environment variable @env{GNUSORT_COMPRESSOR}
+can be set to the name of another program to be used. The program
+specified must compress standard input to standard output when no
+arguments are given to it, and it must decompress standard input to
+standard output when the @option{-d} argument is given to it. To
+disable compression of temporary files, set the variable to the empty
+string. Whitespace characters, and the backslash character are
+reserved for future use in the program name. If the program exits
+with nonzero status, sort will terminate with an error.
+
The following options affect the ordering of output lines. They may be
specified globally or as part of a specific key field. If no key
Index: src/sort.c
===
RCS file: /sources/coreutils/coreutils/src/sort.c,v
retrieving revision 1.345
diff -p -u -r1.345 sort.c
--- src/sort.c 19 Jan 2007 22:28:31 - 1.345
+++ src/sort.c 20 Jan 2007 05:49:29 -
@@ -1,5 +1,5 @@
/* sort - sort lines of text (with all kinds of options).
- Copyright (C) 1988, 1991-2006 Free Software Foundation, Inc.
+ Copyright (C) 1988, 1991-2007 Free Software Foundation, Inc.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published
