bug#20936: suggestion for a 'wart-ish' extension off of 'sort'
I admit the ability to show a summary line might not bethe first thing you'd think a pure-sorting utility might do, but it would be awfully handy if sort had a 'Numeric sum' option (-N -- preferred '-s', but it's already taken) to go with the -h sorting: ala: du -sh *|sort -h|tail 6.0Mfirmware 6.7Mkernel 8.4Mtools 26M net 29M sound 30M Documentation 31M include 37M fs 128March 330Mdrivers --- vs. --- du -sh *|hsort -s|tail -12 6.0Mfirmware 6.7Mkernel 8.4Mtools 26M net 29M sound 30M Documentation 31M include 37M fs 128March 330Mdrivers - 649.4M TOTAL --- I'd donate the code for hsort, but its in perl -- I wrote it several years ago to do what 'sort -h' does, but also put in the option for a summary line -- handy companion for 'human numbers', which would otherwise take alot more typing (I think -- unless there's some hidden switch I don't know about).
bug#20936: suggestion for a 'wart-ish' extension off of 'sort'
Hi, On Tue, Jun 30, 2015 at 12:28:09AM -0700, Linda Walsh wrote: I admit the ability to show a summary line might not bethe first thing you'd think a pure-sorting utility might do, but it would be awfully handy if sort had a 'Numeric sum' option (-N -- preferred '-s', but it's already taken) to go with the -h sorting: ala: du -sh *|sort -h|tail Why not use 'du -shc * | sort -h | tail -n11'? The total produced by du will sort after all the individual parts. Thanks, Erik
bug#20775: cp -a -u destroys files after they are copied
Hello, the bug was observed on Linux this time. Unfortunately I was hit with the bug when backing up several large photo directories with hard-links to one target directory on an external drive. So it was not obvious for me at first, that something was going wrong. From my point of view (without looking at the source) it should be easy to repair by - not attempting to create a hard-link from a file to itself - not deleting a file that is about to be used as the source paramter of a hard-link to be created. best regards Steffen Zahn On Tue, Jun 30, 2015 at 9:02 AM L. A. Walsh coreut...@tlinx.org wrote: I think you'll find this was reported 3 years ago.. bug#10471: Severe or critical - deletes existing files and leaves nothing. (cp) https://lists.gnu.org/archive/html/bug-coreutils/2015-04/msg1.html Unfortunately it was closed it out w/the reason that it was a cygwin/windows-only -- which I disagreed with. I was told the cygwin dev would check it out and if it was in coreutils would move it back to active status (that was 3+ years ago). On 6/8/2015 9:18 PM, Steffen Zahn wrote: Hello, I found that the cp command acts sub-optimal when copying hard-linked files of the same name from several directories to one target directory, it first copies the files then removes them. I cannot see how that can be the intended behaviour. Please fix this. best regards Steffen Zahn sz@gandalf:~ $ cd /tmp sz@gandalf:/tmp $ mkdir 1 2 3 sz@gandalf:/tmp $ touch 1/a sz@gandalf:/tmp $ ln 1/a 2/ sz@gandalf:/tmp $ ls -li 1 2 1: total 0 262424 -rw-r--r-- 2 sz sz 0 Jun 9 06:10 a 2: total 0 262424 -rw-r--r-- 2 sz sz 0 Jun 9 06:10 a sz@gandalf:/tmp $ cp -a -u --verbose 1/* 2/* 3/ '1/a' - '3/a' removed '3/a' cp: cannot create hard link '3/a' to '3/a': No such file or directory sz@gandalf:/tmp $ cp --version
bug#20936: suggestion for a 'wart-ish' extension off of 'sort'
Hi, On Tue, Jun 30, 2015 at 02:35:17AM -0700, Linda Walsh wrote: On 6/30/2015 12:46 AM, Erik Auerswald wrote: du -sh *|sort -h|tail Why not use 'du -shc * | sort -h | tail -n11'? The total produced by du will sort after all the individual parts. Good idea -- didn't know about '-c', but two things, 1 troubling, the other a confusion. If you have a dir named 'total' it can be slightly confusing: You'll always know that the last total is the total of the above. ;-) Ishtar:/tmp/dutest du -shc * |sort -h|tail 1.5Msperl,v 3.6Mtotal 5.0Mtotal Ishtar:/tmp/dutest du -sh * |hsort -s|tail 1.5Msperl,v 3.6Mtotal - 5.1MTOTAL But more a more obvious problem is 'du -shc' seems to be coming up with the wrong number -- i.e. 1.5+3.6 = 5.1, not 5.0. That are probably rounding errors avoided by du, that hsort cannot avoid anymore. Thanks, Erik
bug#20936: suggestion for a 'wart-ish' extension off of 'sort'
On 6/30/2015 3:51 AM, Erik Auerswald wrote: Ishtar:/tmp/dutest du -shc * |sort -h|tail 1.5Msperl,v 3.6Mtotal 5.0Mtotal Ishtar:/tmp/dutest du -sh * |hsort -s|tail 1.5Msperl,v 3.6Mtotal - 5.1MTOTAL But more a more obvious problem is 'du -shc' seems to be coming up with the wrong number -- i.e. 1.5+3.6 = 5.1, not 5.0. That are probably rounding errors avoided by du, that hsort cannot avoid anymore. --- 1) I think you're right, but 2) it still looks odd to see 1.5+3.6=5.0 and not 5.1
bug#20775: cp -a -u destroys files after they are copied
Steffen Zahn wrote: it should be easy to repair Really? Without significantly affecting performance in the usual case? Let's see a patch.
bug#20923: mgetgroups.c vs getgrouplist warning on OS X
Jim Meyering wrote:
same result as before:
OK, let's give up on this approach and try something more direct. I installed
the attached patch; does it work on OS X?
From b1a1450d454698a765a625804fe336305209c696 Mon Sep 17 00:00:00 2001
From: Paul Eggert egg...@cs.ucla.edu
Date: Tue, 30 Jun 2015 09:32:07 -0700
Subject: [PATCH] mgetgroups: port to strict OS X
The previous fix wasn't working, so use a bigger hammer (Bug#20923).
* lib/mgetgroups.c: Ignore -Wpointer-sign diagnostics.
(getgrouplist_gids) [HAVE_GETGROUPLIST]: Remove. All uses removed.
* m4/mgetgroups.m4 (gl_MGETGROUPS): Revert recent changes.
---
ChangeLog| 8
lib/mgetgroups.c | 14 +++---
m4/mgetgroups.m4 | 35 +--
3 files changed, 16 insertions(+), 41 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index cb6ca08..37120cd 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2015-06-30 Paul Eggert egg...@cs.ucla.edu
+
+ mgetgroups: port to strict OS X
+ The previous fix wasn't working, so use a bigger hammer (Bug#20923).
+ * lib/mgetgroups.c: Ignore -Wpointer-sign diagnostics.
+ (getgrouplist_gids) [HAVE_GETGROUPLIST]: Remove. All uses removed.
+ * m4/mgetgroups.m4 (gl_MGETGROUPS): Revert recent changes.
+
2015-06-29 Paul Eggert egg...@cs.ucla.edu
mgetgroups: port to strict OS X
diff --git a/lib/mgetgroups.c b/lib/mgetgroups.c
index dd0c7a5..6acb019 100644
--- a/lib/mgetgroups.c
+++ b/lib/mgetgroups.c
@@ -33,6 +33,12 @@
#include getugroups.h
#include xalloc-oversized.h
+/* Work around an incompatibility of OS X 10.11: getgrouplist
+ accepts int *, not gid_t *, and int and gid_t differ in sign. */
+#if 4 __GNUC__ + (3 = __GNUC_MINOR__)
+# pragma GCC diagnostic ignored -Wpointer-sign
+#endif
+
static gid_t *
realloc_groupbuf (gid_t *g, size_t num)
{
@@ -64,11 +70,6 @@ mgetgroups (char const *username, gid_t gid, gid_t **groups)
gid_t *g;
#if HAVE_GETGROUPLIST
-# if HAVE_GETGROUPLIST_WITH_INT
-# define getgrouplist_gids(g) ((int *) (g))
-# else
-# define getgrouplist_gids(g) (g)
-# endif
/* We prefer to use getgrouplist if available, because it has better
performance characteristics.
@@ -92,8 +93,7 @@ mgetgroups (char const *username, gid_t gid, gid_t **groups)
int last_n_groups = max_n_groups;
/* getgrouplist updates max_n_groups to num required. */
- ng = getgrouplist (username, gid, getgrouplist_gids (g),
- max_n_groups);
+ ng = getgrouplist (username, gid, g, max_n_groups);
/* Some systems (like Darwin) have a bug where they
never increase max_n_groups. */
diff --git a/m4/mgetgroups.m4 b/m4/mgetgroups.m4
index 5747148..42b5cf8 100644
--- a/m4/mgetgroups.m4
+++ b/m4/mgetgroups.m4
@@ -1,4 +1,4 @@
-#serial 7
+#serial 5
dnl Copyright (C) 2007-2015 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -6,38 +6,5 @@ dnl with or without modifications, as long as this notice is preserved.
AC_DEFUN([gl_MGETGROUPS],
[
- AC_CHECK_HEADERS_ONCE([grp.h])
AC_CHECK_FUNCS_ONCE([getgrouplist])
- if test $ac_cv_func_getgrouplist = yes; then
-AC_CACHE_CHECK([for getgrouplist with int signature],
- [gl_cv_getgrouplist_with_int],
- [gl_cv_getgrouplist_with_int=no
- gl_save_c_werror_flag=$ac_c_werror_flag
- ac_c_werror_flag=yes
- AC_COMPILE_IFELSE(
- [AC_LANG_PROGRAM(
-[[#if HAVE_GRP_H
- #include grp.h
- #endif
- int groups[1];
- int ngroups = 1;
-]],
-[[return - getgrouplist (root, 0, groups, ngroups);]])],
- [],
- [AC_COMPILE_IFELSE(
- [AC_LANG_PROGRAM(
- [[#if HAVE_GRP_H
- #include grp.h
-#endif
-int groups[sizeof (gid_t) == sizeof (int) ? 1 : -1];
-int ngroups = 1;
- ]],
- [[return - getgrouplist (root, 0, groups, ngroups);]])],
-[gl_cv_getgrouplist_with_int=yes])])
- ac_c_werror_flag=$gl_save_c_werror_flag])
-if test $gl_cv_getgrouplist_with_int = yes; then
- AC_DEFINE([HAVE_GETGROUPLIST_WITH_INT], 1,
-[Define to 1 if getgrouplist accepts and returns int and not gid_t.])
-fi
- fi
])
--
2.1.0
bug#20936: suggestion for a 'wart-ish' extension off of 'sort'
Hello,
On 06/30/2015 03:28 AM, Linda Walsh wrote:
I admit the ability to show a summary line might not bethe first
thing you'd think a pure-sorting utility might do, but it would be
awfully handy if sort had a 'Numeric sum' option (-N -- preferred
'-s', but it's already taken) to go with the -h sorting:
...
du -sh *|sort -h|tail
A slightly different approach would be to defer the human size to later,
and enable to do any calculation you want manually, then convert to human sizes with
numfmt:
du -s * \
| sort -n \
| awk '{ sum+=$1 ; print } END { print sum, Total }' \
| numfmt --to=iec
One more thing: instead of 'du -s *', perhaps 'du -d1' would work better
(depending on your needs), as it will print sizes used only by directories.
- assaf
bug#20923: mgetgroups.c vs getgrouplist warning on OS X
On Tue, Jun 30, 2015 at 9:36 AM, Paul Eggert egg...@cs.ucla.edu wrote: Jim Meyering wrote: same result as before: OK, let's give up on this approach and try something more direct. I installed the attached patch; does it work on OS X? Perfect. I made latest coreutils use latest from gnulib, and now it all compiles warning-free there with --enable-gcc-warnings and gcc-6.x Thank you!
bug#20936: suggestion for a 'wart-ish' extension off of 'sort'
2015-06-30 13:05:38 -0400, Assaf Gordon: [...] One more thing: instead of 'du -s *', perhaps 'du -d1' would work better (depending on your needs), as it will print sizes used only by directories. Yes, and it will also include hidden files/dirs and won't have problems with filenames starting with -. Add -a if you also want non-directory files. -- Stephane
bug#20936: suggestion for a 'wart-ish' extension off of 'sort'
2015-06-30 12:51:02 +0200, Erik Auerswald: [...] But more a more obvious problem is 'du -shc' seems to be coming up with the wrong number -- i.e. 1.5+3.6 = 5.1, not 5.0. That are probably rounding errors avoided by du, that hsort cannot avoid anymore. [...] Also, du -c gives you the cumulative usage (think of hard links that need to be counted once), not the sum of the above. -- Stephane
bug#20775: cp -a -u destroys files after they are copied
I think you'll find this was reported 3 years ago.. bug#10471: Severe or critical - deletes existing files and leaves nothing. (cp) https://lists.gnu.org/archive/html/bug-coreutils/2015-04/msg1.html Unfortunately it was closed it out w/the reason that it was a cygwin/windows-only -- which I disagreed with. I was told the cygwin dev would check it out and if it was in coreutils would move it back to active status (that was 3+ years ago). On 6/8/2015 9:18 PM, Steffen Zahn wrote: Hello, I found that the cp command acts sub-optimal when copying hard-linked files of the same name from several directories to one target directory, it first copies the files then removes them. I cannot see how that can be the intended behaviour. Please fix this. best regards Steffen Zahn sz@gandalf:~ $ cd /tmp sz@gandalf:/tmp $ mkdir 1 2 3 sz@gandalf:/tmp $ touch 1/a sz@gandalf:/tmp $ ln 1/a 2/ sz@gandalf:/tmp $ ls -li 1 2 1: total 0 262424 -rw-r--r-- 2 sz sz 0 Jun 9 06:10 a 2: total 0 262424 -rw-r--r-- 2 sz sz 0 Jun 9 06:10 a sz@gandalf:/tmp $ cp -a -u --verbose 1/* 2/* 3/ '1/a' - '3/a' removed '3/a' cp: cannot create hard link '3/a' to '3/a': No such file or directory sz@gandalf:/tmp $ cp --version
bug#20936: suggestion for a 'wart-ish' extension off of 'sort'
On 6/30/2015 12:46 AM, Erik Auerswald wrote: du -sh *|sort -h|tail Why not use 'du -shc * | sort -h | tail -n11'? The total produced by du will sort after all the individual parts. Good idea -- didn't know about '-c', but two things, 1 troubling, the other a confusion. If you have a dir named 'total' it can be slightly confusing: Ishtar:/tmp/dutest du -shc * |sort -h|tail 1.5Msperl,v 3.6Mtotal 5.0Mtotal Ishtar:/tmp/dutest du -sh * |hsort -s|tail 1.5Msperl,v 3.6Mtotal - 5.1MTOTAL But more a more obvious problem is 'du -shc' seems to be coming up with the wrong number -- i.e. 1.5+3.6 = 5.1, not 5.0. In my original example, it's off by more: Ishtar:linux/linux-4.1.0 du -sch *|sort -h|tail 6.7Mkernel 8.4Mtools 26M net 29M sound 30M Documentation 31M include 37M fs 128March 330Mdrivers 645Mtotal Ishtar:linux/linux-4.1.0 du -sh *|hsort -s|tail 8.4Mtools 26M net 29M sound 30M Documentation 31M include 37M fs 128March 330Mdrivers - 649.4M TOTAL
