bug#15926: RFE: unlink command already uses 'unlink' call; make 'rm' use 'remove' call
Since there is already an unlink command that corresponds to unlinking a file, but there seems to be no command corresponding to the POSIX remove command, it seems upgrading 'rm' to use the 'remove' POSIX call would be a beneficial move of all the recent POSIX changes. So how about upgrading 'rm' to use the remove function so it would work on empty directories as well.
bug#15926: RFE: unlink command already uses 'unlink' call; make 'rm' use 'remove' call
tag 15926 notabug close 15926 stop On 11/19/2013 11:56 AM, Linda Walsh wrote: Since there is already an unlink command that corresponds to unlinking a file, but there seems to be no command corresponding to the POSIX remove command, it seems upgrading 'rm' to use the 'remove' POSIX call would be a beneficial move of all the recent POSIX changes. So how about upgrading 'rm' to use the remove function so it would work on empty directories as well. Well we have the -d option to rm to explicitly do that. That's a fairly fundamental change that would have backwards compat issues. POSIX is explicit in the handling of directories also: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html thanks, Pádraig.
bug#15926: RFE: unlink command already uses 'unlink' call; make 'rm' use 'remove' call
On 19/11/2013 04:15, Pádraig Brady wrote: tag 15926 notabug close 15926 stop On 11/19/2013 11:56 AM, Linda Walsh wrote: Since there is already an unlink command that corresponds to unlinking a file, but there seems to be no command corresponding to the POSIX remove command, it seems upgrading 'rm' to use the 'remove' POSIX call would be a beneficial move of all the recent POSIX changes. So how about upgrading 'rm' to use the remove function so it would work on empty directories as well. Well we have the -d option to rm to explicitly do that. --- Does the posix remove call require a -d? I thought it made more sense to have rm correspond to remove than create another command called remove that called remove. Are you saying that you think it would be better to have a remove command that does this? As for the posix document you pointed at, I'm suggesting doing step 4 after step 1... same thing posix does, just reordering things a bit. This came up, BTW, because a useless error message was added to find such that if you used find dir/. -type d -empty -delete, it now exits with an error status telling you nothing. other than it didn't delete the . entry from inside the dir (which most people know can't be done anyway). But now there is no way to determine if the find command failed, or if it is just making noise over things it never has done. Alternatively, I suggested adding a -f flag that would occur, positionally before any path arguments (like the -H/-L/-P opts now), that would silence such fails and return a 0 status for delete failures that the OS cannot do. No useful purpose is served by the new error message, and creating it breaks compatibility with scripts.
bug#15927: Bug in date command
date --date=10/20/2013 result in date: invalid date `10/20/2013' version: date (GNU coreutils) 8.13
bug#15926: RFE: unlink command already uses 'unlink' call; make 'rm' use 'remove' call
On 11/19/2013 11:17 AM, Linda Walsh wrote: Well we have the -d option to rm to explicitly do that. --- Does the posix remove call require a -d? Huh? There is no POSIX remove(1), only remove(3), unlink(2), rmdir(2), rm(1), rmdir(1), and unlink(1) (using the traditional notation of which man page section the interface is described in). GNU's 'rm -d' is an extension not required by POSIX, but consistent with BSD implementations. I thought it made more sense to have rm correspond to remove than create another command called remove that called remove. Sorry, but to change that, you'd have to go back in time 30 or 40 years to when rm(1) was first written. People have grown to rely on 'rm(1)' being a wrapper around unlink(2), 'rmdir(1)' being a wrapper around rumdir(2), and 'rm(1) -R' being a wrapper around rmdir(2) or unlink(2) as needed. Are you saying that you think it would be better to have a remove command that does this? No, because remove(3) is just a thin wrapper around unlink(2) or rmdir(2), and most users are already familiar with how to remove files according to their type. I see no reason to add a remove(1). As for the posix document you pointed at, I'm suggesting doing step 4 after step 1... same thing posix does, just reordering things a bit. Sorry, but doing things in rm(1) in a different order than POSIX describes would lead to subtle breakage to lots of existing scripts. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
bug#15927: Bug in date command
tag 15927 needinfo thanks On 11/19/2013 05:23 AM, Claudio Pinto wrote: date --date=10/20/2013 result in date: invalid date `10/20/2013' We need more details, such as your current timezone. I suspect that you are running into a FAQ: https://www.gnu.org/software/coreutils/faq/coreutils-faq.html#The-date-command-is-not-working-right_002e in that you are probably in a timezone where daylight savings changed your time from midnight to 1am at the very start of that particular day, and therefore where there is no effective midnight on that date. But as I don't know what timezone you are in, I haven't managed to reproduce it locally (in my timezone, daylight savings changes occur at 2am rather than midnight). -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
bug#15927: Bug in date command
tag 15927 + moreinfo thanks Claudio Pinto wrote: date --date=10/20/2013 result in date: invalid date `10/20/2013' In what timezone? You didn't give your timezone therefore it is impossible to know for sure but your problem statement matches one of the very common cases where Daylight Saving Time changes and therefore creates an invalid date in your timezone. Since you don't specify a time the time of 00:00 implicit. Better to work with raw dates around 12:00 noon which avoids all known timezone DST changes. Using the raw 'date' output gives ambiguous timezones. Better to use the standardized and unambiguous -R format. $ date -R --date=10/20/2013 Sun, 20 Oct 2013 00:00:00 -0600 $ date -R --date=10/20/2013 12:00 Sun, 20 Oct 2013 12:00:00 -0600 Even better is to always do date calculations in UTC to avoid any DST problems entirely. $ date -u -R --date=10/20/2013 Sun, 20 Oct 2013 00:00:00 + Please see the FAQ for a detailed discussion of date and DST. http://www.gnu.org/software/coreutils/faq/#The-date-command-is-not-working-right_002e Bob
bug#15926: RFE: unlink command already uses 'unlink' call; make 'rm' use 'remove' call
On 19/11/2013 10:34, Eric Blake wrote: On 11/19/2013 11:17 AM, Linda Walsh wrote: Well we have the -d option to rm to explicitly do that. --- Does the posix remove call require a -d? Huh? There is no POSIX remove(1), Since when do you think of a call as being a command? Sorry, but to change that, you'd have to go back in time 30 or 40 years to when rm(1) was first written. People have grown to rely on 'rm(1)' being a wrapper around unlink(2), 'rmdir(1)' being a wrapper around rumdir(2), and 'rm(1) -R' being a wrapper around rmdir(2) or unlink(2) as needed. People relied on rm removing files in a depth first search order for 30-40 years. Posix changed that requiring special checks for .. Scripts relied on that behavior for 30-40 years as well... If you want to use that reasoning, rm should go back to doing depth first deletion and reporting an error with deleting . when it is finished. Sorry, but doing things in rm(1) in a different order than POSIX describes would lead to subtle breakage to lots of existing scripts. I claim not. Come up with 1 case where scripts rely on the current behavior -- to die before doing anything useful, vs. the pre-existing behavior which was to issue an error (suppressible with -f) on the final deletion failing. I am calling your bluff -- show me the scripts (other than a posix conformance test script), that would fail -- subtly or otherwise. I assert they don't exist for 2 reasons. The foremost being that working scripts cannot rely on upon the deletion failure stopping any useful work being done by the command. The 2nd being it was a new change in posix that appeared in gnu utils in only the past few years. The previous 25-35 years of scripts would have relied on it working as *documented* (depth first). Checking pathnames before you start depth first traversal is not strictly depth first. By your own standards for not changing something, rm should be fixed to be the way it was for 30-40 years, as. The problem is, is that by implementing that change, functionality was lost and removed in rm. The earlier version had more functionality. So you can't come up with scripts that rely on missing functionality to get things done. It's like relying on missing water to water your plants or missing food to feed yourself. You can't rely on the absence of a feature to do something positive with it.
bug#15927: Bug in date command
tag 15927 - moreinfo + notabug close 15927 thanks Hello Claudio, Please keep the bug log in the recipient list. That way others in the team on the mailing list can participate in the discussion. Claudio Pinto wrote: Current default time zone: 'America/Sao_Paulo' Local time is now: Ter Nov 19 17:10:40 BRST 2013. Universal Time is now: Tue Nov 19 19:10:40 UTC 2013. Here in Brazil we have the time in an hour early. Yes. And that is the problem. The time you have specified does not exist. That is what makes it an invalid time. $ zdump -v America/Sao_Paulo | grep Oct.*2013 America/Sao_Paulo Sun Oct 20 02:59:59 2013 UTC = Sat Oct 19 23:59:59 2013 BRT isdst=0 gmtoff=-10800 America/Sao_Paulo Sun Oct 20 03:00:00 2013 UTC = Sun Oct 20 01:00:00 2013 BRST isdst=1 gmtoff=-7200 In the America/Sao_Paulo timezone as shown above the seconds of the clock tick to Oct 19 23:59:59 2013 BRT and the very next tick of the clock is Sun Oct 20 01:00:00 2013 BRST. But you are asking for Sun Oct 20 00:00:00 which does not exist in either BRT or BRST. Note that this is not a technology issue. This is an issue of changing the clock for Daylight Saving Time. Most countries do this by an act of government. Removing a difference an hour, with the command below, the error does not happen ... date --date=10/20/2013 01:00 with the command below error appears: date --date=10/20/2013 00:59 Correct. Because those are invalid according to your America/Sao_Paulo timezone. To avoid those errors either use UTC which does not change for Daylight Saving Time or use 12:00 noon to avoid being near the time that it changes. Since this is a bug in usage and not a bug in the date calculations I am going to close the bug report. However if things are still not clear please feel free to follow up with any responses or comments and we can keep discussing it. Please keep the bug log address in the recipient list. Bob
bug#15926: RFE: unlink command already uses 'unlink' call; make 'rm' use 'remove' call
Linda Walsh wrote: but there seems to be no command corresponding to the POSIX remove command, The language here is confusing. There isn't any remove command. There is only the remove(3) library call. man 3 remove it seems upgrading 'rm' to use the 'remove' POSIX call would be a beneficial move of all the recent POSIX changes. That would introduce a significant behavior change and with certainty would introduce bugs. Therefore it shouldn't be done. Eric Blake wrote: Linda Walsh wrote: Pádraig Brady wrote: Linda Walsh wrote: So how about upgrading 'rm' to use the remove function so it would work on empty directories as well. Well we have the -d option to rm to explicitly do that. Does the posix remove call require a -d? Right at this point there is confusion. The only POSIX remove() is the libc library call remove(3). Huh? There is no POSIX remove(1), Yes. Huh? Since when do you think of a call as being a command? We don't. But from what you wrote (Does the posix remove call require a -d?) makes it appear that you think the posix remove(3) library call is a command. Because library calls do not take options like that while commands do. Sorry, but to change that, you'd have to go back in time 30 or 40 years to when rm(1) was first written. People have grown to rely on 'rm(1)' being a wrapper around unlink(2), 'rmdir(1)' being a wrapper around rumdir(2), and 'rm(1) -R' being a wrapper around rmdir(2) or unlink(2) as needed. Agreed. Let's not break this. People relied on rm removing files in a depth first search order for 30-40 years. When using 'rm -rf' they do. But only with the -r option. Never without it. The change you have proposed would for the first time in all of those years have rm remove a directory without the -r option. That would be bad because it would introduce bugs into scripts that are expecting the existing behavior of rm not removing directories. (Unless the -r option is given.) Posix changed that requiring special checks for .. Scripts relied on that behavior for 30-40 years as well... If you want to use that reasoning, rm should go back to doing depth first deletion and reporting an error with deleting . when it is finished. I actually agree with you on that point. ;-) (Regarding 'rm -rf .') Sorry, but doing things in rm(1) in a different order than POSIX describes would lead to subtle breakage to lots of existing scripts. I claim not. Come up with 1 case where scripts rely on the current behavior -- to die before doing anything useful, vs. the pre-existing behavior which was to issue an error (suppressible with -f) on the final deletion failing. I have seen many various script writing styles. Some of them are atrocious indeed. I would expect to at some point see someone use rm on file system objects with abandon thinking that they won't remove directories. Suddenly they will be able to remove directories. That would be a change in behavior. Changing established behavior is bad. I routinely use the opposite side of things myself. I routinely use rmdir on empty directories knowing that if the directory is not empty that it will fail and not remove any actual file content. I could definitely see someone counting on the behavior that rm removes files but never removes directories. (Unless the -r option is given.) I am calling your bluff -- show me the scripts (other than a posix conformance test script), that would fail -- subtly or otherwise. Anything I show would be contrived. But that doesn't mean that changing the behavior should be done. mkdir a b c touch aa bb cc for f in *; do rm $f; done With existing behavior the directories would remain. With the new proposed behavior the directories would be removed. That is a behavior change that should not be made. If you want that behavior then the -d option is present and available. Simply use the -d option. mkdir a b c touch aa bb cc for f in *; do rm -d $f; done That gives you the behavior you are requesting. I assert they don't exist for 2 reasons. The foremost being that working scripts cannot rely on upon the deletion failure stopping any useful work being done by the command. See the above example. The 2nd being it was a new change in posix that appeared in gnu utils in only the past few years. The previous 25-35 years of scripts would have relied on it working as *documented* (depth first). Since you are mentioning depth first I can only think you are talking about 'rm -r'. How does that tie into remove(3)? Checking pathnames before you start depth first traversal is not strictly depth first. Again, what does depth first traversal have to do with remove(3)? By your own standards for not changing something, rm should be fixed to be the way it was for 30-40 years, as. Huh? Stop here. Please explain. Your words here do not follow. The problem is, is that by
bug#15926: RFE: unlink command already uses 'unlink' call; make 'rm' use 'remove' call
On 19/11/2013 12:45, Bob Proulx wrote: Since when do you think of a call as being a command? We don't. But from what you wrote (Does the posix remove call require a -d?) makes it appear that you think the posix remove(3) library call is a command. Because library calls do not take options like that while commands do. Well, that was a bit of metaphore, insomuch as the POSIX calls talk about flag passing with a value of AT_REMOVEDIR to do something different... -d in a call would likely be passed as an option Sorry, but to change that, you'd have to go back in time 30 or 40 years to when rm(1) was first written. People have grown to rely on 'rm(1)' being a wrapper around unlink(2), 'rmdir(1)' being a wrapper around rumdir(2), and 'rm(1) -R' being a wrapper around rmdir(2) or unlink(2) as needed. Agreed. Let's not break this. I'm not see what would break. instead of failing to remove a non-empty directory, it would remove it (if it was empty and permissions allowed). People relied on rm removing files in a depth first search order for 30-40 years. When using 'rm -rf' they do. But only with the -r option. Never without it. --- Absolutely... that's what I was referring to, sorry for unclarity. The change you have proposed would for the first time in all of those years have rm remove a directory without the -r option. That would be bad because it would introduce bugs into scripts that are expecting the existing behavior of rm not removing directories. (Unless the -r option is given.) I would ask how it would fail in a script -- I.e. we are only talking empty directories -- and currently an error would be returned if a script tried to do that. If the script was catching errors or running with -e, the script would terminate under the current implementation. So no script could ever work relying on itself to be fail -- it's a contradiction in terms. Posix changed that requiring special checks for .. Scripts relied on that behavior for 30-40 years as well... If you want to use that reasoning, rm should go back to doing depth first deletion and reporting an error with deleting . when it is finished. I actually agree with you on that point. ;-) (Regarding 'rm -rf .') Well, that's what much of my interest in this area and the newly created bug in find hinges on. Sorry, but doing things in rm(1) in a different order than POSIX describes would lead to subtle breakage to lots of existing scripts. I claim not. Come up with 1 case where scripts rely on the current behavior -- to die before doing anything useful, vs. the pre-existing behavior which was to issue an error (suppressible with -f) on the final deletion failing. I have seen many various script writing styles. Some of them are atrocious indeed. I would expect to at some point see someone use rm on file system objects with abandon thinking that they won't remove directories. Suddenly they will be able to remove directories. That would be a change in behavior. Changing established behavior is bad. - *empty directories*.. lets be clear. I am NOT proposing that rm recursively remove anything without -r... I routinely use the opposite side of things myself. I routinely use rmdir on empty directories knowing that if the directory is not empty that it will fail and not remove any actual file content. I could definitely see someone counting on the behavior that rm removes files but never removes directories. (Unless the -r option is given.) I use the fact that rm doesn't remove non-empty directories as well.. I'm not questioning that behavior. If someone relied on using rm to clean out a directory of any fluff or dander that had built up... I strongly believe that if rm also removed *empty* directories, it would be more likely to be appreciated by soemone who was using it to get rid of all the spurious file entries -- as empty directory entries are often just as spurious -- I'm often cleaning out skeletal trees of directories on my systems... I've even thought about the usefulness of rmdir -r -- which would recursively remove empty dirs starting at the top (with a limitation for -xdev... (I know about rmdir -p a/b/c).. but empty, wider skeletons are more common than skinny+tall dir-structures. (skinny=1 dir wide, many dirs deep). I am calling your bluff -- show me the scripts (other than a posix conformance test script), that would fail -- subtly or otherwise. Anything I show would be contrived. But that doesn't mean that changing the behavior should be done. mkdir a b c touch aa bb cc for f in *; do rm $f; done With existing behavior the directories would remain. With the new proposed behavior the directories would be removed. That is a behavior change that should not be made. Why not? if you want 1 command that does both, why not implement a remove command == but wouldn't it
bug#15926: RFE: unlink command already uses 'unlink' call; make 'rm' use 'remove' call
On 11/19/2013 05:02 PM, Linda Walsh wrote: I'm not see what would break. instead of failing to remove a non-empty directory, it would remove it (if it was empty and permissions allowed). That's what would break. Scripts have been written to assume that 'rm empty_dir' will fail, and your proposal would change that assumption. Falling through to subsequent commands when you used to bail out early has all sorts of subtle implications, that we aren't willing to risk. I would ask how it would fail in a script -- I.e. we are only talking empty directories -- and currently an error would be returned if a script tried to do that. If the script was catching errors or running with -e, the script would terminate under the current implementation. 'set -e' is useless. I highly recommend that script writers avoid it. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
bug#15926: RFE: unlink command already uses 'unlink' call; make 'rm' use 'remove' call
On 11/19/2013 01:45 PM, Bob Proulx wrote: Posix changed that requiring special checks for .. Scripts relied on that behavior for 30-40 years as well... If you want to use that reasoning, rm should go back to doing depth first deletion and reporting an error with deleting . when it is finished. I see nothing in http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html that requires up-front special handling of trailing '.', only a callout to what rmdir() is required to do. Ultimately, rmdir(dir/.) is required to fail, but nothing in POSIX appears to allow us to short-circuit the recursive cleanup before attempting the failing command. I actually agree with you on that point. ;-) (Regarding 'rm -rf .') Yes, the 'rm -rf .' case appears to be a regression in coreutils that is contrary to the behavior required by POSIX. That is: $ mkdir /tmp/foo /tmp/foo/sub $ cd /tmp/foo $ rm -r . rm: cannot remove directory: ‘.’ $ ls sub appears to be a bug in current coreutils, because it should have successfully called rmdir(sub) prior to failing on the attempt to rmdir(.). -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
bug#15926: RFE: unlink command already uses 'unlink' call; make 'rm' use 'remove' call
On 11/20/2013 03:19 AM, Eric Blake wrote: Yes, the 'rm -rf .' case appears to be a regression in coreutils that is contrary to the behavior required by POSIX. That is: $ mkdir /tmp/foo /tmp/foo/sub $ cd /tmp/foo $ rm -r . rm: cannot remove directory: ‘.’ $ ls sub appears to be a bug in current coreutils, because it should have successfully called rmdir(sub) prior to failing on the attempt to rmdir(.). I disagree: see at the top of: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html If either of the files dot or dot-dot are specified as the basename portion of an operand (that is, the final pathname component) or if an operand resolves to the root directory, rm shall write a diagnostic message to standard error and do nothing more with such operands. Maybe cannot remove directory is a bit weak - it's more like refusing to remove dot|dot-dot|root directory. Have a nice day, Berny
