Re: [PATCH] adding two new options to 'cp'
On 08/25/06 11:49, Garance A Drosehn wrote: At 9:05 AM -0400 8/25/06, John Baldwin wrote: On 24 August 2006, Julian Elischer wrote in a cvs-commit log: > > Add an option to allow copying of a hierarchy while > linking he regular files. Very good for commiting this -- except that I'm not quite sure that log-message is parseable! From looking at the man-page update, this option only handles hard-links? What happens with symlinks? No change in functionality - copies them like it normally would (depending on what other options you may have used). It doesn't try to hard link them if that is what you are asking. > Bikeshedded to death on: hackers Submitted by:andersonatcenttech.com MFC in: 1 month It was on my todo list as well. I think I'll still add the -a option. I think that is a good idea, too. Eric -- Eric AndersonSr. Systems AdministratorCentaur Technology Anything that works is better than anything that doesn't. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
At 9:05 AM -0400 8/25/06, John Baldwin wrote: On 24 August 2006, Julian Elischer wrote in a cvs-commit log: > > Add an option to allow copying of a hierarchy while > linking he regular files. Very good for commiting this -- except that I'm not quite sure that log-message is parseable! From looking at the man-page update, this option only handles hard-links? What happens with symlinks? > Bikeshedded to death on: hackers Submitted by:andersonatcenttech.com MFC in: 1 month It was on my todo list as well. I think I'll still add the -a option. I think that is a good idea, too. -- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute; Troy, NY; USA ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Thursday 24 August 2006 16:45, Julian Elischer wrote: > Eric Anderson wrote: > > > On 08/20/06 04:21, Mike Silbersack wrote: > > > >> On Wed, 26 Jul 2006, Eric Anderson wrote: > >> > >>> I'm tired of trying to use rsync or gcp (which doesn't like symlinks > >>> often) to copy trees of files/directories using hard links, so I > >>> added the gcp-ish options -a and -l. > >> > >> > >> ... > >> > >>> Comments? Flames? Committers willing to commit? > >>> > >>> Eric > >> > >> > >> Having just read this thread start to finish, I strongly feel that > >> Eric should be given an award for putting up with all the crap he was > >> given and not losing his temper. > >> > >> In sincerely hope that this thread does not scare off others who have > >> local patches that they are considering contributing to FreeBSD. > >> > >> Mike "Silby" Silbersack > > > > > > > > Alas, no parts of my patch ever were committed. :( > > > > *sigh* - off to other hacking.. > > > > Eric > > > > > Add an option to allow copying of a hierarchy while linking he regular > files. > Bikeshedded to death on: hackers > Submitted by:andersonatcenttech.com > MFC in: 1 month It was on my todo list as well. I think I'll still add the -a option. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Eric Anderson wrote: On 08/20/06 04:21, Mike Silbersack wrote: On Wed, 26 Jul 2006, Eric Anderson wrote: I'm tired of trying to use rsync or gcp (which doesn't like symlinks often) to copy trees of files/directories using hard links, so I added the gcp-ish options -a and -l. ... Comments? Flames? Committers willing to commit? Eric Having just read this thread start to finish, I strongly feel that Eric should be given an award for putting up with all the crap he was given and not losing his temper. In sincerely hope that this thread does not scare off others who have local patches that they are considering contributing to FreeBSD. Mike "Silby" Silbersack Alas, no parts of my patch ever were committed. :( *sigh* - off to other hacking.. Eric Add an option to allow copying of a hierarchy while linking he regular files. Bikeshedded to death on: hackers Submitted by:andersonatcenttech.com MFC in: 1 month CVS: -- CVS: Enter Log. Lines beginning with `CVS:' are removed automatically CVS: CVS: Committing in . CVS: CVS: Modified Files: CVS:cp.1 cp.c extern.h utils.c CVS: -- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On 08/20/06 04:21, Mike Silbersack wrote: On Wed, 26 Jul 2006, Eric Anderson wrote: I'm tired of trying to use rsync or gcp (which doesn't like symlinks often) to copy trees of files/directories using hard links, so I added the gcp-ish options -a and -l. ... Comments? Flames? Committers willing to commit? Eric Having just read this thread start to finish, I strongly feel that Eric should be given an award for putting up with all the crap he was given and not losing his temper. In sincerely hope that this thread does not scare off others who have local patches that they are considering contributing to FreeBSD. Mike "Silby" Silbersack Alas, no parts of my patch ever were committed. :( *sigh* - off to other hacking.. Eric -- Eric AndersonSr. Systems AdministratorCentaur Technology Anything that works is better than anything that doesn't. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Wed, 26 Jul 2006, Eric Anderson wrote: I'm tired of trying to use rsync or gcp (which doesn't like symlinks often) to copy trees of files/directories using hard links, so I added the gcp-ish options -a and -l. ... Comments? Flames? Committers willing to commit? Eric Having just read this thread start to finish, I strongly feel that Eric should be given an award for putting up with all the crap he was given and not losing his temper. In sincerely hope that this thread does not scare off others who have local patches that they are considering contributing to FreeBSD. Mike "Silby" Silbersack ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
In message: <[EMAIL PROTECTED]> John Baldwin <[EMAIL PROTECTED]> writes: : On Monday 31 July 2006 14:53, Mike Meyer wrote: : > Which points up an obvious question: other than compatibility with : > Linux, is there any reason this functionaly shouldn't be added to the : > ln command instead? : : Umm, because ln doesn't copy hierarchies? Using that argument we'd remove : support for hard-links from tar and cpio. I think cp -a is harmless (just as : I use rsync -a all the time rather than rsync -rlptgoD) and cp -l is probably : useful. Really, it is more intuitive to be able to copy a hierarchy using : the 'copy' command (cp) directly rather than a convoluted pair of find | : cpio. In this case I think you might be overly paranoid. :) I tend to agree here. The bloat is minimal, and if there's ever a need to create minimal versions of cp, et al, at that time we can provide ways to optimize for space. We've done it in the past, and as we're pushing down into the embedded space, we may need to do it again. But we may not... Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Julian Elischer wrote:
> Oliver Fromme wrote:
> > Bakul Shah wrote:
> > > Peter Jeremy wrote:
> > > > As a general comment (not addressed to Tim): There _is_ a downside
> > > > to sparsifying files. If you take a sparse file and start filling
> > > > in the holes, the net result will be very badly fragmented and hence
> > > > have very poor sequential I/O performance. If you're never going to
> > > > update a file then making it sparse makes sense, if you will be
> > > > updating it, you will get better performance by making it non-sparse.
> > >
> > > Except for database tables how common is this?
> >
> > For example image files of media, e.g. ISO9660 images
> > or images of hard disk partitions. I often have to handle
> > such images, and I certainly do _not_ want them to be
> > sparse.
>
> well then you'd be silly to go to the extra work fo specifying --sparse
> (or whatever) wouldn't you?
Sure, in that case I wouldn't specify it (and I hope
nobody intends to make it the default, as has been
mentioned in this thread).
> > Before someone adds a bogus "sparse file support" option
> > to cp(1), I would rather prefer that someone fixes the
> > existing -R option which currently doesn't handle hard-
> > links correctly.
>
> It never worked as you suppose.
I know. Probably because nobody cared to implement it,
and there are other tools (cpio) that can do that job.
> Changing it would be a surprise
> (though to me a pleasant one) to many.
I guess most users of cp(1) aren't aware of the flaw.
> > That flaw is documented in the manual page, so it might
> > not count as a "bug", but it's a flaw nevertheless. A lot
> > of people -- even so-called professional admins -- use
> > "cp -Rp" to copy directory hierarchies, and afterwards
> > they wonder why the copy takes up much more space than
> > the original, because all hardlinks have been copied as
> > separate files (if they notice at all).
>
> I ALWAYS use find . -depth -print0|cpio -pdmuv0 {dest}
> or -pdlmuv (poodle-move-0?) if I want links from old to new. because it
> is guaranteed to do that but cp is not.
Yes, exactly. I use:
find -d . | cpio -dump /dest/dir (copy file hierarchy)
or:
find -d . | cpio -dumpl /dest/dir (link file hierarchy)
"-dump" is pretty easy to remember. Of course, if there
might be names with whitespaces, I add the -0 option, too.
I've created aliases "hcp" (hierarchy copy) and "hln"
(hierarchy link) to save typing. That's even shorter
than "cp -al". ;-)
For bourne shells (sh, zsh, bash), these functions work
fine:
hcp() {
local dst
dst=$(realpath "$2") && (
cd -- "$1" && \
find -d . -print0 | cpio -dump0 "$dst"
)
}
hln() {
local dst
dst=$(realpath "$2") && (
cd -- "$1" && \
find -d . -print0 | cpio -dumpl0 "$dst"
)
}
The hcp function _does_ copy hardlinks correctly (unlike
cp), and cpio supports the --sparse option to create
sparse files, so you can add that if you want.
Best regards
Oliver
--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing
Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.
"C++ is over-complicated nonsense. And Bjorn Shoestrap's book
a danger to public health. I tried reading it once, I was in
recovery for months."
-- Cliff Sarginson
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Oliver Fromme wrote:
Bakul Shah wrote:
> Peter Jeremy wrote:
> > As a general comment (not addressed to Tim): There _is_ a downside
> > to sparsifying files. If you take a sparse file and start filling
> > in the holes, the net result will be very badly fragmented and hence
> > have very poor sequential I/O performance. If you're never going to
> > update a file then making it sparse makes sense, if you will be
> > updating it, you will get better performance by making it non-sparse.
>
> Except for database tables how common is this?
For example image files of media, e.g. ISO9660 images
or images of hard disk partitions. I often have to handle
such images, and I certainly do _not_ want them to be
sparse.
well then you'd be silly to go to the extra work fo specifying --sparse
(or whatever) wouldn't you?
Before someone adds a bogus "sparse file support" option
to cp(1), I would rather prefer that someone fixes the
existing -R option which currently doesn't handle hard-
links correctly.
It never worked as you suppose. Changing it would be a surprise
(though to me a pleasant one) to many.
That flaw is documented in the manual page, so it might
not count as a "bug", but it's a flaw nevertheless. A lot
of people -- even so-called professional admins -- use
"cp -Rp" to copy directory hierarchies, and afterwards
they wonder why the copy takes up much more space than
the original, because all hardlinks have been copied as
separate files (if they notice at all).
I ALWAYS use find . -depth -print0|cpio -pdmuv0 {dest}
or -pdlmuv (poodle-move-0?) if I want links from old to new. because it
is guaranteed to do that but cp is not.
Oh by the way: Linux' option for sparse file handling
is "--sparse", and there is no one-letter option (both
-s and -S exist, but have nothing to do with sparse
files). So there wouldn't be an easy way for FreeBSD to
stay compatible with Linux.
Best regards
Oliver
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Bakul Shah wrote: > Peter Jeremy wrote: > > As a general comment (not addressed to Tim): There _is_ a downside > > to sparsifying files. If you take a sparse file and start filling > > in the holes, the net result will be very badly fragmented and hence > > have very poor sequential I/O performance. If you're never going to > > update a file then making it sparse makes sense, if you will be > > updating it, you will get better performance by making it non-sparse. > > Except for database tables how common is this? For example image files of media, e.g. ISO9660 images or images of hard disk partitions. I often have to handle such images, and I certainly do _not_ want them to be sparse. Before someone adds a bogus "sparse file support" option to cp(1), I would rather prefer that someone fixes the existing -R option which currently doesn't handle hard- links correctly. That flaw is documented in the manual page, so it might not count as a "bug", but it's a flaw nevertheless. A lot of people -- even so-called professional admins -- use "cp -Rp" to copy directory hierarchies, and afterwards they wonder why the copy takes up much more space than the original, because all hardlinks have been copied as separate files (if they notice at all). Oh by the way: Linux' option for sparse file handling is "--sparse", and there is no one-letter option (both -s and -S exist, but have nothing to do with sparse files). So there wouldn't be an easy way for FreeBSD to stay compatible with Linux. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "I have stopped reading Stephen King novels. Now I just read C code instead." -- Richard A. O'Keefe ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Wed, Aug 02, 2006 at 05:33:40PM +1000, Peter Jeremy wrote: > On Tue, 2006-Aug-01 23:35:03 -0700, Tim Kientzle wrote: > >The "cheap" solution is to handle it purely on > >extract: Detect blocks of zeros when restoring > >files and seek over them. > > The downside is that you wind up with a sparse file whether or not you > wanted one. No, you wind up with a sparse file if you specify the "-S" option to tar. If you didn't want one, don't specify the option. > Actually, the only real solution to copying sparse files is to add > a system call that can return a map of holes. This would neatly > address the "needs two passes" problem with tar. You don't need two passes-- you advance the file pointer whenever you detect a block of zeros. Note: care must be taken to only do this for newly-created or otherwise truncated files, otherwise the skip ahead wouldn't work. > As a general comment (not addressed to Tim): There _is_ a downside > to sparsifying files. If you take a sparse file and start filling > in the holes, the net result will be very badly fragmented and hence > have very poor sequential I/O performance. If you're never going to > update a file then making it sparse makes sense, if you will be > updating it, you will get better performance by making it non-sparse. Agreed, with a minor correction/elaboration. By "updating" you mean specifically "updating but not appending". And another note: a good way to defragment a file is to sequentially copy it. (The "best" way is to copy the file to a new filesystem, that way you guarantee the blocks allocated to the file are contiguous.) -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Sparsifying Files (was Re: [PATCH] adding two new options to 'cp')
Peter Jeremy wrote: As a general comment (not addressed to Tim): There _is_ a downside to sparsifying files. How people use (or misuse) such a feature is just not my concern. I've not seen anyone on this thread suggest that such sparsification be done by default, and there are certainly rare situations where sparsification is useful. The tar implementation is about 95% system-independent file format work and 5% system-dependent hole mapping. The mapping process is the easy part. (And therefore, for me, the least interesting part.) A system call could make it accurate, but it's not enough code to fuss over either way. Tim ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
> As a general comment (not addressed to Tim): There _is_ a downside > to sparsifying files. If you take a sparse file and start filling > in the holes, the net result will be very badly fragmented and hence > have very poor sequential I/O performance. If you're never going to > update a file then making it sparse makes sense, if you will be > updating it, you will get better performance by making it non-sparse. Except for database tables how common is this? And for such files how important is the sequntial I/O performance? For database tables perhaps there is a size range where not making them sparse helps but for really large tables you wouldn't want to fill in the holes. I suspect that making not writing zeroes the default would actually help overall performance. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Wed, 2 Aug 2006, Peter Jeremy wrote: > As a general comment (not addressed to Tim): There _is_ a downside to > sparsifying files. If you take a sparse file and start filling in the > holes, the net result will be very badly fragmented and hence have very > poor sequential I/O performance. If you're never going to update a file > then making it sparse makes sense, if you will be updating it, you will > get better performance by making it non-sparse. Aha! Thanks for that, Peter. -- Dave, wondering why anyone would *not* want sparse files... ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Tue, 2006-Aug-01 23:35:03 -0700, Tim Kientzle wrote: >The "cheap" solution is to handle it purely on >extract: Detect blocks of zeros when restoring >files and seek over them. The downside is that you wind up with a sparse file whether or not you wanted one. > I simply dislike >the GNU tar approach, in part because it requires >two passes over the file (the map of holes is required >before the file is written). Actually, the only real solution to copying sparse files is to add a system call that can return a map of holes. This would neatly address the "needs two passes" problem with tar. As a general comment (not addressed to Tim): There _is_ a downside to sparsifying files. If you take a sparse file and start filling in the holes, the net result will be very badly fragmented and hence have very poor sequential I/O performance. If you're never going to update a file then making it sparse makes sense, if you will be updating it, you will get better performance by making it non-sparse. -- Peter Jeremy pgpQeKoTW1HxK.pgp Description: PGP signature
Re: [PATCH] adding two new options to 'cp'
Rick C. Petty wrote: While we're at it, I think we should add the -S option to bsdtar. I'm willing to do the work ... I have pretty strong ideas about sparse file support for bsdtar. The "cheap" solution is to handle it purely on extract: Detect blocks of zeros when restoring files and seek over them. That would be pretty easy to implement: just add another option to archive_read_extract() and implement the logic to skip over blocks of zeros. Archiving sparse files as such is harder, although I do have an outline of a technique which would not only handle sparse files, but also allow archiving files whose size is not known in advance (something the GNU tar approach doesn't support). I simply dislike the GNU tar approach, in part because it requires two passes over the file (the map of holes is required before the file is written). Again, if someone is really interested, let me know. Tim Kientzle ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Extended Attributes in Tar (was Re: [PATCH] adding two new options to 'cp')
Rick C. Petty wrote: ... I don't think cp or tar do [handle extended attributes] either. Actually, the OS-independent code for extended attributes has already been implemented in libarchive, thanks to Jaako Heinonen, who also wrote the Linux-specific portions for archive_extract (restore to disk) and tar/write.c (read from disk). All that's needed is for someone to write the FreeBSD-specific portions. I'd be happy to give pointers to anyone who has the time to finish this. Tim Kientzle ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
At 7:17 PM -0500 8/1/06, Rick C. Petty wrote: On Tue, Aug 01, 2006 at 08:09:08PM -0400, Garance A Drosehn wrote: I had understood this option as a request to "copy all the existing holes", which is not the same thing. I.e., I thought we wanted `cp' to create the new file such that it would take up exactly the same number of disk blocks, and have the same number of holes (in exactly the same places) as the original file. Which it currently doesn't, without any sparse option. Yes, but we know it doesn't, and we don't expect it to. I *thought* what this thread was asking was for someone to add some new option to `cp', where that new option would do the above. But I was wrong with that, and apparently people would be happy with a new option which says "sparse-ify the file". > I agree that "sparse-ify" should be easy to implement, and could be useful. I'm not fond of the idea, but I can see how people might want it. I do would not like it, because the user will have to know whether it is appropriate to use on a file-by-file basis. You can't just 'cp -rp' an entire directory, and feel confident that the "Right Thing(TM)" will happen for each file that is being copied. So, if I am copying directories, I'll still have to resort to some other tool to get the job done "Right(TM)". I don't see why not. I don't care if you don't see it. I am just stating my personal opinion, and apparently I am not even doing a good job at that. I will stop now. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Tue, Aug 01, 2006 at 08:09:08PM -0400, Garance A Drosehn wrote: > > I had understood this option as a request to "copy all the > existing holes", which is not the same thing. I.e., I > thought we wanted `cp' to create the new file such that it > would take up exactly the same number of disk blocks, and > have the same number of holes (in exactly the same places) > as the original file. Which it currently doesn't, without any sparse option. A copied file will always be larger than the original (in terms of disk blocks) if the original had any sparseness. > I agree that "sparse-ify" should be easy to implement, and > could be useful. I'm not fond of the idea, but I can see > how people might want it. I do would not like it, because > the user will have to know whether it is appropriate to use > on a file-by-file basis. You can't just 'cp -rp' an entire > directory, and feel confident that the "Right Thing(TM)" > will happen for each file that is being copied. So, if I > am copying directories, I'll still have to resort to some > other tool to get the job done "Right(TM)". I don't see why not. If you're mixing sparse and non-sparse files in a tree and wish to duplicate this precisely, you need dump/restore.. oh, and those only work for UFS filesystems. Whatever the Right Thing is, you should have a good idea whether you wish to sparsify or anti-sparsify the files beneath (current cp does the anti-sparsify). If you're doing a directory copy and cannot choose which is the Right Thing for everything within that directory, then cp(1) certainly is not your choice. > In my case, I want zeros on the disk in the destination > wherever there were zeros on the disk in the source. This may be true with cp(1) as it is now, but certainly the converse is not guaranteed to be true. > In some situations, I don't want the number-of-blocks of a > file to increase every time I change a X'00' to a X'01'. Whereas the opposite situation is preferrable? Hmm, I'm using Y bytes of storage within this directory tree, let's move that to another partition. I'll make that partition at least Y bytes big. Recursive copy-- whoa! Out of space? Darn. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
At 12:19 PM -0500 8/1/06, Rick C. Petty wrote: On Tue, Aug 01, 2006, Garance A Drosehn wrote: > The point is not that you need an explicit switch, the point is that you have to add a lot of code to 'cp' for 'cp' to do the job correctly. Not really. See my example in a previous post. All you need to do is perform an lseek(2) instead of a write(2) if the block you read is all zero. I guess it depends on what the option will be described as doing. If you want an option which will "sparse-ify" any file, then yes that's easy to do. I had understood this option as a request to "copy all the existing holes", which is not the same thing. I.e., I thought we wanted `cp' to create the new file such that it would take up exactly the same number of disk blocks, and have the same number of holes (in exactly the same places) as the original file. I agree that "sparse-ify" should be easy to implement, and could be useful. I'm not fond of the idea, but I can see how people might want it. I do would not like it, because the user will have to know whether it is appropriate to use on a file-by-file basis. You can't just 'cp -rp' an entire directory, and feel confident that the "Right Thing(TM)" will happen for each file that is being copied. So, if I am copying directories, I'll still have to resort to some other tool to get the job done "Right(TM)". In my case, I want zeros on the disk in the destination wherever there were zeros on the disk in the source. In some situations, I don't want the number-of-blocks of a file to increase every time I change a X'00' to a X'01'. It would be nice for `cp' to do that too, but I expect *that* would add too much bloat to `cp'. Not only that, but it just sets us up for the next request for `cp' to do "perfectly-accurate copies" of files. -- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute; Troy, NY; USA ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
> Eric Anderson wrote: > > > It could possibly be bad if you have a real file (say a 10GB file, > > partially filled with zeros - a disk image created with dd for > > instance), and you use cp with something like -spR to recursively copy > > all files. Your destination disk image would then be a sparse file, so > > Incidentally, this is exactly why I've needed it - I like to create disk > images for virtual machines as sparse files, when I know they won't be > much filled, but need the "virtual" space :) [Basic idea from the plan 9 guys] Rather than modify every tool for this may be the OS should avoid writing a block of zeroes? The idea is to check if the first and last word are 0. If so, check the whole block (you can avoid the necessary bounds check by temporarily unzeroing the last word). If all zeroes and there is no existing allocation for the range being written, just advance the current offset (essentially lseek). Else proceed as normal write. This test is very cheap in practice and if you can avoid one write in ten thousand this way you will likely see overall savings. Of course, you still need rsync but it help all local copying, the common case. This being an optimization you don't need to implement a complete solution. For instance writing 1 zero byte in one call and then 4095 zero bytes in another may defeat the optimization. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Mike Meyer wrote: I always think of cp as a tool for making *copies of files*, not for creating an archive of a directory tree. We've got lots of tools that do the latter. Do we really need another one? But, but, but - I only asked for cp to be able to copy my sparse files as sparse files, not do magic with them :) The side-effect that it will sparsify other files is both unavoidable and actually useful. OTOH, for copying/sparisfying single files, dd should be fixed. vsdev:/tmp> l total 2 drwx-- 2 ivoras wheel 512 Jul 9 00:08 mc-ivoras/ vsdev:/tmp> dd if=/dev/zero of=file bs=1024 seek=100 count=1 1+0 records in 1+0 records out 1024 bytes transferred in 0.016648 secs (61509 bytes/sec) vsdev:/tmp> l total 8 -rw-r--r-- 1 ivoras wheel 103424 Aug 1 21:40 file drwx-- 2 ivoras wheel 512 Jul 9 00:08 mc-ivoras/ vsdev:/tmp> du -shc file 6.0Kfile 6.0Ktotal vsdev:/tmp> l total 8 -rw-r--r-- 1 ivoras wheel 103424 Aug 1 21:40 file drwx-- 2 ivoras wheel 512 Jul 9 00:08 mc-ivoras/ vsdev:/tmp> dd if=file of=file2 bs=1024 conv=sparse 101+0 records in 101+0 records out 103424 bytes transferred in 0.130216 secs (794250 bytes/sec) vsdev:/tmp> l total 110 -rw-r--r-- 1 ivoras wheel 103424 Aug 1 21:40 file -rw-r--r-- 1 ivoras wheel 103424 Aug 1 21:41 file2 drwx-- 2 ivoras wheel 512 Jul 9 00:08 mc-ivoras/ vsdev:/tmp> du -shc file* 6.0Kfile 102Kfile2 108Ktotal ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Eric Anderson wrote: It could possibly be bad if you have a real file (say a 10GB file, partially filled with zeros - a disk image created with dd for instance), and you use cp with something like -spR to recursively copy all files. Your destination disk image would then be a sparse file, so Incidentally, this is exactly why I've needed it - I like to create disk images for virtual machines as sparse files, when I know they won't be much filled, but need the "virtual" space :) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Wed, Aug 02, 2006 at 05:04:53AM +1000, Peter Jeremy wrote: > On Tue, 2006-Aug-01 12:51:32 -0500, Eric Anderson wrote: > >string of zeros larger than the block size, or it needs to 'do the right > >thing' and determine if it's sparse or not. > > You can do this by comparing stat.st_size with stat.st_blocks - a > sparse file will have fewer blocks than its size requires. What you > can't do is accurately determine where the holes are. There's an extension for the seek interface in Solaris to do that. Joerg Schily discussed this with some of us BSD developers in Chemnitz/Germany earlier this year. There was someone working on porting it to FreeBSD IIRC. Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Wed, Aug 02, 2006 at 05:04:53AM +1000, Peter Jeremy wrote: > On Tue, 2006-Aug-01 12:51:32 -0500, Eric Anderson wrote: > >string of zeros larger than the block size, or it needs to 'do the right > >thing' and determine if it's sparse or not. > > You can do this by comparing stat.st_size with stat.st_blocks - a > sparse file will have fewer blocks than its size requires. What you > can't do is accurately determine where the holes are. > > Note that st_blksize is not nessarily the allocation blocksize and > therefore is unrelated to the size of holes in the filesystem. Also, > on FreeBSD, the designation of "optimal" is a misnomer and I/O > operations should be much larger than this for optimal efficiency. True, but I've not seen the case where the following is not true: The filesystem's block size (i.e. 8x fragment size) in UFS an integer multiple of st_blksize. My previously-posted example would work just fine. I guess a better solution would be to check every 512-byte chunk and seek if zero. It's up to the underlying filesystem to implement this as a sparse file, if that filesystem supports such a thing. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Tue, 2006-Aug-01 12:51:32 -0500, Eric Anderson wrote: >string of zeros larger than the block size, or it needs to 'do the right >thing' and determine if it's sparse or not. You can do this by comparing stat.st_size with stat.st_blocks - a sparse file will have fewer blocks than its size requires. What you can't do is accurately determine where the holes are. Note that st_blksize is not nessarily the allocation blocksize and therefore is unrelated to the size of holes in the filesystem. Also, on FreeBSD, the designation of "optimal" is a misnomer and I/O operations should be much larger than this for optimal efficiency. -- Peter Jeremy pgpamKbTi5BUJ.pgp Description: PGP signature
Re: [PATCH] adding two new options to 'cp'
Eric Anderson wrote:
On 08/01/06 12:11, Rick C. Petty wrote:
On Tue, Aug 01, 2006 at 05:26:11PM +1000, Peter Jeremy wrote:
On Mon, 2006-Jul-31 22:42:49 +0200, Ivan Voras wrote:
I agree with this, and while you're in there, can you add -s to
copy sparse files (via the usual "if the buffer is all nulls, seek
beyond eof instead of writing" trick)?
Note that it isn's possible to accurately distinguish between a block
of NULs and a hole in the file through the filesystem. The only way
to accurately copy a sparse file is with dump/restore.
Sure it is-- in a number of ways. The most useful way is to do
something
of the sort:
int sd, dd;// assume these are set to source & dest descriptors
unsigned char* zeros;
unsigned char* buffer;
struct stat st;
size_t bytes, offset;
fstat(sd, &st);
zeros = malloc(st.st_blksize);
bzero(zeros, st.st_blksize);
for (offset = 0; offset < st.st_size; offset += bytes)
{
bytes = st.st_blksize;
if (offset + bytes > st.st_size)
bytes = st.st_size - offset;
read(sd, buffer, bytes);
if (0 == memcmp(buffer, zeros, bytes))
lseek(dd, bytes, SEEK_CUR);
else
write(sd, buffer, bytes);
}
Obviously, I didn't add the error checking/handling, but AFAIK this is
essentially what the -S option to gnu's tar does. In this example, you
may not mimic the allocated blocks of a sparse file, but you would
optimize the copy to use as few filesystem blocks as possible.
Wouldn't this be incorrect for files that are really full of zeros?
It would turn them in to sparse files when they shouldn't be,
correct? Is that what happens with other tools?
If you use the sparse option then that is what you are asking it to do.
Eric
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Eric Anderson wrote: On 08/01/06 12:40, Rick C. Petty wrote: On Tue, Aug 01, 2006 at 12:27:54PM -0500, Eric Anderson wrote: Wouldn't this be incorrect for files that are really full of zeros? It would turn them in to sparse files when they shouldn't be, correct? Is that what happens with other tools? Why is this bad? I'm not suggesting that the default implementation should change but that this "-s" option be added to properly optimize for sparse files. A sparse file is one which contains blocks of pure zeros. My example wouldn't vary in how the destination file is read but in how many blocks are allocated in the underlying file system. It could possibly be bad if you have a real file (say a 10GB file, partially filled with zeros - a disk image created with dd for instance), and you use cp with something like -spR to recursively copy all files. Your destination disk image would then be a sparse file, so additional changes and modifications to the file (block allocations) would fragment the image and make it slower. That would be unexpected and probably go unnoticed for some time. I do see the usefulness in this, but I think one needs to be careful to either clearly note in the man page that it will make sparse files from *any* file containing a string of zeros larger than the block size, or it needs to 'do the right thing' and determine if it's sparse or not. the option to copy sparsly is an option and not forced down the user's throat. I agree it would be a useful OPTION. at the moment there is no way to "sparsify" :-) a file easily. As long as the documentation warns that copying it without the sparse option might unexpectedly fill up your disk (or whatever your favourite trap is) then, it is UNIX's job to give you all teh tools you need and your job to decide when and where you shoot yorself in the foot. to "approximatly" quote Terry Lambert: " It is not unix's job to stop you from shooting your foot. If you so choose to do so, then it is UNIX's job to deliver Mr. Bullet to Mr Foot in the most efficient way it knows." Eric ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Tue, Aug 01, 2006 at 02:26:28PM -0400, Mike Meyer wrote: > In <[EMAIL PROTECTED]>, Rick C. Petty <[EMAIL PROTECTED]> typed: > > > Obviously, I didn't add the error checking/handling, but AFAIK this is > > essentially what the -S option to gnu's tar does. > > Yes, but gnu tar doesn't do this accurately, That may be. I haven't looked at their code, I just assumed how they did it. > so you're not doing what > Peter said you couldn't do. You are doing what Ivan asked, though. That depends upon what you mean by "accurately" and what you mean by "sparse". :-P Yes, if you're looking for a block-wise "dump" of a file system's file, you use dump. If you're looking to make a "copy" of a file, optimizing for sparseness, you use rsync. A pretty heavyweight solution when you're trying to copy one file. > I always think of cp as a tool for making *copies of files*, not for > creating an archive of a directory tree. We've got lots of tools that > do the latter. Do we really need another one? I wasn't thinking of the "sparse handling" utility for whole trees, but it would be useful for that also. I'm just looking for a lightweight tool for copying files containing sparse chunks.. Since we switched to bsdtar, the base system has been lacking this feature. To me, this seems more useful (as a base feature) than hard-linking trees. But both have utility. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
In <[EMAIL PROTECTED]>, Rick C. Petty <[EMAIL PROTECTED]> typed: > On Tue, Aug 01, 2006 at 05:26:11PM +1000, Peter Jeremy wrote: > > On Mon, 2006-Jul-31 22:42:49 +0200, Ivan Voras wrote: > > >I agree with this, and while you're in there, can you add -s to copy > > >sparse files (via the usual "if the buffer is all nulls, seek beyond eof > > >instead of writing" trick)? > > > > Note that it isn's possible to accurately distinguish between a block > > of NULs and a hole in the file through the filesystem. The only way > > to accurately copy a sparse file is with dump/restore. > > Sure it is-- in a number of ways. The most useful way is to do something > of the sort: [code elided] > Obviously, I didn't add the error checking/handling, but AFAIK this is > essentially what the -S option to gnu's tar does. Yes, but gnu tar doesn't do this accurately, so you're not doing what Peter said you couldn't do. You are doing what Ivan asked, though. I always think of cp as a tool for making *copies of files*, not for creating an archive of a directory tree. We've got lots of tools that do the latter. Do we really need another one? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Tue, Aug 01, 2006 at 12:51:32PM -0500, Eric Anderson wrote: > On 08/01/06 12:40, Rick C. Petty wrote: > > It could possibly be bad if you have a real file (say a 10GB file, > partially filled with zeros - a disk image created with dd for > instance), and you use cp with something like -spR to recursively copy > all files. Your destination disk image would then be a sparse file, so > additional changes and modifications to the file (block allocations) > would fragment the image and make it slower. That would be unexpected > and probably go unnoticed for some time. I do see the usefulness in > this, but I think one needs to be careful to either clearly note in the > man page that it will make sparse files from *any* file containing a > string of zeros larger than the block size, I completely agree. This is why it would not be the default behavior. Also, gnutar provides this option and it just describes it as "handle sparse files efficiently"-- so a rewording of that to say that by "efficient" we mean that the program will allocate as few blocks as possible in the underlying filesystem. I think a clear description in the man page would suffice. > or it needs to 'do the right > thing' and determine if it's sparse or not. Unfortunately that would require knowledge which the POSIX calls have no knowledge of. I don't believe we should roll filesystem-specific knowledge into cp. However, there is utility in performing a "space-saving" copy, just as a hardlink is quite space-saving. =) "-s" or "-S" could refer to sparse, space, squeeze, shrink, or whatever. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On 08/01/06 12:40, Rick C. Petty wrote: On Tue, Aug 01, 2006 at 12:27:54PM -0500, Eric Anderson wrote: Wouldn't this be incorrect for files that are really full of zeros? It would turn them in to sparse files when they shouldn't be, correct? Is that what happens with other tools? Why is this bad? I'm not suggesting that the default implementation should change but that this "-s" option be added to properly optimize for sparse files. A sparse file is one which contains blocks of pure zeros. My example wouldn't vary in how the destination file is read but in how many blocks are allocated in the underlying file system. It could possibly be bad if you have a real file (say a 10GB file, partially filled with zeros - a disk image created with dd for instance), and you use cp with something like -spR to recursively copy all files. Your destination disk image would then be a sparse file, so additional changes and modifications to the file (block allocations) would fragment the image and make it slower. That would be unexpected and probably go unnoticed for some time. I do see the usefulness in this, but I think one needs to be careful to either clearly note in the man page that it will make sparse files from *any* file containing a string of zeros larger than the block size, or it needs to 'do the right thing' and determine if it's sparse or not. Eric -- Eric AndersonSr. Systems AdministratorCentaur Technology Anything that works is better than anything that doesn't. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Tue, Aug 01, 2006 at 12:27:54PM -0500, Eric Anderson wrote: > > Wouldn't this be incorrect for files that are really full of zeros? It > would turn them in to sparse files when they shouldn't be, correct? Is > that what happens with other tools? Why is this bad? I'm not suggesting that the default implementation should change but that this "-s" option be added to properly optimize for sparse files. A sparse file is one which contains blocks of pure zeros. My example wouldn't vary in how the destination file is read but in how many blocks are allocated in the underlying file system. One of my biggest gripes with cp(1) was how a recursive tree copy was always larger than the source, if the source contained any sparse files. This is similar to your desire to add "-l" for doing hard links. Both have their usefulness. And your "-a" option could be a replacement for "-pRs" if so desired (or "-S", it's all the same to me). While we're at it, I think we should add the -S option to bsdtar. I'm willing to do the work and make patches (to cp & tar), if someone is willing to review and commit them. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On 08/01/06 12:11, Rick C. Petty wrote:
On Tue, Aug 01, 2006 at 05:26:11PM +1000, Peter Jeremy wrote:
On Mon, 2006-Jul-31 22:42:49 +0200, Ivan Voras wrote:
I agree with this, and while you're in there, can you add -s to copy
sparse files (via the usual "if the buffer is all nulls, seek beyond eof
instead of writing" trick)?
Note that it isn's possible to accurately distinguish between a block
of NULs and a hole in the file through the filesystem. The only way
to accurately copy a sparse file is with dump/restore.
Sure it is-- in a number of ways. The most useful way is to do something
of the sort:
int sd, dd; // assume these are set to source & dest descriptors
unsigned char* zeros;
unsigned char* buffer;
struct stat st;
size_t bytes, offset;
fstat(sd, &st);
zeros = malloc(st.st_blksize);
bzero(zeros, st.st_blksize);
for (offset = 0; offset < st.st_size; offset += bytes)
{
bytes = st.st_blksize;
if (offset + bytes > st.st_size)
bytes = st.st_size - offset;
read(sd, buffer, bytes);
if (0 == memcmp(buffer, zeros, bytes))
lseek(dd, bytes, SEEK_CUR);
else
write(sd, buffer, bytes);
}
Obviously, I didn't add the error checking/handling, but AFAIK this is
essentially what the -S option to gnu's tar does. In this example, you
may not mimic the allocated blocks of a sparse file, but you would
optimize the copy to use as few filesystem blocks as possible.
Wouldn't this be incorrect for files that are really full of zeros? It
would turn them in to sparse files when they shouldn't be, correct? Is
that what happens with other tools?
Eric
--
Eric AndersonSr. Systems AdministratorCentaur Technology
Anything that works is better than anything that doesn't.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Tue, Aug 01, 2006 at 01:06:55PM -0400, Garance A Drosehn wrote: > > The point is not that you need an explicit switch, the > point is that you have to add a lot of code to 'cp' for > 'cp' to do the job correctly. Not really. See my example in a previous post. All you need to do is perform an lseek(2) instead of a write(2) if the block you read is all zero. > You are changing the nature of how the command works. No, you're adding features, not changing the behavior for the default case. > I very much doubt you > can implement that *correctly* in a mere 160 bytes of > additional object code! I'd be willing to bet that my aforementioned example would fit in that space-- because you already do an fstat & the read/write code-- the only thing to add is the zero-checking and seek ahead code. > -r-xr-xr-x 1 root wheel 15348 [date] /bin/cp > -r-xr-xr-x 2 root wheel 44288 [date] /sbin/dump > -r-xr-xr-x 2 root wheel 58736 [date] /sbin/restore dump/restore is significantly different in operation than cp. Both do a lot more than just copy files. Also dump/restore is specific to UFS and my example code would handle sparse files for any filesystem. > I realize a lot of these decisions are somewhat subjective. > "One person's feature is another person's bloat". But in > the case of this specific example, I (personally) would > not want 'cp' to implement every detail which is already > handled by dump/restore. I don't think this is what people are asking for. BTW, does dump/restore handle extended attributes? Last I checked, it didn't. But then again I don't think cp or tar do either. Feel free to enlighten me. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Tue, Aug 01, 2006 at 05:26:11PM +1000, Peter Jeremy wrote:
> On Mon, 2006-Jul-31 22:42:49 +0200, Ivan Voras wrote:
> >I agree with this, and while you're in there, can you add -s to copy
> >sparse files (via the usual "if the buffer is all nulls, seek beyond eof
> >instead of writing" trick)?
>
> Note that it isn's possible to accurately distinguish between a block
> of NULs and a hole in the file through the filesystem. The only way
> to accurately copy a sparse file is with dump/restore.
Sure it is-- in a number of ways. The most useful way is to do something
of the sort:
int sd, dd; // assume these are set to source & dest descriptors
unsigned char* zeros;
unsigned char* buffer;
struct stat st;
size_t bytes, offset;
fstat(sd, &st);
zeros = malloc(st.st_blksize);
bzero(zeros, st.st_blksize);
for (offset = 0; offset < st.st_size; offset += bytes)
{
bytes = st.st_blksize;
if (offset + bytes > st.st_size)
bytes = st.st_size - offset;
read(sd, buffer, bytes);
if (0 == memcmp(buffer, zeros, bytes))
lseek(dd, bytes, SEEK_CUR);
else
write(sd, buffer, bytes);
}
Obviously, I didn't add the error checking/handling, but AFAIK this is
essentially what the -S option to gnu's tar does. In this example, you
may not mimic the allocated blocks of a sparse file, but you would
optimize the copy to use as few filesystem blocks as possible.
-- Rick C. Petty
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
At 12:07 PM +0200 8/1/06, Ivan Voras wrote: Peter Jeremy wrote: Note that it isn's possible to accurately distinguish between a block of NULs and a hole in the file through the filesystem. The only way to accurately copy a sparse file is with dump/restore. Hence the need for an explicit switch. The point is not that you need an explicit switch, the point is that you have to add a lot of code to 'cp' for 'cp' to do the job correctly. You are changing the nature of how the command works. I very much doubt you can implement that *correctly* in a mere 160 bytes of additional object code! Note: -r-xr-xr-x 1 root wheel 15348 [date] /bin/cp -r-xr-xr-x 2 root wheel 44288 [date] /sbin/dump -r-xr-xr-x 2 root wheel 58736 [date] /sbin/restore I realize a lot of these decisions are somewhat subjective. "One person's feature is another person's bloat". But in the case of this specific example, I (personally) would not want 'cp' to implement every detail which is already handled by dump/restore. -- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute; Troy, NY; USA ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Tue, 01 Aug 2006, Ivan Voras wrote: > Peter Jeremy wrote: > > >Note that it isn's possible to accurately distinguish between a block > >of NULs and a hole in the file through the filesystem. The only way > >to accurately copy a sparse file is with dump/restore. > > Hence the need for an explicit switch. A switch doesn't help. A sparse file can contain both holes and blocks of NULs. You could, I suppose, do something like -s 100, which would seek past any block of >100 NULs. -- David Taylor ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp' (UPDATE)
On 07/26/06 21:51, Eric Anderson wrote: I'm tired of trying to use rsync or gcp (which doesn't like symlinks often) to copy trees of files/directories using hard links, so I added the gcp-ish options -a and -l. -a is 'archive' mode, which is just a quick form of -PpR. -l is 'link' mode, where regular files get hard linked instead of copied. So, you can mimic an entire tree with something like: cp -al /from/ /to/ and it's fast too! Patch is against 6-STABLE, but works well on 7-CURRENT as well. Patch is here (with man page edits): http://www.googlebit.com/freebsd/patches/cp-patch cd /tmp/ fetch http://www.googlebit.com/freebsd/patches/cp-patch cd /usr/src/ patch < /tmp/cp-patch cd bin/cp make && make install I've made an updated patch available. It fixes the missing information in the usage output, a file descriptor leak, and also now warns when an attempt to copy a socket occurs instead of erroring. This isn't really particular to any of the new arguments, but is more consistent with other tools, and consistent with cp's other options that just warn. The new patch can be found here: http://www.googlebit.com/freebsd/patches/cp-patch-2 Eric -- Eric AndersonSr. Systems AdministratorCentaur Technology Anything that works is better than anything that doesn't. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Peter Jeremy wrote: Note that it isn's possible to accurately distinguish between a block of NULs and a hole in the file through the filesystem. The only way to accurately copy a sparse file is with dump/restore. Hence the need for an explicit switch. This is true in the desktop and server market but is not true in the embedded market and only marginally true for laptops. I don't know about embedded markets (do they need cp, or it's simply kernel+dedicated application?) but I've recently bought a 60GB laptop drive for ~~ $120 (converted back from my currency, may be cheaper in other parts of the world) - that's $2/GB. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Mon, 2006-Jul-31 22:42:49 +0200, Ivan Voras wrote: >I agree with this, and while you're in there, can you add -s to copy >sparse files (via the usual "if the buffer is all nulls, seek beyond eof >instead of writing" trick)? Note that it isn's possible to accurately distinguish between a block of NULs and a hole in the file through the filesystem. The only way to accurately copy a sparse file is with dump/restore. On Mon, 2006-Jul-31 22:37:56 +0200, Ivan Voras wrote: >To badly paraphase [1] a quote: "Here, have a $1, buy yourself a GB of >storage..." :) This is true in the desktop and server market but is not true in the embedded market and only marginally true for laptops. -- Peter Jeremy pgpqSDruA6YGB.pgp Description: PGP signature
Re: [PATCH] adding two new options to 'cp'
John-Mark Gurney wrote: Helping people not be portable w/ other Unixes like Solaris is something we should not do... Can anyone name another major Unix besides Linux that has the -a option? I won't continue this line of discussion more, but it's possible Linux has more traction in these things nowadays. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Ivan Voras wrote this message on Tue, Aug 01, 2006 at 00:07 +0200: > older people who stick with what they know will work - vast majority of > younger admins I know either personally or on the 'net generally know > only Linux. > > Using common options help bring people over, and also saves trouble with > porting shell script code :) Helping people not be portable w/ other Unixes like Solaris is something we should not do... Can anyone name another major Unix besides Linux that has the -a option? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Rick C. Petty wrote: think that more BSD admins are so used to typing "cp -pR" than "cp -a". In Ah, but therein lies one of the points - Percentage of "BSD admins" vs "other-unix-like-systems admins" is slowly but steadily diminishing. In my (I'll admit - not so expansive) experience BSD admins tend to be older people who stick with what they know will work - vast majority of younger admins I know either personally or on the 'net generally know only Linux. Using common options help bring people over, and also saves trouble with porting shell script code :) then save "-a" for something like ACLs. Does cp(1) properly handle ACLs now anyway? It's a good point. I don't know if it works but ACLs should probably be handled by -p. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Mon, Jul 31, 2006 at 11:01:24PM +0200, Ivan Voras wrote: > Rick C. Petty wrote: > > >"-l" may be a useful option, but at what point is the line drawn between > >bloating our base cp and having a gcp port (or using linux_base)?? > > It's like saying "if you need this option, go to Linux" - it just seems > wrong. Not quite. I'm just saying that the "-l" option would be seldomly-used and chances are that the people who would use it are likely to also have linux_base installed. > With all respect to "series of small utilities" way of doing things, I > think the "whatever is shorter to type" takes precedence :) Ah, so then let's install a binary called "a" which is hard-linked to cp and does the equivalent of "cp -a". While we're at it, let's add a "t" command as a shortcut to "tar -f". [/sarcasm] This is what aliases are for. "cp -a" seems to me to be a silly way to say "cp -pR". Come on, it's only one letter shorter (although you could argue the time it takes to hit the shift key counts as a letter). I would think that more BSD admins are so used to typing "cp -pR" than "cp -a". In any case, it seems pointless to waste a letter when an alias is a more appropriate solution. cp(1) uses getopt(3), so there are only so many letters available. It seems more useful to add other bits of functionality and use up letters. If you really want "-a" then it should copy sparse files correctly, preserve hard links within the copied hierarchy (much like rsync's -H option), and handle extended attributes (e.g. ACLs). If not, then save "-a" for something like ACLs. Does cp(1) properly handle ACLs now anyway? > And it's not like the added code will have to be changed/maintained in > the forseeable future - if the basic implementation is ok, the > underlying structures and ideas will not change while UFS semantics are > in use. Actually I have no gripe against "-l"; I was just making a point. I think this hard-link idea is pretty useful. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Monday 31 July 2006 14:53, Mike Meyer wrote: > Which points up an obvious question: other than compatibility with > Linux, is there any reason this functionaly shouldn't be added to the > ln command instead? Umm, because ln doesn't copy hierarchies? Using that argument we'd remove support for hard-links from tar and cpio. I think cp -a is harmless (just as I use rsync -a all the time rather than rsync -rlptgoD) and cp -l is probably useful. Really, it is more intuitive to be able to copy a hierarchy using the 'copy' command (cp) directly rather than a convoluted pair of find | cpio. In this case I think you might be overly paranoid. :) -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Rick C. Petty wrote: "-l" may be a useful option, but at what point is the line drawn between bloating our base cp and having a gcp port (or using linux_base)?? It's like saying "if you need this option, go to Linux" - it just seems wrong. With all respect to "series of small utilities" way of doing things, I think the "whatever is shorter to type" takes precedence :) And it's not like the added code will have to be changed/maintained in the forseeable future - if the basic implementation is ok, the underlying structures and ideas will not change while UFS semantics are in use. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Eric Anderson wrote: -a is 'archive' mode, which is just a quick form of -PpR. -l is 'link' mode, where regular files get hard linked instead of copied. I agree with this, and while you're in there, can you add -s to copy sparse files (via the usual "if the buffer is all nulls, seek beyond eof instead of writing" trick)? I recently had to copy big sparse files and dd with conv=sparse didn't work! I managed with rsync but I feel this functionality is the domain of 'cp' (yes, like it is in Linux - please don't start bikesheading on this detail). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Mike Meyer wrote: snake% uname -a FreeBSD snake.mired.org 6.1-RELEASE FreeBSD 6.1-RELEASE #0: Sun May 7 04:42:56 UTC 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/SMP i386 snake% ls -l /bin/cp -r-xr-xr-x 1 root wheel 15300 May 6 23:56 /bin/cp To badly paraphase [1] a quote: "Here, have a $1, buy yourself a GB of storage..." :) Bloat is bad, but saving me time it takes to type redeems some of it. One of the reasons this binary is so small is because it's dynamically linked, so some space was saved when systems started moving away from static binaries. If nitpicking is in order, this file's size is still under default block size (16 KB) and there's plenty of stuff to fit in there to fill it. [1] butcher, more likely ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
At 12:17 PM -0700 7/31/06, Julian Elischer wrote: Ok I"m going to pipe up here. The feature is cheap, it is useful and it allows people to adopt FreeBSD with less surprises. I will commit this soon unless someone else does it first. This is not the first time we have had a long thread about the '-a' option, at least, and if we do not add it now then we will just debate this again in 8-10 months. The time wasted in the debate is much worse than the 160 bytes wasted on the disk. 160 bytes, on a disk that we allocate room in fragments of what, 4096 byte chunks? I forget now. I think Julian has the right idea. ... assuming there's no bugs in the code. I didn't actually look at the code. :-) -- Garance Alistair Drosehn = [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Institute; Troy, NY; USA ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Mike Meyer wrote: I'm as neutral as I'd be about *any* other addition. I don't have a specific reason to dislike it. But I don't have a specific reason to like it, either. The last time I wanted a hardlinked copy of a directory tree was long enough ago that most (if not all) of the alternative solutions mentioned here didn't exist yet. I suppose I thought the reasons were obvious - to get a hardlinked copy of a directory tree, one must concoct any one of a number of command lines, all using at least one of which is much bigger in size than the patched cp I proposed. Here are some of the commands mentioned so far that are used by people to do the exact same thing: -r-xr-xr-x 1 root wheel 50056 Jul 25 23:08 /usr/bin/bsdtar -r-xr-xr-x 1 root wheel 52600 Jul 25 23:07 /usr/bin/cpio -r-xr-xr-x 1 root wheel 36480 Jul 25 23:08 /usr/bin/find -r-xr-xr-x 1 root wheel 90376 Jul 25 23:06 /bin/pax And here's my patched version of cp: -r-xr-xr-x 1 root wheel 15460 Jul 26 14:52 /bin/cp So yes, you bloat by 160 bytes, but you can then possibly remove your need for one or more utilities that eat up at least twice the space. So are you proposing that we remove one of those utilities? If not, then you are bloating the system. Yeah, it's only by a little bit. But a lot of little bits add up. Ok I"m going to pipe up here. The feature is cheap, it is useful and it allows people to adopt FreeBSD with less surprises. I will commit this soon unless someone else does it first. Now go do something useful :-) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
In <[EMAIL PROTECTED]>, Eric Anderson <[EMAIL PROTECTED]> typed: > On 07/31/06 10:23, Mike Meyer wrote: > > In <[EMAIL PROTECTED]>, Eric Anderson <[EMAIL PROTECTED]> typed: > >> On 07/31/06 09:11, Mike Meyer wrote: > I suppose I'm just missing the reason *not* to commit such a simple and > > n> >> useful set of options. > >>> Feature bloat. Or, more verbosely, this doesn't add any new > >>> functionality to the system, while adding things that we would rather > >>> minimize. > >> This is a really funny reason not to. Honestly, if you believe that, > >> that you probably don't use cp at all, since dd can do it. > > > > Yes, I believe that. Adding features does *not* necessarily improve a > > system. If you want it added, give us *good* reasons to add it. Lack > > of a good reason not to add a feature is *not* a good reason to add > > the feature. > > > > Personally, I'm neutral on this change, other than not wanting FreeBSD > > to bloat any more than it already is. Given good reasons, I'd say > > commit it. The reasons you just provided are specious. > You don't sound neutral at all actually, but ok. I'm as neutral as I'd be about *any* other addition. I don't have a specific reason to dislike it. But I don't have a specific reason to like it, either. The last time I wanted a hardlinked copy of a directory tree was long enough ago that most (if not all) of the alternative solutions mentioned here didn't exist yet. > I suppose I thought > the reasons were obvious - to get a hardlinked copy of a directory tree, > one must concoct any one of a number of command lines, all using at > least one of which is much bigger in size than the patched cp I > proposed. Here are some of the commands mentioned so far that are used > by people to do the exact same thing: > > -r-xr-xr-x 1 root wheel 50056 Jul 25 23:08 /usr/bin/bsdtar > -r-xr-xr-x 1 root wheel 52600 Jul 25 23:07 /usr/bin/cpio > -r-xr-xr-x 1 root wheel 36480 Jul 25 23:08 /usr/bin/find > -r-xr-xr-x 1 root wheel 90376 Jul 25 23:06 /bin/pax > And here's my patched version of cp: > -r-xr-xr-x 1 root wheel 15460 Jul 26 14:52 /bin/cp > > So yes, you bloat by 160 bytes, but you can then possibly remove your > need for one or more utilities that eat up at least twice the space. So are you proposing that we remove one of those utilities? If not, then you are bloating the system. Yeah, it's only by a little bit. But a lot of little bits add up. > The -a option isn't as much of a useful option, but it keeps life nice > and simple for those of us with only FreeBSD systems. If the -a option > is an issue, then ignore it. How does the -a option make life simpler for those of us with only FreeBSD systems? By saving two characters typing if we want to use a tool designed for copying files to archive them instead? > >>> If the functionality is all that useful, then people should have > >>> already created shell code to make this functionality easily available > >>> via the tools that already have it. If nobody needs this functionality > >>> often enough to have done that, is it something that we want to > >>> enshrine in compiled code? > >> To me, I read this as saying: "If it was useful, it would have already > >> been done, and since it isn't, it must not be needed by anyone." > > > > How does "people would have created shell code to make this easy to > > do" equate to "someone would have already added an option for it"? You > > claim that the code provides "useful functionality". If it's useful, > > then people should be using the alternatives frequently. Command lines > > that people use frequently tend to get enshrined in shell scripts, or > > aliases, or shell functions, or whatever. Moving such things into > > commands is a standard path for Unix code, and has been since at least > > v6. So, if you want to take that step, can you show that it's really > > used frequently enough to warrant getting a dedicated option? > People do make scripts to get around it, and they've posted their own > 'heres what I do' on this list in this thread. People even use tools > like rsync for it too. There's a lot of people working around it in > various ways, that's what gave me the reason to post it to the list. I haven't seen any scripts. I recall lots of people pointing out alternatives. None of these are what I would call "work arounds". They're using the available tools to do the job they were designed to do. Your patch means they can use a tool designed to copy files to create links. Which points up an obvious question: other than compatibility with Linux, is there any reason this functionaly shouldn't be added to the ln command instead? > Anyway, it's apparent that those who are against the extra features post > louder than those who could use them, so feel free to ignore the patch > altogether if you wish. That's how technical decisions get made on the internet. http://www.mired.org/consulting.html Independ
Re: [PATCH] adding two new options to 'cp'
On 07/31/06 13:44, Rick C. Petty wrote: On Mon, Jul 31, 2006 at 12:42:02PM -0500, Eric Anderson wrote: On 07/31/06 12:28, Rick C. Petty wrote: In both cases, why don't you just use: /usr/compat/linux/bin/cp Two reasons - it's not in the base system, so a port has to be installed - and linux_base is FC3 now, so if you want to talk about bloat... And the "-l" option is needed in single-user mode? I like not having extra bloat around when I don't even have /usr mounted and am trying to fix a disk or misconfiguration. I'm just arguing the usefulness of having it in the base system vs. using linux_base. The argument that our cp should be equivalent to gcp seems silly to me. I never once said our cp should be equivalent. I just provided a patch that added 2 simple arguments, not the other 10 (or whatever the number is). "-l" may be a useful option, but at what point is the line drawn between bloating our base cp and having a gcp port (or using linux_base)?? I don't know, and I'm not (obviously) the one to make those decisions. "-a" certainly is useless. An alias is far more useful-- even for things in /bin ! I certainly cp and mv mapped to "cp -i" and "mv -i".. one could also argue that the our base versions of these use this option by default. Personally, I prefer to do a post-install patch to add these aliases to /etc/csh.cshrc (actually on my systems: /etc/csh.aliases) and /etc/profile, etc. Another reason is gcp fails to recursively copy a directory that has symlinks in it. That sounds like a bug or at least an oversight. I'm just a FreeBSD user, so what do I know? I'll just do as others have kindly suggested, and keep my patches to my local servers/systems. Eric -- Eric AndersonSr. Systems AdministratorCentaur Technology Anything that works is better than anything that doesn't. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Mon, Jul 31, 2006 at 12:42:02PM -0500, Eric Anderson wrote: > On 07/31/06 12:28, Rick C. Petty wrote: > > > >In both cases, why don't you just use: > > > >/usr/compat/linux/bin/cp > > Two reasons - it's not in the base system, so a port has to be installed > - and linux_base is FC3 now, so if you want to talk about bloat... And the "-l" option is needed in single-user mode? I like not having extra bloat around when I don't even have /usr mounted and am trying to fix a disk or misconfiguration. I'm just arguing the usefulness of having it in the base system vs. using linux_base. The argument that our cp should be equivalent to gcp seems silly to me. "-l" may be a useful option, but at what point is the line drawn between bloating our base cp and having a gcp port (or using linux_base)?? "-a" certainly is useless. An alias is far more useful-- even for things in /bin ! I certainly cp and mv mapped to "cp -i" and "mv -i".. one could also argue that the our base versions of these use this option by default. Personally, I prefer to do a post-install patch to add these aliases to /etc/csh.cshrc (actually on my systems: /etc/csh.aliases) and /etc/profile, etc. > Another reason is gcp fails to recursively copy a directory that has > symlinks in it. That sounds like a bug or at least an oversight. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On 07/31/06 10:23, Mike Meyer wrote: In <[EMAIL PROTECTED]>, Eric Anderson <[EMAIL PROTECTED]> typed: On 07/31/06 09:11, Mike Meyer wrote: In <[EMAIL PROTECTED]>, Eric Anderson <[EMAIL PROTECTED]> typed: The patch doesn't change any current behavior, nor should it be noticed by anyone not looking for it. However, it is useful, and it does make our cp work just like the GNU cp, which eases the migration path for linux->FreeBSD users. Is emulating Linux behavior that good an idea? I mean, if I want Linux, I can download and install a copy. The joke about "Linux is for people who hate Windows; FreeBSD is for people who love Unix" is funny to me *because* it seems to capture the difference between the two systems so accurately. I like Unix/BSD because I feel like the developers respect the user, and are willing to let the user do pretty much anything they need to do, even if there's no obvious reason for them to want that. I detest Windows because the developers treat the the user like an idiot, and write software that does things the way they think the user should want to do them - and make it impossible to do things that the developers don't think users would ever need/want to do. Linux seems to have more of the latter attitude than the former. [And no, I don't think this patch has that attitude; I just don't think that "that's how linux does it" is a valid argument: freebsd isn't linux.] The reasoning was not simply to make it like linux, that's just a side effect. That doesn't make the "to makes it more like Linux" argument a good reason to change FreeBSD. I suppose I'm just missing the reason *not* to commit such a simple and n> >> useful set of options. Feature bloat. Or, more verbosely, this doesn't add any new functionality to the system, while adding things that we would rather minimize. This is a really funny reason not to. Honestly, if you believe that, that you probably don't use cp at all, since dd can do it. Yes, I believe that. Adding features does *not* necessarily improve a system. If you want it added, give us *good* reasons to add it. Lack of a good reason not to add a feature is *not* a good reason to add the feature. Personally, I'm neutral on this change, other than not wanting FreeBSD to bloat any more than it already is. Given good reasons, I'd say commit it. The reasons you just provided are specious. You don't sound neutral at all actually, but ok. I suppose I thought the reasons were obvious - to get a hardlinked copy of a directory tree, one must concoct any one of a number of command lines, all using at least one of which is much bigger in size than the patched cp I proposed. Here are some of the commands mentioned so far that are used by people to do the exact same thing: -r-xr-xr-x 1 root wheel 50056 Jul 25 23:08 /usr/bin/bsdtar -r-xr-xr-x 1 root wheel 52600 Jul 25 23:07 /usr/bin/cpio -r-xr-xr-x 1 root wheel 36480 Jul 25 23:08 /usr/bin/find -r-xr-xr-x 1 root wheel 90376 Jul 25 23:06 /bin/pax And here's my patched version of cp: -r-xr-xr-x 1 root wheel 15460 Jul 26 14:52 /bin/cp So yes, you bloat by 160 bytes, but you can then possibly remove your need for one or more utilities that eat up at least twice the space. The -a option isn't as much of a useful option, but it keeps life nice and simple for those of us with only FreeBSD systems. If the -a option is an issue, then ignore it. If the functionality is all that useful, then people should have already created shell code to make this functionality easily available via the tools that already have it. If nobody needs this functionality often enough to have done that, is it something that we want to enshrine in compiled code? To me, I read this as saying: "If it was useful, it would have already been done, and since it isn't, it must not be needed by anyone." How does "people would have created shell code to make this easy to do" equate to "someone would have already added an option for it"? You claim that the code provides "useful functionality". If it's useful, then people should be using the alternatives frequently. Command lines that people use frequently tend to get enshrined in shell scripts, or aliases, or shell functions, or whatever. Moving such things into commands is a standard path for Unix code, and has been since at least v6. So, if you want to take that step, can you show that it's really used frequently enough to warrant getting a dedicated option? People do make scripts to get around it, and they've posted their own 'heres what I do' on this list in this thread. People even use tools like rsync for it too. There's a lot of people working around it in various ways, that's what gave me the reason to post it to the list. Anyway, it's apparent that those who are against the extra features post louder than those who could use them, so feel free to ignore the patch altogether if you wish. Eric -- -
Re: [PATCH] adding two new options to 'cp'
> the first one because of compatibility with the large base of Linux systems > out there, I'll say it again: Being compatible with some other system is *not* a reason to add something to FreeBSD. Sure, if we decide a feature is useful, then there's a lot to be said for making it compatible with other systems that already offer that functionality. But adding a feature just to add compatibility is nothing but bloat. And it *doesn't matter* how large a base of users that other systems has. I don't run FreeBSD because it's popular; I run it because I believe it's the best solution to my problems. I believe that's true because we - and CSRG before us, and Bell Labs before them - worry more about quality than about popularity (well, at least if you ignore OSI). If I wanted a popular OS, I'd run Windows. If I wanted a popular Unix, I'd run OSX. Turning FreeBSD into Linux is no more desirable than turning it into Windows. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On 07/31/06 12:28, Rick C. Petty wrote: On Mon, Jul 31, 2006 at 05:47:09PM +0200, Juan Rodriguez wrote: My GNU version of "cp" has more than 18 options, the FreeBSD version only has 9. As I usually work with Linux machines, I'm used to "cp -a" and I have always hated to have to look up in the FreeBSD's "cp" manual page for the right options to get the same funtionality. I tend to think that "-a" is option bloating because it's not really needed, but I see "-l" as a new feature for "cp" that might be useful. I agree, -a is bloat. However I don't understand why you say: To sum it up, I think "cp -a" and "cp -l" are both useful, I agree that the "-l" option *may* be useful. the first one because of compatibility with the large base of Linux systems out there, and the second one because I think it's a useful feature for the FreeBSD "cp". In both cases, why don't you just use: /usr/compat/linux/bin/cp Two reasons - it's not in the base system, so a port has to be installed - and linux_base is FC3 now, so if you want to talk about bloat... Another reason is gcp fails to recursively copy a directory that has symlinks in it. Example: $ cd /tmp/ $ mkdir test $ touch test/t $ ln -s t test/b $ ls -al test/ drwxr-xr-x 2 anderson wheel 512 Jul 7 09:39 . drwxrwxrwt 69 root wheel 8704 Jul 7 09:39 .. lrwxr-xr-x 1 anderson wheel 1 Jul 7 09:39 b -> t -rw-r--r-- 1 anderson wheel 0 Jul 7 09:39 t $ gcp -al test/ test2/ $ ls -al test2/ drwxr-xr-x 2 anderson wheel 512 Jul 7 09:39 . drwxrwxrwt 70 root wheel 8704 Jul 7 09:40 .. -rw-r--r-- 3 anderson wheel 0 Jul 7 09:39 b -rw-r--r-- 3 anderson wheel 0 Jul 7 09:39 t ( you can see it made a resolved the symlink and made a hardlink to the symlink destination, not exactly what I would want, but didn't fail) However, now do: $ ln -s /tmp/ test/link-to-dir $ ls -al test/ drwxr-xr-x 2 anderson wheel 512 Jul 7 09:43 . drwxrwxrwt 70 root wheel 8704 Jul 7 09:40 .. lrwxr-xr-x 1 anderson wheel 1 Jul 7 09:39 b -> t lrwxr-xr-x 1 anderson wheel 5 Jul 7 09:43 link-to-dir -> /tmp/ -rw-r--r-- 3 anderson wheel 0 Jul 7 09:39 t $ gcp -al test/ test2/ gcp: cannot create link `test2/test/link-to-dir': Operation not permitted gcp gets an error when copying symlinks that point to directories. Eric -- Eric AndersonSr. Systems AdministratorCentaur Technology Anything that works is better than anything that doesn't. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Mon, Jul 31, 2006 at 05:47:09PM +0200, Juan Rodriguez wrote: > > My GNU version of "cp" has more than 18 options, the FreeBSD > version only has 9. > > As I usually work with Linux machines, I'm used to "cp -a" and I have always > hated to have to look up in the FreeBSD's "cp" manual page for the right > options to get the same funtionality. I tend to think > that "-a" is option bloating because it's not really needed, but I see > "-l" as a new feature for "cp" that might be useful. I agree, -a is bloat. However I don't understand why you say: > To sum it up, I think "cp -a" and "cp -l" are both useful, I agree that the "-l" option *may* be useful. > the first one > because of compatibility with the large base of Linux systems out there, and > the second one because I think it's a useful feature for the FreeBSD "cp". In both cases, why don't you just use: /usr/compat/linux/bin/cp If you're not going to add all 18 options to our cp, then -a shouldn't be added at all. It doesn't provide any useful functionality, and the argument to make it more compatible with Linux is silly unless you add the other 9 options. If you want the linux cp, use the linux cp. -- Rick C. Petty ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Mon, 31 Jul 2006, Mike Meyer wrote: snake% uname -a FreeBSD snake.mired.org 6.1-RELEASE FreeBSD 6.1-RELEASE #0: Sun May 7 04:42:56 UTC 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/SMP i386 snake% ls -l /bin/cp -r-xr-xr-x 1 root wheel 15300 May 6 23:56 /bin/cp flea# ls -l cp -rwxr-xr-x 1 root wheel 15300 Jul 31 12:52 cp flea# ./cp -a cp.o cp.o.2 flea# ./cp usage: cp [-R [-H | -L | -P]] [-f | -i | -n] [-apv] source_file target_file cp [-R [-H | -L | -P]] [-f | -i | -n] [-apv] source_file ... target_directory flea# ls -l /bin/cp -r-xr-xr-x 1 root wheel 15300 Apr 24 13:55 /bin/cp flea# Looks to me with optimizations, the addition of the option seems to add nothing for the -a case at least. Jason -- Jason Slagle /"\ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . \ / ASCII Ribbon Campaign . X - NO HTML/RTF in e-mail . / \ - NO Word docs in e-mail . ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On 7/31/06, Mike Meyer <[EMAIL PROTECTED]> wrote: In <[EMAIL PROTECTED]>, Juan Rodriguez <[EMAIL PROTECTED]> typed: > On 7/31/06, Mike Meyer <[EMAIL PROTECTED]> wrote: > My GNU version of "cp" has more than 18 options, the FreeBSD > version only has 9. And this results in: student% uname -a Linux student.mired.org 2.6.12-9-386 #1 Mon Oct 10 13:14:36 BST 2005 i686 GNU/Linux student% ls -l /bin/cp -rwxr-xr-x 1 root root 51008 2005-09-05 05:14 /bin/cp snake% uname -a FreeBSD snake.mired.org 6.1-RELEASE FreeBSD 6.1-RELEASE #0: Sun May 7 04:42:56 UTC 2006 [EMAIL PROTECTED] :/usr/obj/usr/src/sys/SMP i386 snake% ls -l /bin/cp -r-xr-xr-x 1 root wheel 15300 May 6 23:56 /bin/cp http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. I've just realized that the manual page of FreeBSD's "cp" warns about hard links: -R ...blablabla... Note that cp copies hard linked files as separate files. If you need to preserve hard links, consider using tar(1), cpio(1), or pax(1) instead. -- JFRH ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
In <[EMAIL PROTECTED]>, Juan Rodriguez <[EMAIL PROTECTED]> typed: > On 7/31/06, Mike Meyer <[EMAIL PROTECTED]> wrote: > My GNU version of "cp" has more than 18 options, the FreeBSD > version only has 9. And this results in: student% uname -a Linux student.mired.org 2.6.12-9-386 #1 Mon Oct 10 13:14:36 BST 2005 i686 GNU/Linux student% ls -l /bin/cp -rwxr-xr-x 1 root root 51008 2005-09-05 05:14 /bin/cp snake% uname -a FreeBSD snake.mired.org 6.1-RELEASE FreeBSD 6.1-RELEASE #0: Sun May 7 04:42:56 UTC 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/SMP i386 snake% ls -l /bin/cp -r-xr-xr-x 1 root wheel 15300 May 6 23:56 /bin/cp http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On 7/31/06, Mike Meyer <[EMAIL PROTECTED]> wrote: In <[EMAIL PROTECTED]>, Eric Anderson <[EMAIL PROTECTED]> typed: > On 07/31/06 09:11, Mike Meyer wrote: > > In <[EMAIL PROTECTED]>, Eric Anderson < [EMAIL PROTECTED]> typed: > >> The patch doesn't change any current behavior, nor should it be noticed > >> by anyone not looking for it. However, it is useful, and it does make > >> our cp work just like the GNU cp, which eases the migration path for > >> linux->FreeBSD users. > > Is emulating Linux behavior that good an idea? I mean, if I want > > Linux, I can download and install a copy. The joke about "Linux is for > > people who hate Windows; FreeBSD is for people who love Unix" is funny > > to me *because* it seems to capture the difference between the two > > systems so accurately. I like Unix/BSD because I feel like the > > developers respect the user, and are willing to let the user do pretty > > much anything they need to do, even if there's no obvious reason for > > them to want that. I detest Windows because the developers treat the > > the user like an idiot, and write software that does things the way > > they think the user should want to do them - and make it impossible to > > do things that the developers don't think users would ever need/want > > to do. Linux seems to have more of the latter attitude than the > > former. [And no, I don't think this patch has that attitude; I just > > don't think that "that's how linux does it" is a valid argument: > > freebsd isn't linux.] > The reasoning was not simply to make it like linux, that's just a side > effect. That doesn't make the "to makes it more like Linux" argument a good reason to change FreeBSD. > >> I suppose I'm just missing the reason *not* to commit such a simple and n> >> useful set of options. > > Feature bloat. Or, more verbosely, this doesn't add any new > > functionality to the system, while adding things that we would rather > > minimize. > This is a really funny reason not to. Honestly, if you believe that, > that you probably don't use cp at all, since dd can do it. Yes, I believe that. Adding features does *not* necessarily improve a system. If you want it added, give us *good* reasons to add it. Lack of a good reason not to add a feature is *not* a good reason to add the feature. Personally, I'm neutral on this change, other than not wanting FreeBSD to bloat any more than it already is. Given good reasons, I'd say commit it. The reasons you just provided are specious. > > If the functionality is all that useful, then people should have > > already created shell code to make this functionality easily available > > via the tools that already have it. If nobody needs this functionality > > often enough to have done that, is it something that we want to > > enshrine in compiled code? > To me, I read this as saying: "If it was useful, it would have already > been done, and since it isn't, it must not be needed by anyone." How does "people would have created shell code to make this easy to do" equate to "someone would have already added an option for it"? You claim that the code provides "useful functionality". If it's useful, then people should be using the alternatives frequently. Command lines that people use frequently tend to get enshrined in shell scripts, or aliases, or shell functions, or whatever. Moving such things into commands is a standard path for Unix code, and has been since at least v6. So, if you want to take that step, can you show that it's really used frequently enough to warrant getting a dedicated option? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]" Hello, My GNU version of "cp" has more than 18 options, the FreeBSD version only has 9. As I usually work with Linux machines, I'm used to "cp -a" and I have always hated to have to look up in the FreeBSD's "cp" manual page for the right options to get the same funtionality. I tend to think that "-a" is option bloating because it's not really needed, but I see "-l" as a new feature for "cp" that might be useful. I'm not a unix/linux expert but when I have to copy something, "cp" shows up inmediately in my mind (I almost never use "cpio" and I didn't know "pax", for example). To sum it up, I think "cp -a" and "cp -l" are both useful, the first one because of compatibility with the large base of Linux systems out there, and the second one because I think it's a useful feature for the FreeBSD "cp". This is only my personal experience, though I understand that if you want to protect "cp" for having more than 10 options, these options shouldn't see the sunlight :P -- JFRH ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.
Re: [PATCH] adding two new options to 'cp'
In <[EMAIL PROTECTED]>, Eric Anderson <[EMAIL PROTECTED]> typed: > On 07/31/06 09:11, Mike Meyer wrote: > > In <[EMAIL PROTECTED]>, Eric Anderson <[EMAIL PROTECTED]> typed: > >> The patch doesn't change any current behavior, nor should it be noticed > >> by anyone not looking for it. However, it is useful, and it does make > >> our cp work just like the GNU cp, which eases the migration path for > >> linux->FreeBSD users. > > Is emulating Linux behavior that good an idea? I mean, if I want > > Linux, I can download and install a copy. The joke about "Linux is for > > people who hate Windows; FreeBSD is for people who love Unix" is funny > > to me *because* it seems to capture the difference between the two > > systems so accurately. I like Unix/BSD because I feel like the > > developers respect the user, and are willing to let the user do pretty > > much anything they need to do, even if there's no obvious reason for > > them to want that. I detest Windows because the developers treat the > > the user like an idiot, and write software that does things the way > > they think the user should want to do them - and make it impossible to > > do things that the developers don't think users would ever need/want > > to do. Linux seems to have more of the latter attitude than the > > former. [And no, I don't think this patch has that attitude; I just > > don't think that "that's how linux does it" is a valid argument: > > freebsd isn't linux.] > The reasoning was not simply to make it like linux, that's just a side > effect. That doesn't make the "to makes it more like Linux" argument a good reason to change FreeBSD. > >> I suppose I'm just missing the reason *not* to commit such a simple and n> >> useful set of options. > > Feature bloat. Or, more verbosely, this doesn't add any new > > functionality to the system, while adding things that we would rather > > minimize. > This is a really funny reason not to. Honestly, if you believe that, > that you probably don't use cp at all, since dd can do it. Yes, I believe that. Adding features does *not* necessarily improve a system. If you want it added, give us *good* reasons to add it. Lack of a good reason not to add a feature is *not* a good reason to add the feature. Personally, I'm neutral on this change, other than not wanting FreeBSD to bloat any more than it already is. Given good reasons, I'd say commit it. The reasons you just provided are specious. > > If the functionality is all that useful, then people should have > > already created shell code to make this functionality easily available > > via the tools that already have it. If nobody needs this functionality > > often enough to have done that, is it something that we want to > > enshrine in compiled code? > To me, I read this as saying: "If it was useful, it would have already > been done, and since it isn't, it must not be needed by anyone." How does "people would have created shell code to make this easy to do" equate to "someone would have already added an option for it"? You claim that the code provides "useful functionality". If it's useful, then people should be using the alternatives frequently. Command lines that people use frequently tend to get enshrined in shell scripts, or aliases, or shell functions, or whatever. Moving such things into commands is a standard path for Unix code, and has been since at least v6. So, if you want to take that step, can you show that it's really used frequently enough to warrant getting a dedicated option? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On 07/31/06 09:11, Mike Meyer wrote: In <[EMAIL PROTECTED]>, Eric Anderson <[EMAIL PROTECTED]> typed: The patch doesn't change any current behavior, nor should it be noticed by anyone not looking for it. However, it is useful, and it does make our cp work just like the GNU cp, which eases the migration path for linux->FreeBSD users. Is emulating Linux behavior that good an idea? I mean, if I want Linux, I can download and install a copy. The joke about "Linux is for people who hate Windows; FreeBSD is for people who love Unix" is funny to me *because* it seems to capture the difference between the two systems so accurately. I like Unix/BSD because I feel like the developers respect the user, and are willing to let the user do pretty much anything they need to do, even if there's no obvious reason for them to want that. I detest Windows because the developers treat the the user like an idiot, and write software that does things the way they think the user should want to do them - and make it impossible to do things that the developers don't think users would ever need/want to do. Linux seems to have more of the latter attitude than the former. [And no, I don't think this patch has that attitude; I just don't think that "that's how linux does it" is a valid argument: freebsd isn't linux.] The -a option that was included in my patch was certainly added for compatibility sake and ease of use. I added it in while I was there to make our cp work with that same shortcut as Linux, not because I love linux (which I'm not that fond of), but because I would use it, and it would be helpful. The -l option was a missing feature, and is what gave me the desire to even look at the code. I obviously wouldn't have spent any time on it if I didn't need it bad enough, and I know of other users that work their way around it by using pipes and cpio, or pax, etc, along with hacking the code that uses those tools to change it from a cp command to their concocted version, but that isn't always an option. The reasoning was not simply to make it like linux, that's just a side effect. I suppose I'm just missing the reason *not* to commit such a simple and useful set of options. Feature bloat. Or, more verbosely, this doesn't add any new functionality to the system, while adding things that we would rather minimize. This is a really funny reason not to. Honestly, if you believe that, that you probably don't use cp at all, since dd can do it. In fact, I bet you don't use sysinstall either, right? I hate to be like this, but this is really kind of goofy sounding, as *tons* of tools in FreeBSD are there to make life easier, administration faster, etc. I mean, you use emacs right? Why not just echo the data to the mailer? If the functionality is all that useful, then people should have already created shell code to make this functionality easily available via the tools that already have it. If nobody needs this functionality often enough to have done that, is it something that we want to enshrine in compiled code? To me, I read this as saying: "If it was useful, it would have already been done, and since it isn't, it must not be needed by anyone." Eric -- Eric AndersonSr. Systems AdministratorCentaur Technology Anything that works is better than anything that doesn't. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
In <[EMAIL PROTECTED]>, Eric Anderson <[EMAIL PROTECTED]> typed: > The patch doesn't change any current behavior, nor should it be noticed > by anyone not looking for it. However, it is useful, and it does make > our cp work just like the GNU cp, which eases the migration path for > linux->FreeBSD users. Is emulating Linux behavior that good an idea? I mean, if I want Linux, I can download and install a copy. The joke about "Linux is for people who hate Windows; FreeBSD is for people who love Unix" is funny to me *because* it seems to capture the difference between the two systems so accurately. I like Unix/BSD because I feel like the developers respect the user, and are willing to let the user do pretty much anything they need to do, even if there's no obvious reason for them to want that. I detest Windows because the developers treat the the user like an idiot, and write software that does things the way they think the user should want to do them - and make it impossible to do things that the developers don't think users would ever need/want to do. Linux seems to have more of the latter attitude than the former. [And no, I don't think this patch has that attitude; I just don't think that "that's how linux does it" is a valid argument: freebsd isn't linux.] On a lighter note, if we want to ease the migration path for users from other systems, then we should clearly add software to provide a BSOD at random intervals to make Windows users more comfortable. > I suppose I'm just missing the reason *not* to commit such a simple and > useful set of options. Feature bloat. Or, more verbosely, this doesn't add any new functionality to the system, while adding things that we would rather minimize. If the functionality is all that useful, then people should have already created shell code to make this functionality easily available via the tools that already have it. If nobody needs this functionality often enough to have done that, is it something that we want to enshrine in compiled code? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On 07/27/06 12:44, Doug Barton wrote: Oliver Fromme wrote: Eric Anderson <[EMAIL PROTECTED]> wrote: > I'm tired of trying to use rsync or gcp (which doesn't like symlinks > often) to copy trees of files/directories using hard links, so I added > the gcp-ish options -a and -l. > > -a is 'archive' mode, which is just a quick form of -PpR. -P is the default anyway, so -a would only replace -Rp. I don't think saving one letter justifies introducing a new option. You can use an alias or shell function. > -l is 'link' mode, where regular files get hard linked instead of copied. > > So, you can mimic an entire tree with something like: > > cp -al /from/ /to/ > > and it's fast too! You can do the same with existing tools in a portable (and thus preferable) way: cd /from; find -d . | cpio -dumpl /to While I don't want to stifle anyone's creativity, I agree with Oliver (and other posters) on this one. The Unix way of doing things is small programs that do their jobs well, tied together to accomplish bigger things. Doug Honestly, I agree with the 'Unix way', however if we really took that to heart, there would be no 'cp' at all. We'd use dd instead, along with find, and chmod/chown. But we don't - why? Same reason for adding nearly any option. All the mentions of cpio, pax, tar, etc, are all great, and I know of them, but they are harder to remember, more typing, less 'compatible' in some cases (not all), possibly more prone to human error (typos), etc. The patch doesn't change any current behavior, nor should it be noticed by anyone not looking for it. However, it is useful, and it does make our cp work just like the GNU cp, which eases the migration path for linux->FreeBSD users. I suppose I'm just missing the reason *not* to commit such a simple and useful set of options. Eric -- Eric AndersonSr. Systems AdministratorCentaur Technology Anything that works is better than anything that doesn't. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Eric Anderson wrote: So, you can mimic an entire tree with something like: cp -al /from/ /to/ pax -rwl -pe /from /to is almost what you want. (It requires that /to exist first, though.) If you want to match the 'cp' semantics when /to does not exist, you can use pax's rewrite option: pax -rwl -pe -s|/from|/to| /from / As a bonus, this works on any machine that meets POSIX, unlike every other option that's been mentioned in this thread (tar and cpio were both dropped from the POSIX standard a decade ago). Tim ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Thursday 27 July 2006 13:44, Doug Barton wrote: > Oliver Fromme wrote: > > Eric Anderson <[EMAIL PROTECTED]> wrote: > > > I'm tired of trying to use rsync or gcp (which doesn't like symlinks > > > often) to copy trees of files/directories using hard links, so I added > > > the gcp-ish options -a and -l. > > > > > > -a is 'archive' mode, which is just a quick form of -PpR. > > > > -P is the default anyway, so -a would only replace -Rp. > > I don't think saving one letter justifies introducing a new > > option. You can use an alias or shell function. > > > > > -l is 'link' mode, where regular files get hard linked instead of copied. > > > > > > So, you can mimic an entire tree with something like: > > > > > > cp -al /from/ /to/ > > > > > > and it's fast too! > > > > You can do the same with existing tools in a portable > > (and thus preferable) way: > > > > cd /from; find -d . | cpio -dumpl /to > > While I don't want to stifle anyone's creativity, I agree with Oliver (and > other posters) on this one. The Unix way of doing things is small programs > that do their jobs well, tied together to accomplish bigger things. OTOH, 'cp -al' is a lot less to type. :) It also is not NIH as it is simulating the interface of another system. Maybe I'm just stodgy b/c I never use cpio(8) (it seems to be one of the more cryptic programs). -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Oliver Fromme wrote: > Eric Anderson <[EMAIL PROTECTED]> wrote: > > I'm tired of trying to use rsync or gcp (which doesn't like symlinks > > often) to copy trees of files/directories using hard links, so I added > > the gcp-ish options -a and -l. > > > > -a is 'archive' mode, which is just a quick form of -PpR. > > -P is the default anyway, so -a would only replace -Rp. > I don't think saving one letter justifies introducing a new > option. You can use an alias or shell function. > > > -l is 'link' mode, where regular files get hard linked instead of copied. > > > > So, you can mimic an entire tree with something like: > > > > cp -al /from/ /to/ > > > > and it's fast too! > > You can do the same with existing tools in a portable > (and thus preferable) way: > > cd /from; find -d . | cpio -dumpl /to While I don't want to stifle anyone's creativity, I agree with Oliver (and other posters) on this one. The Unix way of doing things is small programs that do their jobs well, tied together to accomplish bigger things. Doug -- This .signature sanitized for your protection ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Eric Anderson <[EMAIL PROTECTED]> wrote: > I'm tired of trying to use rsync or gcp (which doesn't like symlinks > often) to copy trees of files/directories using hard links, so I added > the gcp-ish options -a and -l. > > -a is 'archive' mode, which is just a quick form of -PpR. -P is the default anyway, so -a would only replace -Rp. I don't think saving one letter justifies introducing a new option. You can use an alias or shell function. > -l is 'link' mode, where regular files get hard linked instead of copied. > > So, you can mimic an entire tree with something like: > > cp -al /from/ /to/ > > and it's fast too! You can do the same with existing tools in a portable (and thus preferable) way: cd /from; find -d . | cpio -dumpl /to Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "With sufficient thrust, pigs fly just fine. However, this is not necessarily a good idea. It is hard to be sure where they are going to land, and it could be dangerous sitting under them as they fly overhead." -- RFC 1925 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On Wed, Jul 26, 2006 at 08:51:13PM -0700 I heard the voice of Julian Elischer, and lo! it spake thus: > > I've always used: > find . -depth |cpio -pdlmv $dest While we're in workarounds, I fake with: % cat ~/bin/tarcp.sh #!/bin/sh cmd1="tar -cf - -C $1 ." cmd2="tar -xvpf - -C $2" echo "$cmd1 | $cmd2" $cmd1 | $cmd2 (which has the twin advantages of telling me what it's doing as it goes, and using two processes so it can pound source and dest disks in full parallel) -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
Eric Anderson wrote: I'm tired of trying to use rsync or gcp (which doesn't like symlinks often) to copy trees of files/directories using hard links, so I added the gcp-ish options -a and -l. -a is 'archive' mode, which is just a quick form of -PpR. -l is 'link' mode, where regular files get hard linked instead of copied. So, you can mimic an entire tree with something like: cp -al /from/ /to/ I've always used: find . -depth |cpio -pdlmv $dest (fully portable) but your addition can't hurt (well it might make the larger program load a bit slower but it isn't that much.) and it's fast too! Patch is against 6-STABLE, but works well on 7-CURRENT as well. Patch is here (with man page edits): http://www.googlebit.com/freebsd/patches/cp-patch cd /tmp/ fetch http://www.googlebit.com/freebsd/patches/cp-patch cd /usr/src/ patch < /tmp/cp-patch cd bin/cp make && make install Patch was done for rsnapshot users mainly (there are quite a few of us). Comments? Flames? Committers willing to commit? Eric ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: [PATCH] adding two new options to 'cp'
On 07/26/06 21:51, Eric Anderson wrote: The patch attached had some junk in it. The patch on the website doesn't have that. :) Oops.. Eric [..snip..] --- etc/mtree/BSD.include.dist 16 Nov 2005 10:50:10 - 1.100.2.2 +++ etc/mtree/BSD.include.dist 26 Jul 2006 03:42:10 - @@ -100,6 +100,8 @@ .. gate .. +journal +.. label .. mirror Index: include/Makefile === RCS file: /alt/ncvs/src/include/Makefile,v retrieving revision 1.244.2.4 diff -u -u -r1.244.2.4 Makefile --- include/Makefile17 Jul 2006 10:09:54 - 1.244.2.4 +++ include/Makefile26 Jul 2006 03:42:10 - @@ -42,8 +42,8 @@ fs/devfs fs/fdescfs fs/fifofs fs/msdosfs fs/ntfs fs/nullfs \ fs/nwfs fs/portalfs fs/procfs fs/smbfs fs/udf fs/umapfs \ fs/unionfs \ - geom/concat geom/eli geom/gate geom/label geom/mirror geom/nop \ - geom/raid3 geom/shsec geom/stripe \ + geom/concat geom/eli geom/gate geom/journal geom/label geom/mirror \ + geom/nop geom/raid3 geom/shsec geom/stripe \ isofs/cd9660 \ netatm/ipatm netatm/sigpvc netatm/spans netatm/uni \ netgraph/atm netgraph/netflow \ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]" -- Eric AndersonSr. Systems AdministratorCentaur Technology Anything that works is better than anything that doesn't. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
[PATCH] adding two new options to 'cp'
I'm tired of trying to use rsync or gcp (which doesn't like symlinks
often) to copy trees of files/directories using hard links, so I added
the gcp-ish options -a and -l.
-a is 'archive' mode, which is just a quick form of -PpR.
-l is 'link' mode, where regular files get hard linked instead of copied.
So, you can mimic an entire tree with something like:
cp -al /from/ /to/
and it's fast too!
Patch is against 6-STABLE, but works well on 7-CURRENT as well.
Patch is here (with man page edits):
http://www.googlebit.com/freebsd/patches/cp-patch
cd /tmp/
fetch http://www.googlebit.com/freebsd/patches/cp-patch
cd /usr/src/
patch < /tmp/cp-patch
cd bin/cp
make && make install
Patch was done for rsnapshot users mainly (there are quite a few of us).
Comments? Flames? Committers willing to commit?
Eric
--
Eric AndersonSr. Systems AdministratorCentaur Technology
Anything that works is better than anything that doesn't.
Index: bin/cp/cp.1
===
RCS file: /alt/ncvs/src/bin/cp/cp.1,v
retrieving revision 1.33
diff -u -u -r1.33 cp.1
--- bin/cp/cp.1 25 Feb 2005 00:40:46 - 1.33
+++ bin/cp/cp.1 26 Jul 2006 19:52:42 -
@@ -45,7 +45,7 @@
.Op Fl H | Fl L | Fl P
.Oc
.Op Fl f | i | n
-.Op Fl pv
+.Op Fl aplv
.Ar source_file target_file
.Nm
.Oo
@@ -53,7 +53,7 @@
.Op Fl H | Fl L | Fl P
.Oc
.Op Fl f | i | n
-.Op Fl pv
+.Op Fl aplv
.Ar source_file ... target_directory
.Sh DESCRIPTION
In the first synopsis form, the
@@ -116,6 +116,10 @@
or
.Xr pax 1
instead.
+.It Fl a
+Archive mode. Same as -PpR
+.It Fl l
+Create hard links to regular files instead of copying.
.It Fl f
For each existing destination pathname, remove it and
create a new file, without prompting for confirmation
Index: bin/cp/cp.c
===
RCS file: /alt/ncvs/src/bin/cp/cp.c,v
retrieving revision 1.51.2.1
diff -u -u -r1.51.2.1 cp.c
--- bin/cp/cp.c 12 Nov 2005 21:21:45 - 1.51.2.1
+++ bin/cp/cp.c 26 Jul 2006 17:49:55 -
@@ -83,7 +83,7 @@
PATH_T to = { to.p_path, emptystring, "" };
-int fflag, iflag, nflag, pflag, vflag;
+int fflag, iflag, lflag, nflag, pflag, vflag;
static int Rflag, rflag;
volatile sig_atomic_t info;
@@ -102,7 +102,7 @@
char *target;
Hflag = Lflag = Pflag = 0;
- while ((ch = getopt(argc, argv, "HLPRfinprv")) != -1)
+ while ((ch = getopt(argc, argv, "HLPRfinprvla")) != -1)
switch (ch) {
case 'H':
Hflag = 1;
@@ -140,6 +140,15 @@
case 'v':
vflag = 1;
break;
+ case 'l':
+ lflag = 1;
+ break;
+ case 'a':
+ Pflag = 1;
+ pflag = 1;
+ Rflag = 1;
+ Hflag = Lflag = 0;
+ break;
default:
usage();
break;
Index: bin/cp/extern.h
===
RCS file: /alt/ncvs/src/bin/cp/extern.h,v
retrieving revision 1.19.8.1
diff -u -u -r1.19.8.1 extern.h
--- bin/cp/extern.h 12 Nov 2005 21:21:45 - 1.19.8.1
+++ bin/cp/extern.h 26 Jul 2006 17:49:55 -
@@ -37,7 +37,7 @@
} PATH_T;
extern PATH_T to;
-extern int fflag, iflag, nflag, pflag, vflag;
+extern int fflag, iflag, lflag, nflag, pflag, vflag;
extern volatile sig_atomic_t info;
__BEGIN_DECLS
Index: bin/cp/utils.c
===
RCS file: /alt/ncvs/src/bin/cp/utils.c,v
retrieving revision 1.45.2.1
diff -u -u -r1.45.2.1 utils.c
--- bin/cp/utils.c 12 Nov 2005 21:21:45 - 1.45.2.1
+++ bin/cp/utils.c 26 Jul 2006 19:39:09 -
@@ -61,7 +61,7 @@
{
static char buf[MAXBSIZE];
struct stat *fs;
- int ch, checkch, from_fd, rcount, rval, to_fd;
+ int ch, checkch, from_fd = 0, rcount, rval, to_fd = 0;
ssize_t wcount;
size_t wresid;
size_t wtotal;
@@ -109,15 +109,20 @@
/* remove existing destination file name,
* create a new file */
(void)unlink(to.p_path);
- to_fd = open(to.p_path, O_WRONLY | O_TRUNC | O_CREAT,
-fs->st_mode & ~(S_ISUID | S_ISGID));
- } else
- /* overwrite existing destination file name */
- to_fd = open(to.p_path, O_WRONLY | O_TRUNC, 0);
- } else
- to_fd = open(to.p_path, O_WRONLY | O_TRUNC | O_CREAT,
- fs->st_mode & ~(S_ISUID | S_ISGID));
-
+ if (!lflag)
+
