Re: Updated: cp -t patch (w/ commentary)
At 6:08 PM -0700 4/25/01, Dima Dorfman wrote: Garance A Drosihn [EMAIL PROTECTED] writes: Dimi has written one or two different patches to xargs. Did ^^^ -- should be 'a', but that's okay. :-) Note that I also wrote: If you need an immediate fix, I'll be happy to change Dimi's patch to use a different letter, and commit the change later tonight. I guess it wasn't obvious, but I meant I was going to be changing a letter in your name. :-) (just a joke, folks...) -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
On Wed, Apr 25, 2001 at 06:08:15PM -0700, Dima Dorfman wrote: Dima Dorfman [EMAIL PROTECTED] P.S. obrien: that's a very clever and unintrusive way of avoiding getting two copies of a message; much better than [EMAIL PROTECTED] P.S. Dima: see the archives for my discussion of my requirements. I welcome your very clever and unintrusive way, but if it is just the statement use procmail, don't bother unless you're going to supply the rules that suit my needs. The standard simplistic delete dups doesn't. BTW, I am not the only one that feels procmail isn't suffient, and uses things like TrimYourCC. Those of us (well, at least me) who actually want a copy of the message in our inbox greatly appreciate it. Thanks! Yes, so I've discovered. I no longer trim them myself. I would just add a request to be removed from the CC line on replies to my emails, but nobody would pay attention to it. -- -- David ([EMAIL PROTECTED]) To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
On Mon, Apr 23, 2001 at 11:33:24AM -0700, John W. De Boskey wrote: After some feedback, I have changed the patch slightly. Rename -d to -t and remove the requirement for the option to have a value. I thought people generally agreed the right fix was to add functionality to `xargs', not `cp' as you aren't scratching the general itch. -- -- David ([EMAIL PROTECTED]) To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
On Mon, Apr 23, 2001 at 11:33:24AM -0700, John W. De Boskey wrote: After some feedback, I have changed the patch slightly. Rename -d to -t and remove the requirement for the option to have a value. I thought people generally agreed the right fix was to add functionality to `xargs', not `cp' as you aren't scratching the general itch. In fact, I think enough people disagreed that xargs(1) should be modified as it can be done with scripts. Personally, I'd prefer Dima's xargs(1) fix. -- -- David ([EMAIL PROTECTED]) -- Brian [EMAIL PROTECTED]brian@[uk.]FreeBSD.org http://www.Awfulhak.org brian@[uk.]OpenBSD.org Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
On Wed, 25 Apr 2001 10:01:18 -0400, John W. De Boskey wrote: I am dealing with a production process that currently runs approximately 10 hours. (on 28 866Mhz processors, 2 Netapps). This process fell into my lap about 2 months ago. Something to consider is that you're trying to solve a fairly specific problem. What everyone's asking for is a general solution. Perhaps there is no general solution to your specific problem. Perhaps shell magic or a perl script is your answer. However, a specific hack to cp(1) is what a lot of people don't like. If FreeBSD contained every little hack every committer had used to address specific problems, it'd be a mess. Ciao, Sheldon. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
Hi David, Brian, Thank you for taking the time to reply. I hope you were able to review the patch also. I am dealing with a production process that currently runs approximately 10 hours. (on 28 866Mhz processors, 2 Netapps). This process fell into my lap about 2 months ago. After studying the process, the 1st item that came to the fore-front was the number of fork/exec pairs occuring for the file copy process. Please note, as written in previous emails, the copy process copies files from multiple directories to a singular directory. I have reduced the runtime of the process so far by a solid hour. My change to cp is the lowest level/minimal change fix which allows me to maintain a O(1) time constraint. I've played with (non-freebsd) versions of xargs already, and found them (the various algorithms in xargs) to be more expensive than the patch to cp. I realize you folks are not here, and cannot examine the processes I have to deal with first hand. I can only simply ask you to trust that the work I and others have done while coming to the conclusion that the cp patch is the best alternative is correct. On a different note, I have spoken with my mentor (seems funny calling him that these days) Jordan, and his response to my email was: I think you should just commit the cp changes and let the xargs weenies debate themselves silly if the want to. :) The two issues are not really related. -Jordan I must say at this point, I tend to agree with him. Basically, my review request was skipped over and folks simply went on to discuss/argue the merits/demerits of various patchs to xargs. The question of whether xargs is appropriate and maintains adequate performance for my particular process seems to have been left on the roadside... I hope I haven't rambled to much. And again, thanks for taking the time to respond. -John - Current List's Original Message - On Mon, Apr 23, 2001 at 11:33:24AM -0700, John W. De Boskey wrote: After some feedback, I have changed the patch slightly. Rename -d to -t and remove the requirement for the option to have a value. I thought people generally agreed the right fix was to add functionality to `xargs', not `cp' as you aren't scratching the general itch. -- -- David ([EMAIL PROTECTED]) To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
John W. De Boskey wrote: Hi David, Brian, Thank you for taking the time to reply. I hope you were able to review the patch also. I am dealing with a production process that currently runs approximately 10 hours. (on 28 866Mhz processors, 2 Netapps). This process fell into my lap about 2 months ago. After studying the process, the 1st item that came to the fore-front was the number of fork/exec pairs occuring for the file copy process. Please note, as written in previous emails, the copy process copies files from multiple directories to a singular directory. I have reduced the runtime of the process so far by a solid hour. My change to cp is the lowest level/minimal change fix which allows me to maintain a O(1) time constraint. I've played with (non-freebsd) versions of xargs already, and found them (the various algorithms in xargs) to be more expensive than the patch to cp. I realize you folks are not here, and cannot examine the processes I have to deal with first hand. I can only simply ask you to trust that the work I and others have done while coming to the conclusion that the cp patch is the best alternative is correct. On a different note, I have spoken with my mentor (seems funny calling him that these days) Jordan, and his response to my email was: I think you should just commit the cp changes and let the xargs weenies debate themselves silly if the want to. :) The two issues are not really related. -Jordan I must say at this point, I tend to agree with him. Basically, my review request was skipped over and folks simply went on to discuss/argue the merits/demerits of various patchs to xargs. The question of whether xargs is appropriate and maintains adequate performance for my particular process seems to have been left on the roadside... I hope I haven't rambled to much. And again, thanks for taking the time to respond. The only thing that I can't understand is why you want this incompatible featute to go into cvs repo. Why you can't make this very specific modification locally and use it on your own, or just make a port of it if you really think that it might be useful to somebody else? -Maxim To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
At 10:01 AM -0400 4/25/01, John W. De Boskey wrote: Hi David, Brian, Thank you for taking the time to reply. I hope you were able to review the patch also. Every time you have asked for people's opinions, they have said that it seems wrong to made add a specific option to the 'cp' command to address a generic problem with the 'xargs' command. You continue to pretend that this is not a valid comment on your proposed change. If you do not want our opinions, then stop asking for them. You then offer to do a similar update to 'mv', again to fix the problem when using 'mv' with 'xargs'. Will you also do updates for 'scp'? How about 'fs setacl'? (an AFS command). Other commands? Why should we fix all these commands to address a problem caused by using them with xargs? Why not fix 'xargs', at which point we don't have to care about any list of commands (even weird ones like 'fs setacl') which have this same problem. I have reduced the runtime of the process so far by a solid hour. My change to cp is the lowest level/minimal change fix which allows me to maintain a O(1) time constraint. I've played with (non-freebsd) versions of xargs already, and found them (the various algorithms in xargs) to be more expensive than the patch to cp. It is inconceivable that the proposed patch to 'xargs' would increase your running time. I don't mean the standard '-I' change, which would certainly destroy performance, but the proposed patch to 'xargs' which solves your specific problem in a general way. I'm still curious as to why you think the proposed change to xargs will cause you ANY performance problem. I simply can not imagine where you would get a performance problem from the -Y idea (though I'm still tempted to change the letter for that proposed option). Dimi has written one or two different patches to xargs. Did you try any of them? (ignore the fact that he used '-I' as the letter for what was supposed to be the NEW option, *that* was a mistake!). How DID that patch effect your running time? I realize you folks are not here, and cannot examine the processes I have to deal with first hand. I can only simply ask you to trust that the work I and others have done while coming to the conclusion that the cp patch is the best alternative is correct. It isn't so much that we don't trust you, we're just wondering why the patch to 'xargs' does not solve the same problem you're trying to solve. We could also ask you to trust us, in that we already know the exact problem you are describing, and we *are* trying to address it. I've hit the exact same problem in the past, it's just that I've always solved it by writing a short script. If we are going to open the floor to adding non-standard options to standard unix commands, then it seems much better to add one option to one command, instead of adding options to a list of commands. On a different note, I have spoken with my mentor (seems funny calling him that these days) Jordan, and his response to my email was: I think you should just commit the cp changes and let the xargs weenies debate themselves silly if they want to. :) The two issues are not really related. -Jordan I must say at this point, I tend to agree with him. I think the problem is that this *discussion* has rambled off in several different directions, many of which have no bearing to your situation. That doesn't mean we aren't honestly trying to come up with a good general solution which *is* directly related to your problem. It just means that we're tossing in a few extra things in addition to the solution for your situation. We should probably fix your problem first, and discuss the rest of it later. The xargs weenies have also offered an explicit patch that could be tried, but that patch is being ignored by you. It is not a matter of talking ourselves to death, it's a matter that we're looking for feedback from anyone who wants to respond to the proposed xargs changes. If you need an immediate fix, I'll be happy to change Dimi's patch to use a different letter, and commit the change later tonight. We'll forget this ask for input stage, if Jordan really finds it so bothersome. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
However, a specific hack to cp(1) is what a lot of people don't like. If FreeBSD contained every little hack every committer had used to address specific problems, it'd be a mess. I was told that the hack everyone is referring to is already implemented in several other operating systems, but I must confess that my searches so far haven't turned up the expected evidence. I did find a rather useful `-u' flag to cp in Redhat 6.2 which FreeBSD could probably stand to adopt, but other than that... I may have to reverse myself on this one if I can't find at least one other vendor which has adopted -t. - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
[.] The xargs weenies have also offered an explicit patch that could be tried, but that patch is being ignored by you. It is not a matter of talking ourselves to death, it's a matter that we're looking for feedback from anyone who wants to respond to the proposed xargs changes. If you need an immediate fix, I'll be happy to change Dimi's patch to use a different letter, and commit the change later tonight. We'll forget this ask for input stage, if Jordan really finds it so bothersome. To be fair to Jordan, I don't think this is aimed at the individuals, but more at the discussion that was filling his mail box... Let's not take a shouldn't-have-been-quoted email out of context eh ? -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] -- Brian [EMAIL PROTECTED]brian@[uk.]FreeBSD.org http://www.Awfulhak.org brian@[uk.]OpenBSD.org Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
It is inconceivable that the proposed patch to 'xargs' would
increase your running time. I don't mean the standard '-I'
change, which would certainly destroy performance, but the
proposed patch to 'xargs' which solves your specific problem
in a general way.
I'm still curious as to why you think the proposed change to
xargs will cause you ANY performance problem. I simply can
not imagine where you would get a performance problem from
the -Y idea (though I'm still tempted to change the letter
for that proposed option).
I suspect that the bulk of the readers of this thread weren't paying
attention.
To summarise, the actual patch that Dima wrote made
blah | xargs -Y {} cp {} somedir
work as expected, replacing {} with as much of stdin as will fit.
It was then suggested that
#! /bin/sh
cmd=$1
last=$2
shift 2
exec $cmd $@ $last
would solve the problem and the only argument against it was that ENV
could corrupt the script and induce an E2BIG. I didn't consider that
argument strong enough, so I stepped out - that's why I'm not writing
this email.
--
Garance Alistair Drosehn= [EMAIL PROTECTED]
Senior Systems Programmer or [EMAIL PROTECTED]
Rensselaer Polytechnic Instituteor [EMAIL PROTECTED]
--
Brian [EMAIL PROTECTED]brian@[uk.]FreeBSD.org
http://www.Awfulhak.org brian@[uk.]OpenBSD.org
Don't _EVER_ lose your sense of humour !
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
aIn message [EMAIL PROTECTED] John W. De Boskey writes: :I must say at this point, I tend to agree with him. Basically, : my review request was skipped over and folks simply went on to : discuss/argue the merits/demerits of various patchs to xargs. The : question of whether xargs is appropriate and maintains adequate : performance for my particular process seems to have been left on : the roadside... : :I hope I haven't rambled to much. And again, thanks for taking : the time to respond. We didn't pass over your patches, but instead pointed out where a more general solution was possible. Often times this ahs happened for features that people suggest. If we just add it to cp, then we'll have to support it forever and it will be yet another difference that we have in our system that will cause problems for other people. So despite what jkh is saying, I'd not commit it. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Updated: cp -t patch (w/ commentary)
Garance A Drosihn [EMAIL PROTECTED] writes: At 10:01 AM -0400 4/25/01, John W. De Boskey wrote: I have reduced the runtime of the process so far by a solid hour. My change to cp is the lowest level/minimal change fix which allows me to maintain a O(1) time constraint. I've played with (non-freebsd) versions of xargs already, and found them (the various algorithms in xargs) to be more expensive than the patch to cp. It is inconceivable that the proposed patch to 'xargs' would increase your running time. I don't mean the standard '-I' change, which would certainly destroy performance, but the proposed patch to 'xargs' which solves your specific problem in a general way. I'm still curious as to why you think the proposed change to xargs will cause you ANY performance problem. I simply can not imagine where you would get a performance problem from the -Y idea (though I'm still tempted to change the letter for that proposed option). I think everything that should have been said in this thread already has been (so I won't repeat it), except for the performace bit. As the author of the patch, I doubt it would hinder performance. All it does is move one part of a loop further down. Instead of doing something once, it does part of that job twice. This job acts on arguments *to xargs* (i.e., argv), and is nothing more than pointer arithmetic and assignment. Unless you give umpteen arguments *to xargs*, you shouldn't notice a difference in execution speed. And as you (gad) said, implementing -I the way SUSv2 defines it would most likely kill performance (most likely it'd also slow down whether you actually use that option or not). Dimi has written one or two different patches to xargs. Did ^^^ -- should be 'a', but that's okay. :-) One patch. Thanks, Dima Dorfman [EMAIL PROTECTED] P.S. obrien: that's a very clever and unintrusive way of avoiding getting two copies of a message; much better than [EMAIL PROTECTED] Those of us (well, at least me) who actually want a copy of the message in our inbox greatly appreciate it. Thanks! To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Updated: cp -t patch (w/ commentary)
Hello, After some feedback, I have changed the patch slightly. Rename -d to -t and remove the requirement for the option to have a value. -t aquire the target from *argv++ instead of argv[argc--] The patch can be found at: http://people.freebsd.org/~jwd/cp-t.patch Some comments: -t was chosen to help match the existing (and be co-existant with) gnu cp option--target-directory=DIRECTORY I have seen many proposals posted with respect to why don't you do it this way? Bascally, they don't do the same thing. cat BigFileList | xargs cp -t targetdir where cat BigFileList is actually a seperate and complex process that I do not own/support. The list of files given to the process spans many subtrees and mount points. The goal is to bring a copy of those files into a single directory. Tar, cpio, and other file copy processes copy hierarchies. Cp is the only program which allows for a simple yet elegant way of transferring files from disparate locations to a single location. cp -s allows for a very simple yet highly effective method of doing what needs to be done. Yes, I am also willing to update mv to work the same way. -John ps: As a humorous aside with respect to standards. I thought FreeBSD set the standard! :-) To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
