Re: Updated: cp -t patch (w/ commentary)

[David O'Brien]

On Wed, Apr 25, 2001 at 06:08:15PM -0700, Dima Dorfman wrote:
>   Dima Dorfman
>   [EMAIL PROTECTED]
> 
> P.S.  obrien: that's a very clever and unintrusive way of avoiding
> getting two copies of a message; much better than [EMAIL PROTECTED]

P.S. Dima: see the archives for my discussion of my requirements.  I
welcome your "very clever and unintrusive way", but if it is just the
statement "use procmail", don't bother unless you're going to supply the
rules that suit my needs.  The standard simplistic delete dups doesn't.

BTW, I am not the only one that feels procmail isn't suffient, and uses
things like "TrimYourCC".

> Those of us (well, at least me) who actually want a copy of the
> message in our inbox greatly appreciate it.  Thanks!

Yes, so I've discovered.  I no longer trim them myself.  I would just add
a request to be removed from the CC line on replies to my emails, but
nobody would pay attention to it.

-- 
-- David  ([EMAIL PROTECTED])

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[Garance A Drosihn]

At 6:08 PM -0700 4/25/01, Dima Dorfman wrote:
>Garance A Drosihn <[EMAIL PROTECTED]> writes:
>  > Dimi has written one or two different patches to xargs.  Did
> ^^^ <-- should be 'a', but that's okay. :-)

Note that I also wrote:

>  > If you need an immediate fix, I'll be happy to change Dimi's
>>  patch to use a different letter, and commit the change later
>  > tonight.

I guess it wasn't obvious, but I meant I was going to be changing
a letter in your name.   :-)

(just a joke, folks...)

-- 
Garance Alistair Drosehn=   [EMAIL PROTECTED]
Senior Systems Programmer   or  [EMAIL PROTECTED]
Rensselaer Polytechnic Instituteor  [EMAIL PROTECTED]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[Dima Dorfman]

Garance A Drosihn <[EMAIL PROTECTED]> writes:
> At 10:01 AM -0400 4/25/01, John W. De Boskey wrote:
> >I have reduced the runtime of the process so far by a solid
> >hour.  My change to cp is the lowest level/minimal change fix
> >which allows me to maintain a O(1) time constraint. I've played
> >with (non-freebsd) versions of xargs already, and found them
> >(the various algorithms in xargs) to be more expensive than the
> >patch to cp.
> 
> It is inconceivable that the proposed patch to 'xargs' would
> increase your running time.  I don't mean the standard '-I'
> change, which would certainly destroy performance, but the
> proposed patch to 'xargs' which solves your specific problem
> in a general way.
> 
> I'm still curious as to why you think the proposed change to
> xargs will cause you ANY performance problem.  I simply can
> not imagine where you would get a performance problem from
> the -Y idea (though I'm still tempted to change the letter
> for that proposed option).

I think everything that should have been said in this thread already
has been (so I won't repeat it), except for the performace bit.  As
the author of the patch, I doubt it would hinder performance.  All it
does is move one part of a loop further down.  Instead of doing
something once, it does part of that job twice.  This job acts on
arguments *to xargs* (i.e., argv), and is nothing more than pointer
arithmetic and assignment.  Unless you give umpteen arguments *to
xargs*, you shouldn't notice a difference in execution speed.

And as you (gad) said, implementing -I the way SUSv2 defines it would
most likely kill performance (most likely it'd also slow down whether
you actually use that option or not).

> Dimi has written one or two different patches to xargs.  Did
^^^ <-- should be 'a', but that's okay. :-)

One patch.

Thanks,

Dima Dorfman
[EMAIL PROTECTED]

P.S.  obrien: that's a very clever and unintrusive way of avoiding
getting two copies of a message; much better than [EMAIL PROTECTED]
Those of us (well, at least me) who actually want a copy of the
message in our inbox greatly appreciate it.  Thanks!

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[Warner Losh]

aIn message <[EMAIL PROTECTED]> "John W. De Boskey" writes:
:I must say at this point, I tend to agree with him. Basically,
: my review request was skipped over and folks simply went on to
: discuss/argue the merits/demerits of various patchs to xargs. The
: question of whether xargs is appropriate and maintains adequate
: performance for my particular process seems to have been left on
: the roadside...
: 
:I hope I haven't rambled to much. And again, thanks for taking
: the time to respond.

We didn't pass over your patches, but instead pointed out where a more
general solution was possible.  Often times this ahs happened for
features that people suggest.  If we just add it to cp, then we'll
have to support it forever and it will be yet another difference that
we have in our system that will cause problems for other people.

So despite what jkh is saying, I'd not commit it.


Warner

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[Brian Somers]

> It is inconceivable that the proposed patch to 'xargs' would
> increase your running time.  I don't mean the standard '-I'
> change, which would certainly destroy performance, but the
> proposed patch to 'xargs' which solves your specific problem
> in a general way.
> 
> I'm still curious as to why you think the proposed change to
> xargs will cause you ANY performance problem.  I simply can
> not imagine where you would get a performance problem from
> the -Y idea (though I'm still tempted to change the letter
> for that proposed option).

I suspect that the bulk of the readers of this thread weren't paying 
attention.

To summarise, the actual patch that Dima wrote made

  blah | xargs -Y {} cp {} somedir

work as expected, replacing {} with as much of stdin as will fit.
It was then suggested that

#! /bin/sh
cmd=$1
last=$2
shift 2
exec "$cmd" "$@" "$last"

would solve the problem and the only argument against it was that ENV 
could corrupt the script and induce an E2BIG.  I didn't consider that 
argument strong enough, so I stepped out - that's why I'm not writing 
this email.

> -- 
> Garance Alistair Drosehn=   [EMAIL PROTECTED]
> Senior Systems Programmer   or  [EMAIL PROTECTED]
> Rensselaer Polytechnic Instituteor  [EMAIL PROTECTED]

-- 
Brian <[EMAIL PROTECTED]>
     
Don't _EVER_ lose your sense of humour !



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[Brian Somers]

[.]
> The "xargs weenies" have also offered an explicit patch that
> could be tried, but that patch is being ignored by you.  It
> is not a matter of talking ourselves to death, it's a matter
> that we're looking for feedback from anyone who wants to
> respond to the proposed xargs changes.
> 
> If you need an immediate fix, I'll be happy to change Dimi's
> patch to use a different letter, and commit the change later
> tonight.  We'll forget this "ask for input" stage, if Jordan
> really finds it so bothersome.

To be fair to Jordan, I don't think this is aimed at the individuals, 
but more at the discussion that was filling his mail box...  Let's 
not take a shouldn't-have-been-quoted email out of context eh ?

> -- 
> Garance Alistair Drosehn=   [EMAIL PROTECTED]
> Senior Systems Programmer   or  [EMAIL PROTECTED]
> Rensselaer Polytechnic Instituteor  [EMAIL PROTECTED]

-- 
Brian <[EMAIL PROTECTED]>
     
Don't _EVER_ lose your sense of humour !



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[Jordan Hubbard]

> However, a specific hack to cp(1) is what a lot of people don't like.
> If FreeBSD contained every little hack every committer had used to
> address specific problems, it'd be a mess.

I was told that the "hack" everyone is referring to is already
implemented in several other operating systems, but I must confess
that my searches so far haven't turned up the expected evidence.  I
did find a rather useful `-u' flag to cp in Redhat 6.2 which FreeBSD
could probably stand to adopt, but other than that...  I may have to
reverse myself on this one if I can't find at least one other vendor
which has adopted -t.

- Jordan

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[Garance A Drosihn]

At 10:01 AM -0400 4/25/01, John W. De Boskey wrote:
>Hi David, Brian,
>
>Thank you for taking the time to reply. I hope you were
>able to review the patch also.

Every time you have asked for people's opinions, they have
said that it seems wrong to made add a specific option to
the 'cp' command to address a generic problem with the
'xargs' command.  You continue to pretend that this is not
a valid comment on your proposed change.  If you do not want
our opinions, then stop asking for them.

You then offer to do a similar update to 'mv', again to fix
the problem when using 'mv' with 'xargs'.  Will you also do
updates for 'scp'?  How about 'fs setacl'?  (an AFS command).
Other commands?  Why should we fix all these commands to
address a problem caused by using them with xargs?  Why not
fix 'xargs', at which point we don't have to care about any
list of commands (even weird ones like 'fs setacl') which
have this same problem.

>I have reduced the runtime of the process so far by a solid
>hour.  My change to cp is the lowest level/minimal change fix
>which allows me to maintain a O(1) time constraint. I've played
>with (non-freebsd) versions of xargs already, and found them
>(the various algorithms in xargs) to be more expensive than the
>patch to cp.

It is inconceivable that the proposed patch to 'xargs' would
increase your running time.  I don't mean the standard '-I'
change, which would certainly destroy performance, but the
proposed patch to 'xargs' which solves your specific problem
in a general way.

I'm still curious as to why you think the proposed change to
xargs will cause you ANY performance problem.  I simply can
not imagine where you would get a performance problem from
the -Y idea (though I'm still tempted to change the letter
for that proposed option).

Dimi has written one or two different patches to xargs.  Did
you try any of them?  (ignore the fact that he used '-I' as
the letter for what was supposed to be the NEW option, *that*
was a mistake!).  How DID that patch effect your running time?

>I realize you folks are not here, and cannot examine the
>processes I have to deal with first hand.  I can only simply
>ask you to trust that the work I and others have done while
>coming to the conclusion that the cp patch is the best
>alternative is correct.

It isn't so much that we don't "trust" you, we're just
wondering why the patch to 'xargs' does not solve the same
problem you're trying to solve.  We could also ask you to
"trust" us, in that we already know the exact problem you
are describing, and we *are* trying to address it.  I've
hit the exact same problem in the past, it's just that I've
always solved it by writing a short script.

If we are going to open the floor to adding non-standard
options to standard unix commands, then it seems much better
to add one option to one command, instead of adding options
to a list of commands.

>On a different note, I have spoken with my mentor
>(seems funny calling him that these days) Jordan, and his
>response to my email was:
>
> I think you should just commit the cp changes and
>let the xargs weenies debate themselves silly if they
>want to. :)  The two issues are not really related.
>
>-Jordan
>
>I must say at this point, I tend to agree with him.

I think the problem is that this *discussion* has rambled
off in several different directions, many of which have
no bearing to your situation.  That doesn't mean we aren't
honestly trying to come up with a good general solution
which *is* directly related to your problem.  It just means
that we're tossing in a few extra things in addition to
the solution for your situation.  We should probably fix
your problem first, and discuss the rest of it later.

The "xargs weenies" have also offered an explicit patch that
could be tried, but that patch is being ignored by you.  It
is not a matter of talking ourselves to death, it's a matter
that we're looking for feedback from anyone who wants to
respond to the proposed xargs changes.

If you need an immediate fix, I'll be happy to change Dimi's
patch to use a different letter, and commit the change later
tonight.  We'll forget this "ask for input" stage, if Jordan
really finds it so bothersome.

-- 
Garance Alistair Drosehn=   [EMAIL PROTECTED]
Senior Systems Programmer   or  [EMAIL PROTECTED]
Rensselaer Polytechnic Instituteor  [EMAIL PROTECTED]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[Maxim Sobolev]

"John W. De Boskey" wrote:

> Hi David, Brian,
>
>Thank you for taking the time to reply. I hope you were
> able to review the patch also.
>
>I am dealing with a production process that currently runs
> approximately 10 hours. (on 28 866Mhz processors, 2 Netapps).
> This process fell into my lap about 2 months ago.
>
>After studying the process, the 1st item that came to the
> fore-front was the number of fork/exec pairs occuring for the
> file copy process. Please note, as written in previous emails,
> the copy process copies files from multiple directories to a
> singular directory.
>
>I have reduced the runtime of the process so far by a solid
> hour.  My change to cp is the lowest level/minimal change fix
> which allows me to maintain a O(1) time constraint. I've played
> with (non-freebsd) versions of xargs already, and found them
> (the various algorithms in xargs) to be more expensive than the
> patch to cp.
>
>I realize you folks are not here, and cannot examine the processes
> I have to deal with first hand.  I can only simply ask you to
> trust that the work I and others have done while coming to the
> conclusion that the cp patch is the best alternative is correct.
>
>On a different note, I have spoken with my mentor (seems funny
> calling him that these days) Jordan, and his response to my
> email was:
>
> 
> I think you should just commit the cp changes and let the
> xargs weenies debate themselves silly if the want to. :)
> The two issues are not really related.
>
> -Jordan
> 
>
>I must say at this point, I tend to agree with him. Basically,
> my review request was skipped over and folks simply went on to
> discuss/argue the merits/demerits of various patchs to xargs. The
> question of whether xargs is appropriate and maintains adequate
> performance for my particular process seems to have been left on
> the roadside...
>
>I hope I haven't rambled to much. And again, thanks for taking
> the time to respond.

The only thing that I can't understand is why you want this incompatible
featute to go into cvs repo. Why you can't make this very specific
modification locally and use it on your own, or just make a port of it if you
really think that it might be useful to somebody else?

-Maxim


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[John W. De Boskey]

Hi David, Brian,

   Thank you for taking the time to reply. I hope you were
able to review the patch also.

   I am dealing with a production process that currently runs
approximately 10 hours. (on 28 866Mhz processors, 2 Netapps).
This process fell into my lap about 2 months ago.

   After studying the process, the 1st item that came to the
fore-front was the number of fork/exec pairs occuring for the
file copy process. Please note, as written in previous emails,
the copy process copies files from multiple directories to a
singular directory.

   I have reduced the runtime of the process so far by a solid
hour.  My change to cp is the lowest level/minimal change fix
which allows me to maintain a O(1) time constraint. I've played
with (non-freebsd) versions of xargs already, and found them
(the various algorithms in xargs) to be more expensive than the
patch to cp.

   I realize you folks are not here, and cannot examine the processes
I have to deal with first hand.  I can only simply ask you to 
trust that the work I and others have done while coming to the
conclusion that the cp patch is the best alternative is correct.


   On a different note, I have spoken with my mentor (seems funny
calling him that these days) Jordan, and his response to my
email was:


I think you should just commit the cp changes and let the
xargs weenies debate themselves silly if the want to. :)
The two issues are not really related.

-Jordan


   I must say at this point, I tend to agree with him. Basically,
my review request was skipped over and folks simply went on to
discuss/argue the merits/demerits of various patchs to xargs. The
question of whether xargs is appropriate and maintains adequate
performance for my particular process seems to have been left on
the roadside...

   I hope I haven't rambled to much. And again, thanks for taking
the time to respond.

-John


- Current List's Original Message -
> On Mon, Apr 23, 2001 at 11:33:24AM -0700, John W. De Boskey wrote:
> >After some feedback, I have changed the patch slightly. Rename
> > -d to -t and remove the requirement for the option to have a
> > value.
> 
> I thought people generally agreed the right fix was to add functionality
> to `xargs', not `cp' as you aren't scratching the general itch.
>  
> -- 
> -- David  ([EMAIL PROTECTED])
> 
> To Unsubscribe: send mail to [EMAIL PROTECTED]
> with "unsubscribe freebsd-current" in the body of the message

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[Sheldon Hearn]

On Wed, 25 Apr 2001 10:01:18 -0400, "John W. De Boskey" wrote:

>I am dealing with a production process that currently runs
> approximately 10 hours. (on 28 866Mhz processors, 2 Netapps).
> This process fell into my lap about 2 months ago.

Something to consider is that you're trying to solve a fairly specific
problem.  What everyone's asking for is a general solution.

Perhaps there is no general solution to your specific problem.  Perhaps
shell magic or a perl script is your answer.

However, a specific hack to cp(1) is what a lot of people don't like.
If FreeBSD contained every little hack every committer had used to
address specific problems, it'd be a mess.

Ciao,
Sheldon.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[Brian Somers]

> On Mon, Apr 23, 2001 at 11:33:24AM -0700, John W. De Boskey wrote:
> >After some feedback, I have changed the patch slightly. Rename
> > -d to -t and remove the requirement for the option to have a
> > value.
> 
> I thought people generally agreed the right fix was to add functionality
> to `xargs', not `cp' as you aren't scratching the general itch.

In fact, I think enough people disagreed that xargs(1) should be 
modified as it can be done with scripts.

Personally, I'd prefer Dima's xargs(1) fix.

> -- 
> -- David  ([EMAIL PROTECTED])

-- 
Brian <[EMAIL PROTECTED]>
     
Don't _EVER_ lose your sense of humour !



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Updated: cp -t patch (w/ commentary)

[David O'Brien]

On Mon, Apr 23, 2001 at 11:33:24AM -0700, John W. De Boskey wrote:
>After some feedback, I have changed the patch slightly. Rename
> -d to -t and remove the requirement for the option to have a
> value.

I thought people generally agreed the right fix was to add functionality
to `xargs', not `cp' as you aren't scratching the general itch.
 
-- 
-- David  ([EMAIL PROTECTED])

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Updated: cp -t patch (w/ commentary)

[John W. De Boskey]

Hello,

   After some feedback, I have changed the patch slightly. Rename
-d to -t and remove the requirement for the option to have a
value.

   -t  aquire the target from *argv++ instead of argv[argc--]

   The patch can be found at:

http://people.freebsd.org/~jwd/cp-t.patch


   Some comments:

   -t  was chosen to help match the existing (and be co-existant
   with) gnu cp option--target-directory=DIRECTORY


   I have seen many proposals posted with respect to "why don't
you do it this way?"  Bascally,  they don't do the same thing.

   cat BigFileList | xargs cp -t targetdir

   where "cat BigFileList" is actually a seperate and complex
process that I do not own/support.

   The list of files given to the process spans many subtrees
and mount points. The goal is to bring a copy of those files 
into a single directory.

   Tar, cpio, and other file copy processes copy hierarchies. Cp
is the only program which allows for a simple yet elegant way
of transferring files from disparate locations to a single location.

   cp -s  allows for a very simple yet highly effective method of
doing what needs to be done.  Yes, I am also willing to update
mv to work the same way.

-John

ps: As a humorous aside with respect to standards. I thought
FreeBSD set the standard! :-)

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message