Re: scp problem with remote filename escaping
On Thu, 12 Apr 2007, Karel Kulhavy wrote: On Wed, Apr 11, 2007 at 10:02:50PM +0100, Stuart Henderson wrote: On 2007/04/11 13:41, Bryan Irvine wrote: snip I agree, spaces in filenames should be avoided. But spaces in filenames are legal, so programs need to support that; this seems like a case scp was never tested against because no one uses files with those names. I scp'd a file called 'a b' to an openbsd server here, then scp'd it back a couple time in different ways. It worked only when using the quotes AND escaping, like so: scp [EMAIL PROTECTED]:a\ b . you have to escape to *both* your local shell, and the remote shell You must not escape to your local shell in case the scp process is called directly by e. g. exec() function in C. If you have to escape to the remote shell, then it should be mentioned in man scp. escape and shell don't occur in man scp and remote doesn't occur in such a context there. If I wrote it, I would do it in a way that scp performed the escaping for the remote shell automatically. Having to supply a different filename depending on where the file is goes against the local-remote transparency that scp is attempting at. What you forget is that scp is implementing the same protocol that rcp uses. The protocol has a lot of shortcomings. See http://www.openssh.com/faq.html#2.10 But it looks like sftp has some problem with spaces in file names as well. ie, this fails: sftp remote:/tmp/a b . In interactive mode, I can specify get 'a b', that works. -Otto
Re: scp problem with remote filename escaping
On Thu, Apr 12, 2007 at 10:44:52AM -0400, Dan Farrell wrote: Wait, so every time documentation is inaccurate or incomplete or simply not to your liking, you're going to call it a bug ``incorrect documentation is a bug'' --http://www.openbsd.org/papers/opencon06-culture.pdf (of the application no less!)? He never said it was the application's fault, just that `file1', `file2', ... are shell expanded by the remote host, but the documentation does not point this out. How about something like below? (I don't love the wording, but hopefully it's a start.) Index: scp.1 === RCS file: /cvs/src/usr.bin/ssh/scp.1,v retrieving revision 1.40 diff -u -r1.40 scp.1 --- scp.1 18 Jul 2006 07:56:28 - 1.40 +++ scp.1 12 Apr 2007 15:47:32 - @@ -58,6 +58,8 @@ .Pp Any file name may contain a host and user specification to indicate that the file is to be copied to/from that host. +The file name component of such an argument is also passed +to the specified host's login shell for expansion and splitting. Copies between two remote hosts are permitted. .Pp The options are as follows:
Re: scp problem with remote filename escaping
A bug of what though? He, in fact, did say it was a bug of the application, but because he felt the documentation was incomplete. All the more without an encoding which depends on where the file actually lies. Sounds like a bug to me - the escaping for the remote shell is not being done correctly? He's not referring to the documentation as the bug, but rather the application itself, but he derived that from his problem with the documentation. If the bug is in the documentation, fine... but address it as such, not as an accusation of the application itself (which others have subsequently proven works correctly for what he was attempting to achieve.) I'm done splitting hairs, danno -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matthew R. Dempsky Sent: Thursday, April 12, 2007 11:59 AM To: OpenBSD Subject: Re: scp problem with remote filename escaping On Thu, Apr 12, 2007 at 10:44:52AM -0400, Dan Farrell wrote: Wait, so every time documentation is inaccurate or incomplete or simply not to your liking, you're going to call it a bug ``incorrect documentation is a bug'' --http://www.openbsd.org/papers/opencon06-culture.pdf (of the application no less!)? He never said it was the application's fault, just that `file1', `file2', ... are shell expanded by the remote host, but the documentation does not point this out. How about something like below? (I don't love the wording, but hopefully it's a start.) Index: scp.1 === RCS file: /cvs/src/usr.bin/ssh/scp.1,v retrieving revision 1.40 diff -u -r1.40 scp.1 --- scp.1 18 Jul 2006 07:56:28 - 1.40 +++ scp.1 12 Apr 2007 15:47:32 - @@ -58,6 +58,8 @@ .Pp Any file name may contain a host and user specification to indicate that the file is to be copied to/from that host. +The file name component of such an argument is also passed +to the specified host's login shell for expansion and splitting. Copies between two remote hosts are permitted. .Pp The options are as follows:
scp problem with remote filename escaping
For the same filename, sometimes you have to specify a different filename to scp, depending on whether the file is on remote system or local one. I have created a remote file whose filename a b is 3 chars long - ASCII codes 97, 32, 98 scp '[EMAIL PROTECTED]:a b' . doesn't work - prints: scp: a: No such file or directory scp: b: No such file or directory I have to type scp '[EMAIL PROTECTED]:a\ b' . which propagates to the scp process as a single argument containing a backslash, as can be seen with touch a\ b (creates a file named a b) scp a\ b d - works scp 'a\ b' d - cp: a\ b: No such file or directory The manual page says I should specify file1 and file2, which (lacking further details) implies the filename is supplied without any further encoding. All the more without an encoding which depends on where the file actually lies. Sounds like a bug to me - the escaping for the remote shell is not being done correctly? Is this the intended behaviour of scp from OpenBSD 4.0? I would tell the version of the scp but scp -version, --version, -V and -v doesn't work and man scp doesn't contain the word version. CL
Re: scp problem with remote filename escaping
Wow. Seriously, I think the real 'bug' is your file naming conventions. Who would anyone specifically want to name a file with a space in it... and if breaks on scp, where else will that screwy naming convention break as well? I'm sure you'll give some really good reason why the files have to be named that way... danno -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Karel Kulhavy Sent: Wednesday, April 11, 2007 3:47 PM To: OpenBSD Subject: scp problem with remote filename escaping Sounds like a bug to me - the escaping for the remote shell is not being done correctly?
Re: scp problem with remote filename escaping
On 4/11/07, Dan Farrell [EMAIL PROTECTED] wrote: From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Karel Kulhavy Sent: Wednesday, April 11, 2007 3:47 PM To: OpenBSD Subject: scp problem with remote filename escaping Sounds like a bug to me - the escaping for the remote shell is not being done correctly? Wow. Seriously, I think the real 'bug' is your file naming conventions. Who would anyone specifically want to name a file with a space in it... and if breaks on scp, where else will that screwy naming convention break as well? I'm sure you'll give some really good reason why the files have to be named that way... I agree, spaces in filenames should be avoided. But spaces in filenames are legal, so programs need to support that; this seems like a case scp was never tested against because no one uses files with those names. -Nick
Re: scp problem with remote filename escaping
On 4/11/07, Nick ! [EMAIL PROTECTED] wrote: On 4/11/07, Dan Farrell [EMAIL PROTECTED] wrote: From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Karel Kulhavy Sent: Wednesday, April 11, 2007 3:47 PM To: OpenBSD Subject: scp problem with remote filename escaping Sounds like a bug to me - the escaping for the remote shell is not being done correctly? Wow. Seriously, I think the real 'bug' is your file naming conventions. But spaces in filenames are legal, so programs need to support that; Er, nevermind. I didn't read closely enough. Karel, single quotes cause backslashes to be backslashes, instead of escape chars (*except* if it's a backslash in front of a single quote, so that you can escape single quotes to include them). The behaviour you're seeing is entirely correct. I still stand by my abstract philosophical standpoint, that forcing certain conventions is not a substitute for bugfixing, but it turns out it doesn't apply here. -Nick
Re: scp problem with remote filename escaping
snip I agree, spaces in filenames should be avoided. But spaces in filenames are legal, so programs need to support that; this seems like a case scp was never tested against because no one uses files with those names. I scp'd a file called 'a b' to an openbsd server here, then scp'd it back a couple time in different ways. It worked only when using the quotes AND escaping, like so: scp [EMAIL PROTECTED]:a\ b . --Bryan
Re: scp problem with remote filename escaping
On Wed, Apr 11, 2007 at 04:17:38PM -0400, Nick ! wrote: On 4/11/07, Dan Farrell [EMAIL PROTECTED] wrote: From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Karel Kulhavy Sent: Wednesday, April 11, 2007 3:47 PM To: OpenBSD Subject: scp problem with remote filename escaping Sounds like a bug to me - the escaping for the remote shell is not being done correctly? Wow. Seriously, I think the real 'bug' is your file naming conventions. Who would anyone specifically want to name a file with a space in it... and if breaks on scp, where else will that screwy naming convention break as well? I'm sure you'll give some really good reason why the files have to be named that way... I agree, spaces in filenames should be avoided. But spaces in filenames are legal, so programs need to support that; this seems like a case scp was never tested against because no one uses files with those names. -Nick scp needs 3 * \ for one space.. scp [EMAIL PROTECTED]:/what/you/want/a\\\ b.txtlocalfile.txt \s -- GnuPG: 5755FB64 Per aspera ad astra.
Re: scp problem with remote filename escaping
On 2007/04/11 13:41, Bryan Irvine wrote: snip I agree, spaces in filenames should be avoided. But spaces in filenames are legal, so programs need to support that; this seems like a case scp was never tested against because no one uses files with those names. I scp'd a file called 'a b' to an openbsd server here, then scp'd it back a couple time in different ways. It worked only when using the quotes AND escaping, like so: scp [EMAIL PROTECTED]:a\ b . you have to escape to *both* your local shell, and the remote shell
Re: scp problem with remote filename escaping
On Wed, Apr 11, 2007 at 10:02:50PM +0100, Stuart Henderson wrote:
On 2007/04/11 13:41, Bryan Irvine wrote:
scp [EMAIL PROTECTED]:a\ b .
you have to escape to *both* your local shell, and the remote shell
This has always seemed silly to me. Does anyone intentionally use
$ scp host:a b .
instead of
$ scp host:{a,b} .
or
$ scp host:a host:b .
or is just that having whatever does the globbing on the host not
split at white space too difficult?
Re: scp problem with remote filename escaping
On Wed, Apr 11, 2007 at 04:33:32PM -0400, Nick ! wrote: Karel, single quotes cause backslashes to be backslashes, instead of escape chars (*except* if it's a backslash in front of a single quote, so that you can escape single quotes to include them). No, backslashes have no special meaning inside single quotes. $ echo '\' \
Re: scp problem with remote filename escaping
On Wed, Apr 11, 2007 at 04:41:41PM -0500, Matthew R. Dempsky wrote:
On Wed, Apr 11, 2007 at 10:02:50PM +0100, Stuart Henderson wrote:
On 2007/04/11 13:41, Bryan Irvine wrote:
scp [EMAIL PROTECTED]:a\ b .
you have to escape to *both* your local shell, and the remote shell
This has always seemed silly to me. Does anyone intentionally use
$ scp host:a b .
instead of
$ scp host:{a,b} .
Does anyone intentionally use
scp '[EMAIL PROTECTED]:`rm -fr /`' /dev/null ?
or
$ scp host:a host:b .
or is just that having whatever does the globbing on the host not
split at white space too difficult?
That 'whatever' is the login shell of the user on the remote host.
(which may or may not do splitting with IFS, globbing, variable expansion -
the usual stuff).
scp is fine. some people are confused.
Re: scp problem with remote filename escaping
I'm sure you'll give some really good reason why the files have to be named that way... Try admining boxes which are used by EvilOS users - all of their files will be called My\ blah.
Re: scp problem with remote filename escaping
I scp'd a file called 'a b' to an openbsd server here, then scp'd it back a couple time in different ways. It worked only when using the quotes AND escaping, like so: scp [EMAIL PROTECTED]:a\ b . That's because of the shell. The shell on the client sees the quotes and doesn't escape the space. The space is escaped when it gets to the SFTP daemon at the other end. Without both, the shell at the client does the escaping and sends a single argument to the scp client. This sends that argument to the server, which presumably sees the space and assumes you're asking for two files called A and B. You need the \ character to arrive unaltered at the scp server. scp [EMAIL PROTECTED]:a\\\ b . should work as well. A
Re: scp problem with remote filename escaping
On 4/11/07, Karel Kulhavy [EMAIL PROTECTED] wrote: For the same filename, sometimes you have to specify a different filename to scp, depending on whether the file is on remote system or local one. I have created a remote file whose filename a b is 3 chars long - ASCII codes 97, 32, 98 Standard shell behavior. Check out the same command but with rcp. You'll see that you get the same results. Read UnixHaters if you want to see some interesting results of this.
Re: scp problem with remote filename escaping
On Wed, Apr 11, 2007 at 04:11:48PM -0400, Dan Farrell wrote: Wow. Seriously, I think the real 'bug' is your file naming conventions. Bug is when behaviour is different from documentation. What is the behaviour and what is the documentation in the case of my file naming conventions? Who would anyone specifically want to name a file with a space in it... I think the question of motivation is irrelevant here. The scp does something else than is written in the manpage. and if breaks on scp, where else will that screwy naming convention break as well? I'm sure you'll give some really good reason why the files have to be named that way... They don't have to be. It's actually completely irrelevant in this case whether they have to be named with spaces or don't. scp does something else than is written in the manpage regardless of the user's motivation. CL danno -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Karel Kulhavy Sent: Wednesday, April 11, 2007 3:47 PM To: OpenBSD Subject: scp problem with remote filename escaping Sounds like a bug to me - the escaping for the remote shell is not being done correctly?
Re: scp problem with remote filename escaping
On Wed, Apr 11, 2007 at 10:55:29PM +0200, Simon Effenberg wrote: On Wed, Apr 11, 2007 at 04:17:38PM -0400, Nick ! wrote: On 4/11/07, Dan Farrell [EMAIL PROTECTED] wrote: From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Karel Kulhavy Sent: Wednesday, April 11, 2007 3:47 PM To: OpenBSD Subject: scp problem with remote filename escaping Sounds like a bug to me - the escaping for the remote shell is not being done correctly? Wow. Seriously, I think the real 'bug' is your file naming conventions. Who would anyone specifically want to name a file with a space in it... and if breaks on scp, where else will that screwy naming convention break as well? I'm sure you'll give some really good reason why the files have to be named that way... I agree, spaces in filenames should be avoided. But spaces in filenames are legal, so programs need to support that; this seems like a case scp was never tested against because no one uses files with those names. -Nick scp needs 3 * \ for one space.. scp needs one (1) \ for one space in case of remote file and zero (0) \ in case of local one. The extra \'s are for bash but bash is irrelevant in this case. It's just one possible method of calling the process. Another method is writing a small C program and using exec. CL scp [EMAIL PROTECTED]:/what/you/want/a\\\ b.txtlocalfile.txt \s -- GnuPG: 5755FB64 Per aspera ad astra.
Re: scp problem with remote filename escaping
On Wed, Apr 11, 2007 at 10:02:50PM +0100, Stuart Henderson wrote: On 2007/04/11 13:41, Bryan Irvine wrote: snip I agree, spaces in filenames should be avoided. But spaces in filenames are legal, so programs need to support that; this seems like a case scp was never tested against because no one uses files with those names. I scp'd a file called 'a b' to an openbsd server here, then scp'd it back a couple time in different ways. It worked only when using the quotes AND escaping, like so: scp [EMAIL PROTECTED]:a\ b . you have to escape to *both* your local shell, and the remote shell You must not escape to your local shell in case the scp process is called directly by e. g. exec() function in C. If you have to escape to the remote shell, then it should be mentioned in man scp. escape and shell don't occur in man scp and remote doesn't occur in such a context there. If I wrote it, I would do it in a way that scp performed the escaping for the remote shell automatically. Having to supply a different filename depending on where the file is goes against the local-remote transparency that scp is attempting at. CL
Re: scp problem with remote filename escaping
On Thu, Apr 12, 2007 at 01:13:16AM +0300, [EMAIL PROTECTED] wrote:
On Wed, Apr 11, 2007 at 04:41:41PM -0500, Matthew R. Dempsky wrote:
On Wed, Apr 11, 2007 at 10:02:50PM +0100, Stuart Henderson wrote:
On 2007/04/11 13:41, Bryan Irvine wrote:
scp [EMAIL PROTECTED]:a\ b .
you have to escape to *both* your local shell, and the remote shell
This has always seemed silly to me. Does anyone intentionally use
$ scp host:a b .
instead of
$ scp host:{a,b} .
Does anyone intentionally use
scp '[EMAIL PROTECTED]:`rm -fr /`' /dev/null ?
Security hole in scp. Send someone an e-mail attachment named
`rm -fr /`. If he uses multiple machines, it's possible he'll want to
copy it and writes a similar command to the above according to the
scp man page. Then he wonders why his files disappeared.
CL
Re: scp problem with remote filename escaping
On Wed, Apr 11, 2007 at 05:01:40PM -0500, Matthew R. Dempsky wrote: On Wed, Apr 11, 2007 at 04:33:32PM -0400, Nick ! wrote: Karel, single quotes cause backslashes to be backslashes, instead of escape chars (*except* if it's a backslash in front of a single quote, so that you can escape single quotes to include them). No, backslashes have no special meaning inside single quotes. $ echo '\' \ indeed. i'll add the verbose G : maybe nick ! just forgot to mention that that is only relevant in the situation where you're chuggin' along using single quotes and then want to escape one of them, so therefore have to close your currently-open single quote and then use the \' jobby out in the open, and then probably start another singlequoted string :) $ echo '$RANDOM'\'x'$RANDOM would yield: $RANDOM'x'? where ? is some number; as one doesn't need to escape the single quote inside double quotes because everyone knows double quotes have TWICE THE POWER! back on topic; i had the impression that the filenames had to be relevant to whatever host they were supposed to be relevant to, and that there was a natural layer of escaping that would have to take place. file a b doesn't need to have its space escaped locally, it's just when it has to be referenced by name on the remote side; so maybe looking at it like that it is more sensible? -- jared
Re: scp problem with remote filename escaping
On Thu, Apr 12, 2007 at 04:05:36AM +0200, Karel Kulhavy wrote: scp needs one (1) \ for one space in case of remote file and zero (0) \ in case of local one. The extra \'s are for bash but bash is irrelevant in this case. It's just one possible method of calling the process. Another method is writing a small C program and using exec. people aren't being specific enough when they're talking about this but it seems most people are trying to say the same thing. the moral of the story is that the shells in question have to not split on the space but treat it as a character of the filename. so you either quote it or escape it both shells need that. these all work: $ scp a b remotehost: $ scp a\ b remotehost: $ scp remotehost:a\ b . $ scp remotehost:\a\ b\ . you can only say that 'zero (0) \ in case of local [file]' (per space) *if* you do use quotes locally. it is more specific to say you use one backslash or quotepair per space per shell. example 3 has the local shell eating the quotes and passing the backslash along so the remote shell escapes the space. example 4 has local shell eating the backslash escaping the space and the remote shell gets the quoted string. just remember that and you don't have any problems any more. -- jared
