Re: more on unescaping escapes
On Tue, 24 Feb 2009 00:46:34 -, bvdp b...@mellowood.ca wrote: Just because I never really thought too much about it :) I'm doing my work on a linux box and my user is on windows ... and he's used to using '\' ... but, you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Erm, no. \x20 is exactly the same as in a string literal. -- Rhodri James *-* Wildebeeste Herder to the Masses -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
On Feb 23, 7:18 pm, bvdp b...@mellowood.ca wrote: Gabriel Genellina wrote: En Mon, 23 Feb 2009 23:31:20 -0200, bvdp b...@mellowood.ca escribió: Gabriel Genellina wrote: En Mon, 23 Feb 2009 22:46:34 -0200, bvdp b...@mellowood.ca escribió: Chris Rebert wrote: On Mon, Feb 23, 2009 at 4:26 PM, bvdp b...@mellowood.ca wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? [...] you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Why is that? \x20 is exactly the same as . It's not like %20 in URLs, that becomes a space only after decoding. I need to use the \x20 because of my parser. I'm reading unquoted lines from a file. The file creater needs to use the form foo\x20bar without the quotes in the file so my parser can read it as a single token. Later, the string/token needs to be decoded with the \x20 converted to a space. So, in my file foo bar (no quotes) is read as 2 tokens; foo\x20bar is one. So, it's not really a problem of what happens when you assign a string in the form foo bar, rather how to convert the \x20 in a string to a space. I think the \\ just complicates the entire issue. Just thinking, if you was reading the string from a file, why were you worried about \\ and \ in the first place? (Ok, you moved to use / so this is moot now). Just cruft introduced while I was trying to figure it all out. Having to figure the \\ and \x20 at same time with file and keyboard input just confused the entire issue :) Having the user set a line like c:\\Program\x20File ... works just fine. I'll suggest he use c:/program\x20files to make it bit simple for HIM, not my parser. Unfortunately, due to some bad design decisions on my part about 5 years ago I'm afraid I'm stuck with the \x20. Thanks. You're confusing the python source with the actual contents of the string. We already do one pass at decoding, which is why \x20 is quite literally no different from a space: '\x20' ' ' However, the interactive interpreter uses repr(x), so various characters that are considered formatting, such as a tab, get reescaped when printing: '\t' '\t' len('\t') 1 It really is a tab that gets stored there, not the escape for one. Finally, if you give python an unknown escape it passes it leaves it as an escape. Then, when the interactive interpreter uses repr(x), it is the backslash itself that gets reescaped: '\P' '\\P' len('\P') 2 list('\P') ['\\', 'P'] What does this all mean? If you want to test your parser with python literals you need to escape them twice, like so: 'c:Program\\x20Filestest' 'c:Program\\x20Filestest' list('c:Program\\x20Filestest') ['c', ':', '\\', '\\', 'P', 'r', 'o', 'g', 'r', 'a', 'm', '\\', 'x', '2', '0', 'F', 'i', 'l', 'e', 's', '\\', '\\', 't', 'e', 's', 't'] 'c:Program\\x20Filestest'.decode('string-escape') 'c:\\Program Files\\test' list('c:Program\\x20Filestest'.decode('string-escape')) ['c', ':', '\\', 'P', 'r', 'o', 'g', 'r', 'a', 'm', ' ', 'F', 'i', 'l', 'e', 's', '\\', 't', 'e', 's', 't'] However, there's an easier way: use raw strings, which prevent python from unescaping anything: r'c:\\Program\x20Files\\test' 'c:Program\\x20Filestest' list(r'c:\\Program\x20Files\\test') ['c', ':', '\\', '\\', 'P', 'r', 'o', 'g', 'r', 'a', 'm', '\\', 'x', '2', '0', 'F', 'i', 'l', 'e', 's', '\\', '\\', 't', 'e', 's', 't'] -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Adam Olsen wrote: On Feb 23, 7:18 pm, bvdp b...@mellowood.ca wrote: Gabriel Genellina wrote: En Mon, 23 Feb 2009 23:31:20 -0200, bvdp b...@mellowood.ca escribió: Gabriel Genellina wrote: En Mon, 23 Feb 2009 22:46:34 -0200, bvdp b...@mellowood.ca escribió: Chris Rebert wrote: On Mon, Feb 23, 2009 at 4:26 PM, bvdp b...@mellowood.ca wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? [...] you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Why is that? \x20 is exactly the same as . It's not like %20 in URLs, that becomes a space only after decoding. I need to use the \x20 because of my parser. I'm reading unquoted lines from a file. The file creater needs to use the form foo\x20bar without the quotes in the file so my parser can read it as a single token. Later, the string/token needs to be decoded with the \x20 converted to a space. So, in my file foo bar (no quotes) is read as 2 tokens; foo\x20bar is one. So, it's not really a problem of what happens when you assign a string in the form foo bar, rather how to convert the \x20 in a string to a space. I think the \\ just complicates the entire issue. Just thinking, if you was reading the string from a file, why were you worried about \\ and \ in the first place? (Ok, you moved to use / so this is moot now). Just cruft introduced while I was trying to figure it all out. Having to figure the \\ and \x20 at same time with file and keyboard input just confused the entire issue :) Having the user set a line like c:\\Program\x20File ... works just fine. I'll suggest he use c:/program\x20files to make it bit simple for HIM, not my parser. Unfortunately, due to some bad design decisions on my part about 5 years ago I'm afraid I'm stuck with the \x20. Thanks. You're confusing the python source with the actual contents of the string. We already do one pass at decoding, which is why \x20 is quite literally no different from a space: '\x20' ' ' However, the interactive interpreter uses repr(x), so various characters that are considered formatting, such as a tab, get reescaped when printing: '\t' '\t' len('\t') 1 It really is a tab that gets stored there, not the escape for one. Finally, if you give python an unknown escape it passes it leaves it as an escape. Then, when the interactive interpreter uses repr(x), it is the backslash itself that gets reescaped: '\P' '\\P' len('\P') 2 list('\P') ['\\', 'P'] What does this all mean? If you want to test your parser with python literals you need to escape them twice, like so: 'c:Program\\x20Filestest' 'c:Program\\x20Filestest' list('c:Program\\x20Filestest') ['c', ':', '\\', '\\', 'P', 'r', 'o', 'g', 'r', 'a', 'm', '\\', 'x', '2', '0', 'F', 'i', 'l', 'e', 's', '\\', '\\', 't', 'e', 's', 't'] 'c:Program\\x20Filestest'.decode('string-escape') 'c:\\Program Files\\test' list('c:Program\\x20Filestest'.decode('string-escape')) ['c', ':', '\\', 'P', 'r', 'o', 'g', 'r', 'a', 'm', ' ', 'F', 'i', 'l', 'e', 's', '\\', 't', 'e', 's', 't'] However, there's an easier way: use raw strings, which prevent python from unescaping anything: r'c:\\Program\x20Files\\test' 'c:Program\\x20Filestest' list(r'c:\\Program\x20Files\\test') ['c', ':', '\\', '\\', 'P', 'r', 'o', 'g', 'r', 'a', 'm', '\\', 'x', '2', '0', 'F', 'i', 'l', 'e', 's', '\\', '\\', 't', 'e', 's', 't'] Thank you. That is very clear. Appreciate your time. -- http://mail.python.org/mailman/listinfo/python-list
more on unescaping escapes
So, we think something is working and send of a bug fix to our client :) I'm not sure I understand this at all and wonder if there is bug? a=c:\\Program\x20Files\\test a 'c:\\Program Files\\test' so far, so good. a.decode(string-escape) 'c:\\Program Files\test' Umm, not so good? The \\ before the P is okay, but the \\t is change to \t and print a.decode(string-escape) c:\Program Filesest Now, decode() converted the \\t to a \t and print expanded the \t to a tab. I would have thought that the \\t would have the same result as the \\P ??? Obviously my brain is missing something (hopefully obvious). -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
bvdp wrote: So, we think something is working and send of a bug fix to our client :) I'm not sure I understand this at all and wonder if there is bug? a=c:\\Program\x20Files\\test a 'c:\\Program Files\\test' so far, so good. a.decode(string-escape) 'c:\\Program Files\test' Umm, not so good? The \\ before the P is okay, but the \\t is change to \t Decoding changes \\x20 to \x20, which is the same as , a space. Decoding changes \\t to \t, which is a tab. Decoding _doesn't_ change \\P to \P because that's not a valid escape sequence. and print a.decode(string-escape) c:\Program Filesest Now, decode() converted the \\t to a \t and print expanded the \t to a tab. \t is already a tab. I would have thought that the \\t would have the same result as the \\P ??? Obviously my brain is missing something (hopefully obvious). Before storing the string (writing it to the file), encode it and then replace with \\x20: C:\Program Files\test becomes: C:\Program Files\test and then: C:\\Program\x20Files\\test After fetching the string (reading it from the file), decode it: C:\\Program\x20Files\\test becomes: C:\Program Files\test -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
I'm getting hopelessly lost in a series of \\\ s :) Let's see if this makes sense: a='c:\\Program Files\\test' a.decode('string-escape') 'c:\\Program Files\test' In this case there are still 2 '\'s before the P; but only 1 before the 't'. Now, when it comes time to open the file windows accepts double '\'s in the filename. So, life is fine. But, the essential part here is that we're lucky we can use '\\' or '\' in a path. What if this wasn't true? The following shows a bit of difference: a='c:\Program Files\test' a 'c:\\Program Files\test' In this case the interpreter has changed the '\P' to '\\P'. And if one lists the string the '\t' really is a tab. No decode() at all in any of this. I guess the general rule would be to double up '\'s in filenames and (in my program's case) to use the \x20 for spaces. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
do you know that a string with the letter r in front doesn't escape slashes? it's intended for regular expressions, but would simplify things for you here too. just do a=r'c:\\Program Files\test' andrew bvdp wrote: I'm getting hopelessly lost in a series of \\\ s :) Let's see if this makes sense: a='c:\\Program Files\\test' a.decode('string-escape') 'c:\\Program Files\test' In this case there are still 2 '\'s before the P; but only 1 before the 't'. Now, when it comes time to open the file windows accepts double '\'s in the filename. So, life is fine. But, the essential part here is that we're lucky we can use '\\' or '\' in a path. What if this wasn't true? The following shows a bit of difference: a='c:\Program Files\test' a 'c:\\Program Files\test' In this case the interpreter has changed the '\P' to '\\P'. And if one lists the string the '\t' really is a tab. No decode() at all in any of this. I guess the general rule would be to double up '\'s in filenames and (in my program's case) to use the \x20 for spaces. Thanks. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
On Mon, 2009-02-23 at 17:00 -0700, bvdp wrote: Let's see if this makes sense: a='c:\\Program Files\\test' a.decode('string-escape') 'c:\\Program Files\test' Hint: try running print a and see what's written - I think that the interpreter adds extra \ characters to escape things and make things more easy to read. i.e. a = c:\\test\\t a 'c:\\test\\t' print a c:\test\t so when it displays strings in the interpreter it includes escape characters, when it is printed though the output is straight to stdout and isn't escaped. Hope that helps, Tim Wintle -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
andrew cooke wrote: do you know that a string with the letter r in front doesn't escape slashes? it's intended for regular expressions, but would simplify things for you here too. just do a=r'c:\\Program Files\test' Yes, I knew that. Unfortunately in my program loop I really don't have the opportuity to use a raw string. But, good reminder. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Tim Wintle wrote: On Mon, 2009-02-23 at 17:00 -0700, bvdp wrote: Let's see if this makes sense: a='c:\\Program Files\\test' a.decode('string-escape') 'c:\\Program Files\test' Hint: try running print a and see what's written - I think that the interpreter adds extra \ characters to escape things and make things more easy to read. i.e. a = c:\\test\\t a 'c:\\test\\t' print a c:\test\t so when it displays strings in the interpreter it includes escape characters, when it is printed though the output is straight to stdout and isn't escaped. Hope that helps, Tim Wintle Not sure if it's more clear or not :) a=c:\\Program\x20Files\\test a 'c:\\Program Files\\test' print a c:\Program Files\test Which is all fine. And I didn't need to use decode(). So, in this case I'm assuming that the interpreter is converting the escapes on assignment. And, in this case the string has single \s in it. So far this is making sense. So, when I do a decode() the \t is converted to a tab, etc. I think my problem may be in viewing an assignment like that above as opposed to reading a line from a file with '\'s and '\x20's in it. In this case the assignment doesn't do the conversion (and that is a good thing!). Now, when I go to save the variable with the escapes I use decode() and it works. Just a matter of consistantly using a method I suppose. Still, confusing between my program reading data from a file and doing testing of strings at the interpreter. Best to do one or the other, not both. -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
On Mon, 23 Feb 2009 22:05:42 -, bvdp b...@mellowood.ca wrote: So, we think something is working and send of a bug fix to our client :) I'm not sure I understand this at all and wonder if there is bug? a=c:\\Program\x20Files\\test a 'c:\\Program Files\\test' so far, so good. a.decode(string-escape) 'c:\\Program Files\test' Umm, not so good? The \\ before the P is okay, but the \\t is change to \t Well yes, that's what you asked it to do. The string-escape decoder reads the string and replaces escape sequences with the corresponding characters. Bear in mind that it's the string as it really is that is being operated on, not the representation of it that you displayed above. In other words: b = a.decode(string-escape) is equivalent to: b = C:\Program Files\test \P isn't a valid escape sequence, so it doesn't get replaced. \t represents a tab, so it does. -- Rhodri James *-* Wildebeeste Herder to the Masses -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
On Tue, 24 Feb 2009 00:26:29 -, bvdp b...@mellowood.ca wrote: So, in this case I'm assuming that the interpreter is converting the escapes on assignment. The compiler converts the escapes on creating its internal representation of the string, before assignment ever gets involved. -- Rhodri James *-* Wildebeeste Herder to the Masses -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
On Mon, Feb 23, 2009 at 4:26 PM, bvdp b...@mellowood.ca wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? They work in Windows from Python in nearly all cases and you can easily interconvert using os.pathsep if you want the path to be pretty when you show it to (or get it from) the user or whatever. Cheers, Chris -- Follow the path of the Iguana... http://rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Chris Rebert wrote: On Mon, Feb 23, 2009 at 4:26 PM, bvdp b...@mellowood.ca wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? They work in Windows from Python in nearly all cases and you can easily interconvert using os.pathsep if you want the path to be pretty when you show it to (or get it from) the user or whatever. Cheers, Chris Just because I never really thought too much about it :) I'm doing my work on a linux box and my user is on windows ... and he's used to using '\' ... but, you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Thanks for the suggestion! -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Bear in mind that it's the string as it really is that is being operated on, not the representation of it that you displayed Yes, that is the confusion ... what is displayed and what's actually in the string. I think I understand it all now :) Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
bvdp wrote: Not sure if it's more clear or not :) a=c:\\Program\x20Files\\test a 'c:\\Program Files\\test' print a c:\Program Files\test Which is all fine. And I didn't need to use decode(). So, in this case I'm assuming that the interpreter is converting the escapes on assignment. And, in this case the string has single \s in it. Strictly speaking, the compiler is converting the escapes when it uses the literal to create a string value. Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
En Mon, 23 Feb 2009 22:46:34 -0200, bvdp b...@mellowood.ca escribió: Chris Rebert wrote: On Mon, Feb 23, 2009 at 4:26 PM, bvdp b...@mellowood.ca wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? They work in Windows from Python in nearly all cases and you can easily interconvert using os.pathsep if you want the path to be pretty when you show it to (or get it from) the user or whatever. Just because I never really thought too much about it :) I'm doing my work on a linux box and my user is on windows ... and he's used to using '\' ... but, you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Why is that? \x20 is exactly the same as . It's not like %20 in URLs, that becomes a space only after decoding. py '\x20' == ' ' True py '\x20' is ' ' True (ok, the last line might show False, but being True means that both are the very same object) -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Gabriel Genellina wrote: En Mon, 23 Feb 2009 22:46:34 -0200, bvdp b...@mellowood.ca escribió: Chris Rebert wrote: On Mon, Feb 23, 2009 at 4:26 PM, bvdp b...@mellowood.ca wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? They work in Windows from Python in nearly all cases and you can easily interconvert using os.pathsep if you want the path to be pretty when you show it to (or get it from) the user or whatever. Just because I never really thought too much about it :) I'm doing my work on a linux box and my user is on windows ... and he's used to using '\' ... but, you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Why is that? \x20 is exactly the same as . It's not like %20 in URLs, that becomes a space only after decoding. py '\x20' == ' ' True py '\x20' is ' ' True (ok, the last line might show False, but being True means that both are the very same object) I need to use the \x20 because of my parser. I'm reading unquoted lines from a file. The file creater needs to use the form foo\x20bar without the quotes in the file so my parser can read it as a single token. Later, the string/token needs to be decoded with the \x20 converted to a space. So, in my file foo bar (no quotes) is read as 2 tokens; foo\x20bar is one. So, it's not really a problem of what happens when you assign a string in the form foo bar, rather how to convert the \x20 in a string to a space. I think the \\ just complicates the entire issue. -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
En Mon, 23 Feb 2009 23:31:20 -0200, bvdp b...@mellowood.ca escribió: Gabriel Genellina wrote: En Mon, 23 Feb 2009 22:46:34 -0200, bvdp b...@mellowood.ca escribió: Chris Rebert wrote: On Mon, Feb 23, 2009 at 4:26 PM, bvdp b...@mellowood.ca wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? [...] you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Why is that? \x20 is exactly the same as . It's not like %20 in URLs, that becomes a space only after decoding. I need to use the \x20 because of my parser. I'm reading unquoted lines from a file. The file creater needs to use the form foo\x20bar without the quotes in the file so my parser can read it as a single token. Later, the string/token needs to be decoded with the \x20 converted to a space. So, in my file foo bar (no quotes) is read as 2 tokens; foo\x20bar is one. So, it's not really a problem of what happens when you assign a string in the form foo bar, rather how to convert the \x20 in a string to a space. I think the \\ just complicates the entire issue. Just thinking, if you was reading the string from a file, why were you worried about \\ and \ in the first place? (Ok, you moved to use / so this is moot now). -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Gabriel Genellina wrote: En Mon, 23 Feb 2009 23:31:20 -0200, bvdp b...@mellowood.ca escribió: Gabriel Genellina wrote: En Mon, 23 Feb 2009 22:46:34 -0200, bvdp b...@mellowood.ca escribió: Chris Rebert wrote: On Mon, Feb 23, 2009 at 4:26 PM, bvdp b...@mellowood.ca wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? [...] you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Why is that? \x20 is exactly the same as . It's not like %20 in URLs, that becomes a space only after decoding. I need to use the \x20 because of my parser. I'm reading unquoted lines from a file. The file creater needs to use the form foo\x20bar without the quotes in the file so my parser can read it as a single token. Later, the string/token needs to be decoded with the \x20 converted to a space. So, in my file foo bar (no quotes) is read as 2 tokens; foo\x20bar is one. So, it's not really a problem of what happens when you assign a string in the form foo bar, rather how to convert the \x20 in a string to a space. I think the \\ just complicates the entire issue. Just thinking, if you was reading the string from a file, why were you worried about \\ and \ in the first place? (Ok, you moved to use / so this is moot now). Just cruft introduced while I was trying to figure it all out. Having to figure the \\ and \x20 at same time with file and keyboard input just confused the entire issue :) Having the user set a line like c:\\Program\x20File ... works just fine. I'll suggest he use c:/program\x20files to make it bit simple for HIM, not my parser. Unfortunately, due to some bad design decisions on my part about 5 years ago I'm afraid I'm stuck with the \x20. Thanks. -- http://mail.python.org/mailman/listinfo/python-list