Re: String escaping utility for Python (was: Rawest raw string literals)
On 23 April 2017 at 05:03, MRAB wrote: > On 2017-04-22 23:30, Mikhail V wrote: >> >> On 20 April 2017 at 23:54, MRAB wrote: >> > On 2017-04-20 22:03, Mikhail V wrote: >> >> >> >> On 20 April 2017 at 22:43, Random832 wrote: >> >>> [snip] >> >>> >> >>> The best solution I can think of is to have a text editor designed to >> >>> parse a string literal, spawn a nested editor with the unescaped >> >>> contents of that string literal, and then re-escape it back to place >> >>> in >> >>> the code. If we had that, then we wouldn't even need raw strings. >> >> >> >> >> >> Yes exactly, it would be cool to have such a satellite app >> >> which can escape and unescape strings according to rules. >> >> And which can also convert unicode literals to their ascii >> >> analogues and back on the fly, this would very useful >> >> for programming. >> >> Probably it is a good idea to even include such thing >> >> in Python package. So it would be a small standalone app >> >> running parallel with text editor making it to copy paste strings. >> >> >> > I'm sure it's possible in, say, Emacs. >> > >> > The editor that I use (EditPad Pro) can call external tools, so I could: >> > >> > 1. Select the string literal (easy when it is syntax-aware, so I can >> > select >> > all of the literal with 2 keypresses). >> > >> > 2. Call the external tool (1 keypress), to open, say, a simple tkinter >> > app. >> > >> > 3. Edit the unescaped text (unescape with ast.literal_eval, re-escape >> > with >> > 'ascii'). >> > >> > 4. Close the external tool, and the selection is replaced. >> >> I have done a quick google search and could not find >> such utility for Python. >> >> I am very interested in having such utility. >> And I think it would be fair that such utility >> should be made by the Python team so that >> all syntax nuances will be correctly implemented. >> >> The purpose is simple: reduce manual work to escape special >> characters in string literals (and escape non-ASCII characters). >> >> Simple usage scenario: >> - I have a long command-line string in some text editor. >> - Copy this string and paste into the utility edit box >> - In the second edit box same string with escaped characters >>appears (i.e tab becomes \t, etc) >> - Further, if I edit the text in the second edit box, >>an unescaped string appears in the first box. >> >> Possible toggle options, e.g. : >> - 'asciify' non-ascii characters >> >> It could be not only useful to eliminate boilerplate typing, >> but also a great way to learn string rules for Python learners. >> > Here's a very simple tkinter GUI app. It only goes one way (plain to escaped > (asciified)), but it shows what's possible with very little code. > > > #! python3.6 > # -*- coding: utf-8 -*- > import tkinter as tk > > class App(tk.Tk): > def __init__(self): > tk.Tk.__init__(self) > self.title('Escaper') > > tk.Label(self, text='Plain string').pack() > > self.plain_box = tk.Text(self) > self.plain_box.pack() > self.plain_box.focus() > > tk.Label(self, text='Escaped string').pack() > > self.escaped_box = tk.Text(self) > self.escaped_box.pack() > > self.after(100, self.on_tick) > > def on_tick(self): > plain_string = self.plain_box.get('1.0', 'end')[ : -1] > > escaped_string = ascii(plain_string) > > self.escaped_box.delete('1.0', 'end') > self.escaped_box.insert('1.0', escaped_string) > > self.after(100, self.on_tick) > > App().mainloop() > Thank you for sharing! Works fine on Win 7. So that is what I mean, even such simple app is uncomparably better that any workarounds and already covers most cases. Unfortunately I am not proficient in tkinter so I hope that will gain some support from other folks and then could be pinned somewhere and be accessible for everyone. Mikhail -- https://mail.python.org/mailman/listinfo/python-list
Re: String escaping utility for Python (was: Rawest raw string literals)
On 2017-04-22 23:30, Mikhail V wrote: On 20 April 2017 at 23:54, MRAB wrote: > On 2017-04-20 22:03, Mikhail V wrote: >> >> On 20 April 2017 at 22:43, Random832 wrote: >>> [snip] >>> >>> The best solution I can think of is to have a text editor designed to >>> parse a string literal, spawn a nested editor with the unescaped >>> contents of that string literal, and then re-escape it back to place in >>> the code. If we had that, then we wouldn't even need raw strings. >> >> >> Yes exactly, it would be cool to have such a satellite app >> which can escape and unescape strings according to rules. >> And which can also convert unicode literals to their ascii >> analogues and back on the fly, this would very useful >> for programming. >> Probably it is a good idea to even include such thing >> in Python package. So it would be a small standalone app >> running parallel with text editor making it to copy paste strings. >> > I'm sure it's possible in, say, Emacs. > > The editor that I use (EditPad Pro) can call external tools, so I could: > > 1. Select the string literal (easy when it is syntax-aware, so I can select > all of the literal with 2 keypresses). > > 2. Call the external tool (1 keypress), to open, say, a simple tkinter app. > > 3. Edit the unescaped text (unescape with ast.literal_eval, re-escape with > 'ascii'). > > 4. Close the external tool, and the selection is replaced. I have done a quick google search and could not find such utility for Python. I am very interested in having such utility. And I think it would be fair that such utility should be made by the Python team so that all syntax nuances will be correctly implemented. The purpose is simple: reduce manual work to escape special characters in string literals (and escape non-ASCII characters). Simple usage scenario: - I have a long command-line string in some text editor. - Copy this string and paste into the utility edit box - In the second edit box same string with escaped characters appears (i.e tab becomes \t, etc) - Further, if I edit the text in the second edit box, an unescaped string appears in the first box. Possible toggle options, e.g. : - 'asciify' non-ascii characters It could be not only useful to eliminate boilerplate typing, but also a great way to learn string rules for Python learners. Here's a very simple tkinter GUI app. It only goes one way (plain to escaped (asciified)), but it shows what's possible with very little code. #! python3.6 # -*- coding: utf-8 -*- import tkinter as tk class App(tk.Tk): def __init__(self): tk.Tk.__init__(self) self.title('Escaper') tk.Label(self, text='Plain string').pack() self.plain_box = tk.Text(self) self.plain_box.pack() self.plain_box.focus() tk.Label(self, text='Escaped string').pack() self.escaped_box = tk.Text(self) self.escaped_box.pack() self.after(100, self.on_tick) def on_tick(self): plain_string = self.plain_box.get('1.0', 'end')[ : -1] escaped_string = ascii(plain_string) self.escaped_box.delete('1.0', 'end') self.escaped_box.insert('1.0', escaped_string) self.after(100, self.on_tick) App().mainloop() -- https://mail.python.org/mailman/listinfo/python-list
Re: String escaping utility for Python (was: Rawest raw string literals)
On Sun, Apr 23, 2017 at 12:30 PM, eryk sun wrote: > The X terminals that I've used make it easy to copy text to the > clipboard. For Windows, it's a pain prior to Windows 10 since the > legacy console only does rectangular selection. The Windows 10 console > does line-wrapped selection. This is true, but if you're using Python for simple text manipulation as I suggest here, rectangular selection is usually fine. At worst, you have to rejoin the lines after you paste. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: String escaping utility for Python (was: Rawest raw string literals)
On Sun, Apr 23, 2017 at 2:06 AM, Mikhail V wrote: > > But are you joking, right? Even if it worked, how can this be convinient, > e.g. in console one cannot even select and copy paste easily. The X terminals that I've used make it easy to copy text to the clipboard. For Windows, it's a pain prior to Windows 10 since the legacy console only does rectangular selection. The Windows 10 console does line-wrapped selection. -- https://mail.python.org/mailman/listinfo/python-list
Re: String escaping utility for Python (was: Rawest raw string literals)
On Sun, Apr 23, 2017 at 12:06 PM, Mikhail V wrote: > Don't know, all I see is "SyntaxError: invalid syntax" if I paste > there some text. > Try to paste e.g. this: > "ffmpeg -i "D:\VIDEO\exp\intro.mp4" -vf "crop=1280:720:0:40, > scale=640:360" -pix_fmt yuv420p "D:\ART\0MASTER_UMST\yt_pico.mp4"" Oh. I should have said that my example was for Python 3. If you're using Python 2, use raw_input() instead. Or just switch to Python 3. > But are you joking, right? Even if it worked, how can this be convinient, > e.g. in console one cannot even select and copy paste easily. Get a better console. Even in Windows, the default console is fully capable of copying and pasting text, but you can do better than the default. On every Linux desktop I've used, the console is beautifully easy to use (GNOME, LXDE, Mate, Xfce, and probably a few others as well). > Probably one can make a python script which takes clipboard contents > then place the conversion result back to clipboard. > Like: > - copy some text to clipboard > - run the script, which replace the clipboard contents with result > - paste text > > I haven't tried that, but even this would be very inconvenient and > limited in comparison > to a GUI utility. You can do that too if you want to. I don't know how you'd do that with tkinter, but it ought to be possible, and then you wouldn't need any third party libraries. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: String escaping utility for Python (was: Rawest raw string literals)
On 23 April 2017 at 02:33, Chris Angelico wrote: > On Sun, Apr 23, 2017 at 10:19 AM, Mikhail V wrote: >> On 23 April 2017 at 00:48, Chris Angelico wrote: >>> On Sun, Apr 23, 2017 at 8:30 AM, Mikhail V wrote: The purpose is simple: reduce manual work to escape special characters in string literals (and escape non-ASCII characters). Simple usage scenario: - I have a long command-line string in some text editor. - Copy this string and paste into the utility edit box - In the second edit box same string with escaped characters appears (i.e tab becomes \t, etc) - Further, if I edit the text in the second edit box, an unescaped string appears in the first box. >>> >>> Easy. >>> >> input() >>> This string has "quotes" of 'various' «styles», and \backslashes\ too. >>> 'This string has "quotes" of \'various\' «styles», and \\backslashes\\ too.' >>> >>> The repr of a string does pretty much everything you want. If you want >>> a nice GUI, you can easily put one together that uses repr() to escape >>> and ast.literal_eval() to unescape. >> >> I am sorry, could you elaborate what have you shown here? >> So in Python console I can become escaped string, but what >> commands do you use? I never use Python console actually :/ > > You type "input()" at the Python console, then type the string you > want. It will be echoed back in representation form, with everything > correctly escaped. > >> And yes the idea is to have a nice GUI. And the idea is exactly opposite >> to "everyone let's roll an own tool". Obviously I can spend day >> or two and create such a tool, e.g. with PyQt. >> But since the task is very common and quite unambiguos I think it is >> a good reason for a standard official tool. > > Or you could spend two seconds firing up the Python REPL, which has > all the tools you need right there :) > Don't know, all I see is "SyntaxError: invalid syntax" if I paste there some text. Try to paste e.g. this: "ffmpeg -i "D:\VIDEO\exp\intro.mp4" -vf "crop=1280:720:0:40, scale=640:360" -pix_fmt yuv420p "D:\ART\0MASTER_UMST\yt_pico.mp4"" But are you joking, right? Even if it worked, how can this be convinient, e.g. in console one cannot even select and copy paste easily. Probably one can make a python script which takes clipboard contents then place the conversion result back to clipboard. Like: - copy some text to clipboard - run the script, which replace the clipboard contents with result - paste text I haven't tried that, but even this would be very inconvenient and limited in comparison to a GUI utility. Mikhail -- https://mail.python.org/mailman/listinfo/python-list
Re: String escaping utility for Python (was: Rawest raw string literals)
On Sun, Apr 23, 2017 at 10:19 AM, Mikhail V wrote: > On 23 April 2017 at 00:48, Chris Angelico wrote: >> On Sun, Apr 23, 2017 at 8:30 AM, Mikhail V wrote: >>> The purpose is simple: reduce manual work to escape special >>> characters in string literals (and escape non-ASCII characters). >>> >>> Simple usage scenario: >>> - I have a long command-line string in some text editor. >>> - Copy this string and paste into the utility edit box >>> - In the second edit box same string with escaped characters >>> appears (i.e tab becomes \t, etc) >>> - Further, if I edit the text in the second edit box, >>> an unescaped string appears in the first box. >> >> Easy. >> > input() >> This string has "quotes" of 'various' «styles», and \backslashes\ too. >> 'This string has "quotes" of \'various\' «styles», and \\backslashes\\ too.' >> >> The repr of a string does pretty much everything you want. If you want >> a nice GUI, you can easily put one together that uses repr() to escape >> and ast.literal_eval() to unescape. > > I am sorry, could you elaborate what have you shown here? > So in Python console I can become escaped string, but what > commands do you use? I never use Python console actually :/ You type "input()" at the Python console, then type the string you want. It will be echoed back in representation form, with everything correctly escaped. > And yes the idea is to have a nice GUI. And the idea is exactly opposite > to "everyone let's roll an own tool". Obviously I can spend day > or two and create such a tool, e.g. with PyQt. > But since the task is very common and quite unambiguos I think it is > a good reason for a standard official tool. Or you could spend two seconds firing up the Python REPL, which has all the tools you need right there :) >>> PS: >>> Also I remember now about the python-ideas thread >>> on entering unicode characters with decimals instead of >>> hex values. It was met somewhat negatively but then it turned out >>> that in recent Python version it can be done with f-strings. >>> E.g. a string : >>> >>> s="абв" >>> one can write as: >>> s = f"{1072:c}{1073:c}{1074:c}" >>> instead of traditional hex: >>> "\u0430\u0431\u0432" >>> >>> It was told however this is not normal usage. >>> Still I find it very helpful, so if this is correct syntax, I'd >>> personally find such a conversion option also very useful. >> >> Most of the world finds the hex form MUCH more logical, since Unicode >> is built around 16s and 256s and such. Please don't proliferate more >> messes - currently, the only place I can think of where decimal is >> supported is HTML character entities, and hex is equally supported >> there. >> >> Of course, the best way to represent most non-ASCII characters is as >> themselves - s="абв" from your example. The main exception is >> combining characters and related incomplete forms, such as this table >> of diacritical marks more-or-less lifted from an app of mine: >> >> { >> "\\`":"\u0300","\\'":"\u0301","\\^":"\u0302","\\~":"\u0303", >> "\\-":"\u0304","\\@":"\u0306","\\.":"\u0307","\\\"":"\u0308", >> "\\o":"\u030A","\\=":"\u030B","\\v":"\u030C","\\<":"\u0326", >> "\\,":"\u0327","\\k":"\u0328", >> } >> >> All of them are in the 03xx range. Much easier than pointing out that >> they're in the range 768 to 879. Please stick to hex. > > I don't insist on decimals, I want to use decimals for my own pleasure > in own projects, may I? > And don't worry in my whole life I will not produce so many software > that will significantly increase the 'messes'. > (Anyway I've got used already to decimals somehow, ord(char), etc., > so for me it's too late for the ugly hex) Will your projects ever be shared with anyone else? If so, please use the standard. In your own projects, you're welcome to shoot yourself in the foot, but I'm not going to help you. I'm going to encourage hex for Unicode. It's not too late for you to adjust your mind to the standard. And I strongly recommend it. There are good reasons for hex, and the sooner you change, the easier it'll be. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: String escaping utility for Python (was: Rawest raw string literals)
On 23 April 2017 at 00:48, Chris Angelico wrote: > On Sun, Apr 23, 2017 at 8:30 AM, Mikhail V wrote: >> The purpose is simple: reduce manual work to escape special >> characters in string literals (and escape non-ASCII characters). >> >> Simple usage scenario: >> - I have a long command-line string in some text editor. >> - Copy this string and paste into the utility edit box >> - In the second edit box same string with escaped characters >> appears (i.e tab becomes \t, etc) >> - Further, if I edit the text in the second edit box, >> an unescaped string appears in the first box. > > Easy. > input() > This string has "quotes" of 'various' «styles», and \backslashes\ too. > 'This string has "quotes" of \'various\' «styles», and \\backslashes\\ too.' > > The repr of a string does pretty much everything you want. If you want > a nice GUI, you can easily put one together that uses repr() to escape > and ast.literal_eval() to unescape. I am sorry, could you elaborate what have you shown here? So in Python console I can become escaped string, but what commands do you use? I never use Python console actually :/ And yes the idea is to have a nice GUI. And the idea is exactly opposite to "everyone let's roll an own tool". Obviously I can spend day or two and create such a tool, e.g. with PyQt. But since the task is very common and quite unambiguos I think it is a good reason for a standard official tool. > >> PS: >> Also I remember now about the python-ideas thread >> on entering unicode characters with decimals instead of >> hex values. It was met somewhat negatively but then it turned out >> that in recent Python version it can be done with f-strings. >> E.g. a string : >> >> s="абв" >> one can write as: >> s = f"{1072:c}{1073:c}{1074:c}" >> instead of traditional hex: >> "\u0430\u0431\u0432" >> >> It was told however this is not normal usage. >> Still I find it very helpful, so if this is correct syntax, I'd >> personally find such a conversion option also very useful. > > Most of the world finds the hex form MUCH more logical, since Unicode > is built around 16s and 256s and such. Please don't proliferate more > messes - currently, the only place I can think of where decimal is > supported is HTML character entities, and hex is equally supported > there. > > Of course, the best way to represent most non-ASCII characters is as > themselves - s="абв" from your example. The main exception is > combining characters and related incomplete forms, such as this table > of diacritical marks more-or-less lifted from an app of mine: > > { > "\\`":"\u0300","\\'":"\u0301","\\^":"\u0302","\\~":"\u0303", > "\\-":"\u0304","\\@":"\u0306","\\.":"\u0307","\\\"":"\u0308", > "\\o":"\u030A","\\=":"\u030B","\\v":"\u030C","\\<":"\u0326", > "\\,":"\u0327","\\k":"\u0328", > } > > All of them are in the 03xx range. Much easier than pointing out that > they're in the range 768 to 879. Please stick to hex. I don't insist on decimals, I want to use decimals for my own pleasure in own projects, may I? And don't worry in my whole life I will not produce so many software that will significantly increase the 'messes'. (Anyway I've got used already to decimals somehow, ord(char), etc., so for me it's too late for the ugly hex) Mikhail -- https://mail.python.org/mailman/listinfo/python-list
Re: String escaping utility for Python (was: Rawest raw string literals)
On Sun, Apr 23, 2017 at 8:30 AM, Mikhail V wrote: > The purpose is simple: reduce manual work to escape special > characters in string literals (and escape non-ASCII characters). > > Simple usage scenario: > - I have a long command-line string in some text editor. > - Copy this string and paste into the utility edit box > - In the second edit box same string with escaped characters > appears (i.e tab becomes \t, etc) > - Further, if I edit the text in the second edit box, > an unescaped string appears in the first box. Easy. >>> input() This string has "quotes" of 'various' «styles», and \backslashes\ too. 'This string has "quotes" of \'various\' «styles», and \\backslashes\\ too.' The repr of a string does pretty much everything you want. If you want a nice GUI, you can easily put one together that uses repr() to escape and ast.literal_eval() to unescape. > PS: > Also I remember now about the python-ideas thread > on entering unicode characters with decimals instead of > hex values. It was met somewhat negatively but then it turned out > that in recent Python version it can be done with f-strings. > E.g. a string : > > s="абв" > one can write as: > s = f"{1072:c}{1073:c}{1074:c}" > instead of traditional hex: > "\u0430\u0431\u0432" > > It was told however this is not normal usage. > Still I find it very helpful, so if this is correct syntax, I'd > personally find such a conversion option also very useful. Most of the world finds the hex form MUCH more logical, since Unicode is built around 16s and 256s and such. Please don't proliferate more messes - currently, the only place I can think of where decimal is supported is HTML character entities, and hex is equally supported there. Of course, the best way to represent most non-ASCII characters is as themselves - s="абв" from your example. The main exception is combining characters and related incomplete forms, such as this table of diacritical marks more-or-less lifted from an app of mine: { "\\`":"\u0300","\\'":"\u0301","\\^":"\u0302","\\~":"\u0303", "\\-":"\u0304","\\@":"\u0306","\\.":"\u0307","\\\"":"\u0308", "\\o":"\u030A","\\=":"\u030B","\\v":"\u030C","\\<":"\u0326", "\\,":"\u0327","\\k":"\u0328", } All of them are in the 03xx range. Much easier than pointing out that they're in the range 768 to 879. Please stick to hex. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
String escaping utility for Python (was: Rawest raw string literals)
On 20 April 2017 at 23:54, MRAB wrote: > On 2017-04-20 22:03, Mikhail V wrote: >> >> On 20 April 2017 at 22:43, Random832 wrote: >>> [snip] >>> >>> The best solution I can think of is to have a text editor designed to >>> parse a string literal, spawn a nested editor with the unescaped >>> contents of that string literal, and then re-escape it back to place in >>> the code. If we had that, then we wouldn't even need raw strings. >> >> >> Yes exactly, it would be cool to have such a satellite app >> which can escape and unescape strings according to rules. >> And which can also convert unicode literals to their ascii >> analogues and back on the fly, this would very useful >> for programming. >> Probably it is a good idea to even include such thing >> in Python package. So it would be a small standalone app >> running parallel with text editor making it to copy paste strings. >> > I'm sure it's possible in, say, Emacs. > > The editor that I use (EditPad Pro) can call external tools, so I could: > > 1. Select the string literal (easy when it is syntax-aware, so I can select > all of the literal with 2 keypresses). > > 2. Call the external tool (1 keypress), to open, say, a simple tkinter app. > > 3. Edit the unescaped text (unescape with ast.literal_eval, re-escape with > 'ascii'). > > 4. Close the external tool, and the selection is replaced. I have done a quick google search and could not find such utility for Python. I am very interested in having such utility. And I think it would be fair that such utility should be made by the Python team so that all syntax nuances will be correctly implemented. The purpose is simple: reduce manual work to escape special characters in string literals (and escape non-ASCII characters). Simple usage scenario: - I have a long command-line string in some text editor. - Copy this string and paste into the utility edit box - In the second edit box same string with escaped characters appears (i.e tab becomes \t, etc) - Further, if I edit the text in the second edit box, an unescaped string appears in the first box. Possible toggle options, e.g. : - 'asciify' non-ascii characters It could be not only useful to eliminate boilerplate typing, but also a great way to learn string rules for Python learners. PS: Also I remember now about the python-ideas thread on entering unicode characters with decimals instead of hex values. It was met somewhat negatively but then it turned out that in recent Python version it can be done with f-strings. E.g. a string : s="абв" one can write as: s = f"{1072:c}{1073:c}{1074:c}" instead of traditional hex: "\u0430\u0431\u0432" It was told however this is not normal usage. Still I find it very helpful, so if this is correct syntax, I'd personally find such a conversion option also very useful. Mikhail -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 2017-04-22 23:13, Mikhail V wrote: > k = r"abc > def" > > gives an error: > > k = r"abc > ^ > SyntaxError: EOL while scanning string literal > > So one could define a rule, that a raw string *must* > be terminated by the sequence quote + newline. > In theory. Though one can do k = r"""abc def ghi jkl mno pqrs \tuv wxyz """ which keeps the "\" while allowing multi-line strings. -tkc -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On Sun, Apr 23, 2017 at 7:13 AM, Mikhail V wrote: > So one could define a rule, that a raw string *must* > be terminated by the sequence quote + newline. > In theory. Then how would you pass one as a function parameter? func(r"" ) Ugh. Ugly. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 20 April 2017 at 18:40, Grant Edwards wrote: > On 2017-04-20, Mikhail V wrote: >> On 20 April 2017 at 17:59, Grant Edwards wrote: >>> On 2017-04-20, Mikhail V wrote: Quite often I need raw string literals for concatenating console commands. I want to input them exactly as they are in python sources. There is r"" string, but it is obviously not enough because e.g. this: s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg" " >>> >>>s = r'ffmpeg -i "\\server-01\D\SER_Bigl.mpg" ' >>> >>> Does that do what you want? >> >> Yes but it still needs to watch out if there is no ' inside or vice >> versa with " characters if use r"". I would like a universal >> solution. > > IOW, you want something that just reads your mind. > > How can there exist a "universal solution" even in theory? > > There has to be some sort of "end of literal" terminator character > sequence. That means there has to be some sort of escaping mechanism > when that "end of literal" sequence appears in the literal itself. > In *theory* one can define what characters can be part of the raw string. In Python e.g. the newline character cannot be part of the r"" string (unless a line ends with \): k = r"abc def" gives an error: k = r"abc ^ SyntaxError: EOL while scanning string literal So one could define a rule, that a raw string *must* be terminated by the sequence quote + newline. In theory. Mikhail -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
Tim Chase writes: > On 2017-04-21 08:23, Jussi Piitulainen wrote: >> Tim Chase writes: >>> Bash: >>> cat <>> "single and double" with \ and / >>> EOT >>> >>> PS: yes, bash's does interpolate strings, so you still need to do >>> escaping within, but the arbitrary-user-specified-delimiter idea >>> still holds. >> >> If you put any quote characters in the initial EOT, it doesn't. >> Quote removal on the EOT determines the actual EOT at the end. >> >> cat <<"EOT" >> Not expanding any $amount here >> EOT > > Huh, I just tested it and you're 100% right on that. But I just > re-read over that section of my `man bash` page and don't see anything > that stands out as detailing this. Is there something I missed in the > docs? It's in this snippet, yanked straight from the man page: The format of here-documents is: <<[-]word here-document delimiter No parameter expansion, command substitution, arithmetic expansion, or pathname expansion is performed on word. If any characters in word are quoted, the delimiter is the result of quote removal on word, and the lines in the here-document are not expanded. If word -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 2017-04-21 08:23, Jussi Piitulainen wrote: > Tim Chase writes: >> Bash: >> cat <> "single and double" with \ and / >> EOT >> >> PS: yes, bash's does interpolate strings, so you still need to do >> escaping within, but the arbitrary-user-specified-delimiter idea >> still holds. > > If you put any quote characters in the initial EOT, it doesn't. > Quote removal on the EOT determines the actual EOT at the end. > > cat <<"EOT" > Not expanding any $amount here > EOT Huh, I just tested it and you're 100% right on that. But I just re-read over that section of my `man bash` page and don't see anything that stands out as detailing this. Is there something I missed in the docs? Thanks for this little tip (filing away for future use) -tkc -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
Tim Chase writes: > A number of tools use a custom quote-string: > > Bash: > > cat < "single and double" with \ and / > EOT [snip] > PS: yes, bash's does interpolate strings, so you still need to do > escaping within, but the arbitrary-user-specified-delimiter idea still > holds. If you put any quote characters in the initial EOT, it doesn't. Quote removal on the EOT determines the actual EOT at the end. cat <<"EOT" Not expanding any $amount here EOT -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 2017-04-21 01:11, Tim Chase wrote: On 2017-04-20 16:40, Grant Edwards wrote: How can there exist a "universal solution" even in theory? There has to be some sort of "end of literal" terminator character sequence. That means there has to be some sort of escaping mechanism when that "end of literal" sequence appears in the literal itself. A number of tools use a custom quote-string: Bash: cat < Perl 5 also does this. [snip] -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 2017-04-20 16:40, Grant Edwards wrote: > How can there exist a "universal solution" even in theory? > > There has to be some sort of "end of literal" terminator character > sequence. That means there has to be some sort of escaping > mechanism when that "end of literal" sequence appears in the > literal itself. A number of tools use a custom quote-string: Bash: cat
Re: Rawest raw string literals
On 2017-04-20 22:03, Mikhail V wrote: On 20 April 2017 at 22:43, Random832 wrote: On Thu, Apr 20, 2017, at 16:01, Grant Edwards wrote: On 2017-04-20, MRAB wrote: > There _is_ a "universal solution"; it's called a Hollerith constant. :-) Wow, I haven't seen one of those in a _long_ time -- probably about 45 years. I think the first FORTAN implementation I used was WATFIV, which had just introduced the character type. But, books/classes on FORTRAN all still covered Hollerith constants. The IMAP protocol uses a similar kind of construct (the length is enclosed in braces) Even ignoring the maintenance difficulty, I don't think it's possible to syntax highlight something like that on most common editors. The best solution I can think of is to have a text editor designed to parse a string literal, spawn a nested editor with the unescaped contents of that string literal, and then re-escape it back to place in the code. If we had that, then we wouldn't even need raw strings. Yes exactly, it would be cool to have such a satellite app which can escape and unescape strings according to rules. And which can also convert unicode literals to their ascii analogues and back on the fly, this would very useful for programming. Probably it is a good idea to even include such thing in Python package. So it would be a small standalone app running parallel with text editor making it to copy paste strings. I'm sure it's possible in, say, Emacs. The editor that I use (EditPad Pro) can call external tools, so I could: 1. Select the string literal (easy when it is syntax-aware, so I can select all of the literal with 2 keypresses). 2. Call the external tool (1 keypress), to open, say, a simple tkinter app. 3. Edit the unescaped text (unescape with ast.literal_eval, re-escape with 'ascii'). 4. Close the external tool, and the selection is replaced. -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On Fri, Apr 21, 2017 at 7:37 AM, Stefan Ram wrote: > Mikhail V writes: >>But the less probable it is, the more complex or ugly would the tag >>become. >>E.g. curly braces {} seems to be much less frequent characters >>for filenames and command line arguments. > > When one uses brackets to delimit string literals, > on even can allow /nested/ brackets to be part of > the string without escapes, e.g., the literal > > [abc[def]ghi] > > can stand for the string > > abc[def]ghi > > . Only unpaired brackets then have to be escaped > (and occurences of the escape symbol at the end > of such a string). If you're going to use brackets like that, I'd prefer a multi-character delimiter: [[[abcdef]]] But you have to be careful about the nesting, because that makes it difficult to create a string with an imbalanced set of delimiters. You'll end up with messy and complicated edge cases. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 20 April 2017 at 22:43, Random832 wrote: > On Thu, Apr 20, 2017, at 16:01, Grant Edwards wrote: >> On 2017-04-20, MRAB wrote: >> > There _is_ a "universal solution"; it's called a Hollerith constant. :-) >> >> Wow, I haven't seen one of those in a _long_ time -- probably about 45 >> years. I think the first FORTAN implementation I used was WATFIV, >> which had just introduced the character type. But, books/classes on >> FORTRAN all still covered Hollerith constants. > > The IMAP protocol uses a similar kind of construct (the length is > enclosed in braces) > > Even ignoring the maintenance difficulty, I don't think it's possible to > syntax highlight something like that on most common editors. > > The best solution I can think of is to have a text editor designed to > parse a string literal, spawn a nested editor with the unescaped > contents of that string literal, and then re-escape it back to place in > the code. If we had that, then we wouldn't even need raw strings. Yes exactly, it would be cool to have such a satellite app which can escape and unescape strings according to rules. And which can also convert unicode literals to their ascii analogues and back on the fly, this would very useful for programming. Probably it is a good idea to even include such thing in Python package. So it would be a small standalone app running parallel with text editor making it to copy paste strings. Mikhail -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 20 April 2017 at 19:27, Chris Angelico wrote: > On Fri, Apr 21, 2017 at 2:26 AM, wrote: >> I find this:- >> >> s = r"ffmpeg -i '\\server-01\D\SER_Bigl.mpg' " >> >> vastly superior. > > It's semantically different though. I don't know whether single quotes > are valid in that context, on Windows. > For ffmpeg on Windows at least, both single and double quotes work equally. So yes, if I keep the rule to use single quotes only inside commands then I'd be fine with r"". I simply don't like single quotes, and tend to use doble quotes everywhere for better readability. Well it is just somewhat unfortunate, because literals use the quotes as a open/close tag, which are very common character in strings. So the idea was to choose something that is not so frequent and much less probable to appear in a string. But the less probable it is, the more complex or ugly would the tag become. E.g. curly braces {} seems to be much less frequent characters for filenames and command line arguments. Mikhail -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On Thu, Apr 20, 2017, at 16:01, Grant Edwards wrote: > On 2017-04-20, MRAB wrote: > > There _is_ a "universal solution"; it's called a Hollerith constant. :-) > > Wow, I haven't seen one of those in a _long_ time -- probably about 45 > years. I think the first FORTAN implementation I used was WATFIV, > which had just introduced the character type. But, books/classes on > FORTRAN all still covered Hollerith constants. The IMAP protocol uses a similar kind of construct (the length is enclosed in braces) Even ignoring the maintenance difficulty, I don't think it's possible to syntax highlight something like that on most common editors. The best solution I can think of is to have a text editor designed to parse a string literal, spawn a nested editor with the unescaped contents of that string literal, and then re-escape it back to place in the code. If we had that, then we wouldn't even need raw strings. -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 2017-04-20, MRAB wrote: > On 2017-04-20 17:40, Grant Edwards wrote: > >> There has to be some sort of "end of literal" terminator character >> sequence. That means there has to be some sort of escaping mechanism >> when that "end of literal" sequence appears in the literal itself. >> > There _is_ a "universal solution"; it's called a Hollerith constant. :-) Wow, I haven't seen one of those in a _long_ time -- probably about 45 years. I think the first FORTAN implementation I used was WATFIV, which had just introduced the character type. But, books/classes on FORTRAN all still covered Hollerith constants. -- Grant Edwards grant.b.edwardsYow! Look! A ladder! at Maybe it leads to heaven, gmail.comor a sandwich! -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 2017-04-20 17:40, Grant Edwards wrote: On 2017-04-20, Mikhail V wrote: On 20 April 2017 at 17:59, Grant Edwards wrote: On 2017-04-20, Mikhail V wrote: Quite often I need raw string literals for concatenating console commands. I want to input them exactly as they are in python sources. There is r"" string, but it is obviously not enough because e.g. this: s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg" " s = r'ffmpeg -i "\\server-01\D\SER_Bigl.mpg" ' Does that do what you want? Yes but it still needs to watch out if there is no ' inside or vice versa with " characters if use r"". I would like a universal solution. IOW, you want something that just reads your mind. How can there exist a "universal solution" even in theory? There has to be some sort of "end of literal" terminator character sequence. That means there has to be some sort of escaping mechanism when that "end of literal" sequence appears in the literal itself. There _is_ a "universal solution"; it's called a Hollerith constant. :-) -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
No escaping is not something possible, in your suggested syntax ") is ambigous. E.g. raw("abcd")") is ambigous. Any sequence delimited string involves escaping, the only thing that wouldnt would be size-defined strings but they are impractical. Le 20/04/2017 à 18:03, Mikhail V a écrit : On 20 April 2017 at 17:55, Chris Angelico wrote: On Fri, Apr 21, 2017 at 1:44 AM, Mikhail V wrote: What I think: why there is no some built-in function, for example like: s = raw("ffmpeg -i "\\server-01\D\SER_Bigl__"") which would just need *one* quote sign in the beginning and on the end. Would it be useful, what do you think? I think looks better than triple quote. In the past there were quite a lot additions to string manipulation, probably there is already something like this. What would be the argument passed to this function? ChrisA Yeah that is right, I cant imagine how this would work. But I think you get the idea- I'd want something dedicated to input raw strings with cute syntax and *no* escaping at all. Mikhail -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On Thu, Apr 20, 2017 at 5:27 PM, Chris Angelico wrote: > On Fri, Apr 21, 2017 at 2:26 AM, wrote: >> I find this:- >> >> s = r"ffmpeg -i '\\server-01\D\SER_Bigl.mpg' " >> >> vastly superior. > > It's semantically different though. I don't know whether single quotes > are valid in that context, on Windows. On Windows, whether forward slash can be used as a path separator and whether single quotes escape spaces and special characters depends on the programs involved. If you use the lpApplicationName parameter of CreateProcess (i.e. `executable` for Popen), the system doesn't have to look at the command line. Otherwise if the path of the executable in the command line contains spaces, it needs to be quoted using double quotes; single quotes have no special significance. If it has to search for the executable, it calls SearchPath with the API's executable search path (i.e. the directory of the calling application, the current directory [in legacy mode], system directories, plus the PATH environment variable) and .EXE as the optional file extension to append. Beyond that, the application can implement any command-line parsing rules. In practice most programs use the CRT's argv parameter from the `[w]main` entry point. Microsoft's CRT delimits arguments using spaces; quotes arguments using double quotes; and escapes double quotes using backslash [1]. If you're using the cmd.exe shell to run this, spaces and special characters are escaped with double quotes, or with '^' outside of quotes - except there's no way to completely escape "%" in a cmd.exe command line (in a batch script, percent can be escaped by doubling it as %%). If quoted, the path to the executable can use forward slash as a path delimiter. cmd.exe implements its own custom search for the executable, which includes the current directory (in legacy mode) and PATH. It also tries appending all of the extensions in PATHEXT instead of just .EXE. Since it uses the lpApplicationName parameter when it calls CreateProcess, the system doesn't have to re-parse the command line to locate the executable. [1]: http://msdn.microsoft.com/en-us/library/17w5ykft.aspx -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On Fri, Apr 21, 2017 at 2:26 AM, wrote: > I find this:- > > s = r"ffmpeg -i '\\server-01\D\SER_Bigl.mpg' " > > vastly superior. It's semantically different though. I don't know whether single quotes are valid in that context, on Windows. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On Thursday, April 20, 2017 at 4:59:48 PM UTC+1, Grant Edwards wrote: > On 2017-04-20, Mikhail V wrote: > > Quite often I need raw string literals for concatenating console commands. > > I want to input them exactly as they are in python sources. > > > > There is r"" string, but it is obviously not enough because e.g. this: > > s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg" " > >s = r'ffmpeg -i "\\server-01\D\SER_Bigl.mpg" ' > > Does that do what you want? > > -- > Grant Edwards grant.b.edwardsYow! And then we could sit > at on the hoods of cars at > gmail.comstop lights! I find this:- s = r"ffmpeg -i '\\server-01\D\SER_Bigl.mpg' " vastly superior. Kindest regards. Mark Lawrence. -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 2017-04-20, Mikhail V wrote: > On 20 April 2017 at 17:44, Mikhail V wrote: >> Quite often I need raw string literals for concatenating console commands. >> I want to input them exactly as they are in python sources. >> >> There is r"" string, but it is obviously not enough because e.g. this: >> s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg" " >> >> is not valid. >> >> The closest I've found is triple quote literal: >> s = r"""ffmpeg -i "\\server-01\D\SER_Bigl__" """ >> >> This is what I use now, still there is problem: last quote inside the string >> needs escaping or a space character before closing triple quote, >> otherwise there is again an error. >> >> What I think: why there is no some built-in function, for example like: >> s = raw("ffmpeg -i "\\server-01\D\SER_Bigl__"") >> >> which would just need *one* quote sign in the beginning and on the end. >> Would it be useful, what do you think? I think looks better than triple >> quote. >> In the past there were quite a lot additions to string manipulation, >> probably there is already something like this. > > A function probably would not be possible to implement directly, > but probably such a syntax for example: > s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg""r > > i.e. an opening and closing "r" particle. What do you do when the sting '"r' appears in the literal? All you've done is change the literal terminating character sequence. You still have to have a mechanism to escape it. -- Grant Edwards grant.b.edwardsYow! World War Three can at be averted by adherence gmail.comto a strictly enforced dress code! -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 2017-04-20, Mikhail V wrote: > On 20 April 2017 at 17:59, Grant Edwards wrote: >> On 2017-04-20, Mikhail V wrote: >>> Quite often I need raw string literals for concatenating console commands. >>> I want to input them exactly as they are in python sources. >>> >>> There is r"" string, but it is obviously not enough because e.g. this: >>> s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg" " >> >>s = r'ffmpeg -i "\\server-01\D\SER_Bigl.mpg" ' >> >> Does that do what you want? > > Yes but it still needs to watch out if there is no ' inside or vice > versa with " characters if use r"". I would like a universal > solution. IOW, you want something that just reads your mind. How can there exist a "universal solution" even in theory? There has to be some sort of "end of literal" terminator character sequence. That means there has to be some sort of escaping mechanism when that "end of literal" sequence appears in the literal itself. -- Grant Edwards grant.b.edwardsYow! I brought my BOWLING at BALL -- and some DRUGS!! gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On Fri, Apr 21, 2017 at 2:16 AM, Mikhail V wrote: > On 20 April 2017 at 17:59, Grant Edwards wrote: >> On 2017-04-20, Mikhail V wrote: >>> Quite often I need raw string literals for concatenating console commands. >>> I want to input them exactly as they are in python sources. >>> >>> There is r"" string, but it is obviously not enough because e.g. this: >>> s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg" " >> >>s = r'ffmpeg -i "\\server-01\D\SER_Bigl.mpg" ' >> >> Does that do what you want? >> > > Yes but it still needs to watch out if there is no ' inside or vice versa > with " characters if use r"". I would like a universal solution. There's no universal solution, by definition. The nearest you'll get is something like the PostgreSQL "dollar string": $SomeTag$any text$SomeTag$ You can choose any tag you like for SomeTag and it'll match only the corresponding tag, so this can be nested. But it's long and ugly, especially when you don't need to actually nest it. Python is not a shell scripting language, and trying to treat it as one is usually going to get you into tangles like this. Much better, IMO, is to treat Python as a scripting language, where higher-level constructs like lists are the norm. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 20 April 2017 at 17:59, Grant Edwards wrote: > On 2017-04-20, Mikhail V wrote: >> Quite often I need raw string literals for concatenating console commands. >> I want to input them exactly as they are in python sources. >> >> There is r"" string, but it is obviously not enough because e.g. this: >> s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg" " > >s = r'ffmpeg -i "\\server-01\D\SER_Bigl.mpg" ' > > Does that do what you want? > Yes but it still needs to watch out if there is no ' inside or vice versa with " characters if use r"". I would like a universal solution. -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On Fri, Apr 21, 2017 at 2:03 AM, Mikhail V wrote: > On 20 April 2017 at 17:55, Chris Angelico wrote: >> On Fri, Apr 21, 2017 at 1:44 AM, Mikhail V wrote: >>> What I think: why there is no some built-in function, for example like: >>> s = raw("ffmpeg -i "\\server-01\D\SER_Bigl__"") >>> >>> which would just need *one* quote sign in the beginning and on the end. >>> Would it be useful, what do you think? I think looks better than triple >>> quote. >>> In the past there were quite a lot additions to string manipulation, >>> probably there is already something like this. >> >> What would be the argument passed to this function? >> >> ChrisA > > > Yeah that is right, I cant imagine how this would work. > But I think you get the idea- I'd want something dedicated > to input raw strings with cute syntax and *no* escaping > at all. Yep. It can't be a function; it has to be a piece of language syntax. So then the question is: what language syntax is appropriate? And honestly, I hardly ever need this kind of thing - partly because I wouldn't call ffmpeg this way. I'd do it like this: cmd = ["ffmpeg", "-i", r"\\server-01\D\SER_Bigl.mpg"] with the separate arguments as, well, separate arguments. Far less need for double escaping that way, and you can use different escaping rules for different arguments if you need to. (Actually, I usually use forward slashes, so even raw string lits are unnecessary.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 20 April 2017 at 17:55, Chris Angelico wrote: > On Fri, Apr 21, 2017 at 1:44 AM, Mikhail V wrote: >> What I think: why there is no some built-in function, for example like: >> s = raw("ffmpeg -i "\\server-01\D\SER_Bigl__"") >> >> which would just need *one* quote sign in the beginning and on the end. >> Would it be useful, what do you think? I think looks better than triple >> quote. >> In the past there were quite a lot additions to string manipulation, >> probably there is already something like this. > > What would be the argument passed to this function? > > ChrisA Yeah that is right, I cant imagine how this would work. But I think you get the idea- I'd want something dedicated to input raw strings with cute syntax and *no* escaping at all. Mikhail -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 2017-04-20, Mikhail V wrote: > Quite often I need raw string literals for concatenating console commands. > I want to input them exactly as they are in python sources. > > There is r"" string, but it is obviously not enough because e.g. this: > s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg" " s = r'ffmpeg -i "\\server-01\D\SER_Bigl.mpg" ' Does that do what you want? -- Grant Edwards grant.b.edwardsYow! And then we could sit at on the hoods of cars at gmail.comstop lights! -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On 20 April 2017 at 17:44, Mikhail V wrote: > Quite often I need raw string literals for concatenating console commands. > I want to input them exactly as they are in python sources. > > There is r"" string, but it is obviously not enough because e.g. this: > s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg" " > > is not valid. > > The closest I've found is triple quote literal: > s = r"""ffmpeg -i "\\server-01\D\SER_Bigl__" """ > > This is what I use now, still there is problem: last quote inside the string > needs escaping or a space character before closing triple quote, > otherwise there is again an error. > > What I think: why there is no some built-in function, for example like: > s = raw("ffmpeg -i "\\server-01\D\SER_Bigl__"") > > which would just need *one* quote sign in the beginning and on the end. > Would it be useful, what do you think? I think looks better than triple quote. > In the past there were quite a lot additions to string manipulation, > probably there is already something like this. A function probably would not be possible to implement directly, but probably such a syntax for example: s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg""r i.e. an opening and closing "r" particle. -- https://mail.python.org/mailman/listinfo/python-list
Re: Rawest raw string literals
On Fri, Apr 21, 2017 at 1:44 AM, Mikhail V wrote: > What I think: why there is no some built-in function, for example like: > s = raw("ffmpeg -i "\\server-01\D\SER_Bigl__"") > > which would just need *one* quote sign in the beginning and on the end. > Would it be useful, what do you think? I think looks better than triple quote. > In the past there were quite a lot additions to string manipulation, > probably there is already something like this. What would be the argument passed to this function? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Rawest raw string literals
Quite often I need raw string literals for concatenating console commands. I want to input them exactly as they are in python sources. There is r"" string, but it is obviously not enough because e.g. this: s = r"ffmpeg -i "\\server-01\D\SER_Bigl.mpg" " is not valid. The closest I've found is triple quote literal: s = r"""ffmpeg -i "\\server-01\D\SER_Bigl__" """ This is what I use now, still there is problem: last quote inside the string needs escaping or a space character before closing triple quote, otherwise there is again an error. What I think: why there is no some built-in function, for example like: s = raw("ffmpeg -i "\\server-01\D\SER_Bigl__"") which would just need *one* quote sign in the beginning and on the end. Would it be useful, what do you think? I think looks better than triple quote. In the past there were quite a lot additions to string manipulation, probably there is already something like this. Mikhail -- https://mail.python.org/mailman/listinfo/python-list