[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Change by Serhiy Storchaka : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Serhiy Storchaka added the comment: New changeset 4d840e428ab1a2712f219c5e4008658cbe15892e by Miss Islington (bot) in branch '3.8': [3.8] bpo-42318: Fix support of non-BMP characters in Tkinter on macOS (GH-23281). (GH-23784) (GH-23787)

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Serhiy Storchaka added the comment: New changeset 28bf6ab61f77c69b732a211c398ac882bf3f65f4 by Serhiy Storchaka in branch '3.9': [3.9] bpo-42318: Fix support of non-BMP characters in Tkinter on macOS (GH-23281). (GH-23784)

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Change by miss-islington : -- nosy: +miss-islington nosy_count: 5.0 -> 6.0 pull_requests: +22644 pull_request: https://github.com/python/cpython/pull/23787 ___ Python tracker

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Change by Serhiy Storchaka : -- pull_requests: +22641 pull_request: https://github.com/python/cpython/pull/23784 ___ Python tracker ___

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Serhiy Storchaka added the comment: Oh, the fix is not backported yet. Automatically backporting does not work because of renames in the supporting test library. -- ___ Python tracker

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Serhiy Storchaka added the comment: New changeset a26215db11cfcf7b5f55cab9e91396761a0e0bcf by Serhiy Storchaka in branch 'master': bpo-42318: Fix support of non-BMP characters in Tkinter on macOS (GH-23281) https://github.com/python/cpython/commit/a26215db11cfcf7b5f55cab9e91396761a0e0bcf

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Change by Serhiy Storchaka : -- keywords: +patch pull_requests: +22175 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/23281 ___ Python tracker

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Terry J. Reedy added the comment: Yash, this is specifically a macOS issue. Printing astral chars in tkinter/IDLE on Windows and Linux has 'worked' (details not important) for over a year. -- nosy: +terry.reedy ___ Python tracker

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Ronald Oussoren added the comment: > And yet one question. What do you see if you print '\udcf0\udc9f\udc98\udc80' > in IDLE? This prints a smiley emoji, likewise for printing chr(128516) -- ___ Python tracker

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Serhiy Storchaka added the comment: And yet one question. What do you see if you print '\udcf0\udc9f\udc98\udc80' in IDLE? -- ___ Python tracker ___

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Serhiy Storchaka added the comment: You can ignore msg380917. It was written before I read msg380908. Now I have the needed information. Thank you. -- ___ Python tracker ___

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Ronald Oussoren added the comment: When I assign root.clipboard_get() to "v" I get: >>> print(ascii(v)) '\udced\udca0\udcbd\udced\udcb8\udc84' >>> print(v) ?? This is with Tk 8.6.10. -- ___ Python tracker

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Serhiy Storchaka added the comment: Well, try copy  (or other text with color emoji) to clipboard and run the following code: import tkinter root = tkinter.Tk() print(ascii(root.clipboard_get())) -- ___ Python tracker

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Ronald Oussoren added the comment: BTW. The unicodeFromTclStringAndSize() basically undoes the special treatment of \0 in Modified UTF-8 [1]. That page says that all known implementation of MUTF-8 treat surrogate pairs the same as CESU-8 [2], which is UTF-8 with characters outside of the

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Ronald Oussoren added the comment: @Pixmew: I get this error with Tk 8.6.10 on macOS 11. With Tk 8.6.8 on macOS 10.15 (from the python.org installer) I get the behaviour described in msg380906. 8.6.10 is the version of Tk we'd like to switch to for the "universal2", it is the latest

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Ronald Oussoren added the comment: With 8.6.10: >>> print(ascii("")) raises the SyntaxError mentioned earlier >>> print(ascii(input())) works and prints: '\udced\udca0\udcbd\udced\udcb8\udc84' In an editor window I don't get spurious text, but syntax colouring is a bit off: The text after

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Yash Shete added the comment: Well for me in Python 3.9.0 print("") prints  and print(ascii("")) prints '\U0001f600' It does not Raises error "utf-8' codec can't encode characters in position 7-12: surrogates not allowed." as you are suggesting -- nosy: +Pixmew

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Ronald Oussoren added the comment: With 8.6.8 both "hang", in that the Shell window no longer accepts input. I've checked that ``print(input())`` works when I don't use an emoji. Interestingly enough, pasting ``print(ascii(""))`` into an edit window does work, I can continue editing, but

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Serhiy Storchaka added the comment: Well, it is likely the same syntax error. Then what will print print(ascii(input())) when you paste  and press Enter? -- ___ Python tracker

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

Serhiy Storchaka added the comment: Just to be sure, what is the result of pasting and executing the following code on Tk 8.6.8 and 8.6.10? print(ascii("")) -- nosy: +serhiy.storchaka versions: +Python 3.10, Python 3.8, Python 3.9 ___ Python

[issue42318] [tkinter] surrogate pairs in Tcl/Tk string when pasting an emoji in a text widget

New submission from Ronald Oussoren : As mentioned in msg380552: I get an SyntaxError with message "utf-8' codec can't encode characters in position 7-12: surrogates not allowed." when I paste a smiley emoji in an IDLE interactive shell and try to execute that line, for example using: >>>