[issue40863] bytes.decode changes/destroys line endings on windows
Matthias Naegler added the comment: I forgot something important. Using open with 'ab' works. >>> ...above code... with open("./test_binary.txt", "ab") as myfile: ... myfile.write(s.encode('utf-8')) -- ___ Python tracker <https://bugs.python.org/issue40863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40863] bytes.decode changes/destroys line endings on windows
Matthias Naegler added the comment: Thanks Steven for your fast response. > The best way to see what your string actually contains is the print the repr: You are right. bytes.decode is correct. Im not a python expert, so thanks for the note about "repr". With repr(...) everything looks fine. Nevertheless, I get an additional \r in my output. Not sure if it is a problem of python, windows or just me. I get the following output with the python interpretor: Python 3.8.3rc1 (tags/v3.8.3rc1:802eb67, Apr 29 2020, 21:39:14) [MSC v.1924 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.stdout.encoding 'utf-8' >>> b = b'A\r\nA' >>> s = b.decode('utf-8') >>> print(b) b'A\r\nA' >>> print(repr(s)) 'A\r\nA' >>> print(s) A A >>> sys.stdout.write(s) A A4 >>> with open("./test.txt", "a") as myfile: ... myfile.write(s) This all looks right. But the file doesn't (see attached screenshot). I also get an additional \r in the output file if i run the script throught "python test.py > piped.txt" -- Added file: https://bugs.python.org/file49215/test.png ___ Python tracker <https://bugs.python.org/issue40863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40863] bytes.decode changes/destroys line endings on windows
New submission from Matthias Naegler : ``` # 0x0D, 13 = /r # 0x0A, 10 = /n print('test "\\r\\n"') print('-') b = bytes([0x41, 0x0D, 0x0A]) print("bytes: %s" % b) print("string: %s" % b.decode('utf8'), end='') # expected string: "A\r\n" # ressult string: "A\r\r\n" print('test "\\n"') print('--') b = bytes([0x41, 0x0A]) print("bytes: %s" % b) print("string: %s" % b.decode('utf8'), end='') # expected string: "A\n" # ressult string: "A\r\n" ``` It seems like bytes.decode always replaces "\n" with "\r\n". Tested with Windows 10 with Python: - 3.6.0 - 3.7.7 - 3.8.3-rc1 - 3.8.3 - 2.7.18 (works as expected) -- components: Unicode, Windows messages: 370711 nosy: Matthias Naegler, ezio.melotti, paul.moore, steve.dower, tim.golden, vstinner, zach.ware priority: normal severity: normal status: open title: bytes.decode changes/destroys line endings on windows type: behavior versions: Python 3.6, Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue40863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com