Re: File write, weird behaviour
On 2/19/2023 6:10 PM, Mats Wichmann wrote: On 2/19/23 14:06, Dieter Maurer wrote: Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500: ... Example 2 (weird behaviour) file = open("D:\Programming\Python\working_with_files\cities.txt", 'r+') ## contains list cities # the following code DOES NOT add new record TO THE BEGINNING of the file IF FOLLOWED BY readline() and readlines()# Expected behaviour: new content should be added to the beginning of the file (as in Example 1) file.write("new city\n") file.readlines() file.close() I could not find anything in documentation to explain this strange behaviour. Why is this happening? The effect of "r+" (and friends) is specified by the C standard. The Linux doc (of `fopen`) tells us that ANSI C requires that a file positioning command (e.g. `seek`) must intervene between input and output operations. Your example above violates this condition. Therefore, weird behavior is to be expected. If this isn't sufficiently described, someone should raise an issue against the Python docs. I know that many concepts are "inherited from" environments generally in the POSIX space and the C language, because that's where Python was hatched (all of which makes perfect sense to me, who's been working in those spaces for...ever), but a Python programmer shouldn't have to read the ISO C standard (which is not free, although you can find copies on-line), or the POSIX standard (which also is not free, though manpages for systems like Linux cover the same material), in order to figure out how Python is going to work. The Python docs say that text file behavior is done by Python, not C, and so its behavior will be consistent across all platforms. I didn't find any words about this case, though. -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
On 2/19/23 14:06, Dieter Maurer wrote: Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500: ... Example 2 (weird behaviour) file = open("D:\Programming\Python\working_with_files\cities.txt", 'r+') ## contains list cities # the following code DOES NOT add new record TO THE BEGINNING of the file IF FOLLOWED BY readline() and readlines()# Expected behaviour: new content should be added to the beginning of the file (as in Example 1) file.write("new city\n") file.readlines() file.close() I could not find anything in documentation to explain this strange behaviour. Why is this happening? The effect of "r+" (and friends) is specified by the C standard. The Linux doc (of `fopen`) tells us that ANSI C requires that a file positioning command (e.g. `seek`) must intervene between input and output operations. Your example above violates this condition. Therefore, weird behavior is to be expected. If this isn't sufficiently described, someone should raise an issue against the Python docs. I know that many concepts are "inherited from" environments generally in the POSIX space and the C language, because that's where Python was hatched (all of which makes perfect sense to me, who's been working in those spaces for...ever), but a Python programmer shouldn't have to read the ISO C standard (which is not free, although you can find copies on-line), or the POSIX standard (which also is not free, though manpages for systems like Linux cover the same material), in order to figure out how Python is going to work. -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500: > ... >Example 2 (weird behaviour) > >file = open("D:\Programming\Python\working_with_files\cities.txt", >'r+') ## contains list cities ># the following code DOES NOT add new record TO THE BEGINNING of the >file IF FOLLOWED BY readline() and readlines()# Expected behaviour: >new content should be added to the beginning of the file (as in >Example 1) >file.write("new city\n") > >file.readlines() >file.close() > >I could not find anything in documentation to explain this strange >behaviour. Why is this happening? The effect of "r+" (and friends) is specified by the C standard. The Linux doc (of `fopen`) tells us that ANSI C requires that a file positioning command (e.g. `seek`) must intervene between input and output operations. Your example above violates this condition. Therefore, weird behavior is to be expected. -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
On 2/19/2023 2:31 PM, Chris Angelico wrote: On Mon, 20 Feb 2023 at 06:24, Thomas Passin wrote: On 2/19/2023 1:53 PM, Chris Angelico wrote: On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov wrote: Example 1 (works as expected) file = open("D:\Programming\Python\working_with_files\cities.txt", 'r+') ## contains list cities Side note: You happened to get lucky with P, w, and c, but for the future, I recommend using forward slashes for your paths: open("D:/Programming/Python/working_with_files/cities.txt", "r+") Otherwise, you may run into annoying and hard-to-solve problems. Or alternatively, you'll upgrade to a newer Python and start getting warnings, which would at least tell you that there's a problem. Or use r'...' strings. If you are copying a path to clipboard from Windows Explorer - a fairly common operation - it's much easier to prepend the "r" than to change all the backslashes to forward slashes. Yep, either way. I personally prefer using forward slashes since there's no way to end an r-string with a single backslash, which is often useful when building a path: # won't work path = r"c:\path\to\some\" open(path + "file") # will work path = "c:/path/to/some/" open(path + "file") Never hit that one before. To be classified as "Learn something new every day"! I've been using pathlib more lately. It does clever things with "/" - >>> from pathlib import PurePath >>> p1 = PurePath('c:/this') / 'is' / 'a' /'test' >>> p1 PureWindowsPath('c:/this/is/a/test') # on Linux, would be a PurePosixPath and also >>> p2 = PurePath('c:/this/') /'is' /'a' /'test' >>> p2 PureWindowsPath('c:/this/is/a/test') Despite the apparent forward slashes, the right separators get used for the OS. And the Paths can be used with open(), etc. -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
On 2023-02-19 19:31, Chris Angelico wrote: On Mon, 20 Feb 2023 at 06:24, Thomas Passin wrote: On 2/19/2023 1:53 PM, Chris Angelico wrote: > On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov > wrote: >> >> Example 1 (works as expected) >> >> file = open("D:\Programming\Python\working_with_files\cities.txt", >> 'r+') ## contains list cities > > Side note: You happened to get lucky with P, w, and c, but for the > future, I recommend using forward slashes for your paths: > > open("D:/Programming/Python/working_with_files/cities.txt", "r+") > > Otherwise, you may run into annoying and hard-to-solve problems. Or > alternatively, you'll upgrade to a newer Python and start getting > warnings, which would at least tell you that there's a problem. Or use r'...' strings. If you are copying a path to clipboard from Windows Explorer - a fairly common operation - it's much easier to prepend the "r" than to change all the backslashes to forward slashes. Yep, either way. I personally prefer using forward slashes since there's no way to end an r-string with a single backslash, which is often useful when building a path: # won't work path = r"c:\path\to\some\" open(path + "file") # will work path = "c:/path/to/some/" open(path + "file") When I build a path, I use os.path.join, so it's only a problem if one of the strings refers to the root of a drive, e.g. "C:\\". -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
On Mon, 20 Feb 2023 at 06:24, Thomas Passin wrote: > > On 2/19/2023 1:53 PM, Chris Angelico wrote: > > On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov > > wrote: > >> > >> Example 1 (works as expected) > >> > >> file = open("D:\Programming\Python\working_with_files\cities.txt", > >> 'r+') ## contains list cities > > > > Side note: You happened to get lucky with P, w, and c, but for the > > future, I recommend using forward slashes for your paths: > > > > open("D:/Programming/Python/working_with_files/cities.txt", "r+") > > > > Otherwise, you may run into annoying and hard-to-solve problems. Or > > alternatively, you'll upgrade to a newer Python and start getting > > warnings, which would at least tell you that there's a problem. > > Or use r'...' strings. If you are copying a path to clipboard from > Windows Explorer - a fairly common operation - it's much easier to > prepend the "r" than to change all the backslashes to forward slashes. > Yep, either way. I personally prefer using forward slashes since there's no way to end an r-string with a single backslash, which is often useful when building a path: # won't work path = r"c:\path\to\some\" open(path + "file") # will work path = "c:/path/to/some/" open(path + "file") ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
On 2023-02-19 12:59:43 -0500, Thomas Passin wrote: > On 2/19/2023 11:57 AM, Axy via Python-list wrote: > > Looks like the data to be written is buffered, so actual write takes > > place after readlines(), when close() flushes buffers. > > > > See io package documentation, BufferedIOBase. > > > > The solution is file.flush() after file.write() > > Another possibility, from the Python docs: > > "...TextIOWrapper (i.e., files opened with mode='r+') ... To disable > buffering in TextIOWrapper, consider using the write_through flag for > io.TextIOWrapper.reconfigure()" That actually doesn't help (I tried it before writing my answer). The binary layer below the text layer also buffers ... > Also from the docs: > > "Warning: Calling f.write() without using the with keyword or calling > f.close() might result in the arguments of f.write() not being completely > written to the disk, even if the program exits successfully." He does call file.close(): > > > file.close() so that doesn't seem relevant. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
On 2/19/2023 1:53 PM, Chris Angelico wrote: On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov wrote: Example 1 (works as expected) file = open("D:\Programming\Python\working_with_files\cities.txt", 'r+') ## contains list cities Side note: You happened to get lucky with P, w, and c, but for the future, I recommend using forward slashes for your paths: open("D:/Programming/Python/working_with_files/cities.txt", "r+") Otherwise, you may run into annoying and hard-to-solve problems. Or alternatively, you'll upgrade to a newer Python and start getting warnings, which would at least tell you that there's a problem. Or use r'...' strings. If you are copying a path to clipboard from Windows Explorer - a fairly common operation - it's much easier to prepend the "r" than to change all the backslashes to forward slashes. -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov wrote: > > Example 1 (works as expected) > > file = open("D:\Programming\Python\working_with_files\cities.txt", > 'r+') ## contains list cities Side note: You happened to get lucky with P, w, and c, but for the future, I recommend using forward slashes for your paths: open("D:/Programming/Python/working_with_files/cities.txt", "r+") Otherwise, you may run into annoying and hard-to-solve problems. Or alternatively, you'll upgrade to a newer Python and start getting warnings, which would at least tell you that there's a problem. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
On 2/19/2023 11:57 AM, Axy via Python-list wrote: Looks like the data to be written is buffered, so actual write takes place after readlines(), when close() flushes buffers. See io package documentation, BufferedIOBase. The solution is file.flush() after file.write() Another possibility, from the Python docs: "...TextIOWrapper (i.e., files opened with mode='r+') ... To disable buffering in TextIOWrapper, consider using the write_through flag for io.TextIOWrapper.reconfigure()" Also from the docs: "Warning: Calling f.write() without using the with keyword or calling f.close() might result in the arguments of f.write() not being completely written to the disk, even if the program exits successfully." I realize that in the example, close() was called ... but not immediately after the write(). Can't deny, such a behaviour looks utterly weird. Try to avoid designing your software this way. Axy. On 19/02/2023 14:03, Azizbek Khamdamov wrote: Example 1 (works as expected) file = open("D:\Programming\Python\working_with_files\cities.txt", 'r+') ## contains list cities # the following code adds new record to the beginning of the file, expected behaviour file.write("new city\n") file.close() Example 2 (weird behaviour) file = open("D:\Programming\Python\working_with_files\cities.txt", 'r+') ## contains list cities # the following code DOES NOT add new record TO THE BEGINNING of the file IF FOLLOWED BY readline() and readlines()# Expected behaviour: new content should be added to the beginning of the file (as in Example 1) file.write("new city\n") file.readlines() file.close() I could not find anything in documentation to explain this strange behaviour. Why is this happening? -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
On 2023-02-19 14:03, Azizbek Khamdamov wrote: Example 1 (works as expected) file = open("D:\Programming\Python\working_with_files\cities.txt", 'r+') ## contains list cities # the following code adds new record to the beginning of the file, expected behaviour file.write("new city\n") file.close() Example 2 (weird behaviour) file = open("D:\Programming\Python\working_with_files\cities.txt", 'r+') ## contains list cities # the following code DOES NOT add new record TO THE BEGINNING of the file IF FOLLOWED BY readline() and readlines()# Expected behaviour: new content should be added to the beginning of the file (as in Example 1) file.write("new city\n") file.readlines() file.close() I could not find anything in documentation to explain this strange behaviour. Why is this happening? It works correctly if you use file.flush() or file.tell() before switching from writing to reading. -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
On 2023-02-19 16:57:02 +, Axy via Python-list wrote: > Looks like the data to be written is buffered, so actual write takes place > after readlines(), when close() flushes buffers. > > See io package documentation, BufferedIOBase. > > The solution is file.flush() after file.write() Or alternatively, file.seek() to the intended position when switching between reading and writing. (The C standard says you have to do this. I can't find it in the Python docs, but apparently Python behaves the same.) > On 19/02/2023 14:03, Azizbek Khamdamov wrote: > > Example 2 (weird behaviour) > > > > file = open("D:\Programming\Python\working_with_files\cities.txt", > > 'r+') ## contains list cities > > # the following code DOES NOT add new record TO THE BEGINNING of the > > file IF FOLLOWED BY readline() and readlines()# Expected behaviour: > > new content should be added to the beginning of the file (as in > > Example 1) > > file.write("new city\n") Also note that you can't ADD anything at the beginning (or in the middle) of a file. You will overwrite existing content if you try this. You can only add at the end of the file. If you want to insert something, you have to rewrite everything from that position. (So typically, for small files you wouldn't update a file in place, you would just replace it completely. For large data sets which need to be updated you would generally use some kind of database.) hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: File write, weird behaviour
Looks like the data to be written is buffered, so actual write takes place after readlines(), when close() flushes buffers. See io package documentation, BufferedIOBase. The solution is file.flush() after file.write() Can't deny, such a behaviour looks utterly weird. Try to avoid designing your software this way. Axy. On 19/02/2023 14:03, Azizbek Khamdamov wrote: Example 1 (works as expected) file = open("D:\Programming\Python\working_with_files\cities.txt", 'r+') ## contains list cities # the following code adds new record to the beginning of the file, expected behaviour file.write("new city\n") file.close() Example 2 (weird behaviour) file = open("D:\Programming\Python\working_with_files\cities.txt", 'r+') ## contains list cities # the following code DOES NOT add new record TO THE BEGINNING of the file IF FOLLOWED BY readline() and readlines()# Expected behaviour: new content should be added to the beginning of the file (as in Example 1) file.write("new city\n") file.readlines() file.close() I could not find anything in documentation to explain this strange behaviour. Why is this happening? -- https://mail.python.org/mailman/listinfo/python-list