[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi added the comment: I verified that launching the script from a CMD.EXE window instead of from Cygwin also reproduces the "can't overwrite ZZZ.pyc". Here's a sample of "python -v" showing this: # C:\depot\central\ntf\tools\bin\build1.pyc has bad mtime import build1 # from C:\depot\central\ntf\tools\bin\build1.py # can't create C:\depot\central\ntf\tools\bin\build1.pyc If you would like some help, I can discuss this with Windows expert programmers here and see whether they can suggest a solution you might find acceptable. Thank you. -- ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi added the comment: Re: chmod 0666 inappropriate: I agree that leaving a file 0666 is inappropriate, however theoretically the code should never leave the file 0666 since the unlink will delete the file. The code was intended to show what the problem is by demonstrating a solution. Note that the line immediately following the chmod 0666 is an unlink call, which will delete the file if the user had sufficient permission to chmod the file to 0666. I suppose the meta-level description of the problem is: [1] Without the chmod 0666, the unlink will fail if the file is 0400 or similar. [2] With the chmod 0666, the unlink succeeds, and the file will be recreated with new permissions (not 0666: I believe my tests show 0444) after the post-unlink file creation. [3] Someone familiar with windows should analyze, propose a solution, test it under all relevant scenarios of file/group ownership (and again the chmod is only required on Windows), and on XP/Vista/Win7. The only problem I can see with the chmod 0666 is if for some reason the file still exists with bad permissions after the unlink. Since this is a low-overhead operation compared to the time to regenerate the .pyc file, I think that it would be reasonable to have an #ifdef MS_WINDOWS, if the file exists after the unlink(), print a big warning message that will always be visible even if -v is not used when invoking Python. This is getting complicated enough that you may want to see if you can get someone else to review it for ideas about how to address the chmod/unlink code. I am certainly not a Windows expert hence I'm making a best Unix-like attempt to analyze and test fixes. Good luck and thanks. -- ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi added the comment: I realized that my patch, with a chmod(..., 0600), may not work under certain conditions where the original file is owned by a different user but the directory is group- or world-writable. A more likely correct fix would use chmod(..., 0666). At any rate: someone should test the various combinations of whether this user or another user owns the original .pyc file. -- ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi added the comment:
I have a fix for this. The code is Windows-only. It has been verified to work
for attached test script
showpycreadonlysleep.sh. It simply adds a "chmod" call. The issue is that the
unlink() call silently fails
if the file is readonly, but unlink() succeeds if chmod() makes the file
writable.
Out company would really appreciate having this fix integrated into 2.6 since
we use the ActiveState 2.6
builds on Windows. I haven't test this on non-Windows platforms but it should
not change the behavior on
non-Windows platforms if the #ifdef MS_WINDOWS is correct. The diff is below:
$ svn diff
Index: import.c
===
--- import.c(revision 72946)
+++ import.c(working copy)
@@ -840,6 +840,7 @@
static FILE *
open_exclusive(char *filename, mode_t mode)
{
+
#if defined(O_EXCL)&&defined(O_CREAT)&&defined(O_WRONLY)&&defined(O_TRUNC)
/* Use O_EXCL to avoid a race condition when another process tries to
write the same file. When that happens, our open() call fails,
@@ -848,6 +849,9 @@
writable, the file will never be written. Oh well.
*/
int fd;
+#ifdef MS_WINDOWS
+(void) chmod(filename, 0600);
+#endif
(void) unlink(filename);
fd = open(filename, O_EXCL|O_CREAT|O_WRONLY|O_TRUNC
#ifdef O_BINARY
It may be appropriate to document that the chmod() is only required on Windows,
and therefore it is only
called on Windows to avoid slowing down non-Windows platforms.
--
___
Python tracker
<http://bugs.python.org/issue6074>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi added the comment: I may have a reason to analyze that would explain why this fails on Windows: the non-Windows code path in import.c's open_exclusive() contians an unlink(filename), but the Windows path does not. I'm going to try to build Python on Windows and add an unlink for both code paths. More later. -- ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi added the comment: Re: Ubuntu: I can't reproduce the problem. At this point, I consider this problem Windows XP-only. Re: Windows & bash: I decided to verify that this problem occurs even if running outside of Bash in the native (and wonderful) shell CMD.EXE. It *does* occur. Here's the state: [1] Run my script in Bash, which fails as in my previous append from today. [2] Start CMD.EXE. Then run a "touch b.py" in Bash, then verify using Bash "ls -l" that b.py is protection 444 (readonly). [3] In CMD.EXE, run "python -v a.py" (I don't know how to use something like grep in CMD.EXE, so I just manually copy/pasted the "python -v" lines.) The following lines in CMD.EXE's output show that the problem still occurs in CMD.EXE: # C:\Documents and Settings\psimanyi\b.pyc has bad mtime import b # from C:\Documents and Settings\psimanyi\b.py # can't create C:\Documents and Settings\psimanyi\b.pyc b # clear __builtin__._ If you are unable to reproduce this on XP, I will be happy to help. I would not be surprised if this had something to do with the granularity of mtime or some other oddity of Windows, because: if I delete the sleep in the Bash script, then rerun the Bash script very quickly using "control-P control-J" typed ahead, sometimes the Bash script fails and sometimes it succeeds! (Nice work, Windows.) Thanks much. -- ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
Peter Simanyi added the comment: Oops, I was not accurate in reporting behavior: I now believe that this fails on Windows XP, but may work fine on Linux. Below are my results, showing reproducible failure on XP. The most important thing about this is: on XP, once the .pyc file is "broken", Python will not overwrite it, and it must be *manually* removed. This is bad! I will append Ubuntu results in a few minutes. To reproduce the problem, I had to modify the script to include a "sleep", see attached showpyreadonlysleep.sh (cygwin bash script). Here's the results of running showpyreadonlysleep.sh, then manually reproducing the problem with "touch/python": $ ./showpycreadonlysleep.sh umask = 0022 CYGWIN_NT-5.1 psimanyi_xp 1.5.25(0.156/4/2) 2008-06-12 19:34 i686 Cygwin Python 2.6.2 Running python.exe from: /cygdrive/c/Python26/python -rw-r--r-- 1 psimanyi mkgroup-l-d 10 May 26 12:43 b.py b -rwx--+ 1 psimanyi mkgroup-l-d 137 May 26 12:43 b.pyc b -r-x--+ 1 psimanyi mkgroup-l-d 137 May 26 12:43 b.pyc # c:\Documents and Settings\psimanyi\b.pyc has bad mtime import b # from c:\Documents and Settings\psimanyi\b.py # can't create c:\Documents and Settings\psimanyi\b.pyc And now, running manually: $ touch b.py psima...@psimanyi_xp ~ $ python -v a.py 2>&1 | grep 'b\.py' # c:\Documents and Settings\psimanyi\b.pyc has bad mtime import b # from c:\Documents and Settings\psimanyi\b.py # can't create c:\Documents and Settings\psimanyi\b.pyc psima...@psimanyi_xp ~ $ touch b.py psima...@psimanyi_xp ~ $ python -v a.py 2>&1 | grep 'b\.py' # c:\Documents and Settings\psimanyi\b.pyc has bad mtime import b # from c:\Documents and Settings\psimanyi\b.py # can't create c:\Documents and Settings\psimanyi\b.pyc Again, this shows that without removing the .pyc manually, the pyc will not be overwritten, and therefore program startup will be slow! -- Added file: http://bugs.python.org/file14083/showpycreadonlysleep.sh ___ Python tracker <http://bugs.python.org/issue6074> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6074] .pyc files created readonly if .py file is readonly, python won't overwrite
New submission from Peter Simanyi :
Some source code control tools, like Perforce, by default sync files readonly,
and it's useful to
leave them readonly so that you can mark files changed by making them writeable
even if they're not
checked out (e.g. working offline).
When python implicitly compiles a .py file that's readonly, it will create a
.pyc file that's
readonly. This would be fine, except that when the .py file changes, python
*silently* refuses to
overwrite a .pyc file that's readonly, even if the directory is completely
writable by the user
executing Python.
The attached script shows this behavior. Running "python -v" will produce the
following line
showing this:
import b # from c:\Documents and Settings\psimanyi\tmp\b.py
# can't create c:\Documents and Settings\psimanyi\tmp\b.pyc
And because I hoped this was a Windows-only problem, I tested on OSX (Leopard)
and Ubuntu (9.10
alpha): it occurs on all three platforms.
Thanks!
I fixed showpycreadonly.sh after attaching it, so to be sure you have the
current version, it
follows:
rm -f [ab].py{,c}
echo 'import b' > a.py
echo 'print "b"' > b.py
ls -l b.py
python a.py
ls -l b.pyc
rm b.pyc
chmod -w b.py
python a.py
ls -l b.pyc
touch b.py
python -v a.py 2>&1 | grep b.py
--
components: Interpreter Core
files: showpycreadonly.sh
messages: 88138
nosy: pdsimanyi
severity: normal
status: open
title: .pyc files created readonly if .py file is readonly, python won't
overwrite
type: behavior
versions: Python 2.6
Added file: http://bugs.python.org/file14025/showpycreadonly.sh
___
Python tracker
<http://bugs.python.org/issue6074>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
