[issue23120] installation order of 32bit and 64bit python seems to matter

2014-12-27 Thread Peter Santoro

New submission from Peter Santoro:

It appears that installation order matters when installing both 32bit and 64bit 
versions of Python.  If you install the 32bit version first, the 64bit version 
will uninstall the 32bit version.  Here are the steps I used:

1. Starting point (Windows 7 64bit with latest MS patches):
   Python 3.3.5 32bit (c:\bin32\python33, required for support of older systems 
that will be upgraded to 3.4.3 when it's available)
   Python 3.4.1 32bit (c:\bin32\python34)
   Python 3.4.1 64bit (c:\bin\python34, included in PATH)
2. Uninstalled
   Python 3.4.1 32bit - ran as expected
   Python 3.4.1 64bit - ran as expected
3. Attempted Install (32bit install before 64bit install)
   Python 3.4.2 32bit (c:\bin32\python34) - ran as expected
   Python 3.4.2 64bit (c:\bin\python34) - install dialog had red warning 
message at top stating that previous version will be uninstalled; if you 
continue, the 64bit install deletes the Python 3.4.2 32bit install
4. Work around installation order (64bit install before 32bit install):
   Python 3.4.2 64bit (c:\bin\python34) - ran as expected
   Python 3.4.2 32bit (c:\bin32\python34) - ran as expected

Notes:

I've reported other strange issues with multiple versions of python installed, 
but they were deemed different than the original reported bug and were not 
further discussed.  It's quite possible that the combination of python versions 
installed/uninstalled over time on this system has caused some 
bad/unexpected/untested state (e.g. Windows registry entries).

--
components: Installation
messages: 233131
nosy: pe...@psantoro.net
priority: normal
severity: normal
status: open
title: installation order of 32bit and 64bit python seems to matter
type: behavior
versions: Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23120
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21427] installer not working

2014-06-21 Thread Peter Santoro

Peter Santoro added the comment:

I believe I may have hit a related issue yesterday.  I'm using Python 3.3.5 (32 
and 64 bit) and 3.4.1 (32 and 64 bit) releases all on the same Windows 
7SP1/64bit PC (patched with latest MS updates).  The Tkinter applications that 
I wrote and have been using with 3.x for a while now stopped working sometime 
after installing 3.4.1 - I only noticed this behavior yesterday.  All the 
Tkinter apps run fine as .py files (but with an associated console window); 
however, they no longer run as .pyw files.  Instead, they silently fail.

I checked my file associations/types and I believe they are correct.  The 
py.exe and pyw.exe files are in the Windows directory.  None of my colleagues 
have this issue, but they only have one Python installation (3.4.1 64bit).  My 
users also only have one Python version installed (3.3.5 or 3.4.1) and they are 
not experiencing this issue.

I did search for a empty file on the path with the same name as the python 
script that I was trying to execute, as this will lead to silent failures - but 
I found none.

I also tried using the Python Windows installer repair facility on the Python 
3.4.1 64bit install, but that didn't help.  I then uninstalled and reinstalled 
my Python 3.4.1 64bit release, but that didn't help either.

I did notice that the py.exe and pyw.exe files in my Windows directory were not 
identical to a colleague's Python 3.4.1 64bit PC.

I then ran pyw.exe via windbg (e.g. windbg pyw.exe -3.4 irtool.pyw, windbg 
pyw.exe -3.4-32 irtool.pyw) to see if there were obvious errors.  I discovered 
that my pyw.exe silently fails whenever it is instructed to use the 3.4.1 
release.  However, when I specified Python 3.3.5 (pyw.exe -3.3 or -3.3-32) my 
Tkinter applications ran fine and as expected, without an associated console 
window.

I'm reasonably certain that I did not intentionally modify the py.exe and 
pyw.exe files in my Windows directory.

Also, all my scripts start with the following: 
#!/usr/bin/python3

Unfortunately, I'm not able to uninstall 3.3.5, until we upgrade all users to 
3.4.x (probably shortly after 3.4.2 is released).

--
nosy: +pe...@psantoro.net

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21427
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20907] behavioral differences between shutil.unpack_archive and ZipFile.extractall

2014-03-20 Thread Peter Santoro

Peter Santoro added the comment:

It seems clear to me that the logic in shutil._unpack_zipfile that silently 
skips paths that start with '/' (indicates absolute path) or that contain 
references to the parent directory ('..') was added to prevent malicious zip 
files from making potential malicious/unwanted modifications to the filesystem 
(perhaps at a time when zipfile did not itself contain such logic).  This 
conservative approach works, but it can have unexpected results.  For example, 
if all entries in a zip file contain these invalid characters, then 
shutil._unpack_zipfile appears to do nothing (i.e. the zip file is not 
unpacked).  This is good (except for the silent part), if the zip file is truly 
malicious.  However, I recently had to deal with thousands of zip files created 
by well known software vendors where hundreds of the zip files were created 
incorrectly and contained these invalid characters.  These files were not 
malicious, but they were created improperly. Note that shutil._unpack_zipfile 
silently fai
 led to unzip these files, but by using ZipFile.extractall I could unzip them.

It appears that most unzipping software today either either ignores (sometimes 
silently) potentially malicious zip entries (e.g. Windows 7 Explorer displays 
an invalid zip file error) or it attempts to filter out/replace known bad 
characters so that the zip entries can be extracted (e.g. WinZip, gnu unzip).  
I created this issue because the Python library uses both approaches, which may 
need rethinking.

The newer logic in ZipFile._extract_member, which is used by 
ZipFile.extractall, takes a different approach.  Instead of silently ignoring 
potentially malicious zip entries, it attempts to filter out or replace known 
invalid characters before extracting the zip entries.

From the Python zipfile docs:
---
If a member filename is an absolute path, a drive/UNC sharepoint and leading 
(back)slashes will be stripped, e.g.: ///foo/bar becomes foo/bar on Unix, and 
C:\foo\bar becomes foo\bar on Windows. And all .. components in a member 
filename will be removed, e.g.: ../../foo../../ba..r becomes foo../ba..r. On 
Windows illegal characters (:, , , |, , ?, and *) replaced by underscore (_).
---

As ZipFile._extract_member filters out/replaces more invalid characters than 
shutil._unpack_zipfile handles, one could argue that the (apparent older) 
approach used by shutil._unpack_zipfile is less safe.

The approaches used by shutil._unpack_zipfile and ZipFile.extractall to deal 
with potentially malicious zip file entries are different.  This issue could be 
closed if not deemed important by the Python core developers or it could be 
handled by documentation and/or coding changes.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20907
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20907] behavioral differences between shutil.unpack_archive and ZipFile.extractall

2014-03-13 Thread Peter Santoro

New submission from Peter Santoro:

Since Python 3.3.1, ZipFile.extractall was enhanced to better handle absolute 
paths and illegal characters.  The associated logic within 
shutil._unpack_zipfile essentially skips zip members with these issues.

If a zip file contains all absolute paths, ZipFile.extractall works as expected 
(i.e. the zip file is unpacked), but shutil._unpack_zipfile (normally called 
indirectly via shutil.unpack_archive) appears to do nothing (i.e. it silently 
fails to unpack the zip file).

The attached patch attempts to unify the behavior of extracting zip files 
between shutil.unpack_archive with ZipFile.extractall.

--
components: Library (Lib)
files: shutil.diff
keywords: patch
messages: 213374
nosy: pe...@psantoro.net
priority: normal
severity: normal
status: open
title: behavioral differences between shutil.unpack_archive and 
ZipFile.extractall
type: behavior
versions: Python 3.3, Python 3.4
Added file: http://bugs.python.org/file34393/shutil.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20907
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20907] behavioral differences between shutil.unpack_archive and ZipFile.extractall

2014-03-13 Thread Peter Santoro

Peter Santoro added the comment:

I've attached a zip file which contains a test script and test zip files for 
the previously submitted Python 3.3.5 patch.  See the included README.txt for 
more information.  To view the contents of the included bad.zip file, use the 
following command:

 unzip -l bad.zip

--
Added file: http://bugs.python.org/file34408/test_unpack.zip

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20907
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18577] lru_cache enhancement: lru_timestamp helper function

2014-02-02 Thread Peter Santoro

Peter Santoro added the comment:

As requested, I published this for review on 
http://code.activestate.com/recipes/578817-lru_timestamp-cache-entry-aging-for-functoolslru_c/

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18577
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20102] shutil._make_zipfile possible resource leak

2014-01-01 Thread Peter Santoro

New submission from Peter Santoro:

Now that zipfile.ZipFile supports the context manager protocol, shouldn't
shutil._make_zipfile make use of it to avoid the possibility of the archive 
file not being closed properly if an exception occurs?  It should be noted that 
shutil._unpack_zipfile does use try/finally to ensure that files are closed.

--
components: Library (Lib)
files: shutil.diff
keywords: patch
messages: 207132
nosy: pe...@psantoro.net
priority: normal
severity: normal
status: open
title: shutil._make_zipfile possible resource leak
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file33291/shutil.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20102
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18577] lru_cache enhancement: lru_timestamp helper function

2013-07-29 Thread Peter Santoro

Peter Santoro added the comment:

I updated my proposed lru_timestamp function with the following change:

1) raise TypeError instead of ValueError

--
Added file: http://bugs.python.org/file31079/lru.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18577
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18577] lru_cache enhancement: lru_timestamp helper function

2013-07-28 Thread Peter Santoro

New submission from Peter Santoro:

The attached proposed lru_timestamp function provides developers with more 
control over how often lru_cache entries are refreshed.  Doc string follows:

def lru_timestamp(refresh_interval=60):
 Return a timestamp string for @lru_cache decorated functions.

The returned timestamp is used as the value of an extra parameter
to @lru_cache decorated functions, allowing for more control over
how often cache entries are refreshed. The lru_timestamp function
should be called with the same refresh_interval value for a given
lru_cache decorated function. 

Positional arguments:
refresh_interval -- 1-1440 minutes (default 60) as int or float



Rationale:

Some functions have input parameters that rarely change, but yet return 
different results over time.  It would be nice to have a ready-made solution to 
force lru_cache entries to be refreshed at specified time intervals.

An common example is using a stable userid to read user information from a 
database.  By itself, the lru_cache decorator can be used to cache the user 
information and prevent unnecessary i/o.  However, if a given user's 
information is updated in the database, but the associated lru_cache entry has 
not yet been discarded, the application will be using stale data.  The 
lru_timestamp function is a simple, ready-made helper function that gives the 
developer more control over the age of lru_cache entries in such situations.

Sample usage:

@functools.lru_cache()
def user_info(userid, timestamp):
# expensive database i/o, but value changes over time
# the timestamp parameter is normally not used, it is
# for the benefit of the @lru_cache decorator
pass

# read user info from database, if not in cache or
# older than 120 minutes
info = user_info('johndoe', functools.lru_timestamp(120))

--
components: Library (Lib)
files: lru.py
messages: 193820
nosy: pe...@psantoro.net
priority: normal
severity: normal
status: open
title: lru_cache enhancement: lru_timestamp helper function
type: enhancement
versions: Python 3.3
Added file: http://bugs.python.org/file31063/lru.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18577
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18577] lru_cache enhancement: lru_timestamp helper function

2013-07-28 Thread Peter Santoro

Peter Santoro added the comment:

I updated my proposed lru_timestamp function with the following changes:

1) restricted refresh_interval to int type
2) updated doc string

Updated doc string follows:

def lru_timestamp(refresh_interval=60):
 Return a timestamp string for @lru_cache decorated functions.

The returned timestamp is used as the value of an extra parameter
to @lru_cache decorated functions, allowing for more control over
how often cache entries are refreshed. The lru_timestamp function
should be called with the same refresh_interval value for a given
@lru_cache decorated function.  The returned timestamp is for the
benefit of the @lru_cache decorator and is normally not used by
the decorated function.

Positional arguments:
refresh_interval -- in minutes (default 60), values less than 1
are coerced to 1, values more than 1440 are
coerced to 1440



--
Added file: http://bugs.python.org/file31064/lru.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18577
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18224] pyvenv pydoc.py script causing AttributeErrors on Windows

2013-06-17 Thread Peter Santoro

Peter Santoro added the comment:

As requested, I've attached a small test script called shadow.py.  Steps to 
reproduce:

1) pyvenv.py bugtest
2) copy the attached shadow.py script to bugtest and bugtest\scripts
3) cd bugtest
4) run shadow.py (first entry in sys.path is refers to bugtest directory per 
Python docs; finds the systems's pydoc module)
5) run bugtest\shadow.py (first entry in sys.path refers to bugtest\scripts 
directory per Python docs; finds the bugtest\scripts pydoc module instead of 
the system's pydoc module)

According to the Python documentation 
(http://docs.python.org/3/library/sys.html#sys.path):

As initialized upon program startup, the first item of this list, path[0], is 
the directory containing the script that was used to invoke the Python 
interpreter. If the script directory is not available (e.g. if the interpreter 
is invoked interactively or if the script is read from standard input), path[0] 
is the empty string, which directs Python to search modules in the current 
directory first. Notice that the script directory is inserted before the 
entries inserted as a result of PYTHONPATH.

Maybe I'm missing something here, but isn't this problem caused by the fact 
that Python initializes sys.path[0] to contain the directory of the executing 
script and that having pydoc.py in that same directory (i.e. the venv's scripts 
directory) shadows the system's pydoc.py module?  On Linux, I didn't have this 
problem, because the pydoc script doesn't have the .py extension.  However, if 
you rename the pydoc script on Linux to pydoc.py, the same problem occurs.

I don't think a pydoc.py (or any other .py file which shadows a system module) 
can exist in the venv scripts (or bin) directory without shadowing/breaking the 
system provided module.  Maybe a pydoc.exe or pydoc.bat file is needed on 
Windows?  Another option would be to rename the pydoc.py file to something like 
pydocs.py, but that would be incompatible with other platforms and the existing 
documentation.

--
Added file: http://bugs.python.org/file30620/shadow.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18224
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18224] pyvenv pydoc.py script causing AttributeErrors on Windows

2013-06-15 Thread Peter Santoro

New submission from Peter Santoro:

I've recently hit an issue with pyvenv in Python 3.3.2 that is causing 
AttributeErrors in other packages on Windows (see 
https://groups.google.com/forum/?fromgroups#!topic/pylons-discuss/FpOSMDpdvy4). 
 Here's what I believe is going on:

On Windows, the pyvenv pydoc script has a .py extension - so import finds it 
instead of the system's pydoc module.  On Linux, the pyvenv pydoc script 
doesn't have an extension - so import finds the system's pydoc module.

I believe the Windows pyvenv pydoc.py script should be renamed to something 
like pydocs.py to prevent AttributeErrors.

--
components: Windows
messages: 191210
nosy: pe...@psantoro.net
priority: normal
severity: normal
status: open
title: pyvenv pydoc.py script causing AttributeErrors on Windows
type: behavior
versions: Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18224
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com