[issue3566] httplib persistent connections violate MUST in RFC2616 sec 8.1.4.

2015-04-05 Thread R. David Murray

R. David Murray added the comment:

Thanks, Martin and Demian.  I tweaked the patch slightly before commit, so I've 
uploaded the diff.  After thinking about it I decided that it does indeed make 
sense that the new exception subclass both ConnectionResetError and 
BadStatusLine, exactly because it *isn't* a pure ConnectionError, it is a 
synthetic one based on getting a '' response when we are expecting a status 
line.  So I tweaked the language to not mention backward compatibility.  I also 
tweaked the language of the docs, comments and error message to make it clear 
that the issue is that the server closed the connection (I understand why you 
changed it to 'shut down', but I think 'the server closed the connection' is 
both precise enough and more intuitive).

If you have any issues with the changes I made, let me know.

--
resolution:  - fixed
stage: commit review - resolved
status: open - closed
Added file: http://bugs.python.org/file38839/RemoteDisconnected.v6.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3566
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23496] Steps for Android Native Build of Python 3.4.2

2015-04-05 Thread Cyd Haselton

Cyd Haselton added the comment:

Latest error:

gcc --sysroot=/usr/gcc-4.9.2/sysroot -c -Wno-unused-result -Wsign-compare 
-Wunreachable-code -DNDEBUG -g  -O3 -Wall -Wstrict-prototypes
-Werror=declaration-after-statement   -I. -IInclude -I./Include
-DPy_BUILD_CORE -o Objects/unicodeobject.o Objects/unicodeobject.c
Objects/unicodeobject.c:45:23: fatal error: androidfn.h: No such file or 
directory
 #include androidfn.h
   ^
compilation terminated.
make: *** [Objects/unicodeobject.o] Error 1

No idea what androidfn.h is, but will troubleshoot in the AM

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23496
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Marc-Andre Lemburg

Marc-Andre Lemburg added the comment:

On 05.04.2015 23:25, R. David Murray wrote:
 
 MAL: then what you are arguing for is that the SSL changes in 2.7.9 should 
 not have happened.  Which is an argument that Antoine and I at least are 
 sympathetic to :)

I think those changes were probably fine for many Python users,
just not all of them. I'm only arguing to get some easy way to
disable these enforced checks which doesn't require patching Python.

(So I guess I'm kind of standing in the middle between Antoine and
you on one side and Donald on the other side ;-))

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread R. David Murray

R. David Murray added the comment:

Actually I was in favor of an environment variable (or something like that) 
from the start, because it could be set per-process (making it as close to 
per-application as we can get from upstream).  But a global config file I think 
is a bad idea (at least in the form so far suggested).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue3566] httplib persistent connections violate MUST in RFC2616 sec 8.1.4.

2015-04-05 Thread Roundup Robot

Roundup Robot added the comment:

New changeset eba80326ba53 by R David Murray in branch 'default':
#3566: Clean up handling of remote server disconnects.
https://hg.python.org/cpython/rev/eba80326ba53

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3566
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



Re: [issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread M.-A. Lemburg
On 05.04.2015 22:49, Donald Stufft wrote:
 
 Donald Stufft added the comment:
 
 I don't consider monkey patching a proper way to configure a Python
 installation.
 
 The point is that that TLS validation on/off isn't conceptually a Python level
 configuration option, that's going to be a per application configuration
 option. The monkeypatching is simply an escape hatch to give people time to
 update their applications (or pressure whoever wrote the application) to
 support the configuration option that really belongs at the application
 level. It *should* feel improper because the entire concept of a Python level
 on/off switch isn't proper and making it feel more proper by adding an 
 official
 API or config file for doing it is only giving footguns out to people.

People upgrading to a new patch level Python release will *not*
expect or want to have to change their application to adapt to
it. That's simply not within the scope of a patch level release.

Furthermore, old applications such as Zope will (most likely) not
receive such updates.

Please accept that there's a whole universe out there where people
don't continually update their applications, but still want to
benefit from bug fixes to their underlying libs and tools. The
world is full of legacy systems, regardless of whether we like it
or not. There's no good or bad about this. It's just a fact of
life.

What I'm arguing for is a way for admins of such older systems
to be able to receive bug fixes for Python 2.7.x *without*
having to change the applications.

Using an environment setting and adding that to the application's
user account settings is an easy way to resolve the issue in
situations where other options are not feasible or simply not
deemed needed (Zope has been working without any egg verification
for years).

-- 
Marc-Andre Lemburg
eGenix.com

___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



ANN: polynice 0.7 - a nice(1) like utility for throttling processes

2015-04-05 Thread garabik-news-2005-05
polynice is a nice(1)-like command line utility for unix systems to
throttle long running processes beyond what can be achieved by nice(1),
by repeatedly suspending and resuming the process.

It is written for python3, though there is some python2.7 compatibility.

Author:
Radovan Garabík

URL:
http://kassiopeia.juls.savba.sk/~garabik/software/polynice.html

License:
GPL (v2)

Notable changes:
* This is the first public release under the name `polynice' - the
  utility has been renamed from `verynice' due to name clash with an
  existing software.
* polynice can now emulate timeout(1) - terminate process after given
  time has passed
* MacOS X improvements
* better python2 compatiblity

-- 
 ---
| Radovan Garabík http://kassiopeia.juls.savba.sk/~garabik/ |
| __..--^^^--..__garabik @ kassiopeia.juls.savba.sk |
 ---
Antivirus alert: file .signature infected by signature virus.
Hi! I'm a signature virus! Copy me into your signature file to help me spread!
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: XML Parsing

2015-04-05 Thread Ben Finney
Sepideh Ghanavati sepideh...@gmail.com writes:

 I know basic of python and I have an xml file created from csv

What XML schema defines the document's format?

Without knowing the schema, parsing will be unreliable.

What created the document? Why is it relevant that the document was
“created from CSV”?

 which has three attributes category, definition and definition
 description.

What do you mean by “attributes”?

In Python, an attribute has a specific meaning.

In XML, an attribute has a rather different meaning.

Neither of those meanings seems to apply to “the XML document has three
attributes”. XML documents don't have attributes; differnt XML elements
in a document have different attributes.

 I want to parse through xml file and identify actors, constraints,
 principal from the text.

How are those defined in the document's schema?

 However, I am not sure what is the best way to go. Any suggestion?

You should:

* Learn some more about XML
  URL:http://www.xmlobjective.com/the-basic-principles-of-xml/.

* Learn exactly what formal document schema defines the document
  URL:https://en.wikipedia.org/wiki/XML_schema. If the document isn't
  accompanied by a specification of exactly what its schema is, you're
  going to have a difficult time.

-- 
 \“If I melt dry ice, can I swim without getting wet?” —Steven |
  `\Wright |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Help with pipes, buffering and pseudoterminals

2015-04-05 Thread Cameron Simpson

On 05Apr2015 12:20, Daniel Ellis ellis...@gmail.com wrote:

I have a small little tool I'd like to make.  It essentially takes piped input, 
modifies the text in some way, and immediately prints the output.  The problem 
I'm having is that any output I pipe to the program seems to be buffered, 
removing the desired effect.


That depends on the upstream program; it does the buffering. The pipe itself 
presents received data downstream immediately.


However, as you've seen, almost every program buffers its standard output if 
the output is not a tty; this is automatic in the stdio C library and results 
in more fficient use of I/O.



From what I understand, I need to somehow have the input be retrieved via a 
pseudoterminal.


This is a rather gross hack, though sometimes all you can do. While some 
programs have an option to force unbuffered output, most do not. Attaching 
their output to a pty is one way to encourage them to at least line buffer 
their output.


However, you should bear in mind that the reason that programs line buffer to a 
terminal is that they presume they are in an interactive situation with a 
person watching. The program _may_ act differently in other ways as well, such 
as asking question it might not otherwise ask in batch mode (where it might 
cautiously not ask and presume no).


Also, output sent through a pty is subject to the line discipline in the 
terminal; temrinals are funny things with much historical behaviour. At the 
least you pobably want your pty in raw mode to avoid all sorts of stuff that 
can be done to your data.



The problem that I'm having is that most examples on the internet seem to 
assume I would like to launch a program in a forked pty process, which doesn't 
really fit my use case.


Indeed not, but not to worry. You don't need to fork.


I've tried a number of things, but I seem to be unable to get even a basic 
understanding of how to use the pty module.


Have you every used a pty from C? Do you know how ptys work? (master side, 
slave side, etc).



Here's a piece of code I whipped up just to try to get a feel for what is going 
on when I use pty.fork, but it doesn't seem to do what I think it should:

   import pty
   import os
   import sys

   pid, fd = pty.fork()
   print pid, fd
   sys.stdout.flush()
   os.read(fd, 1024)

This only seems to print from the parent process.


The documentation for pty.fork says:

 Return value is (pid, fd). Note that the child gets pid 0, and the fd is 
invalid.

So the child cannot used fd. It further says that the child has its stdin and 
stdout attached to the pty, and that the pty is the child's controlling 
terminal (this means it is affected by things like typing ^C at the pty, 
etc).


I read that I need to do the os.read call for the fork to happen.  I've also 

tried printing *after* the os.read call.

Don't try to adapt fork-based tutorials to your needs. Understand ptys directly 
first.



I realize this does very little to solve my overall goal, but I figure 
understanding what is going on is probably a worthwhile first step.


What you probably want to use is pty.openpty() instead. No fork. You will get 
back file descriptors for the master and slave sides of the pty. Then you can 
use these with the subprocess module to connect your input program. Or, 
guessing from your opening sentence, you can write a wrapper script whose whole 
purpose is to run a program on a pty.


Regarding terminology: a pseudoterminal (pty) is a device that looks like a 
traditional serial terminal. All terminal emulators like xterm use one, and so 
do other programs presenting a terminal session such as the sshd process 
handling an interactive remote login.


When you call pty.openpty() you are handed two file descriptors: one for the 
master side of the pty and one for the slave side. The slave side is the side 
that looks like a terminal, and is what a typical use would connect a child 
process to. The master side is the other side of the pty. When a program writes 
to the slave side, the output is available for read on the master side, much 
like a pipe. When a program writes to the master side, the output is available 
for read on the slave side, _as_ _if_ _typed_ at the terminal.


A pty is not necessarily going to solve your problem unless you can get your 
input via the pty. From the sounds of it you're in this situation:


 command-generating-output | your-program

such that your input is attached to a pipe, and because 
command-generating-output is attached to a pipe it is block buffering its 
output, hence your problem.


You can't undo that situation after the fact.

To solve your problem via a pty you need to contrive to set up 
command-generating-output already attached to a pty. One way to do that is 
for your-program to open a pty and itself invoke command-generating-output 
with its output via the pty, which is why so many tutorials suppose a fork 
situation.


One typical away to do that is to pass 

Re: Is it possible to deliver different source distributions for different Python versions?

2015-04-05 Thread Steven D'Aprano
On Mon, 6 Apr 2015 06:38 am, Dave Hein wrote:

 I would like to distribute a python package with different code for
 Python 2.* than for Python 3.*. (Mostly this is because of different
 unicode string handling).
 
 There is nothing in to setuptools or PyPi that directly supports
 this scenario.
 
 But perhaps there could be some script run at install time that moves
 the correct source code to the right location? In other works, if I
 included both source code versions in the distribution (in a src2 and
 a src3 subdirectory) then a function invoked at install time could
 detect the python version and copy the appropriate source code to the
 right location.
 
 Is that at all possible? Is there some install time hook that lets me
 supply custom installation code?

I'm not aware of any standard solution to that, but I'm not a setuptools
expert. setup.py files are Python code, so you can put any code you like in
them. But, as far as I am concerned, having the installer pick and choose
what source files to include is not a good solution. Instead, you should
pick one of these two alternatives:


(1) Supply a separate package for 2.x and 3.x, each with their own
installer. The installer confirms that it is running under the correct
version of Python, and just installs.


(2) Or supply a single package with a single installer that works under both
2.x and 3.x. (This is my preference.)

One way of handling the second case is to only support 3.3 or better: that
way, you can still use u... for Unicode strings. Hardly anyone used 3.1
or 3.2, and 3.0 is no longer supported, so it is quite reasonable to insist
people upgrade to 3.3.

Another way to handle the second case is to use conditional imports:


# mymodule2
mystring = uäπЖ☃  # Also works in 3.3 or better.

# mymodule3
mystring = äπЖ☃


# main application
from __future__ import print_function
from future_builtins import *
if sys.version  '3':
import mymodule2 as mymodule
else:
import mymodule3 as mymodule
print(mymodule.mystring)



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Is it possible to deliver different source distributions for different Python versions?

2015-04-05 Thread Mark Lawrence

On 05/04/2015 21:38, Dave Hein wrote:

I would like to distribute a python package with different code for
Python 2.* than for Python 3.*. (Mostly this is because of different
unicode string handling).

There is nothing in to setuptools or PyPi that directly supports
this scenario.

But perhaps there could be some script run at install time that moves
the correct source code to the right location? In other works, if I
included both source code versions in the distribution (in a src2 and
a src3 subdirectory) then a function invoked at install time could
detect the python version and copy the appropriate source code to the
right location.

Is that at all possible? Is there some install time hook that lets me
supply custom installation code?

--
Dave Hein



I can't help directly but have you looked here 
https://packaging.python.org/en/latest/ ?


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list


[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Donald Stufft

Donald Stufft added the comment:

 I don't consider monkey patching a proper way to configure a Python
 installation.

The point is that that TLS validation on/off isn't conceptually a Python level
configuration option, that's going to be a per application configuration
option. The monkeypatching is simply an escape hatch to give people time to
update their applications (or pressure whoever wrote the application) to
support the configuration option that really belongs at the application
level. It *should* feel improper because the entire concept of a Python level
on/off switch isn't proper and making it feel more proper by adding an official
API or config file for doing it is only giving footguns out to people.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23496] Steps for Android Native Build of Python 3.4.2

2015-04-05 Thread Cyd Haselton

Cyd Haselton added the comment:

issue_20306.patch won't install; attempting to do so yields the following:

patching file configure.ac  
 Hunk 56 FAILED 4944/4944.
 AC_MSG_RESULT($ENSUREPIP)
 AC_SUBST(ENSUREPIP)
 
+AC_CHECK_MEMBER([struct passwd.pw_gecos],  
 +  [AC_DEFINE(HAVE_PASSWD_GECOS_FIELD, 1, [Define if pwd.h 
defines field passwd.pw_gecos])],   
 +  [], 
  +  [[#include pwd.h]])
+   
  # generate output files
 AC_CONFIG_FILES(Makefile.pre Modules/Setup.config Misc/python.pc 
Misc/python-config.sh)
 AC_CONFIG_FILES([Modules/ld_so_aix], [chmod +x Modules/ld_so_aix])

For the time being I'll just use --without-ensurepip when configuring.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23496
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread R. David Murray

R. David Murray added the comment:

MAL: then what you are arguing for is that the SSL changes in 2.7.9 should not 
have happened.  Which is an argument that Antoine and I at least are 
sympathetic to :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



Re: Help with pipes, buffering and pseudoterminals

2015-04-05 Thread Nobody
On Sun, 05 Apr 2015 12:20:48 -0700, Daniel Ellis wrote:

 This only seems to print from the parent process.  I read that I need to
 do the os.read call for the fork to happen.  I've also tried printing
 *after* the os.read call.

The child process has its std{in,out,err} attached to the newly-created
pty, so that's where the output from the child's print goes.

You'll see that output if the parent prints the string returned from the
os.read() call.

-- 
https://mail.python.org/mailman/listinfo/python-list


[issue18553] os.isatty() is not Unix only

2015-04-05 Thread Berker Peksag

Changes by Berker Peksag berker.pek...@gmail.com:


--
stage: needs patch - resolved

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18553
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



Re: XML Parsing

2015-04-05 Thread Stefan Behnel
Sepideh Ghanavati schrieb am 06.04.2015 um 04:26:
 I know basic of python and I have an xml file created from csv which has
 three attributes category, definition and definition description.
 I want to parse through xml file and identify actors, constraints,
 principal from the text. However, I am not sure what is the best way to
 go. Any suggestion?

If it's really generated from a CSV file, you could also parse that instead:

https://docs.python.org/3/library/csv.html

Admittedly, CSV files are simple, but they also have major problems,
especially when it comes to detecting their character encoding and their
specific format (tab/comma/semicolon/space/whatever separated, with or
without escaping, quoted values, ...). Meaning, you can easily end up
reading nonsense from the file instead of the content that was originally
put into it.

So, if you want to parse from XML instead, use ElementTree:

https://docs.python.org/3/library/xml.etree.elementtree.html

Stefan

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Is it possible to deliver different source distributions for different Python versions?

2015-04-05 Thread Stefan Behnel
Dave Hein schrieb am 05.04.2015 um 22:38:
 I would like to distribute a python package with different code for
 Python 2.* than for Python 3.*. (Mostly this is because of different
 unicode string handling).
 
 There is nothing in to setuptools or PyPi that directly supports
 this scenario.
 
 But perhaps there could be some script run at install time that moves
 the correct source code to the right location? In other works, if I
 included both source code versions in the distribution (in a src2 and
 a src3 subdirectory) then a function invoked at install time could
 detect the python version and copy the appropriate source code to the
 right location.
 
 Is that at all possible? Is there some install time hook that lets me
 supply custom installation code?

Sure. You can simply change the directory in which distutils looks for your
Python code:

https://docs.python.org/2/distutils/setupscript.html#listing-whole-packages

However, in general, you shouldn't be doing this. It's usually easier
(definitely in the long-term) to keep your sources cross-Py2.x/3.x
compatible, maybe with the help of tools like six or python-future,
than to try to keep separate source trees in sync.

http://python-future.org/

Stefan


-- 
https://mail.python.org/mailman/listinfo/python-list


[issue23870] pprint collections classes

2015-04-05 Thread Raymond Hettinger

Raymond Hettinger added the comment:

This seems like a reasonable intermediate step.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23870
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18704] IDLE: Integrate external code analysis tools

2015-04-05 Thread Terry J. Reedy

Changes by Terry J. Reedy tjre...@udel.edu:


--
superseder:  - IDLE: Ability to run 3rd party code checkers

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18704
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



XML Parsing

2015-04-05 Thread Sepideh Ghanavati
Hi, 

I know basic of python and I have an xml file created from csv which has three 
attributes category, definition and definition description. I want to 
parse through xml file and identify actors, constraints, principal from the 
text. However, I am not sure what is the best way to go. Any suggestion?
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: emacs for python web development

2015-04-05 Thread Sayth
Thanks, got the python bit down is just the Web for flask and django. Getting 
the templates and snippets to work in a good flow is where I am looking for 
advice. 

Cheers

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: [issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread M.-A. Lemburg
FWIW: I just ran into a situation where the new approach resulted
in pip, setuptools and zc.buildout not working anymore.

This was on an AIX system which did come with CA root certificates
at all.

Now, I knew how to fix this, but the solution was not
an obvious one. I had to use truss to figure out where OpenSSL
was looking for certificates and the added the Mozilla cert
bundle from our egenix-pyopenssl package to make things work
again.

This was on a system where Python 2.7.3 had been installed
previously. After the upgrade to Python 2.7.9 nothing worked
anymore.

Again: Please let the users decide what level of security they
want to apply. We can point users to solutions, but in the end
have to respect their own decisions. Note that staying with
Python 2.7.8 is a much worse approach than disabling the checks.

-- 
Marc-Andre Lemburg
eGenix.com

___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Antoine Pitrou

Antoine Pitrou added the comment:

 This was on a system where Python 2.7.3 had been installed
 previously. After the upgrade to Python 2.7.9 nothing worked
 anymore.

Who did the upgrade and with which binaries?
If you're compiling Python from source, especially for an exotic system, well, 
you're supposed to read the release notes :-)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23872] Typo in response in smtpd

2015-04-05 Thread Benjamin Peterson

Changes by Benjamin Peterson benja...@python.org:


--
versions:  -Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23872
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23872] Typo in response in smtpd

2015-04-05 Thread Roundup Robot

Roundup Robot added the comment:

New changeset cc2c7aa2d7a6 by Benjamin Peterson in branch '3.4':
fix extended command syntax (closes #23872)
https://hg.python.org/cpython/rev/cc2c7aa2d7a6

New changeset 2c89c1c34e19 by Benjamin Peterson in branch 'default':
merge 3.4 (#23872)
https://hg.python.org/cpython/rev/2c89c1c34e19

--
nosy: +python-dev
resolution:  - fixed
stage: commit review - resolved
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23872
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Nick Coghlan

Nick Coghlan added the comment:

PEP 476 *has* a mechanism in it that was supposed to deal with this problem, 
thus leaving *end users* in full control of the decision on when they upgrade 
their security infrastructure rather than having that decision arbitrarily 
imposed on them by a vendor or an upstream community project regardless of 
whether or not it's appropriate for their particular situation. Unfortunately, 
it turned out I was wrong about the viability of the approach in PEP 476, hence 
this suggestion to revisit the question.

There is *no* suggestion of changing the default behaviour away from that 
defined in PEP 476, the part I would like to revisit is merely the section on 
configurability, where the goal is to be able to deploy All of PEP 476 
*except* the change in default certificate verification behaviour. The 
approach in the PEP works for folks deploying upstream Python directly, and I 
*thought* it would work for the redistributor case as well. It's the latter 
point I was wrong about.

This is a level of consideration of their needs that folks are willing to pay 
for, but it's also an expensive one to provide, so it doesn't make sense for 
upstream to provide it for free. Rather, I am asking the upstream development 
community to work with commercial redistributors to come to an accommodation 
that actually meets end users upgrade needs, rather than leaving them stuck on 
a legacy Python version with no viable path forward to more secure 
infrastructure. (Telling end users just upgrade anyway when complex systems 
and large scale deployments are involved doesn't work - this is why Microsoft 
ended up having to support Windows XP for 12 years)

I thought proposing a useful new feature for Python 3.5 and then proposing a 
subsequent backport would be the easiest path forward, but I now suspect a PEP 
specifically targeting an improved network security transition plan for the 
benefit of folks managing infrastructure upgrades in the 2.7.x series may be a 
better option.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23841] py34 OrderedDict is using weakref for root reference

2015-04-05 Thread Antoine Pitrou

Antoine Pitrou added the comment:

weakrefs are traditionally cleared by the cyclic GC before calling any __del__ 
method. This used to be mandatory to eschew situations where a weakref callback 
could see cleared objects, but produces the side effect that __del__ methods 
can see dead weakrefs. This is also true pre-3.4, by the way, but perhaps the 
OP's __del__ wasn't called at all (if it was part of the cycle, the object 
would end up in gc.garbage instead)?

We could perhaps reverse the order, but then weakref callbacks may see 
__del__'ed objects. Hard to say which one is better, and keeping the 
traditional behaviour means less compatibility hassles.

Some of this is discussed in Modules/gc_weakref.txt.

In general, __del__ methods may see strange errors. For example, at interpreter 
shutdown, your __del__ method can be called while some modules have started 
unloading. Python 3.4 definitely tries to improve on all this, but there's no 
perfect solution, and some of the improvements might also backfire in some rare 
cases :-)

--
nosy: +tim.peters
versions: +Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23841
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23841] py34 OrderedDict is using weakref for root reference

2015-04-05 Thread Antoine Pitrou

Antoine Pitrou added the comment:

Concretely, the possible workarounds are:

- don't do anything complex in your __del__
- be prepared to deal with unexpected errors in your __del__
- starting from Python 3.4, don't define __del__ and use weakref.finalize() 
instead: https://docs.python.org/3/library/weakref.html#weakref.finalize

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23841
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23496] Steps for Android Native Build of Python 3.4.2

2015-04-05 Thread Cyd Haselton

Cyd Haselton added the comment:

Hey Ryan,
Just now patching downloaded/unzipped tip and was wondering if there was an 
order in which patches should be applied.

I ask because i'm getting the following when applying the 
android_segfault_fix.patch

/bld/python/cpython-3.4/cpython-3.4/Python $ patch  android_segfault_fix.patch
patching file frozenmain.c
Hunk 3 FAILED 53/56.
 }

 setlocale(LC_ALL, );
+#endif
 for (i = 0; i  argc; i++) {
 argv_copy[i] = Py_DecodeLocale(argv[i], NULL);
 argv_copy2[i] = argv_copy[i];
patching file pylifecycle.c
patch: can't open 'pylifecycle.c': No such file or directory

Thoughts?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23496
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23496] Steps for Android Native Build of Python 3.4.2

2015-04-05 Thread Cyd Haselton

Cyd Haselton added the comment:

UPDATE: I found the file in github, under master, in Python/. It's not in the 
3.4 or origin/3.4 branches...aren't we working on those? Or does it not matter?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23496
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23496] Steps for Android Native Build of Python 3.4.2

2015-04-05 Thread Ryan Gonzalez

Ryan Gonzalez added the comment:

I thought this was for the tip, i.e. the 3.5 dev...?

But I created the patches in the order that I wrote the descriptions in the 
comment. So you might want to use that order.

If that fails, I can figure out the revision I was at when I created the 
patches.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23496
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20173] Derby #4: Convert 53 sites to Argument Clinic across 5 files

2015-04-05 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Here is reworked patch for the _codecs module. No optional groups are used, all 
parameters have sensible defaults. Default encoding is utf-8, default errors 
is strict or None (if function accepts None), default mapping is None. 
Unified names and coding style, improved docstrings.

--
nosy: +serhiy.storchaka
stage: needs patch - patch review
Added file: http://bugs.python.org/file38838/codecs_clinic.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20173
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23871] turning itertools.{repeat, count} into indexable iterables

2015-04-05 Thread Antti Haapala

Antti Haapala added the comment:

well, they wouldn't and shouldn't behave like range. range is a sequence 
whereas count or repeat wouldn't necessarily be sequences. (they can be 
infinite and thus not having length). And the count shouldn't be *reiterable* 
because that is why it exists (otherwise we could just use range). For repeat, 
indexing hardly matters.

--
nosy: +ztane

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23871
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20159] Derby #7: Convert 51 sites to Argument Clinic across 3 files - Derby: Convert the ElementTree module to use Argument Clinic

2015-04-05 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Updated patch. Converted __init__ methods and removed explicit declarations of 
self parameters. SubElement and Element.__init__ still are not converted, as 
they need the support of **kwargs.

--
Added file: http://bugs.python.org/file38837/etree_clinic_2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20159
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Antoine Pitrou

Antoine Pitrou added the comment:

By the way, if a vendor wants vendor-specific behaviour, forking the standard 
library is a normal price to pay.
(in this case, the diff wouldn't be large, and it's made against an extremely 
stable upstream branch)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23871] turning itertools.{repeat, count} into indexable iterables

2015-04-05 Thread Serhiy Storchaka

Changes by Serhiy Storchaka storch...@gmail.com:


--
nosy: +rhettinger

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23871
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Donald Stufft

Donald Stufft added the comment:

On it's own I think this switch is a bad idea because it's too big of a hammer. 
Someone shouldn't accidentally disable TLS verification in pip for instance 
because they wanted to disable TLS verification for some random tool that only 
hit internal TLS but which didn't have it's own off switch written into it. A 
lot of tools are written in Python and it's hard for a user to really know what 
the full extent of toggling this switch on their system will be, especially 
given that they may have no idea which other tools are incidentally written in 
python (pip is not a good example of this, but there are lots of tools that are 
written in Python but which the fact they are written in Python isn't important 
or maybe even obvious).

I think keyed by site is wrong too, again because the scope is wrong. Opting 
out of security at the Python level filters down into tons of random 
applications that the end user may or may not be aware is even written in 
Python.

Part of the benefit of the current opt out mechanism is that it feels a 
little dirty to opt in in that fashion, and it should because globally opting 
out is breaking the security expectations that any application has now with the 
latest versions of Python, and adding a cleaner way of doing this is 
essentially giving people a nicer footgun (in the long term).

Now, I recognize that there is legacy systems at play here that are going to be 
around for a long time and that who this proposal is really being aimed to 
helping. My question would be, why can't those downstreams simply carry this 
patch? The attached patch is relatively tiny so it shouldn't be a very large 
burden, the largest being documenting and making people aware that such a thing 
exists on that platform. If there's enough downstreams who would reasonably 
have a reason to apply said patch maybe an addendum (or a new PEP) can be added 
suggesting that downstreams should apply said patch.

The tl;dr of my opinion is that if it weren't for legacy systems, I don't think 
anyone would propose this feature, and given the security sensitive nature of 
it I think it's best to treat this feature as a quirk of those legacy systems 
rather than a fully supported API of Python.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Nick Coghlan

Nick Coghlan added the comment:

The discussion isn't on python-ideas yet because I wanted to get a better sense 
of what might be politically feasible before putting this question to a broader 
audience. I agree it needs to move there eventually (likely during or after 
PyCon), and will almost certainly lead to a PEP (3.5b1 is slated for late May, 
so we have 6-7 weeks to resolve the question in time for that if anything is 
going to change for 3.5)

To be absolutely clear, nobody is thinking of reintroducing silent security 
failures anywhere - the ultimate aim of posting this draft patch is to start 
down the path to defining a new Python 3.5 feature that could then by pitched 
for a PEP 466 style backport to Python 2.7 to provide a potentially smoother 
upgrade path from the pre-PEP-476 status quo to the shiny new PEP 476 future 
for the benefits of folks that take both security concerns and backwards 
compatibility concerns at least as seriously as python-dev do, but are serving 
a very different audience and hence may need to make different trade-offs 
between these considerations.

The use sitecustomize.py to monkeypatch in the old behaviour section in PEP 
476 was *intended* to provide that upgrade path, but it turned out not to work 
as well as I hoped it would as it turns out that approach effectively requires 
forking the standard library to let a vendor manage the migration on behalf of 
their customers by offering a bridging opt-in period. Changing the standard 
library's behaviour to this degree would be a genuinely drastic option, so I 
consider it vastly superior to backport a supported behaviour from a later 
version of Python (along the lines of the network security backports in PEP 
466) than it would be to invent something custom that has no upstream support.

This does mean spending more time upfront coming up with a way of designing the 
feature that the core development community considers to be useful 
independently of backporting considerations (e.g. bringing the STARTTLS 
migration into the framework could be useful, as the sad state of email server 
certificate validity means that even upstream CPython is going to need to leave 
that off by default for the time being). That additional time investment is 
likely to be worthwhile when the pay-off is avoiding a long-lived behavioural 
fork.

As for *why* such an opt-in bridging period might be needed by some 
organisations, one of the key issues to consider is the likely desire to do a 
global upgrade to an updated Python version as soon as possible, *without* 
risking breaking currently working services in an end-user visible way, and 
then handling the security configuration change on a service-by-service basis 
as a subsequent step, in conjunction with any necessary upgrades to the related 
security infrastructure.

Splitting the two activities (Python upgrade, service network security upgrade) 
this way is potentially desirable even if you have control of all of the 
affected Python applications, but it may be absolutely essential if you're 
running a proprietary bytecode-only Python application in the system Python, or 
simply aren't authorised to make application level changes to an affected 
service.

The rationale for introducing a configuration or marker file for this is to 
allow the *default* behaviour in the absence of such a file to be the standard 
PEP 476 behaviour. An opt-in bridging period can then be implemented by 
publishing a default configuration file that globally opts out, with system 
administrators selectively opting in.

Eventually the default configuration can potentially be changed or removed such 
that certificates are verified by default, by which time services that 
genuinely need to be opted out should already have the appropriate 
configuration settings set.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Nick Coghlan

Nick Coghlan added the comment:

As far as Alex's post goes, it's simply wrong, and I wish he had spoken to me 
about his frustrations with the significant challenges of infrastructure 
maintenance in large established organisations before posting it. Red Hat's 
been fighting the battle for better enterprise infrastructure management for 20 
years at this point (including in the US public sector: 
https://www.redhat.com/en/technologies/industries/government), but like almost 
all institutional reform, it's very slow going.

We offer plenty of options for folks to upgrade faster, and it's much easier 
for us when they do: 
http://www.curiousefficiency.org/posts/2015/04/stop-supporting-python26.html

So if you care about getting security enhancements rolled out in a way that 
means people responsible for infrastructure management in large organisations 
will actually adopt them, rather than dismissing them out of hand as too 
risky, please take a moment to consider that we might have some idea what 
we're talking about.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23830] Add AF_IUCV support to sockets

2015-04-05 Thread Armin Rigo

Armin Rigo added the comment:

The PyArg_ParseTuple() size arguments should be of type Py_ssize_t instead of 
int.

--
nosy: +arigo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23830
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Antoine Pitrou

Antoine Pitrou added the comment:

Le 05/04/2015 12:25, Nick Coghlan a écrit :
 
 This does mean spending more time upfront coming up with a way of
 designing the feature that the core development community considers to
 be useful independently of backporting considerations (e.g. bringing the
 STARTTLS migration into the framework could be useful, as the sad state
 of email server certificate validity means that even upstream CPython is
 going to need to leave that off by default for the time being).

I'm curious about statistics about e-mail servers, even though unrelated
to this issue.

 Splitting the two activities (Python upgrade, service network
 security
 upgrade) this way is potentially desirable even if you have control of
 all of the affected Python applications, but it may be absolutely
 essential if you're running a proprietary bytecode-only Python
 application in the system Python, or simply aren't authorised to make
 application level changes to an affected service.

True, but this is a repeat of the PEP 476 discussion. Something has
changed in the meantime: PEP 476 was accepted and its code has shipped
in an official release. There hasn't been any major (or even minor) outcry.

Speaking as someone who opposed PEP 476, I now support us moving forward
instead of trying to eschew the PEP's deliberate effects.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21114] wsgiref.simple_server doesn't handle multi-line headers correctly

2015-04-05 Thread Tom Tanner

Tom Tanner added the comment:

Any chance to get this into 2.7.10?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21114
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21718] sqlite3 cursor.description seems to rely on incomplete statement parsing for detection

2015-04-05 Thread Tom Tanner

Tom Tanner added the comment:

Are you going to merge it into 2.7.10?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21718
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Donald Stufft

Donald Stufft added the comment:

 Now, I knew how to fix this, but the solution was not
 an obvious one. I had to use truss to figure out where OpenSSL
 was looking for certificates and the added the Mozilla cert
 bundle from our egenix-pyopenssl package to make things work
 again.

You also could have passed the --cert flag to pip to tel pip specifically where
to look for them (also available via environment variable and config file)
though I'm guessing it wasn't actually pip itself that had a problem because
we ship our own CA file and we don't actually rely on the stdlib to have
validated TLS. Unless you were using an old pip I guess.

 Again: Please let the users decide what level of security they
 want to apply. We can point users to solutions, but in the end
 have to respect their own decisions. Note that staying with
 Python 2.7.8 is a much worse approach than disabling the checks.

Sure, and nobody has ever advocated to make it impossible to disable the TLS
verification. For me it's entirely about the scope of the setting. I don't
think that a Python wide setting is the right scope. That's a knob that has
an extremely large scope of which end users are most likely not going to be
completely aware of the total impact of adjusting that knob. This isn't even
something that they could reasonably audit their system with _today_ and then
say OK I've looked at everything that uses Python and I'm happy for it not to
verify because if they every install anything else that uses Python (whether
they know it uses Python or not) they have to re-evaluate that decision they
made all over again, but with no indicator that they need to do that.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



Re: Permission denied when opening a file that was created concurrently by os.rename (Windows)

2015-04-05 Thread Dave Angel

On 04/05/2015 01:45 PM, Alexey Izbyshev wrote:

Hello!

I've hit a strange problem that I reduced to the following test case:
* Run several python processes in parallel that spin in the following loop:
while True:
   if os.path.isfile(fname):
 with open(fname, 'rb') as f:
   f.read()
 break
* Then, run another process that creates a temporary file and then
renames it to the name than other processes are expecting
* Now, some of the reading processes occasionally fail with Permission
denied OSError

I was able to reproduce it on two Windows 7 64-bit machines. It seems
when the file appears on the filesystem it is still unavailable to
reading, but I have no idea how it can happen. Both source and
destination files are in the same directory, and the destination doesn't
exist before calling os.rename. Everything I could find indicates that
os.rename should be atomic under this conditions even on Windows, so
nobody should be able to observe the destination in unaccessible state.

I know that I can workaround this problem by removing useless
os.path.isfile() check and wrapping open() with try-except, but I'd like
to know the root cause of the problem. Please share you thoughts.

The test case is attached, the main file is test.bat. Python is expected
to be in PATH. Stderr of readers is redirected to *.log. You may need to
run several times to hit the issue.

Alexey Izbyshev,
research assistant,
ISP RAS



The attachment is missing;  please just include it inline, after 
reducing it to a reasonably minimal sample.


My guess is that the process that does the os.rename is not closing the 
original file before renaming it.  So even though the rename is atomic, 
the file is still locked by the first process.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list


[issue23496] Steps for Android Native Build of Python 3.4.2

2015-04-05 Thread Cyd Haselton

Cyd Haselton added the comment:

From previous post:
**
How does this sound?
* I'll clone the fork of the 3.4 branch I made in github, build and patch.
* Ryan will (as best as he can) grab said patches, regenerate them for the bug 
tracker or forward port them to 3.5 (in github? to Mercurial)
**

For some reason I was thinking that the 3.4 branch I cloned was a dev branch.  
I'll start again with a download of 3.5 dev.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23496
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



Permission denied when opening a file that was created concurrently by os.rename (Windows)

2015-04-05 Thread Alexey Izbyshev

Hello!

I've hit a strange problem that I reduced to the following test case:
* Run several python processes in parallel that spin in the following 
loop:

while True:
  if os.path.isfile(fname):
with open(fname, 'rb') as f:
  f.read()
break
* Then, run another process that creates a temporary file and then 
renames it to the name than other processes are expecting
* Now, some of the reading processes occasionally fail with Permission 
denied OSError


I was able to reproduce it on two Windows 7 64-bit machines. It seems 
when the file appears on the filesystem it is still unavailable to 
reading, but I have no idea how it can happen. Both source and 
destination files are in the same directory, and the destination doesn't 
exist before calling os.rename. Everything I could find indicates that 
os.rename should be atomic under this conditions even on Windows, so 
nobody should be able to observe the destination in unaccessible state.


I know that I can workaround this problem by removing useless 
os.path.isfile() check and wrapping open() with try-except, but I'd like 
to know the root cause of the problem. Please share you thoughts.


The test case is attached, the main file is test.bat. Python is 
expected to be in PATH. Stderr of readers is redirected to *.log. You 
may need to run several times to hit the issue.


Alexey Izbyshev,
research assistant,
ISP RAS

--
https://mail.python.org/mailman/listinfo/python-list


Re: Permission denied when opening a file that was created concurrently by os.rename (Windows)

2015-04-05 Thread Chris Angelico
On Mon, Apr 6, 2015 at 3:45 AM, Alexey Izbyshev izbys...@ispras.ru wrote:
 The test case is attached, the main file is test.bat. Python is expected to
 be in PATH. Stderr of readers is redirected to *.log. You may need to run
 several times to hit the issue.

You have an interesting-looking problem, but the attachment didn't
arrive. Is it short enough to include in-line as text in your email?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


[issue19511] lib2to3 Grammar file is no longer a Python 3 superset

2015-04-05 Thread Gregory P. Smith

Gregory P. Smith added the comment:

This was fixed in 3.4.1:

https://hg.python.org/cpython/log/094615256a54/Lib/lib2to3/Grammar.txt

i'm leaving this open to update the devguide.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19511
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Marc-Andre Lemburg

Marc-Andre Lemburg added the comment:

On 05.04.2015 18:28, Donald Stufft wrote:
 
 Donald Stufft added the comment:
 
 Now, I knew how to fix this, but the solution was not
 an obvious one. I had to use truss to figure out where OpenSSL
 was looking for certificates and the added the Mozilla cert
 bundle from our egenix-pyopenssl package to make things work
 again.
 
 You also could have passed the --cert flag to pip to tel pip specifically 
 where
 to look for them (also available via environment variable and config file)
 though I'm guessing it wasn't actually pip itself that had a problem because
 we ship our own CA file and we don't actually rely on the stdlib to have
 validated TLS. Unless you were using an old pip I guess.

I was working on a Zope installation using zc.buildout, so
basically setuptools, and yes, it was an older version as well.

But this is only an example of an application not working anymore
because the system's OpenSSL could not verify certificates.
In this case, no root CA certs were available. On older systems
with proper root CA certs, it's likely that the newer CA certs
needed to verify the PyPI certificates are not installed...
and yes: those system do exist and are in active use, simply because
they cannot be upgraded for other reasons :-)

 Again: Please let the users decide what level of security they
 want to apply. We can point users to solutions, but in the end
 have to respect their own decisions. Note that staying with
 Python 2.7.8 is a much worse approach than disabling the checks.
 
 Sure, and nobody has ever advocated to make it impossible to disable the TLS
 verification. For me it's entirely about the scope of the setting. I don't
 think that a Python wide setting is the right scope. That's a knob that has
 an extremely large scope of which end users are most likely not going to be
 completely aware of the total impact of adjusting that knob. This isn't even
 something that they could reasonably audit their system with _today_ and then
 say OK I've looked at everything that uses Python and I'm happy for it not to
 verify because if they every install anything else that uses Python (whether
 they know it uses Python or not) they have to re-evaluate that decision they
 made all over again, but with no indicator that they need to do that.

I'd be fine with having a knob that says: don't check the certificates
but warn me about instances where the certificates are not checked
(using the warning framework).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



Help with pipes, buffering and pseudoterminals

2015-04-05 Thread Daniel Ellis
I have a small little tool I'd like to make.  It essentially takes piped input, 
modifies the text in some way, and immediately prints the output.  The problem 
I'm having is that any output I pipe to the program seems to be buffered, 
removing the desired effect.

From what I understand, I need to somehow have the input be retrieved via a 
pseudoterminal.  The problem that I'm having is that most examples on the 
internet seem to assume I would like to launch a program in a forked pty 
process, which doesn't really fit my use case.

I've tried a number of things, but I seem to be unable to get even a basic 
understanding of how to use the pty module.  Here's a piece of code I whipped 
up just to try to get a feel for what is going on when I use pty.fork, but it 
doesn't seem to do what I think it should:

import pty 
import os
import sys 

pid, fd = pty.fork()

print pid, fd
sys.stdout.flush()

os.read(fd, 1024)

This only seems to print from the parent process.  I read that I need to do the 
os.read call for the fork to happen.  I've also tried printing *after* the 
os.read call.  

I realize this does very little to solve my overall goal, but I figure 
understanding what is going on is probably a worthwhile first step.
-- 
https://mail.python.org/mailman/listinfo/python-list


[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Antoine Pitrou

Antoine Pitrou added the comment:

Le 05/04/2015 21:26, Marc-Andre Lemburg a écrit :
 
 But this is only an example of an application not working anymore
 because the system's OpenSSL could not verify certificates.
 In this case, no root CA certs were available. On older systems
 with proper root CA certs, it's likely that the newer CA certs
 needed to verify the PyPI certificates are not installed...
 and yes: those system do exist and are in active use, simply because
 they cannot be upgraded for other reasons :-)

Let's sum it up:

- the machine can't be upgraded, but you are upgrading Python by hand
(hand-compiled?)

- OpenSSL is installed but there are no root CA certs (?!)

- the machine probably isn't ever doing a single verified HTTPS access,
for the previous reason, and nobody cares about it

- you want to be able to use unauthenticated HTTPS to download and
install software from the Internet

And, since this is an AIX machine, I'm presuming this isn't a hobbyist's
setup, but an enterprise system with paid-for support and licenses,
right? And you want the python-dev community to care for that broken
situation by bearing the cost of additional maintenance and security
risk in implementing the new configuration options?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Marc-Andre Lemburg

Marc-Andre Lemburg added the comment:

On 05.04.2015 21:36, Antoine Pitrou wrote:
 
 And you want the python-dev community to care for that broken
 situation by bearing the cost of additional maintenance and security
 risk in implementing the new configuration options?

No, I want to be able to easily disable the newly added
checks in 2.7.9+ to get systems such as these behave the
same as with 2.7.8, since without this option, people
using these system are going to be forced to stick with
buggy 2.7.8 systems.

It's rather unusual that a patch level release completely
breaks an existing setup. I understand why this was done,
but in the light of backwards compatibility, it's a huge
issue for people.

PS: Python installations in Zope systems are often custom
installs, not system installs of Python. The AIX system
I'm referencing here still has Python 2.6 as system
Python version.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Donald Stufft

Donald Stufft added the comment:

 No, I want to be able to easily disable the newly added
 checks in 2.7.9+ to get systems such as these behave the
 same as with 2.7.8, since without this option, people
 using these system are going to be forced to stick with
 buggy 2.7.8 systems.

Why is the monkeypatch in sitecustomize.py unacceptable? I understand why it's
unacceptable to Nick and rkuska, they are a vendor and they don't want to write
sitecustomize.py when the machine operator may want to use that file, however
you're the machine operator in this case.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23871] turning itertools.{repeat, count} into indexable iterables

2015-04-05 Thread R. David Murray

R. David Murray added the comment:

I agree with Antti.  If Raymond disagrees he can reopen :)

(There is a reason it is called *iter*tools.  As Antti says, range is 
documented as being a *sequence* type.)

--
nosy: +r.david.murray
resolution:  - rejected
stage:  - resolved
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23871
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread Marc-Andre Lemburg

Marc-Andre Lemburg added the comment:

On 05.04.2015 21:48, Donald Stufft wrote:
 
 Donald Stufft added the comment:
 
 No, I want to be able to easily disable the newly added
 checks in 2.7.9+ to get systems such as these behave the
 same as with 2.7.8, since without this option, people
 using these system are going to be forced to stick with
 buggy 2.7.8 systems.
 
 Why is the monkeypatch in sitecustomize.py unacceptable? I understand why it's
 unacceptable to Nick and rkuska, they are a vendor and they don't want to 
 write
 sitecustomize.py when the machine operator may want to use that file, however
 you're the machine operator in this case.

I don't consider monkey patching a proper way to configure a Python
installation.

I could simply patch the Python installation directly, but this is just
me. I'm talking about sys admins out there who don't know enough about
Python to be able to patch Python or write a sitecutomize.py which
uses monkey patching to fix the issue.

I also cannot recommend to our customers to monkey patch
Python just to get it running again. This is not what folks
expect from a production quality system :-)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



Re: Permission denied when opening a file that was created concurrently by os.rename (Windows)

2015-04-05 Thread Terry Reedy

On 4/5/2015 1:45 PM, Alexey Izbyshev wrote:

Hello!

I've hit a strange problem that I reduced to the following test case:
* Run several python processes in parallel that spin in the following loop:



while True:
   if os.path.isfile(fname):
 with open(fname, 'rb') as f:
   f.read()
 break


Besides anything else, I would consider adding a minimal sleep in the loop.


* Then, run another process that creates a temporary file and then
renames it to the name than other processes are expecting
* Now, some of the reading processes occasionally fail with Permission
denied OSError


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list


[issue23857] Make default HTTPS certificate verification setting configurable via global ini file

2015-04-05 Thread R. David Murray

R. David Murray added the comment:

Really these arguments make it sound like 2.7.9 never should have happened.

Given that it did, Nick has not addressed the question of why the vendors 
maintaining this simple patch (given that it addresses what he sees as their 
need) is not a viable option.

I do *not* see the proposed patch as an acceptable feature for 3.5, and I 
think I'm far from alone, so I suspect that it is a non-starter for following 
Nick's proposed path.

Could there be a related feature that would be both acceptable and worthwhile?  
Yes.  But someone will have to figure out what it is and propose it :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23857
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21718] sqlite3 cursor.description seems to rely on incomplete statement parsing for detection

2015-04-05 Thread R. David Murray

R. David Murray added the comment:

It looks to me like a patch that could be merged as a bug fix.

--
stage:  - commit review
versions: +Python 2.7, Python 3.4, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21718
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



Is it possible to deliver different source distributions for different Python versions?

2015-04-05 Thread Dave Hein
I would like to distribute a python package with different code for
Python 2.* than for Python 3.*. (Mostly this is because of different
unicode string handling).

There is nothing in to setuptools or PyPi that directly supports
this scenario.

But perhaps there could be some script run at install time that moves
the correct source code to the right location? In other works, if I
included both source code versions in the distribution (in a src2 and
a src3 subdirectory) then a function invoked at install time could
detect the python version and copy the appropriate source code to the
right location.

Is that at all possible? Is there some install time hook that lets me
supply custom installation code?

--
Dave Hein
-- 
https://mail.python.org/mailman/listinfo/python-list