[issue4769] b64decode should accept strings or bytes

[HiroakiKawai]

Changes by HiroakiKawai :


--
nosy: +kawai

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3613] base64.encodestring does not actually accept strings

[HiroakiKawai]

Changes by HiroakiKawai :


--
nosy: +kawai

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4842] int('3L') still valid in Python 3.0

[Martin v. Löwis]

Martin v. Löwis  added the comment:

We should really start maintaining a specification of the pickle
format(s). Pickle is designed to be independent of the Python version,
although protocol extensions may be added over time. In such a
specification, it would say that the format of the L code is "ascii
decimal digits, followed by L".

The patches look fine to me, please apply.

A further change might be that on pickling a long in text mode, the I
code could be used if the value is in range(-2**31,2**31). However, this
is independent of the issue at hand.

--
assignee:  -> marketdickinson
nosy: +loewis
resolution:  -> accepted

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4962] urlparse & nfs url (rfc 2224)

[Cédric BRINER]

Cédric BRINER  added the comment:

> Do you encounter any errors or weird behaviors while using nfs url?

Do you mean: Do I have problem using the python module in conjunction of
urlparse ?
No, because, I'm not yet using it. But I would like to do it. And I find
it strange that it can not find the host by itself.

>('nfs', '', '//server//a/b/c/d/e/f', '', '', '')
>If you have done the research already, can you please explain what
>difference will adding 'nfs' to uses_netloc do in urlparse.py.

As much as I've looked at the code, the uses_netloc provide an easy way
to tell urlparse.py if such scheme as nfs is composed
by a net location (host[:port]). Which is the case. Modifying
uses_netloc by addind 'nfs' into it will transform the following:
nfsurl : 'nfs://server//a/b/c/d/e/f'
in  urlparse from:
('nfs', '', '//server//a/b/c/d/e/f', '', '', '')
to
('nfs', 'server', '//a/b/c/d/e/f', '', '', '')

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4474] PyUnicode_FromWideChar incorrect for characters outside the BMP (unix only)

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

On 2009-01-18 22:59, Mark Dickinson wrote:
> Mark Dickinson  added the comment:
> 
> Looks good to me.
> 
> I'm not in a position to test with 16-bit wchar_t, but I can't see why 
> anything would go wrong.  I think we can take our chances: check this in 
> and watch the buildbots for signs of trouble.
> 
> Some minor whitespace issues in the unicodeobject.c part of the patch 
> (mixing of tabs and spaces, one brace indented oddly), but those can 
> easily be taken care of before committing;  not worth regenerating the 
> patch for.
> 
> Marc-André, is it okay with you to check this in?

I'd structure the patch differently, ie. put the whole support code
into a single #ifndef Py_UNICODE_WIDE section as part of the
#ifdef HAVE_USABLE_WCHAR_T pre-processor statement.

Also note that on platforms with 16-bit wchar_t, the comparison
(0x < *w) will always be false, so an additional check for
(Py_UNICODE_SIZE > 2) is needed.

BTW: Please always use upper-case hex literals, or at leat don't
mix the case within the same function.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com



::: Try our new mxODBC.Connect Python Database Interface for free ! 

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4996] io.TextIOWrapper calls buffer.read1()

[HiroakiKawai]

New submission from HiroakiKawai :

The documentation says io.TextIOWrapper wraps io.BufferedIOBase raw 
stream. In the code, io.TextIOWrapper.read(), 
io.TextIOWrapper._read_chunk() calls buffer.read1() which seems 
expecting buffer to be an instance of io.BufferedReader. I'm not sure 
which is correct right now.

--
components: Library (Lib)
messages: 80152
nosy: kawai
severity: normal
status: open
title: io.TextIOWrapper calls buffer.read1()
type: behavior
versions: Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3582] thread_nt.c update

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Note, this has been ported to py3k in http://svn.python.org/view?
view=rev&rev=68543

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4997] xml.sax.saxutils.XMLGenerator should write to io.RawIOBase.

[HiroakiKawai]

New submission from HiroakiKawai :

xml.sax.saxutils.XMLGenerator._write tests the argument by 
isinstance(text, str), but this is problematic in Python 3.0. 
XMLGenerator accepts encoding and the produced file is encoded by that 
encoding, i.e., the XML is a binary sequence. So IMHO, the XMLGenerator 
constructor argument should be a subclass of io.RawIOBase.

--
components: Library (Lib), XML
messages: 80155
nosy: kawai
severity: normal
status: open
title: xml.sax.saxutils.XMLGenerator should write to io.RawIOBase.
versions: Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2563] embed manifest in windows extensions

[Mark Hammond]

Mark Hammond  added the comment:

Given bug 4120, this seems the most appropriate resolution...

--
resolution:  -> out of date
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4972] let's equip ftplib.FTP with __enter__ and __exit__

[Tarek Ziadé]

Tarek Ziadé  added the comment:

positive feedbacks on python-ideas, so I'll start to write the patches.

targets : 

   - smtplib.SMTP
   - imaplib.IMAP4
   - ftplib.FTP

first patch : smtplib

(will do ftplib and imaplib as well, then propose this enhancement to
python-dev)

--
keywords: +patch
Added file: http://bugs.python.org/file12792/smtplib.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4995] sqlite3 module gives SQL logic error only in transactions

[Muayyad Alsadi]

Muayyad Alsadi  added the comment:

same thing

[als...@pc1 ~]$ python
Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38) 
[GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> cn=sqlite3.connect(':memory:')
>>> c=cn.cursor()
>>> c.execute('BEGIN TRANSACTION')

>>> c.execute('create temp table tmp_main (id integer, b text)')

>>> cn.commit()
>>> c.execute('insert into tmp_main (id) values (10);')
Traceback (most recent call last):
  File "", line 1, in 
sqlite3.OperationalError: SQL logic error or missing database
>>> 

to make it easy for you to try it yourself

import sqlite3
cn=sqlite3.connect(':memory:')
c=cn.cursor()
c.execute('BEGIN TRANSACTION')
c.execute('create temp table tmp_main (id integer, b text)')
cn.commit() # this was added on your request
c.execute('insert into tmp_main (id) values (10);')

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4336] Fix performance issues in xmlrpclib

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

note, this has been ported to Py3k in http://svn.python.org/view?
view=rev&rev=68458

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4929] smptlib.py can raise socket.error

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Note, this has been ported to py3k in http://svn.python.org/view?
view=rev&rev=68736

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4448] should socket readline() use default_bufsize instead of _rbufsize?

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Hi,
I'm reawakening this because http://bugs.python.org/issue4879 needs to 
be ported to py3k.
In py3k, a socket.fileobject() is still created with bufsize(0), 
although now the reasoning is different:

 def __init__(self, sock, debuglevel=0, strict=0, method=None):
# XXX If the response includes a content-length header, we
# need to make sure that the client doesn't read more than the
# specified number of bytes.  If it does, it will block until
# the server times out and closes the connection.  (The only
# applies to HTTP/1.1 connections.)  Since some clients access
# self.fp directly rather than calling read(), this is a little
# tricky.
self.fp = sock.makefile("rb", 0)

I think that this is just a translation of the old comment, i.e. a 
warning that some people may choose to call .recv() on the underlying 
socket.
Now, this should be far more difficult now, with the newfangled IO 
library and all, and since the sock.makefile() is now a SocketIO object 
which inherits from RawIOBase and all that.  It's tricky to excracth 
the socket to do .recv() on it.  So, I don't think we need to fear 
buffering for readline() anymore.

Or, is the comment about someone doing a HTTPResponse.fp.read() in 
stead of a HTTPResponse.read()?  In that case, I don't see the 
problem.  Of course, anyone reading N characters from a socket stream 
may cause blocking.

My proposal is to remove the comment above and use default buffering 
for the fileobject.  Any thoughts?

--
versions: +Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4998] fractions are mutable

[Somelauw]

New submission from Somelauw :

>>> f = Fraction()
>>> f.a = 5
>>> f.__slots__
('_numerator', '_denominator')
>>> f.a
5
>>> f.__dict__
{}

When I create my own object, this doesn't happen.

>>> class Slots:
__slots__ = ("slot1", "slot2")


>>> a = Slots()
>>> a.slot3 = 6
Traceback (most recent call last):
  File "", line 1, in 
a.slot3 = 6
AttributeError: 'Slots' object has no attribute 'slot3'
>>> 

In python2 this only happens when __slots__ is a tuple. (When __slots__ 
is a list, it works correctly)
>>> class Slots:
__slots__ = ("slot1", "slot2")


>>> a = Slots()
>>> a.slot3 = 8
>>> 

Here is a copy-paste from the python3 documentation:
Without a __dict__ variable, instances cannot be assigned new variables 
not listed in the __slots__ definition. Attempts to assign to an 
unlisted variable name raises AttributeError. If dynamic assignment of 
new variables is desired, then add '__dict__' to the sequence of strings 
in the __slots__ declaration.

Any non-string iterable may be assigned to __slots__. Mappings may also 
be used; however, in the future, special meaning may be assigned to the 
values corresponding to each key.

--
components: Library (Lib)
messages: 80161
nosy: Somelauw
severity: normal
status: open
title: fractions are mutable
versions: Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4999] multiprocessing.Queue does not order objects

[Frédéric Sagnes]

New submission from Frédéric Sagnes :

Objects contained in a multiprocessing.Queue object are not comming out
of the queue in the same order as they went in. For instance, if I put
in object1, object2 and object3 in this very time sequence from multiple
processes, they can end up comming out of the queue as object2, object1
then object3 instead of the original order.

When using the threading module instead of multiprocessing everything is
fine.

The provided test script adds strings to the queue with timestamps.
These messages are not ordered by timestamp when they are printed. This
is an output of the test script with format "[...@time] message":

[2...@00406] Got lock
[2...@02424] Released lock
[2...@02426] Got lock
[2...@04439] Released lock
[...]
[2...@16459] Released lock
[2...@16461] Got lock
[2...@18464] Got lock
[2...@18462] Released lock
[2...@20466] Released lock

Using print to print the message immediatly prints the messages in the
right order.

See this mailing-list thread for details:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/11a5c4ce4ff4382d/033dcd3607eacbf9

--
components: Extension Modules
files: testthreads.py
messages: 80162
nosy: ndfred
severity: normal
status: open
title: multiprocessing.Queue does not order objects
versions: Python 2.6
Added file: http://bugs.python.org/file12793/testthreads.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4995] sqlite3 module gives SQL logic error only in transactions

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Ok, I've just tried. While it fails in Python 2.5.2, it works in 2.6.1
and in trunk (what will become 2.7). So I suggest you upgrade to 2.6.1,
or simply drop the "BEGIN TRANSACTION" statement in this particular case.

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4999] multiprocessing.Queue does not order objects

[Frédéric Sagnes]

Frédéric Sagnes  added the comment:

Tested using Python 2.6.1 on Mac OS 10.5 and Linux 2.6.26

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4957] os.ftruncate raises IOError instead of OSError

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

submitted as r68763

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4996] io.TextIOWrapper calls buffer.read1()

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Indeed, read1() is not documented as a standard method of either IOBase,
RawIOBase or BufferedIOBase. I suggest that it becomes a standard method
of IOBase, with a default implementation simply calling read(n).

It also means unbuffered stdio as it was recently committed doesn't work
for stdin:

$ ./python -u
Python 3.1a0 (py3k:68756, Jan 19 2009, 01:17:26) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.stdin.read(1)
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/antoine/py3k/__svn__/Lib/io.py", line 1739, in read
eof = not self._read_chunk()
  File "/home/antoine/py3k/__svn__/Lib/io.py", line 1565, in _read_chunk
input_chunk = self.buffer.read1(self._CHUNK_SIZE)
AttributeError: 'FileIO' object has no attribute 'read1'
>>> 

I had been misguided by the fact that the interpreter prompt did work,
but it doesn't seem to use sys.stdin...

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4996] io.TextIOWrapper calls buffer.read1()

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Indeed, read1() is not documented as a standard method of either IOBase,
RawIOBase or BufferedIOBase. I suggest that it becomes a standard method
of IOBase, with a default implementation simply calling read(n).

It also means unbuffered stdio as it was recently committed doesn't work
for stdin:

$ ./python -u
Python 3.1a0 (py3k:68756, Jan 19 2009, 01:17:26) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.stdin.read(1)
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/antoine/py3k/__svn__/Lib/io.py", line 1739, in read
eof = not self._read_chunk()
  File "/home/antoine/py3k/__svn__/Lib/io.py", line 1565, in _read_chunk
input_chunk = self.buffer.read1(self._CHUNK_SIZE)
AttributeError: 'FileIO' object has no attribute 'read1'
>>> 

I had been misguided by the fact that the interpreter prompt did work,
but it doesn't seem to use sys.stdin...

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4996] io.TextIOWrapper calls buffer.read1()

[Antoine Pitrou]

Changes by Antoine Pitrou :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5000] multiprocessing - Pool.map() slower about 5 times than map() on 2 cores machine

[0x666]

New submission from 0x666 :

I think something wrong with implementation of multiprocessing module.
I`ve run this very simple test on my machine (core 2, vista):
import multiprocessing as mul
from time import time

def f(x):
return x*x

if __name__ == '__main__':
print " testing multiprocessing on ",mul.cpu_count(),"cores
--"
print ""

elements = 10

pool = mul.Pool(processes=mul.cpu_count())
t1 = time()
res_par = pool.map(f, range(elements))
t2 = time()
res_seq = map(f, range(elements))
t3 = time()
res_app = [pool.apply_async(f,(x,)) for x in range(elements)]
res_app = [result.get() for result in res_app]
t4 = time()

print len(res_seq),"elements","map() time",(t3-t2),"s"
print len(res_par),"elements","pool.map() time",(t2-t1),"s"
print len(res_app),"elements","pool.apply_async() time", (t4-t3),"s"

print
raw_input("press enter to exit...")
__
Results:
 testing multiprocessing on  2 cores ---

10 elements map() time 0.0269 s
10 elements pool.map() time 0.108 s
10 elements pool.apply_async() time 10.567 s


IMHO, execution on 2 cores should be 1.x - 2 times faster than compared
with non-parallel execution. (at least in simple cases).
If you dont believe in this, check http://www.parallelpython.com/
module (demo example sum_primes.py), which fits very well this idea.

So how it can be that parallel pool.map() method executes in about 5
times SLOWER, than ordinary map() function ?
So please correct multiprocessing package to work in more-less
perfomance predictable way (like parallelpython).

--
components: Library (Lib)
messages: 80168
nosy: 0x666
severity: normal
status: open
title: multiprocessing - Pool.map() slower about 5 times than map() on 2 cores 
machine
type: performance
versions: Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4999] multiprocessing.Queue does not order objects

[Jesse Noller]

Changes by Jesse Noller :


--
assignee:  -> jnoller
nosy: +jnoller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Jesse Noller  added the comment:

Attached is a patch+test for this condition, which is not used if we're 
running on windows.

Added file: http://bugs.python.org/file12794/issue_3321.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Changes by Jesse Noller :


Removed file: http://bugs.python.org/file10861/_multiprocessing_connection.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Changes by Jesse Noller :


Removed file: http://bugs.python.org/file11007/test_multiprocessing.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Changes by Jesse Noller :


Removed file: http://bugs.python.org/file11485/another_solution.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Jesse Noller  added the comment:

Curse you hard-tabs. Here's the new patch w/ fixed comment

Added file: http://bugs.python.org/file12795/issue_3321.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Changes by Jesse Noller :


Removed file: http://bugs.python.org/file12794/issue_3321.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4944] os.fsync() doesn't work as expect in Windows

[Javen Wang]

Javen Wang  added the comment:

I'm quite certain that the makefile is generated before the "make"
program is launched in separated process. Follow is the original code
where the makefile is created (line 14) and a task is put in queue (line
19). It's executed in the main thread (no parallel execution here).
There's no way that the process is launched before the makefile generation. 
 
01for Module in Pa.Platform.Modules:
02Ma = ModuleAutoGen(Wa, Module, BuildTarget, ToolChain, Arch,
self.PlatformFile)
03if Ma == None:
04continue
05# generate AutoGen files and Makefile here
06if self.Target not in ['clean', 'cleanlib', 'cleanall', 'run',
'fds']:
07# for target which must generate AutoGen code and makefile
08if not self.SkipAutoGen or self.Target == 'genc':
09Ma.CreateCodeFile(True)
10if self.Target == "genc":
11continue
12   
13if not self.SkipAutoGen or self.Target == 'genmake':
14Ma.CreateMakeFile(True)
15if self.Target == "genmake":
16continue
17
18# Generate build task for the module which will be launched
separately
19Bt = BuildTask.New(ModuleMakeUnit(Ma, self.Target))

I think the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS should not
be used for fsync in the Python core code (in posixmodule.c, right?).
Because fsync is mostly used to solve file operation issue in parallel
execution context, it should not give out CPU time to other threads or
processes when it's running.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4804] Python on Windows disables all C runtime library assertions

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Okay, here is a second patch.
open now verifies the 'mode' string properly, and all uses of a 'fp' 
turn off the CRT error handling temporarily, while holding the GIL

Added file: http://bugs.python.org/file12796/crterror.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5000] multiprocessing - Pool.map() slower about 5 times than map() on 2 cores machine

[Antoine Pitrou]

Antoine Pitrou  added the comment:

You should try with something less trivial than your "f" function. For
such a short function, it seems expectable that the dispatch overhead
will dominate the actual computation time.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Jesse Noller  added the comment:

Removed raise TestSkip per code review from bpeterson

Added file: http://bugs.python.org/file12797/issue_3321.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Changes by Jesse Noller :


Removed file: http://bugs.python.org/file12795/issue_3321.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Jesse Noller  added the comment:

Committed patch as r68768 to python-trunk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4991] os.fdopen doesn't raise on invalid file descriptors

[Benjamin Peterson]

Benjamin Peterson  added the comment:

Well, the Windows bot is failing but not from test_fileio, so I'll go
ahead and merge it. r68767.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5000] multiprocessing - Pool.map() slower about 5 times than map() on 2 cores machine

[Jesse Noller]

Changes by Jesse Noller :


--
assignee:  -> jnoller
nosy: +jnoller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[STINNER Victor]

STINNER Victor  added the comment:

@jnoller: Hey, you removed my patch! My patch used fstat() in 
Connection constructor, whereas your just check file descriptor bounds 
in the poll() method. And what is the "save" new argument? Is it 
related to this issue?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Jesse Noller  added the comment:

The save was needed for the Py_BLOCK_THREADS call.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Jesse Noller  added the comment:

Ugh, I didn't mean to chuck your original patch, but it also wasn't 
correct for win32

Additionally, if you close the handle from underneath it, it behaves 
properly:

>>> obj.poll()
Traceback (most recent call last):
  File "", line 1, in 
IOError: [Errno 9] Bad file descriptor
>>>

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5000] multiprocessing - Pool.map() slower about 5 times than map() on 2 cores machine

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

The multiprocessing module indeed has some overhead:

- the processes are spawned when needed. Before you perform performance
timings, you should "warm up" the Pool with a line like
pool.map(f, range(mul.cpu_count()))
(starting a process is a slowish operation specially on Windows)
This reduces timings by a factor of two.

- the dispatch overhead of multiprocessing is certainly greater than a
single multiplication. multiprocessing is for CPU-bound functions!
And do not forget that you have *tree* processes here: two from the
Pool, and your main program.

As Antoine said, try with this function instead:

def f(x):
for i in range(10):
x = x * x
return x

And the timings are much better...

--
assignee: jnoller -> 
nosy: +amaury.forgeotdarc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5000] multiprocessing - Pool.map() slower about 5 times than map() on 2 cores machine

[Jesse Noller]

Jesse Noller  added the comment:

My results don't match yours. (8 cores, Mac OS/X):

 testing multiprocessing on  8 cores --

10 elements map() time 0.0444118976593 s
10 elements pool.map() time 0.0366489887238 s
10 elements pool.apply_async() time 24.3125801086 s

Now, this could be for a variety of reasons: More cores, different OS 
(which means different speed at which processes can be forked) and so 
on. As Antoine/Amaury point out you really need a use case that is large 
enough to offset the cost of forking the processes in the first place.

I also ran this on an 8 core Ubuntu box with kernel 2.6.22.19 and 
py2.6.1 and
16gb of ram:

 testing multiprocessing on  8 cores --

10 elements map() time 0.0258889198303 s
10 elements pool.map() time 0.0339770317078 s
10 elements pool.apply_async() time 11.0373139381 s

OS/X is pretty snappy when it comes for forking. 

Now, if you cut the example you provided over to Amaury's example, you 
see a significant difference:

OS/X, 8 cores:

 testing multiprocessing on  8 cores --

10 elements map() time 30.704061985 s
10 elements pool.map() time 4.95880293846 s
10 elements pool.apply_async() time 23.6090102196 s

Ubuntu, kernel 2.6.22.19 and py2.6.1:

 testing multiprocessing on  8 cores --

10 elements map() time 38.3818569183 s
10 elements pool.map() time 5.65878105164 s
10 elements pool.apply_async() time 14.1757941246 s

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5000] multiprocessing - Pool.map() slower about 5 times than map() on 2 cores machine

[Jesse Noller]

Jesse Noller  added the comment:

Closing as not an issue.

--
resolution:  -> invalid

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4998] __slots__ on Fraction is useless

[Benjamin Peterson]

Benjamin Peterson  added the comment:

The problem is that Fraction inherits from a class without __slots__
(Rationale), so it's useless. I suggest that the __slots__ be removed or
Rationale.register() is used instead of inheritance.

--
assignee:  -> jyasskin
nosy: +benjamin.peterson, jyasskin
title: fractions are mutable -> __slots__ on Fraction is useless

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5000] multiprocessing - Pool.map() slower about 5 times than map() on 2 cores machine

[Benjamin Peterson]

Changes by Benjamin Peterson :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Changes by Jesse Noller :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4065] _multiprocessing doesn't build on macosx 10.3

[Jesse Noller]

Changes by Jesse Noller :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[STINNER Victor]

STINNER Victor  added the comment:

Why don't you check the file descriptor directly in connection_new()? 
conn->handle is read only and so can't be changed before the call to 
poll(). So other methods will also be "protected" and the error will 
be raised earlier.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Jesse Noller  added the comment:

That's an enhancement - not a bad idea, I just noticed that this issue is 
pretty close to issue http://bugs.python.org/issue3311 as well.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[STINNER Victor]

STINNER Victor  added the comment:

jnoller> issue #3311

Oh, I forgot this issue :-) But the fix doesn't solve #3311, because 
it is disabled on Windows and only protect poll() method.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3321] _multiprocessing.Connection() doesn't check handle

[Jesse Noller]

Jesse Noller  added the comment:

Oh, I agree - I think we should update 3311 with the enhancement to move 
the check to connection_new

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4996] io.TextIOWrapper calls buffer.read1()

[Benjamin Peterson]

Benjamin Peterson  added the comment:

IOBase doesn't even define read(), though! I think we should make it
part of BufferIOBase.

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4996] io.TextIOWrapper calls buffer.read1()

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3881] IDLE won't start in custom directory.

[Zlm]

Zlm  added the comment:

I reinstalled Python to "C:\Program Files\Python\" (I wrote this path in
the installer) and IDLE works fine. I can access the Python folder by
going to "C:\Programas\Python".

Then, I unninstalled and installed again to "C:\Programas\Python". IDLE
doesn't work. I can still access the Python folder by going to
"C:\Programas\Python".

Jacob, can you try installing it to "C:\Program Files\Python" to sse if
that works?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4705] python3.0 -u: unbuffered stdout

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Reopening, since sys.stdin is actually broken in unbuffered mode:

$ ./python -u
Python 3.1a0 (py3k:68756, Jan 19 2009, 01:17:26) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.stdin.read(1)
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/antoine/py3k/__svn__/Lib/io.py", line 1739, in read
eof = not self._read_chunk()
  File "/home/antoine/py3k/__svn__/Lib/io.py", line 1565, in _read_chunk
input_chunk = self.buffer.read1(self._CHUNK_SIZE)
AttributeError: 'FileIO' object has no attribute 'read1'
>>> 

What I propose is that stdin be always opened in buffered mode (even
with -u), since I don't see how the behaviour can differ for a read-only
non-seekable stream.

--
resolution: fixed -> 
stage: commit review -> committed/rejected
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4804] Python on Windows disables all C runtime library assertions

[Martin v. Löwis]

Martin v. Löwis  added the comment:

Kristjan, please understanding that setting the CRT error handling is
not thread-safe, and trying to do it in a fine-grained way can cause all
kinds of crazy races. With your patch, the following sequencce of events
is possible if two threads T1 and T2 simultaneously try to access the CRT:

1. T1 saves the old mode (O), installs its own mode (N1)
2. T1 releases the GIL, invokes CRT operation
3. T2 starts running, saves the old mode (N1), installs its own mode (N2)
4. T1 completes, restores the old mode (O)
5. T2 completes, restores the old mode (N1)

As a net result, the original error handling is *not* restored.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5001] Remove assertion-based checking in multiprocessing

[Jesse Noller]

New submission from Jesse Noller :

Right now, the multiprocessing code is littered with statements like:

assert self._popen is None, 'cannot start a process twice'
assert self._parent_pid == os.getpid(), \
   'can only start a process object created by current 
process'
assert not _current_process._daemonic, \
   'daemonic processes are not allowed to have children'

These will obviously be stripped out if running in optimized mode - 
however its not cool to rely on these anyway, the code should be 
refactored to proper checks, e.g.:

if not hasattr(lock, 'acquire'):
raise AttributeError("'%r' has no method 'acquire'" % lock)

--
assignee: jnoller
components: Library (Lib)
messages: 80192
nosy: jnoller
priority: normal
severity: normal
status: open
title: Remove assertion-based checking in multiprocessing
versions: Python 2.7, Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4708] os.pipe should return inheritable descriptors (Windows)

[Jesse Noller]

Changes by Jesse Noller :


--
nosy: +jnoller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3807] _multiprocessing build fails when configure --without-threads

[Jesse Noller]

Changes by Jesse Noller :


--
assignee:  -> jnoller
nosy: +jnoller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3272] Multiprocessing hangs when multiprocessing.Pool methods are called

[Jesse Noller]

Changes by Jesse Noller :


--
assignee:  -> jnoller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3273] multiprocessing and meaningful errors

[Jesse Noller]

Changes by Jesse Noller :


--
assignee:  -> jnoller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3283] multiprocessing.connection doesn't import AuthenticationError, while using it

[Jesse Noller]

Changes by Jesse Noller :


--
assignee:  -> jnoller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3093] Namespace polution from multiprocessing

[Jesse Noller]

Changes by Jesse Noller :


--
assignee:  -> jnoller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4996] io.TextIOWrapper calls buffer.read1()

[STINNER Victor]

STINNER Victor  added the comment:

Usage of read1():
 * _BytesIO.read1(n) (_bytesio.c): calls bytesio_read(n)
 * _BytesIO.read1(n) (io.py): return self.read(n)
 * BufferedReader._read_unlocked(n=None): 
   - if n is None or n==-1, call self.raw.read() in a "while 
True: ..." until EOF
   - else, call self.raw.read() until we get enough bytes
 * BufferedReader.read1(n): calls self._read_unlocked(...) but make 
sure that we don't call self.raw.read() more than once
 * BufferedRWPair.read1(n): return self.reader.read1(n)
 * BufferedRandom.read1(n): self.flush(); return BufferedReader.read1
(self, n)
 * TextIOWrapper._read_chunk() calls self.buffer.read1
(self._CHUNK_SIZE)

IOBase, RawIOBase, FileIO, BufferedIOBase, TextIOBase, TextIOWrapper, 
StringIO have no read1() method.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4944] os.fsync() doesn't work as expect in Windows

[Gabriel Genellina]

Gabriel Genellina  added the comment:

> I'm quite certain that the makefile is generated before
> the "make"
> program is launched in separated process. Follow is the
> original code
> where the makefile is created (line 14) and a task is put
> in queue (line
> 19). It's executed in the main thread (no parallel
> execution here).
> There's no way that the process is launched before the
> makefile generation. 

It isn't obvious that this is the case, at least to me, due to those "if" and 
"continue" in the code.
Try instrumenting it a bit and see what happens; a few "print" around lines 14 
and 19 should suffice. 

Also, try to come with a *short* example that shows the problem. Remove all 
irrelevant details: if you write a "constant" makefile and still fails, omit 
all dependences for makefile generation; if you invoke a simple .exe (like the 
one I posted) and it still fails, replace the call to nmake.exe. And so on, 
until you can't remove anything. It may happen that you find yourself the 
problem doing this.

Another strategy is to start with a simple example that works (like the one I 
posted) and adding more and more things until it doesn't work anymore.

With the code as it is now, it's difficult to say whether this is a Python bug 
or not; there are tons of other factors involved.

> I think the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS
> should not
> be used for fsync in the Python core code (in
> posixmodule.c, right?).
> Because fsync is mostly used to solve file operation issue
> in parallel
> execution context, it should not give out CPU time to other
> threads or
> processes when it's running.

Why not? All I/O operations release the GIL, and that's a good thing. In this 
case, if (as you assert) the other thread that accesses the file hasn't started 
yet, it doesn't matter if the GIL is released or not.

  Yahoo! Cocina
Recetas prácticas y comida saludable
http://ar.mujer.yahoo.com/cocina/

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4705] python3.0 -u: unbuffered stdout

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Here is a patch.

Added file: http://bugs.python.org/file12798/unbuffered-stdin.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4804] Python on Windows disables all C runtime library assertions

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

I understand thread-safe.  This usage is safe from Python threads 
because it is all done in the context of the GIL.

It is, however, unsafe if some other random thread is also modifying 
these settings during runtime.

In your example, T2 doesn't hold the GIL and so, this is the scenario 
that I believe you are invoking.

This is however not likely to be the case because these settings are 
normally left alone.  The only reason we have to worry about this is 
because we are allowing through file descriptors that are out of the 
control of compiled C code.  Nobody is likely to be messing with this 
stuff except for us.

Furthermore, since your argument assumes a rogue thread modifying the 
CRT settings, this thread may just as well be active during startup 
when we are modifying these values under the current system.  So there 
is not a fundamental difference here, only a difference in scale.

I think that the drawbacks of modifying the CRT behaviour unexpectedly 
for all code in the process far outweigh the risk of there being 
another unknown thread also heavily modifying these obscure settings.

Now, I can see a compromise we could make.  I could make this a compile 
time choice.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4804] Python on Windows disables all C runtime library assertions

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Ooops, ignore my last comment.
I just realized the point you were making Martin, and it doesn't 
involve rogue threads at all.

Hmm, back to the drawing board, I suppose.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4996] io.TextIOWrapper calls buffer.read1()

[STINNER Victor]

STINNER Victor  added the comment:

Short example to reproduce the problem:
---
import io, os
fd = os.open("/etc/issue", os.O_RDONLY)
raw = open(fd, "rb", buffering=0)
text = io.TextIOWrapper(raw, line_buffering=False)
print(text.read(1))
---

Traceback (most recent call last):
  File "x.py", line 6, in 
print(text.read(1))
  File "/home/SHARE/SVN/py3k/Lib/io.py", line 1739, in read
eof = not self._read_chunk()
  File "/home/SHARE/SVN/py3k/Lib/io.py", line 1565, in _read_chunk
input_chunk = self.buffer.read1(self._CHUNK_SIZE)
AttributeError: 'FileIO' object has no attribute 'read1'

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4996] io.TextIOWrapper calls buffer.read1()

[Benjamin Peterson]

Benjamin Peterson  added the comment:

On Mon, Jan 19, 2009 at 11:59 AM, STINNER Victor  wrote:
>
> STINNER Victor  added the comment:
>
> Short example to reproduce the problem:
> ---
> import io, os
> fd = os.open("/etc/issue", os.O_RDONLY)
> raw = open(fd, "rb", buffering=0)
> text = io.TextIOWrapper(raw, line_buffering=False)
> print(text.read(1))

You can only use TextIOWrapper over a buffered stream, so this example
is invalid anyway.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4978] allow unicode keyword args

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

The patch works when the unicode contains only ascii, but crashes with
the following input:

>>> def f(hehe): return 42
... 
>>> f(**{u'hehe': 1})
42
>>> f(**{u'héhé': 1})
Segmentation fault

The two PyString_AsString(keyword) calls are now wrong and should be
changed.

--
nosy: +amaury.forgeotdarc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4996] io.TextIOWrapper calls buffer.read1()

[STINNER Victor]

STINNER Victor  added the comment:

I don't understand the motivation of having two different methods 
for "raw" streams: read1() and read(). I would prefer to have only a 
method read() (because "read" is the most common name, whereas read1() 
is only used on Python3), but read() will have to follow the read1() 
rule: at most one syscall. If someone requires a read() method with 
the "one syscall" rule, he can use a raw stream. If you don't care, 
use high level classes (TextIOWrapper, BufferedReader, etc.).

_fileio._FileIO.read() and _socket.socket.recv_into() already respect 
the syscall rule.

Since BytesIO and StringIO don't use kernel object, the syscall rule 
is meaningles.

Note: socket.SocketIO() implements RawIOBase but it has no read() nor 
write() methods ;-) That's maybe why IOBase has no read()/write() 
method.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4972] let's equip ftplib.FTP with __enter__ and __exit__

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

What is the rationale for swallowing all socket exceptions except 
"Connection reset by peer" in __exit__? In any case, it is better to use 
errno.ECONNRESET instead of literal 54.

Note that SMTP.quit() calls SMTP.close(), so in the normal termination 
case, close will be called twice.  This is not a real problem since 
SMTP.close() is a noop on a closed SMTP object, but it does not look 
right.

The double call to close() also makes error path harder to analyze.  It 
appears that if a socket error is raised in the first call to close, it 
gets caught only to be raised again in the second call (assuming a 
persistent error).

--
nosy: +belopolsky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4972] let's equip ftplib.FTP with __enter__ and __exit__

[Tarek Ziadé]

Tarek Ziadé  added the comment:

> What is the rationale for swallowing all socket exceptions except 
> "Connection reset by peer" in __exit__? 

I am catching just the error that raises if the connection is closed
when calling quit()


> In any case, it is better to use errno.ECONNRESET instead of literal 54.

Right

> Note that SMTP.quit() calls SMTP.close(), so in the normal termination 
> case, close will be called twice.  This is not a real problem since 
> SMTP.close() is a noop on a closed SMTP object, but it does not look 
> right.

Right i'll fix that

Thanks for teh feedback
The double call to close() also makes error path harder to analyze.  It 
appears that if a socket error is raised in the first call to close, it 
gets caught only to be raised again in the second call (assuming a 
persistent error).

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4972] let's equip ftplib.FTP with __enter__ and __exit__

[Tarek Ziadé]

Changes by Tarek Ziadé :


Added file: http://bugs.python.org/file12799/ftplib.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5002] multiprocessing/pipe_connection.c compiler warning (conn_poll)

[Hirokazu Yamamoto]

New submission from Hirokazu Yamamoto :

I got compiler warning "conn_poll takes too many arguments".
I hope the attached patch is right fix.

--
components: Library (Lib)
files: pipe_connection_c.patch
keywords: patch
messages: 80204
nosy: ocean-city
severity: normal
status: open
title: multiprocessing/pipe_connection.c compiler warning (conn_poll)
type: compile error
versions: Python 2.7
Added file: http://bugs.python.org/file12800/pipe_connection_c.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5000] multiprocessing - Pool.map() slower about 5 times than map() on 2 cores machine

[0x666]

0x666  added the comment:

Thanks for quick response and for informative answers, especially 
thanks to Antoine Pitrou/Amaury Forgeot.

p.s.
By seting bigger range - range(15) and elements = 1000, I was able to 
get speed-up factor up to 1.8. Wow, I`m amazed :-)
Good library, keep it up.
BTW, about warp-up,- forking first processes. User can forget 
to "warpup" pool. It would be cool if pool somehow warm-ups itself in 
initialization phase (in step mul.Pool(processes=mul.cpu_count())).
Maybe you can define another initialization parameter=function ? (if 
not None, first processes should be spawned, not waiting map function) 
or something like that. But it is only cosmetics :-)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4978] allow unicode keyword args

[Benjamin Peterson]

Benjamin Peterson  added the comment:

This patch should fix that.

Added file: http://bugs.python.org/file12801/unicode_keyword_args.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5003] Error while installing Python-3

[Sravani Gogineni]

New submission from Sravani Gogineni :

i was trying to install python 3 using the folowing command 
$ python setup.py --home=~/python3

and i got the below error:

  File "setup.py", line 232
except (CCompilerError, DistutilsError) as why:
 ^
SyntaxError: invalid syntax

Could anyone please help me with the problem?

Thanks,
Vani

--
components: Installation
messages: 80207
nosy: gogine...@corning.com
severity: normal
status: open
title: Error while installing Python-3
versions: Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5003] Error while installing Python-3

[Antoine Pitrou]

Antoine Pitrou  added the comment:

setup.py is not meant to be called directly (it will be called by the
Python 3 interpreter as part of the build process and is, therefore,
written in Python 3 syntax).

The proper command sequence to build and install Python is:

./configure --prefix=~/python3
make
make install

--
nosy: +pitrou
resolution:  -> invalid
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4804] Python on Windows disables all C runtime library assertions

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Here is what I think is a better attempt at the selective-disabling.  I 
did away with the messy macros and added a function in errors.c.  This 
maintans a global 'level' variable, implicitly guarded by the GIL.  Now 
only the thread that first enters a guarded block will set the hendler, 
and the thread that is last to leave will reset it.  This should 
guarantee that between the two macros, our custom handler is set, and 
that no thread pulls the rug from underneath another.

Added file: http://bugs.python.org/file12802/crterror.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4972] let's equip ftplib.FTP with __enter__ and __exit__

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

On Mon, Jan 19, 2009 at 2:01 PM, Tarek Ziadé  wrote:
>
> Tarek Ziadé  added the comment:
>
>> What is the rationale for swallowing all socket exceptions except
>> "Connection reset by peer" in __exit__?
>
> I am catching just the error that raises if the connection is closed
> when calling quit()
>

I see. I misread the double negative "except errno NOT equals 54", but
I still don't see the rationale for that exception.  I any case, I
don't think your patch implements that because SMTP transforms socket
errors into SMTPServer* errors in send():

if self.sock:
try:
self.sock.sendall(str)
except socket.error:
self.close()
raise SMTPServerDisconnected('Server not connected')
else:
raise SMTPServerDisconnected('please run connect() first')

so you will never see a socket error from quit().

Furthermore, I don't think you should ignore return code from quit():
you should raise an error if it returns anything but 221.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4862] utf-16 BOM is not skipped after seek(0)

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I support Amaury's suggestion (actually I implemented it in the io-c
branch). Resetting the decoder when seeking to the beginning of the
stream is a reasonable way to deal with those incremental decoders for
which the start state is something else than (b"", 0).

(and, you're right, opening in append mode is a different problem...)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4998] __slots__ on Fraction is useless

[Mark Dickinson]

Mark Dickinson  added the comment:

I believe that __slots__ was used for performance (memory, speed) reasons 
here rather than for preventing random attribute assignments.  But maybe 
inheriting from Rational invalidates those reasons as well...

--
nosy: +marketdickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4998] __slots__ on Fraction is useless

[Mark Dickinson]

Mark Dickinson  added the comment:

The Decimal class has the same issue in py3k (but not in the trunk).

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4448] should socket readline() use default_bufsize instead of _rbufsize?

[Gabriel Genellina]

Changes by Gabriel Genellina :


--
nosy: +gagenellina

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4804] Python on Windows disables all C runtime library assertions

[Martin v. Löwis]

Martin v. Löwis  added the comment:

> I understand thread-safe.  This usage is safe from Python threads 
> because it is all done in the context of the GIL.

No, it is not. See below.

> In your example, T2 doesn't hold the GIL and so, this is the scenario 
> that I believe you are invoking.

Assume that T2 is a Python thread. First, it doesn't hold the GIL.
However, when T1 releases the GIL to invoke the CRT, T2 can acquire
the GIL (which it had been waiting for, anyway), and then proceed
as described.

> Furthermore, since your argument assumes a rogue thread modifying the 
> CRT settings

No, it does not. Regular Python threads can break under this patch.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4804] Python on Windows disables all C runtime library assertions

[Martin v. Löwis]

Martin v. Löwis  added the comment:

> Ooops, ignore my last comment.

(sorry, too late - I only read this after responding to the earlier one)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4922] set.add and set.discard are not conformant to collections.MutableSet interface

[Georg Brandl]

Changes by Georg Brandl :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5004] socket.getfqdn() doesn't cope properly with purely DNS-based setups

[Daniel Frnake]

New submission from Daniel Frnake :

On Linux and presumably on other POSIX-like systems, socket.getfqdn()
doesn't work if a system resolves its own FQDN using DNS rather than
/etc/hosts.

My system's FQDN is 'fugue.tank.wellohorld.com'.  My /etc/hosts is empty
except for loopback entries, and /etc/resolv.conf contains the line
'domain tank.wellohorld.com'.  This is sufficient for 'hostname -f' to
do the Right Thing, but socket.getfqdn() simply returns 'fugue':

dfra...@fugue:~/Python-2.6.1$ hostname
fugue
dfra...@fugue:~/Python-2.6.1$ hostname -f
fugue.tank.wellohorld.com
dfra...@fugue:~/Python-2.6.1$ ./python
Python 2.6.1 (r261:67515, Jan 19 2009, 13:56:59)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.getfqdn()
'fugue'
>>>
dfra...@fugue:~/Python-2.6.1$ echo -e '$a\n172.17.0.120
fugue.tank.wellohorld.com fugue\n.\nwq' | sudo ed /etc/hosts
305
350
dfra...@fugue:~/Python-2.6.1$ ./python
Python 2.6.1 (r261:67515, Jan 19 2009, 13:56:59)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.getfqdn()
'fugue.tank.wellohorld.com'
>>>
dfra...@fugue:~/Python-2.6.1$

--
components: Library (Lib)
messages: 80216
nosy: dfranke
severity: normal
status: open
title: socket.getfqdn() doesn't cope properly with purely DNS-based setups
type: behavior
versions: Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5004] socket.getfqdn() doesn't cope properly with purely DNS-based setups

[Daniel Franke]

Changes by Daniel Franke :


--
components: +Extension Modules -Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4978] allow unicode keyword args

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

Sorry, but _PyUnicode_AsDefaultEncodedString returns either a borrowed 
reference (if 
errors==NULL) or a new reference (if errors!=NULL).
In either case it's wrong to DECREF the string when you simply take the 
buffer's 
address:

>>> f(**{u'someLongString':2})
Traceback (most recent call last):
  File "", line 1, in 
TypeError: f() got an unexpected keyword argument 'ÛÛÛgString'

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5005] 3.0 sqlite doc: most refers to pysqlite2, use 2.x syntax.

[Terry J. Reedy]

New submission from Terry J. Reedy :

1. Several examples start with
>>> from pysqlite2 import dbapi2 as sqlite3

Traceback (most recent call last):
  File "", line 1, in 
from pysqlite2 import dbapi2 as sqlite3
ImportError: No module named pysqlite2

I presume that should be just 'import sqlite3'.

2. Print statements need conversion and future import deleted (2to3?).

3. Cut and paste crashes due to 'bad' blank line.  For instance,
# A minimal SQLite shell for experiments
...
>>> while True:
... line = raw_input()
... if line == "":
... break
... buffer += line
... if sqlite3.complete_statement(buffer):
... try:
... buffer = buffer.strip()
... cur.execute(buffer)
...
  File "", line 10

^
IndentationError: unexpected unindent
>>> if buffer.lstrip().upper().startswith("SELECT"):
  File "", line 1
if buffer.lstrip().upper().startswith("SELECT"):
^
IndentationError: unexpected indent

--
messages: 80218
nosy: tjreedy
severity: normal
status: open
title: 3.0 sqlite doc: most refers to pysqlite2, use 2.x syntax.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5002] multiprocessing/pipe_connection.c compiler warning (conn_poll)

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

On Windows only, since r68768. Patch seems OK to me.

--
assignee:  -> jnoller
nosy: +amaury.forgeotdarc, jnoller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4995] sqlite3 module gives SQL logic error only in transactions

[Muayyad Alsadi]

Muayyad Alsadi  added the comment:

can you please tell me how to detect the version so that my application
will not crash on Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38) 

which of these values should I use
apilevel = '2.0'
sqlite_version = '3.5.9'
sqlite_version_info = (3, 5, 9)
version = '2.3.2'
version_info = (2, 3, 2)
x = '9'

that are different in your python 2.6
for example
import sqlite3
if sqlite3.version_info<(3,0,0): c.execute('BEGIN TRANSACTION')

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5006] Duplicate UTF-16 BOM if a file is open in append mode

[STINNER Victor]

New submission from STINNER Victor :

Copy/paste of message79330 from the issue #4862:
--
>>> f = open('utf16.txt', 'w', encoding='utf-16')
>>> f.write('abc')
3
>>> f.close()

>>> f = open('utf16.txt', 'a', encoding='utf-16')
>>> f.write('def')
3
>>> f.close()
>>> open('utf16.txt', 'r', encoding='utf-16').read()
'abc\ufeffdef'
--

--
messages: 80221
nosy: haypo
severity: normal
status: open
title: Duplicate UTF-16 BOM if a file is open in append mode
versions: Python 3.0, Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4862] utf-16 BOM is not skipped after seek(0)

[STINNER Victor]

STINNER Victor  added the comment:

I opened a different issue (#5006) for the duplicate BOM in append 
mode.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5006] Duplicate UTF-16 BOM if a file is open in append mode

[STINNER Victor]

STINNER Victor  added the comment:

Bug is reproductible with:
 * Python 2.5 : charset utf-8-sig and utf-16 for codecs.open()
 * trunk : charset utf-8-sig, utf-16 and utf-32 for codecs.open()
 * py3k : charset utf-8-sig, utf-16 and utf-32 for open()

With utf-7 or utf-8, no BOM is written.

Note: with UTF-32, the BOM is 4 bytes long (0xff 0xfe 0x00 0x00 on 
little endian) but it's still the character (BOM) \ufeff (little 
endian).

--
versions: +Python 2.6, Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5007] urllib2 HTTPS connection failure (BadStatusLine Exception)

[ak]

New submission from ak :

https://www.orange.sk/ is served by an Oracle HTTPS server, and works
with firefox or opera but not urllib2.

code snippet:


import cookiejar
import urllib2

cookiejar = cookielib.LWPCookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookiejar))
url = 'https://www.orange.sk/'
req = urllib2.Request(url, None)
s=opener.open(req)
 

Here's some output with debuglevel=1 :

>>> opener.open(urllib2.Request('http://www.orange.sk/', None, headers))

reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Mon, 19 Jan 2009 21:44:03 GMT
header: Server: Oracle-Application-Server-10g/10.1.3.1.0 Oracle-HTTP-
Server
header: Set-Cookie:
JSESSIONID=0a19055a30d630c427bda71d4e26a37ca604b9f590dc.e3eNaNiRah4Pe3aSch8Sc3yOc40;
path=/web
header: Expires: Mon, 19 Jan 2009 21:44:13 GMT
header: Surrogate-Control: max-age="10"
header: Content-Type: text/html; charset=ISO-8859-2
header: X-Cache: MISS from www.orange.sk
header: Connection: close
header: Transfer-Encoding: chunked
>

>>> opener.open(urllib2.Request('https://www.orange.sk/', None, headers))

reply: ''
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.5/urllib2.py", line 381, in open
response = self._open(req, data)
  File "/usr/lib/python2.5/urllib2.py", line 399, in _open
'_open', req)
  File "/usr/lib/python2.5/urllib2.py", line 360, in _call_chain
result = func(*args)
  File "/usr/lib/python2.5/urllib2.py", line 1115, in https_open
return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.5/urllib2.py", line 1080, in do_open
r = h.getresponse()
  File "/usr/lib/python2.5/httplib.py", line 928, in getresponse
response.begin()
  File "/usr/lib/python2.5/httplib.py", line 385, in begin
version, status, reason = self._read_status()
  File "/usr/lib/python2.5/httplib.py", line 349, in _read_status
raise BadStatusLine(line)
httplib.BadStatusLine 

As you can see the reply from the server seems empty (which results in
the BadStatusLine exception)

--
components: Library (Lib)
messages: 80224
nosy: ak
severity: normal
status: open
title: urllib2 HTTPS connection failure (BadStatusLine Exception)
type: behavior
versions: Python 2.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com