Re: [Python-Dev] Py3k and asyncore/asynchat

2008-03-31 Thread Neal Norwitz 
On Sun, Mar 30, 2008 at 7:44 PM, Josiah Carlson
<[EMAIL PROTECTED]> wrote:
>
>  I haven't really had time to update the tests/documentation, but
>  again, I wasn't experiencing any strange test failures with
>  asyncore/asynchat, nor have I been able to find the buildbot failures
>  that you are referring to.  Could someone please link the failures
>  that are not related to being unable to discover a port number?

3 different platforms, 3 different problems:

http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/2790/step-test/0
http://www.python.org/dev/buildbot/all/x86%20FreeBSD%203%203.0/builds/0/step-test/0
http://www.python.org/dev/buildbot/all/x86%20XP-4%203.0/builds/643

>  According to the release schedule, we should have at least a couple
>  more months for documentation and tests to be updated (I can get
>  patches ready for alpha 3).

When you get the patches with tests and doc, I'll be happy to check in.

n
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] FW: [issue2513] 64bit cross compilation on windows

2008-03-31 Thread Mark Hammond 
FYI, I've uploaded a patch that provides for cross-compilation on Windows 
between 32 and 64 bit platforms - all comments invited!

Thanks,

Mark
-Original Message-
From: Mark Hammond [mailto:[EMAIL PROTECTED] 
Sent: Sunday, 30 March 2008 6:01 PM
To: [EMAIL PROTECTED]
Subject: [issue2513] 64bit cross compilation on windows


New submission from Mark Hammond <[EMAIL PROTECTED]>:

I've taken the liberty of adding Trent, Christian and Martin to the nosy
list as I know they are actively, if reluctantly interested in this.

This patch allows the distutils to cross-compile on Windows.  It has
been tested on x86 and amd64 platforms, with both platforms successfully
able to natively and cross-compile extension modules and create binary
distributions.

To cross-compile, specify '--plat-name' to the build command (valid
values are 'win32', 'win-amd64' and 'win-ia64').  This option name was
chosen to be consistent with the bdist_dumb command.  I've included the
docs I added below (which are also in the patch), but note that as with
native compilation using distutils, it's not necessary to set any
environment variables or do anything else special with your environment
to make this work.

The patch also adds a x64 target for the 'bdist_wininst' target, which
it creates as distutils/command/wininst-9.0-amd64.exe.  This executable
is necessary even for bdist_wininst to work natively on x64, but is
still included here for simplicity.

To assist with testing, I've also added a distutils setup.py script to
the PC/example_nt directory.  This is capable of creating bdist_wininst
executables for both native and cross platforms; 'setup.py build
--platname=win-amd64 bdist_wininst' will create an amd64 installer on an
x86 machine.

The patch has not been tested with a Visual Studio environment without
cross-compile tools installed - it will obviously fail, but its not
clear how ugly this failure will be.

Below is the text I added to docs/distutils/builtdist.rst:

  Cross-compiling on Windows
  =

  Starting with Python 2.6, distutils is capable of cross-compiling
between Windows platforms.  In practice, this means that with the
correct tools installed, you can use a 32bit version of Windows to
create 64bit extensions and vice-versa.

  To build for an alternate platform, specify the :option:`--plat-name`
option to the build command.  Valid values are currently 'win32',
'win-amd64' and 'win-ia64'.  For example, on a 32bit version of Windows,
you could execute::

 python setup.py build --plat-name=win-amd64

  to build a 64bit version of your extension.  The Windows Installers
also support this option, so the command::

 python setup.py build --plat-name=win-amd64 bdist_wininst

  would create a 64bit installation executable on your 32bit version of
Windows.

  Note that by default, Visual Studio 2008 does not install 64bit
compilers or tools.  You may need to reexecute the Visual Studio setup
process and select these tools.

--
assignee: mhammond
components: Distutils
files: windows-cross-compile.patch
keywords: 64bit, patch
messages: 64743
nosy: Trent.Nelson, ctheune, loewis, mhammond
severity: normal
status: open
title: 64bit cross compilation on windows
type: feature request
versions: Python 2.6
Added file: http://bugs.python.org/file9900/windows-cross-compile.patch

__
Tracker <[EMAIL PROTECTED]>

__

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] No time for svn merge

2008-03-31 Thread Facundo Batista 
2008/3/30, "Martin v. Löwis" <[EMAIL PROTECTED]>:

> > If you'd like, I can merge the rest.
>
> If you have the time to figure it all out, sure.
>  I found that quite a tedious task, and had to spent
>  on some patches quite a long time to figure out what
>  they do, and what the 3.x equivalent should be.

Me for myself, I thought that the trunk -> 3k merge was easier!

Sometimes I commited changes to the trunk, don't worrying about 3k at
all,  thinking it was a mostly automatic process.

Now that I know this, I will start patching both trunk & 3k simultaneously...

Regards,

-- 
.Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] No time for svn merge

2008-03-31 Thread Christian Heimes 
Facundo Batista schrieb:
> Me for myself, I thought that the trunk -> 3k merge was easier!
> 
> Sometimes I commited changes to the trunk, don't worrying about 3k at
> all,  thinking it was a mostly automatic process.
> 
> Now that I know this, I will start patching both trunk & 3k simultaneously...

In most cases it's easy. Usually it takes me less than 20 minutes per
day to merge the chances from trunk -> py3k. In this particular case
several obstacles come together. The changes in the AST and parser code
aren't trivial, I'm not familiar with the internals of the AST and
parser code and my free time is very limited.

Please don't patch trunk and py3k simultaneously. It may confuse svn and
may make future merges harder. You should apply the patch to the trunk
and either merge or block the revision with svnmerge.

Example:

# Commit a patch to the trunk
~$ cd python/trunk
trunk$ svn ci -m "Message"
...
Committed revision 1234.

# Merge the revision to the py3k branch
trunk $ cd ../py3k
py3k $ svnmerge.py merge -r 1234
...

# resolve conflicts
...

# commit the merge
py3k $ svn ci -F svnmerge-commit-message.txt


Christian
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] Python source code on Bazaar vcs

2008-03-31 Thread Barry Warsaw 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

- -BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mar 30, 2008, at 3:35 AM, Martin v. Löwis wrote:

> Likewise, if you are accessing the repository over bzr+ssh, in  
> general,
> you can just ssh to the machine, and do a regular ls. In the specific
> setup, regular ssh is restricted to running "bzr serve", which
> (apparently) has no support for directory browsing.

I suppose we could enable sftp access to user branches on  
code.python.org.  You wouldn't want to use sftp:// protocol to do bzr,  
even though you could, because bzr+ssh is more efficient.

- - -Barry

- -BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR+/04HEjvBPtnXfVAQI6gAQApMnG/h4qCoSx5rxd6US03DN01hh2ef2N
l7tNnQCJMAd2+By5z/cn/BY2vehZmVZEFhgNsGzhAOHZNbsbFG2kBdJkT5PLCOQN
ZGqdYCgbMbHhDLsPG0XfWorOgpeER/7WlesF/VXGbbGiZqbWYFZVbprM6M7n4O/y
V5RzsxpHO/s=
=Po3X
- -END PGP SIGNATURE-
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR/DZ1XEjvBPtnXfVAQJm6gQAjnNOy8HjE90WjbzPgJtSS+S6W9Z2lBGm
IFxSFElDv6D59G67qikdHHlJfn/eCSrl1QB158AucGMCIWDRFM/d5UOTfv6a7vcQ
pLpJvWD1DqvvbvKZi7CkRIpcTn80YATAhJ4mOTaJ7zlC6m9OHJwkLZnxDRJdmrkJ
xv5hZzqVTS0=
=fcSk
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] No time for svn merge

2008-03-31 Thread Facundo Batista 
2008/3/31, Christian Heimes <[EMAIL PROTECTED]>:

> In most cases it's easy. Usually it takes me less than 20 minutes per
>  day to merge the chances from trunk -> py3k. In this particular case
>  several obstacles come together. The changes in the AST and parser code
>  aren't trivial, I'm not familiar with the internals of the AST and
>  parser code and my free time is very limited.
>
>  Please don't patch trunk and py3k simultaneously. It may confuse svn and
>  may make future merges harder. You should apply the patch to the trunk
>  and either merge or block the revision with svnmerge.

Mmmm... so, I'll do this. I'll just commit on the trunk, but know
people that I'm here if you need to ask anything about commited stuff
or whatever ("here" means through IM, and normally hanging around in
#python-dev or #pyar in freenode).

Regards,

-- 
.Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Next alphas, this Wednesday

2008-03-31 Thread Barry Warsaw 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Just another heartbeat reminder that I intend to release the next  
alphas for Python 2.6 and 3.0 this Wednesday April 2nd, at  
approximately 6pm Eastern time (UTC 2200).

Current status: the buildbots for both the trunk and 3.0 look  
relatively good, though a few are purple or red:

http://www.python.org/dev/buildbot/stable/

There is currently one release blocker bug for 3.0:

http://bugs.python.org/issue2507

I'll be looking at these in more detail later today.  If you have  
time, feel free to comment on the bug or send a follow up about the  
stable buildbots.  Please try to be very conservative in your commits  
to the trunk and 3.0 over the next few days.   Concentrate on fixing  
existing code rather than committing new features.  Your release  
manager thanks you for your diligence! :)

Cheers,
- -Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBR/Dh0HEjvBPtnXfVAQL3HgQAnk/eoRyfrF0RQcCKCfhNyfpfc7KGbPMW
Y46xZy/yzySPbUDLP4YhTs3hhjt4hfZEHYagWV50Yy0Wtak0vDMj1Fr+8jxIptR0
Qh0zQhop2OQPZVT5S5TOgOVlXAj9nYITDpfnFfogoNo4PaEG8wNvlBZKcxN+8cfv
KkSrM6Sm4Rw=
=rOZw
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] Python source code on Bazaar vcs

2008-03-31 Thread 
[EMAIL PROTECTED] wrote:
> Benjamin> Once you've pushed the branches, is there a way to remove them?
> 
> Related question: is there a way to view the various branches in a non-local
> repository?

IIUC, conceptually, no. A branch is not *in* a repository; a branch *is*
a repository (*). So your question is almost equivalent to "is there a
way to find out all clones of a repository that have ever been made?",
to which the answer is "no".

Now, if you have a convention of where you put the branches, you should
be able to find them later. E.g. if they are all in a directory tree
that is exposed through http, you can use a web browser to see them,
e.g. by going to

http://code.python.org/python/users/skip/

Likewise, if you are accessing the repository over bzr+ssh, in general,
you can just ssh to the machine, and do a regular ls. In the specific
setup, regular ssh is restricted to running "bzr serve", which
(apparently) has no support for directory browsing.

Regards,
Martin
___
Python-3000 mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-3000
Unsubscribe: 
http://mail.python.org/mailman/options/python-3000/fumanchu%40aminus.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Thread-safe file objects, the return

2008-03-31 Thread Antoine Pitrou 

Hello,

It seems this subject has had quite a bit of history. Tim Peters demonstrated
the problem in 2003 in this message:
http://mail.python.org/pipermail/python-dev/2003-June/036537.html

In short, Python file objects release the GIL before calling any C stdlib
function on their embedded FILE pointer. Unfortunately, if another thread
calls fclose on the FILE pointer concurrently, the contents pointed to can
become garbage and the interpreter process crashes. Just by using the same
file object in two threads running pure Python code, you can crash the
interpreter.

(another, easier-to-solve problem is that the FILE pointer stored in the
file object could become NULL at the point it is used by another thread.
If that was the only problem you could just store the FILE pointer in a
local variable before releasing the GIL et voilà)

There was some discussion at the time about the possible resolution. I've
tried to fix the problem, and I've come to what I think is a satisfying
solution, which I can sum up as the following bullet points:
 * Each file object gets a dedicated counter, which is incremented before
the bject releases the GIL and decremented after the GIL is taken again; thus
this counter keeps track of how many running "unlocked" sections of code are
using that particular file object. (please note the counter doesn't need its
own lock, since it is only modified in GIL-protected sections)
 * In the close() method, if the aforementioned counter is greater than 0,
we refuse to call fclose and instead raise an IOError.

This may seem like a worrying semantic change, but I don't think it is, for the
following reasons:
 1) if we closed the FILE pointer anyway, the interpreter would likely crash
because another thread would be using garbage data (that's what we are trying
to fix after all!)
 2) if close() raises an IOError, it can be called again later, or at worse
fclose will be called when the file object is garbage collected
 3) close() can already raise an IOError if fclose fails for whatever reason
(although for sure it's probably very rare)
 4) it doesn't seem wrong to notify the programmer that his code is very
unsafe

The patch is attached at http://bugs.python.org/issue815646 . It addresses
(or at least I hope it does) all potential problems with pure Python code,
threads, and the file object. It doesn't try to fix C extensions using the
PyFile_AsFile API and doing their own dirty things with the FILE pointer. It
could be a second step if the approach is accepted, but as noted in the 2003
discussions it would probably involve a new API. Whether we want to introduce
such an API in Python 2.x while Python 3.0 has a different IO model anyway
is left open to discussion :)

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] refleaks on trunk

2008-03-31 Thread Neal Norwitz 
test_io is the only leaky test on trunk that I know of.  I narrowed
down the leaks to the code below.

It's possible there are other leaks in test_io.

n
--
import sys, gc
import _fileio, io

class FileIO(_fileio._FileIO, io.RawIOBase):
def close(self):
io.RawIOBase.close(self)

def main():
  class MyFileIO(FileIO): pass
  f = MyFileIO('tt-test', "w")
  f.close()

for i in range(10):
  main()
  print(gc.collect())
  print(sys.gettotalrefcount())
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] fixing tests on windows

2008-03-31 Thread Neal Norwitz 
The Windows buildbots are still failing because some tests keep files
opened.  This causes subsequent tests which use the same file to fail.

Here is a recent run which had a failure early on:
http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1209/step-test/0

I'm assuming the first failure (test_bufio) was due to an open file.
(Can't tell, no error msg.)  That means the problem was in that test
or an earlier test.  The only earlier tests are:

test_import
test_sys
test_descr
test_xdrlib
test_urllibnet
test_binhex
test_strptime
test_importhooks
test_copy
test_hmac
test_genericpath
test_complex
test_timeout
test_quopri
test_marshal
test_zipfile
test_mutants
test_csv
test_ucn
test_hash
test_wsgiref
test_mmap
test_ftplib
test_pickletools
test_codecmaps_cn

Of those tests, only some of them reference TESTFN which is the probable file:

test_import 22
test_descr 3
test_urllibnet 2
test_binhex 2
test_genericpath 44
test_complex 3
test_marshal 33
test_zipfile 136
test_mutants 7
test_mmap 32

I inspected test_mmap which had been modified somewhat recently and
tried to fix a few things.  I'm not sure I helped and there are
probably more problems lurking.

This issue may not be noticed in the default test run.  It definitely
occurs when the tests are run in a random order as they are on the
buildbots.  Try running:  ./python.exe ./Lib/test/regrtest.py -r

That should help provoke more errors.

As always, patches are greatly appreciated.

Cheers,
n
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com