Re: [Python-Dev] Taking over the Mercurial Migration
Am 30.06.2010 07:37, schrieb "Martin v. Löwis": > It seems that both Dirkjan and Brett are very caught up > with real life for the coming months. So I suggest that > some other committer who favors the Mercurial transition > steps forward and takes over this project. > > If nobody volunteers, I propose that we release 3.2 > from Subversion, and reconsider Mercurial migration > next year. IIUC, Dirkjan is only caught up for another month. I have no problems with releasing a first 3.2 alpha from SVN and then switching, so I propose that we target the migration for August -- I can help in the second half of August if needed. Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle security and remote logging
Guido van Rossum python.org> writes: > As for protocol buffers, assuming its absence (so far from the > stdlib is the only objection, how hard would it be to make the logging > package "prepared" so that if one *did* have protocol buffers > installed, it would be a one-line config setting to use them? I envisage that if protocol buffers were available, and if support for them in logging was to be added, this could be done via an optional keyword arg to the SocketHandler which sets a handler attribute, which would then be used in makePickle to make the required serialized form. @anatoly: The documentation just mentions HMAC as an example; the levels of paranoia to be applied are different for different people, different times and different situations ;-) I assume that someone reading the docs could readily see that they could substitute "sign the pickle" with some alternative strategy in makePickle. You could implement marshal, protocol buffers etc. right now just by overriding SocketHandler.makePickle in your custom class. An alternative strategy would be to provide an optional serializer=None callable in the SocketHandler constructor. If specified, then makePickle would call this serializer with the LogRecord instance as the only argument, and use the return value as the serialized form, instead of calling pickle.dumps. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On 05:24 am, mar...@v.loewis.de wrote: Seems to work fine. So this I don't understand. Any ideas, anyone? Didn't we discuss this before? The buildbot slave has no controlling terminal anymore, hence it cannot open /dev/tty. If you are curious, just patch your checkout to output the exact errno (e.g. to stdout), and trigger a build through the web. Could the test be rewritten (or supplemented) to use a pty? Most or perhaps all of the same operations should be supported. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mailbox module - timings and functionality changes
R. David Murray wrote: > On Tue, 29 Jun 2010 17:02:14 -0400, Steve Holden wrote: >> Guido van Rossum wrote: >> >>> - wrap the binary stream in a text stream >> "wrap" how? The ultimate destiny of the text is twofold: > > I would imagine Guido is talking about an io.TextIOWrapper...in other > words, take the binary file you've just finished grabbing info > from, and reread it as a text file in order to grab the actual > message content. > > If you have messages in your files that are using an 8bit content > transfer encoding, then you (currently) will have some problems > unless the charset happens to be the one you use when you wrap > the binary stream as a text stream. > http://bugs.python.org/issue9124 regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
Guido van Rossum wrote:
> On Tue, Jun 29, 2010 at 7:55 PM, Bill Janssen wrote:
> > My Leopard and Tiger PPC buildbots are momentarily green! But I'm
> > looking into why I'm skipping some tests. My buildbots are up-to-date
> > OS-wise and very vanilla, with the latest applicable Xcode.
> >
> > 4 skips unexpected on darwin:
> > test_gdb test_ioctl test_readline test_ttk_guionly
> >
> > Three of these (gdb, readline, ttk_guionly) are just bad predictions of
> > which tests should skip on Darwin, I think -- gdb is only version 6, so
> > that test won't run, readline doesn't get built, ttk doesn't work
> > without Tcl/Tk 8.5.
>
> So it looks like you gould get readline and ttk to run and pass by
> separately downloading and installing readline (I've done this many
> times before) and Tcl/Tk (no idea but I suppose it should work).
Sure. But the skips should be expected "on Darwin", since a vanilla OS
X system apparently won't have the necessary bits. At the very least,
regrtest.py should test for these conditions and add them to the
"expected skips" list if necessary. I'll work up a patch.
> > But the the skip of test_ioctl baffles me.
> >
> > "test_ioctl skipped -- Unable to open /dev/tty"
> >
> > But when I log in via ssh and try it with the system python:
> >
> > ~ wjanssen$ python
> > python
> > Python 2.5.1 (r251:54863, Jun 17 2009, 20:37:34)
> > [GCC 4.0.1 (Apple Inc. build 5465)] on darwin
> > Type "help", "copyright", "credits" or "license" for more information.
> open("/dev/tty")
> > open("/dev/tty")
> >
>
> >
> > Seems to work fine. So this I don't understand. Any ideas, anyone?
>
> Maybe the buildbot runs the tests as a tty-less daemon process. If you
> ask me it's pretty crazy to have a test that requires a tty. But there
> you have it -- and it's the same in Python 3. (But then again, who
> knows, I might have written that test. ;-)
So, my question then is, why are these skips "unexpected"? Seems to me
that if this is the case, this test will never run on any platform.
Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
Martin v. Löwis wrote: > > Seems to work fine. So this I don't understand. Any ideas, anyone? > > Didn't we discuss this before? Possibly, but I don't recall doing so. > The buildbot slave has no controlling > terminal anymore, hence it cannot open /dev/tty. If you are curious, > just patch your checkout to output the exact errno (e.g. to stdout), > and trigger a build through the web. So, why is skipping this test "unexpected"? I see "x86 Tiger" is also showing this as an unexpected skip. Should I just add it to the list of expected skips on Darwin? Actually, will it run on any platform? Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
exar...@twistedmatrix.com wrote:
> Could the test be rewritten (or supplemented) to use a pty? Most or
> perhaps all of the same operations should be supported.
Buildbot seems to be explicitly not using a PTY. From the the top of
the test output:
make buildbottest
in dir /Users/buildbot/buildarea/trunk.parc-leopard-1/build (timeout 1800 secs)
watching logfiles {}
argv: ['make', 'buildbottest']
[...]
closing stdin
using PTY: False
I believe this is specified by the build master.
This test seems to work on Ubuntu and FreeBSD, though.
Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mailbox module - timings and functionality changes
On Tue, 29 Jun 2010 20:05:29 -0400 "R. David Murray" wrote: > > I would imagine Guido is talking about an io.TextIOWrapper...in other > words, take the binary file you've just finished grabbing info > from, and reread it as a text file in order to grab the actual > message content. This sounds a bit suboptimal to me (and introduces race conditions if e.g. the file is replaced with another one before you reopen it). You could instead decode the binary data by yourself, especially if you have already stored that data somewhere. Also, please note that values used by seek() and tell() on text I/O are "opaque cookies". While they can happen to match the raw binary file position, it is a mere coincidence (or an implementation detail, at your will). Therefore, reusing tell() values of a binary file to seek() a TextIOWrapper accessing the same file is wrong. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On Wed, 30 Jun 2010 09:00:09 PDT Bill Janssen wrote: > > So, my question then is, why are these skips "unexpected"? Seems to me > that if this is the case, this test will never run on any platform. You can change the value of the "usepty" option in your buildbot.tac. (you will also have to restart the buildslave process) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mailbox module - timings and functionality changes
On Wed, Jun 30, 2010 at 9:42 AM, Antoine Pitrou wrote: > On Tue, 29 Jun 2010 20:05:29 -0400 > "R. David Murray" wrote: >> >> I would imagine Guido is talking about an io.TextIOWrapper...in other >> words, take the binary file you've just finished grabbing info >> from, and reread it as a text file in order to grab the actual >> message content. > > This sounds a bit suboptimal to me (and introduces race conditions if > e.g. the file is replaced with another one before you reopen it). You > could instead decode the binary data by yourself, especially if you > have already stored that data somewhere. That's why I proposed not reopening but wrapping. Of course the contents of the file could still change, but that's a limitation of how the mailbox module works -- it builds a TOC and expects the file not to change. > Also, please note that values used by seek() and tell() on > text I/O are "opaque cookies". While they can happen to match the > raw binary file position, it is a mere coincidence (or an > implementation detail, at your will). Therefore, reusing tell() values > of a binary file to seek() a TextIOWrapper accessing the same file > is wrong. Well, um, I actually designed it carefully so that bytes offsets *would* work as text offsets in those cases where they make sense at all. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] TextIOWrapper.tell()
On Wed, 30 Jun 2010 10:03:49 -0700 Guido van Rossum wrote: > > > Also, please note that values used by seek() and tell() on > > text I/O are "opaque cookies". While they can happen to match the > > raw binary file position, it is a mere coincidence (or an > > implementation detail, at your will). Therefore, reusing tell() values > > of a binary file to seek() a TextIOWrapper accessing the same file > > is wrong. > > Well, um, I actually designed it carefully so that bytes offsets > *would* work as text offsets in those cases where they make sense at > all. Ah, this is embarrassing. I always assumed it was an implementation detail since neither the PEP nor the module docs say otherwise. PEP 3116 clearly says: “Unlike with raw I/O, the units for .seek() are not specified - some implementations (e.g. StringIO) use characters and others (e.g. TextIOWrapper) use bytes.” And also: “.seek(pos: object, whence: int = 0) -> int Seek to position pos. If pos is non-zero, it must be a cookie returned from .tell() and whence must be zero.” “it must be a cookie returned from .tell()” here seems to imply that non-zero values of other origin should not be used. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] TextIOWrapper.tell()
On Wed, Jun 30, 2010 at 10:20 AM, Antoine Pitrou wrote: > On Wed, 30 Jun 2010 10:03:49 -0700 > Guido van Rossum wrote: >> >> > Also, please note that values used by seek() and tell() on >> > text I/O are "opaque cookies". While they can happen to match the >> > raw binary file position, it is a mere coincidence (or an >> > implementation detail, at your will). Therefore, reusing tell() values >> > of a binary file to seek() a TextIOWrapper accessing the same file >> > is wrong. >> >> Well, um, I actually designed it carefully so that bytes offsets >> *would* work as text offsets in those cases where they make sense at >> all. > > Ah, this is embarrassing. I always assumed it was an implementation > detail since neither the PEP nor the module docs say otherwise. > > PEP 3116 clearly says: > > “Unlike with raw I/O, the units for .seek() are not specified - some > implementations (e.g. StringIO) use characters and others (e.g. > TextIOWrapper) use bytes.” > > And also: > > “.seek(pos: object, whence: int = 0) -> int > > Seek to position pos. If pos is non-zero, it must be a cookie > returned from .tell() and whence must be zero.” > > “it must be a cookie returned from .tell()” here seems to imply that > non-zero values of other origin should not be used. Guilty as charged. I really did take care that it would work, but forgot to mention it. I guess we can depend on this property *inside* the stdlib (as long as there are tests for each piece of code depending on it that would break if it ever changed) but should not advertise it widely. Note that it doesn't go the other way -- due to encoding state, text streams can certainly return cookies that make no sense to binary streams. But text streams take byte offsets too and do the best they can. (Obviously if a byte offset points in the middle of a multibyte character all bets are off.) The C stdlib has a similar thing -- while AFAIK POSIX lseek() really is required to return and take byte offsets, this is not required for fseek() and ftell() according to the C std -- but I think it's still a pretty safe bet, and I betcha lots of apps are making this assumption. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
Am 30.06.2010 13:32, schrieb exar...@twistedmatrix.com: > On 05:24 am, mar...@v.loewis.de wrote: >>> Seems to work fine. So this I don't understand. Any ideas, anyone? >> >> Didn't we discuss this before? The buildbot slave has no controlling >> terminal anymore, hence it cannot open /dev/tty. If you are curious, >> just patch your checkout to output the exact errno (e.g. to stdout), >> and trigger a build through the web. > > Could the test be rewritten (or supplemented) to use a pty? Most or > perhaps all of the same operations should be supported. I'm not sure. It uses TIOCGPGRP, basically to establish that ioctl can also put results into a Python array (IIUC). This goes back to http://bugs.python.org/555817 Somebody rewriting it would need to make sure the original test purpose is still met. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
I'm trying to catch up on this thread, so I may collapse some responses or refer to points others have brought up. On Jun 24, 2010, at 05:53 PM, Scott Dial wrote: >If the package has .so files that aren't compatible with other version >of python, then what is the motivation for placing that in a shared >location (since it can't actually be shared)? I think Matthias has described the motivation for the Debian/Ubuntu case, and James describes Python's current search algorithm for a packages .py[c] and .so files. There are a few points that you've made that I want to respond to. You claim that versioned .so files scheme is "more complicated" than multiple version-specific search paths (if I understand your counter proposal correctly). It all depends on your point of view. From mine, a 100 line patch that almost nobody but (some) distros will care about or be affected by, and that only changes a fairly obscure build-time configuration, is much simpler than trying to make version-specific search paths work. If you build Python from source, you do not care about this patch and you'll never see its effects. If you get Python on a distribution that only gives you one version of Python at a time, you also will probably never care or see the effects of this patch. If you're a Debian or Ubuntu user who wants to use Python 3.2 and 3.3, you *might* care about it, but most likely it'll just work behind the scenes. If you're a Python packager or work on the Python infrastructure for one of those platforms, then you will care. About just sharing the py files. You say that would be acceptable to you, but it's actually a pretty big deal. If you're supporting two versions of Python, then every distro Python package doubles in size. Even with compression, you're talking longer download times and probably more critically, you've greatly increased CDROM space pressures. The Ubuntu CDROM is already essentially at capacity so doubling the size of all Python packages (most of which btw do not have extension modules) makes such an approach impossible. Moving to a DVD image has been discussed, but it is currently believed not in the best interest of users, especially on slow links, to do so at this time. The versioned .so approach will of course increase the size of packages by twice the contained .so file size, and that's already an uncomfortable but acceptable increase. It's acceptable because of the gain users get by having multiple versions of Python available and the fact that there aren't nearly as many extension modules as there are Python files. Doubling the size of .py files as well isn't acceptable. >But the only motivation for doing this with .pyc files is that the .py >files are able to be shared, since the .pyc is an on-demand-generated, >version-specific artifact (and not the source). The .so file is created >offline by another toolchain, is version-specific, and presumably you >are not suggesting that Python generate it on-demand. Definitely not. pyc files are generated upon installation of the distro package, but of course the .so files must be compiled on a build machine and included in the distro package. The whole process is much simpler if the versioned .so files can just live in the same directory. >For packages that have .so files, won't the distro already have to build >multiple copies of that package for all version of Python? So, why can't >it place them in separate directories that are version-specific at that >time? This is not the same as placing .py files that are >version-agnostic into a version-agnostic location. It's not a matter of "could", it's a matter of simplicity, and I think versioned .so files are the simplest solution given all the constraints. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Jun 26, 2010, at 10:22 PM, Matthias Klose wrote: >On 24.06.2010 22:46, Barry Warsaw wrote: >> So, we could say that PEP 384 compliant extension modules would get written >> without a version specifier. IOW, we'd treat foo.so as using the ABI. It >> would then be up to the Python runtime to throw ImportErrors if in fact we >> were loading a legacy, non-PEP 384 compliant extension. > >Is it realistic to never break the ABI? I would think of having the ABI >encoded in the file name as well, and only bump the ABI if it does change. >With the "versioned .so files" proposal an ABI bump is necessary with every >python version, with PEP 384 the ABI bump will be decoupled from the python >version. You're right that the ABI will break, requiring a bump, and I think you're right that this means that PEP 384 compliant shared libraries would have to have a version number in their file name (assuming the versioned .so proposal is accepted). The problem is that we would need two version numbers, one for extension modules that are not PEP 384 complaint (and thus get bumped for every new Python version), and one for modules that are PEP 384 compliant (and thus only get bumped once in a while). The reason is that I think it will always be the case that we will have PEP 384 compliant and non-compliant extension modules. Perhaps identifying the underlying problems will lead to a more acceptable patch for Python. My patch tries to take a simple (perhaps too simplistic) solution, and I'm not married to it, but I think the general idea of versioned .so files is the right one. 1. The file name extensions that Python searches for are hardcoded and compiled in. dyload_shlib.c hard codes the file name pattern that extension modules must have in order for Python to load them. They must be .so or module.so. This gets compiled into Python at build time and there's no way for a distro (or anyone else who builds Python from source) to extend the file name patterns without modifying the source code. 2. The extension that distutils writes for shared libraries is dictated by build-time options and cannot be overridden. When you ./configure Python, autoconf figures out what shared library extension your platform uses. It substitutes this into a Makefile variable. That Makefile gets installed into your system with the base Python package and distutils parses the Makefile looking for this variable. When distutils calls your platform compiler, it uses this Makefile variable as the file name extension to use for your shared library. You cannot change this or override it to get distutils to write some other file name extension, well. Of these two problems, #1 is more serious because we have to modify the Python source code to hack in addition shared library search suffixes. #2 can be worked around by renaming the .so file after the build. The disadvantage of this though is that if you're a local packager, you'll have to remember to do the same thing if you want multiple Python version support, because distutils won't take care of it for you. Maybe that's okay, in which case it would still be good to address #1. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Jun 25, 2010, at 04:53 AM, Scott Dial wrote: >My suggestion was that a package that contains .so files should not be >shared (e.g., the entire lxml package should be placed in a >version-specific path). Matthias outlined some of the pitfalls with this approach. >The motivation for this PEP was to simplify the installation python packages >for distros; it was not to reduce the number of .py files on the disk. As others have pointed out, versioned so files is not part of PEP 3147. That PEP does reduce the number of py files on disk, which as I explained in a previous follow, is an important consideration. >Placing .so files together does not simplify that install process in any >way. I disagree of course. :) >You will still have to handle such packages in a special way. You must still >compile the package multiple times for each relevant version of python (with >special tagging that I imagine distutils can take care of) and, worse yet, No, distutils cannot take care of this. There is no way currently to tell distutils to generate a .so file with anything but the platform-specific way of spelling "shared library". >you have created a more trick install than merely having multiple search >paths (e.g., installing/uninstalling lxml for *one* version of python is >actually more difficult in this scheme). That's not a use case we care about. If you have Python 3.2 and 3.3 installed on your system, why would you want lxml installed for one but not the other? And even if for some reason you did, the only way to do that would be in a way similar to handling the PEP 3147 pyc files. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On 04:44 pm, solip...@pitrou.net wrote: On Wed, 30 Jun 2010 09:00:09 PDT Bill Janssen wrote: So, my question then is, why are these skips "unexpected"? Seems to me that if this is the case, this test will never run on any platform. You can change the value of the "usepty" option in your buildbot.tac. (you will also have to restart the buildslave process) But don't do this. The usepty option is completely unrelated to the suggestion I was making. Flipping it to True will only cause other things to break and have no impact on this test. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On 04:26 pm, jans...@parc.com wrote:
exar...@twistedmatrix.com wrote:
Could the test be rewritten (or supplemented) to use a pty? Most or
perhaps all of the same operations should be supported.
Buildbot seems to be explicitly not using a PTY. From the the top of
the test output:
make buildbottest
in dir /Users/buildbot/buildarea/trunk.parc-leopard-1/build (timeout
1800 secs)
watching logfiles {}
argv: ['make', 'buildbottest']
[...]
closing stdin
using PTY: False
This output is telling you that the build slave isn't giving the child
processes it creates a pty. What I had in mind was writing the test to
create a new pty, instead of trying to use the controlling tty. So
basically, the two things are completely unrelated and this buildbot
configuration isn't hurting anything (and in fact is likely helping
quite a few things, so I suggest leaving it alone).
I believe this is specified by the build master.
This test seems to work on Ubuntu and FreeBSD, though.
That's interesting. I wonder if those slaves are able to open /dev/tty
for some reason? The slave is supposed to detach from the controlling
terminal when it daemonizes. There could be a bug in that code, I
suppose, or the slaves could be running without daemonization for some
reason. The operators would have to tell us about that, I think. Or,
another possibility is that /dev/tty doesn't work how I expect it to and
on Ubuntu and FreeBSD it can be opened even if you don't have a
controlling terminal. Hopefully not, though.
Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Jun 26, 2010, at 06:50 PM, Scott Dial wrote: >On 6/26/2010 4:06 PM, Matthias Klose wrote: >> On 25.06.2010 22:12, James Y Knight wrote: >>> On Jun 25, 2010, at 4:53 AM, Scott Dial wrote: Placing .so files together does not simplify that install process in any way. You will still have to handle such packages in a special way. >>> >>> This is a good point, but I think still falls short of a solution. For a >>> package like lxml, indeed you are correct. Since debian needs to build >>> it once per version, it could just put the entire package (.py files and >>> .so files) into a different per-python-version directory. >> >> This is what is currently done. This will increase the size of packages >> by duplicating the .py files, or you have to install the .py in a common >> location (irrelevant to sys.path), and provide (sym)links to the >> expected location. > >"This is what is currently done" and "provide (sym)links to the >expected location" are conflicting statements. I think Matthias was referring to "what is currently done" to your statement "debian needs to build it once per version". Providing symlinks is how we are able to make it appear that there are version-specific py files without actually doing so. >If you are symlinking .py files from a shared location, then that is not the >same as "just install the package into a version-specific location". What >motivation is there for preferring symlinks? This reduces .py file duplications in distro packages. >Who cares if a ditro package install yields duplicate .py files? Nor am >I motivated by having to carry duplicate .py files in a distribution >package (I imagine the compression of duplicate .py files is amazing). It might be amazing, but it's still a significant overhead. As I've described, multiply that by all the py files in all the distro packages containing Python source code, and then still try to fit it on a CDROM. >What happens to the distro packaging if a python package splits the >codebase between 2.x and 3.x (meaning they have distinct .py files)? The Debian/Ubuntu approach to Python 2/3 support is to provide them in separate distro packages. E.g. for Python package foo, you would have Debuntu package python-foo (for the Python 2.x version) and python3-foo. We do not share source between Python 2 and 3 versions, at least not yet . This doesn't hurt us much because the number of Python packages that are source compatible between the two is pretty low (Benjamin's 'six' package might change that :), and not much depends on Python 3 yet. >As someone else mentioned, how is virtualenv going to interact with packages >that install like this? This is a good question, but I *think* it won't affect it much at all. To test for sure I'd either need a Python 3 compatible virtualenv or backport my patch to Python 2.6 and 2.7. But still, I'm not sure it would matter since the same shared library import suffix is used in either case. I actually think version-specific search paths would have a greater impact on virtualenv. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Jun 25, 2010, at 11:58 AM, Brett Cannon wrote: >> Placing .so files together does not simplify that install process in any >> way. You will still have to handle such packages in a special way. You >> must still compile the package multiple times for each relevant version >> of python (with special tagging that I imagine distutils can take care >> of) and, worse yet, you have created a more trick install than merely >> having multiple search paths (e.g., installing/uninstalling lxml for >> *one* version of python is actually more difficult in this scheme). > >This is meant to be used by distros in a programmatic fashion, so my >response is "so what?" Their package management system is going to >maintain the directory, not a person. You and I are not going to be >using this for anything. This is purely meant for Linux OS vendors >(maybe OS X) to manage their installs through their package software. >I honestly do not expect human beings to be mucking around with these >installs (and I suspect Barry doesn't either). Spot on. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Jun 26, 2010, at 10:45 PM, Matthias Klose wrote:
>Having non-conflicting extension names is a schema which already is used on
>some platforms (debug builds on Windows). The question for me is, if just a
>renaming of the .so files is acceptable for upstream, or if distributors
>should implement this on their own, as something like:
>
> if ext_path.startswith('/usr/') and not ext_path.startswith('/usr/local/'):
> load_ext('foo.2.6.so')
> else:
> load_ext('foo.so')
>
>I fear this will cause issues when e.g. virtualenv environments start copying
>parts from the system installation instead of symlinking it.
I concur. I think my patch will have much less impact on virtualenv and
similar tools because there's nothing much magical about it. It just says "oh
there's another file suffix you should consider when looking for a shared
library", which as you point out is already done on Windows.
-Barry
signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Jun 25, 2010, at 03:42 PM, Scott Dial wrote: >On 6/25/2010 2:58 PM, Brett Cannon wrote: >> I assume you are talking about PEP 3147. You're right that the PEP was >> for pyc files and that's it. No one is talking about rewriting the >> PEP. > >Yes, I am making reference to PEP 3147. I make reference to that PEP >because this change is of the same order of magnitude as the .pyc >change, and we asked for a PEP for that, and if this .so stuff is an >extension of that thought process, then it should either be reflected by >that PEP or a new PEP. I think it's not nearly on the order of magnitude as PEP 3147. One way to measure that is the size of the patch required to implement the feature and ensure the test suite still works. My versioned so patch is *way* smaller. I actually think because this is almost exclusively an extension to a build-time configuration option, and doesn't really change the language, a PEP shouldn't be necessary. But by the same token, I'm willing to write a new one (and *not* touch PEP 3147) just so that we have a point of reference to record the discussion and decision. So I'll do that. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Jun 25, 2010, at 12:35 AM, M.-A. Lemburg wrote: >Scott Dial wrote: >> On 6/24/2010 5:09 PM, Barry Warsaw wrote: What use case does this address? >>> If you want to make it so a system can install a package in just one location to be used by multiple Python installations, then the version number isn't enough. You also need to distinguish debug builds, profiling builds, Unicode width (see issue8654), and probably several other ./configure options. >>> >>> This is a good point, but more easily addressed. Let's say a distro makes >>> three Python 3.2 variants available, one "normal" build, a debug build, and >>> UCS2 and USC4 versions of the above. All we need to do is choose a >>> different >>> .so ABI tag (see previous follow) for each of those builds. My updated >>> patch >>> (coming soon) allows you to define that tag to configure. So e.g. >> >> Why is this use case not already addressed by having independent >> directories? And why is there an incentive to co-mingle these >> version-punned files with version-agnostic ones? > >I don't think this is a good idea. After a while your Python >lib directories would need some serious dusting off to make them >maintainable again. > >Disk space is cheap so setting up dedicated directories for each >variant will result in a much easier to manage installation. > >If you want a really clever setup, use hard links between those >directory (you can also use symlinks if you like). >Then a change in one Python file will automatically >propagate to all other variant dirs without any maintenance >effort. Together with PYTHONHOME this makes a really nice >virtualenv-like environment. Note that I do believe there is a difference between what users maintaining their own Python installations might want, and what a distro needs to maintain its entire Python stack. So while dedicated directories might make more sense if you're maintaining your own Python built from source, it doesn't make as much sense for a distro, as described in previous responses by Matthias. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On 05:29 pm, mar...@v.loewis.de wrote: Am 30.06.2010 13:32, schrieb exar...@twistedmatrix.com: On 05:24 am, mar...@v.loewis.de wrote: Seems to work fine. So this I don't understand. Any ideas, anyone? Didn't we discuss this before? The buildbot slave has no controlling terminal anymore, hence it cannot open /dev/tty. If you are curious, just patch your checkout to output the exact errno (e.g. to stdout), and trigger a build through the web. Could the test be rewritten (or supplemented) to use a pty? Most or perhaps all of the same operations should be supported. I'm not sure. It uses TIOCGPGRP, basically to establish that ioctl can also put results into a Python array (IIUC). This goes back to http://bugs.python.org/555817 Somebody rewriting it would need to make sure the original test purpose is still met. Absolutely. And even so, it may still make sense to run the test against both /dev/tty and a pty (or whatever subset of those things can be acquired in the testing environment). You can do a TIOCGPGRP on a new pty (created by os.openpty) but it produces somewhat less interesting results than doing it on /dev/tty. FIONREAD might be a nice alternative. It produces interesting (ie, non- zero) values in an easily predictable/controllable way (it tells you how many bytes are in the read buffer). Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On 06:46 pm, exar...@twistedmatrix.com wrote: On 04:44 pm, solip...@pitrou.net wrote: On Wed, 30 Jun 2010 09:00:09 PDT Bill Janssen wrote: So, my question then is, why are these skips "unexpected"? Seems to me that if this is the case, this test will never run on any platform. You can change the value of the "usepty" option in your buildbot.tac. (you will also have to restart the buildslave process) But don't do this. The usepty option is completely unrelated to the suggestion I was making. Flipping it to True will only cause other things to break and have no impact on this test. Ah, sorry. I confused myself. The option is related. But it will also break other things, so I still would recommend looking for other solutions. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On Wed, Jun 30, 2010 at 09:03, Bill Janssen wrote: > Martin v. Löwis wrote: > >> > Seems to work fine. So this I don't understand. Any ideas, anyone? >> >> Didn't we discuss this before? > > Possibly, but I don't recall doing so. > >> The buildbot slave has no controlling >> terminal anymore, hence it cannot open /dev/tty. If you are curious, >> just patch your checkout to output the exact errno (e.g. to stdout), >> and trigger a build through the web. > > So, why is skipping this test "unexpected"? I see "x86 Tiger" is also > showing this as an unexpected skip. Should I just add it to the list of > expected skips on Darwin? Actually, will it run on any platform? The whole "unexpected" skipping is somewhat of a mess. In an ideal situation modules that are optionally built should be allowed to skip, and on a per-platform basis certain OS-specific tests (whether they be exclusive to a specific OS or run on all OSs except Windows) should be skipped. Otherwise any import failure should be a test failure. The "unexpected" test skipping was meant to solve both of these situations, but in an imperfect way. My PSF grant proposal to work on Python full-time for two to three months after my Ph.D. is complete (assuming the PSF gives me the grant this would start most likely in November or December) includes cleaning up the test suite and this would be the first thing I tackle. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
> The whole "unexpected" skipping is somewhat of a mess. In an ideal > situation modules that are optionally built should be allowed to skip, While this may be the wide-spread interpretation, it is definitely *not* the original intention of the feature. When Tim Peters added it, he wanted it to tell him whether he did the Windows build correctly, INCLUDING ALL OPTIONAL PACKAGES that can possibly work on Windows. If you try to generalize this beyond Windows, then the only skips that are expected are the ones for tests that absolutely cannot work on the platform - i.e. Unix tests on Windows, and Windows tests on Unix. Otherwise, if you can get it to pass by installing additional software, Tim did *not* mean this to be an expected skip. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
Martin v. Löwis wrote: > > The whole "unexpected" skipping is somewhat of a mess. In an ideal > > situation modules that are optionally built should be allowed to skip, > > While this may be the wide-spread interpretation, it is definitely *not* > the original intention of the feature. > > When Tim Peters added it, he wanted it to tell him whether he did the > Windows build correctly, INCLUDING ALL OPTIONAL PACKAGES that can > possibly work on Windows. If you try to generalize this beyond Windows, > then the only skips that are expected are the ones for tests that > absolutely cannot work on the platform - i.e. Unix tests on Windows, > and Windows tests on Unix. Otherwise, if you can get it to pass by > installing additional software, Tim did *not* mean this to be an > expected skip. Perfectly reasonable, good to know. So on my OS X buildbots I should update gdb, tcl/tk, and readline, so that those tests can run. Probably be good to put a note in the regrtest.py comments to this effect, as I don't see a PEP about testing or buildbots. Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
Barry Warsaw wrote: > On Jun 25, 2010, at 12:35 AM, M.-A. Lemburg wrote: > >> Scott Dial wrote: >>> On 6/24/2010 5:09 PM, Barry Warsaw wrote: > What use case does this address? > If you want to make it so a system can install a package in just one > location to be used by multiple Python installations, then the version > number isn't enough. You also need to distinguish debug builds, profiling > builds, Unicode width (see issue8654), and probably several other > ./configure options. This is a good point, but more easily addressed. Let's say a distro makes three Python 3.2 variants available, one "normal" build, a debug build, and UCS2 and USC4 versions of the above. All we need to do is choose a different .so ABI tag (see previous follow) for each of those builds. My updated patch (coming soon) allows you to define that tag to configure. So e.g. >>> >>> Why is this use case not already addressed by having independent >>> directories? And why is there an incentive to co-mingle these >>> version-punned files with version-agnostic ones? >> >> I don't think this is a good idea. After a while your Python >> lib directories would need some serious dusting off to make them >> maintainable again. >> >> Disk space is cheap so setting up dedicated directories for each >> variant will result in a much easier to manage installation. >> >> If you want a really clever setup, use hard links between those >> directory (you can also use symlinks if you like). >> Then a change in one Python file will automatically >> propagate to all other variant dirs without any maintenance >> effort. Together with PYTHONHOME this makes a really nice >> virtualenv-like environment. > > Note that I do believe there is a difference between what users maintaining > their own Python installations might want, and what a distro needs to maintain > its entire Python stack. So while dedicated directories might make more sense > if you're maintaining your own Python built from source, it doesn't make as > much sense for a distro, as described in previous responses by Matthias. Fair enough. I haven't followed the thread closely, so Matthias will probably already have answered this: The Python default installation dir for libs (including site-packages) is $prefix/lib/pythonX.X, so you already have separate and properly versioned directory paths. What difference would the extra version on the .so file make in such a setup ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 30 2010) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ 2010-07-19: EuroPython 2010, Birmingham, UK18 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On Wed, Jun 30, 2010 at 12:53, "Martin v. Löwis" wrote: >> The whole "unexpected" skipping is somewhat of a mess. In an ideal >> situation modules that are optionally built should be allowed to skip, > > While this may be the wide-spread interpretation, it is definitely *not* > the original intention of the feature. > > When Tim Peters added it, he wanted it to tell him whether he did the > Windows build correctly, INCLUDING ALL OPTIONAL PACKAGES that can > possibly work on Windows. If you try to generalize this beyond Windows, > then the only skips that are expected are the ones for tests that > absolutely cannot work on the platform - i.e. Unix tests on Windows, > and Windows tests on Unix. Otherwise, if you can get it to pass by > installing additional software, Tim did *not* mean this to be an > expected skip. Interesting. Do you use it that way when you make the Windows build? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On Thu, Jul 1, 2010 at 5:53 AM, "Martin v. Löwis" wrote: > When Tim Peters added it, he wanted it to tell him whether he did the > Windows build correctly, INCLUDING ALL OPTIONAL PACKAGES that can > possibly work on Windows. If you try to generalize this beyond Windows, > then the only skips that are expected are the ones for tests that > absolutely cannot work on the platform - i.e. Unix tests on Windows, > and Windows tests on Unix. Otherwise, if you can get it to pass by > installing additional software, Tim did *not* mean this to be an > expected skip. Note that it works this way on Linux as well. On Kubuntu (for example) you need another half dozen or so additional *-dev packages installed to avoid unexpected test skips. Cheers, Nick. P.S. For anyone curious, I posted the list of extra packages you need here: http://boredomandlaziness.blogspot.com/2010/01/kubuntu-dev-packages-to-build-python.html -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On Wed, Jun 30, 2010 at 14:52, Nick Coghlan wrote: > On Thu, Jul 1, 2010 at 5:53 AM, "Martin v. Löwis" wrote: >> When Tim Peters added it, he wanted it to tell him whether he did the >> Windows build correctly, INCLUDING ALL OPTIONAL PACKAGES that can >> possibly work on Windows. If you try to generalize this beyond Windows, >> then the only skips that are expected are the ones for tests that >> absolutely cannot work on the platform - i.e. Unix tests on Windows, >> and Windows tests on Unix. Otherwise, if you can get it to pass by >> installing additional software, Tim did *not* mean this to be an >> expected skip. > > Note that it works this way on Linux as well. On Kubuntu (for example) > you need another half dozen or so additional *-dev packages installed > to avoid unexpected test skips. So it isn't that it's "unexpected", it's that a dependency is missing. So it seems the terminology needs to get tweaked. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
>> When Tim Peters added it, he wanted it to tell him whether he did the >> Windows build correctly, INCLUDING ALL OPTIONAL PACKAGES that can >> possibly work on Windows. If you try to generalize this beyond Windows, >> then the only skips that are expected are the ones for tests that >> absolutely cannot work on the platform - i.e. Unix tests on Windows, >> and Windows tests on Unix. Otherwise, if you can get it to pass by >> installing additional software, Tim did *not* mean this to be an >> expected skip. > > Interesting. Do you use it that way when you make the Windows build? You want a honest answer? I usually don't run the test suite on Windows, and trust that the packaging will tell me if an extension module failed to build (and otherwise trust that if the setup worked for the release candidate, it will also work for the final release). Independent of that, I also decided to entirely ignore the notion of expected skipped test (so even if I would run the test suite, I wouldn't bother if one was reported as skipped). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On Wed, Jun 30, 2010 at 15:01, "Martin v. Löwis" wrote: >>> When Tim Peters added it, he wanted it to tell him whether he did the >>> Windows build correctly, INCLUDING ALL OPTIONAL PACKAGES that can >>> possibly work on Windows. If you try to generalize this beyond Windows, >>> then the only skips that are expected are the ones for tests that >>> absolutely cannot work on the platform - i.e. Unix tests on Windows, >>> and Windows tests on Unix. Otherwise, if you can get it to pass by >>> installing additional software, Tim did *not* mean this to be an >>> expected skip. >> >> Interesting. Do you use it that way when you make the Windows build? > > You want a honest answer? Of course. You're amongst friends. =) > I usually don't run the test suite on Windows, > and trust that the packaging will tell me if an extension module failed > to build (and otherwise trust that if the setup worked for the release > candidate, it will also work for the final release). Seems reasonable. > > Independent of that, I also decided to entirely ignore the notion of > expected skipped test (so even if I would run the test suite, I wouldn't > bother if one was reported as skipped). Yeah, I remember you bringing this up years ago. I think that "unexpected" is a bad term and should be renamed to reflect the fact that the test was skipped because an optional third-party package was not included. Or simply output why a certain test was skipped at the end of the test suite run (I know it is outputted when the individual test is skipped, but it would be easier to read that info at the end during the summary of results). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Jun 30, 2010, at 10:35 PM, M.-A. Lemburg wrote: >The Python default installation dir for >libs (including site-packages) is $prefix/lib/pythonX.X, so you >already have separate and properly versioned directory paths. > >What difference would the extra version on the .so file make in >such a setup ? So on Debian (inherted by Ubuntu), that would be /usr/lib/pythonX.Y and in fact if you look there, you'll see the normal standard library layout you'd expect. There is no site-packages under there though because we install add-ons to dist-packages[*] but functionally it's what you'd expect. However, if you look inside dist-packages, you'll see something a little weird. All the .py files .egg-info files will actually be symlinks into a parallel hierarchy under /usr/share/pyshared. Also, under dist-packages you'll see a directory layout that looks like it normally would under a standard site-packages layout, except that again, all the .py files are symlinked to the pyshared location. The .so and .pyc files in dist-packages will be the actual .so and .pyc files. Why is that? Well, it's because the .py files can be shared but the .pyc and .so files can't. Getting rid of these crufty symlink farms was the initial motivation for PEP 3147. In a PEP 3147 world, pure-Python packages don't need the directory /usr/lib/pythonX.Y/dist-packages. We can just install packages to /usr/share/pyshared and none of the .pyc files will collide between Python versions. It makes distro packaging simpler (no symlinks to keep track of or post-install linking and post-removal clean up to do) and smaller (only one .py file for all supported Python versions). All we have to do is a post-install byte-compilation and we're done. When we remove such a package we only *have* to remove the .py source file because PEP 3147 will not load a __pycache__ pyc if the source is missing (though it's easy enough to clean up the pyc files too). Throw extension modules into the mix and we have the file collision problem again. We can't install _foo.so into /usr/share/pyshared because that's going to be Python ABI-specific. If we can ABI-tag the .so and ensure it will get imported, then we're back to the simpler layout where /usr/share/pyshared is the destination for all installed Python packages. -Barry [*] That's done as a compromise between Debian's interpretation of the FHS and the conflict of that interpretation to from-source installs of Python. Specifically, Debian puts /usr/local/lib/pythonX.Y/dist-packages on the *system* Python's path because that's where it expects system administrators to put their own add-ons to the system Python. That used to be /usr/local/lib/pythonX.Y/site-packages but then if you did a from-source altinstall for example of the same Python version, it would be possible for a "/usr/local/bin/pythonX.Y setup.py install" of a third-party package to break your system Python. Not good! (And yes, it happened to me :). signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On Thu, Jul 1, 2010 at 7:55 AM, Brett Cannon wrote: > So it isn't that it's "unexpected", it's that a dependency is missing. > So it seems the terminology needs to get tweaked. More that the phrase "expected skip" isn't clearly defined and people sometimes guess wrong as to what it means. As Martin pointed out, there are two possible meanings: "will never work on this OS" and "won't work with just the base OS install". Currently, the "expected skip" list is based purely on the former, but developers occasionally interpret it as the latter (as Bill did in this case). I will note that the first list is much easier to keep up to date, since the latter may vary significantly based on vendor decisions as to what they install by default (a fairly significant factor in the Linux and *BSD worlds). Adding "(Were all optional modules built successfully?)" to the end of the "skips were unexpected" line in the regrtest output may be enough to eliminate the confusion. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On Jul 01, 2010, at 07:52 AM, Nick Coghlan wrote: >Note that it works this way on Linux as well. On Kubuntu (for example) >you need another half dozen or so additional *-dev packages installed >to avoid unexpected test skips. > >Cheers, >Nick. > >P.S. For anyone curious, I posted the list of extra packages you need >here: >http://boredomandlaziness.blogspot.com/2010/01/kubuntu-dev-packages-to-build-python.html As an aside, I wonder if it would make sense to create an Ubuntu/Debian package that essentially just depended on all those -dev packages. That way you could install this python-uber-dev package and get everything you needed to build a sumo Python. Hmm, /me goes to hack on his PPA. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On Wed, Jun 30, 2010 at 15:20, Nick Coghlan wrote: > On Thu, Jul 1, 2010 at 7:55 AM, Brett Cannon wrote: >> So it isn't that it's "unexpected", it's that a dependency is missing. >> So it seems the terminology needs to get tweaked. > > More that the phrase "expected skip" isn't clearly defined and people > sometimes guess wrong as to what it means. As Martin pointed out, > there are two possible meanings: "will never work on this OS" and > "won't work with just the base OS install". Currently, the "expected > skip" list is based purely on the former, but developers occasionally > interpret it as the latter (as Bill did in this case). > > I will note that the first list is much easier to keep up to date, > since the latter may vary significantly based on vendor decisions as > to what they install by default (a fairly significant factor in the > Linux and *BSD worlds). > > Adding "(Were all optional modules built successfully?)" to the end of > the "skips were unexpected" line in the regrtest output may be enough > to eliminate the confusion. Probably. So I would still want to shift the test-specific info into the tests instead of regrtest and raise a subclass od SkippedTest (or whatever the name of the exception is) to signify that there is a difference. This would also do away with the possibility of having a test get silently skipped by an ImportError even though the module should definitely be available (didn't that bite you once, or was that someone else?). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
Brett Cannon wrote: > On Wed, Jun 30, 2010 at 15:01, "Martin v. Löwis" > wrote: When Tim Peters added it, he wanted it to tell him whether he did the Windows build correctly, INCLUDING ALL OPTIONAL PACKAGES that can possibly work on Windows. If you try to generalize this beyond Windows, then the only skips that are expected are the ones for tests that absolutely cannot work on the platform - i.e. Unix tests on Windows, and Windows tests on Unix. Otherwise, if you can get it to pass by installing additional software, Tim did *not* mean this to be an expected skip. >>> Interesting. Do you use it that way when you make the Windows >>> build? >> You want a honest answer? > > Of course. You're amongst friends. =) > >> I usually don't run the test suite on Windows, and trust that the >> packaging will tell me if an extension module failed to build (and >> otherwise trust that if the setup worked for the release candidate, >> it will also work for the final release). > > Seems reasonable. > >> Independent of that, I also decided to entirely ignore the notion >> of expected skipped test (so even if I would run the test suite, I >> wouldn't bother if one was reported as skipped). > > Yeah, I remember you bringing this up years ago. I think that > "unexpected" is a bad term and should be renamed to reflect the fact > that the test was skipped because an optional third-party package was > not included. A "dependency skip", perhaps? > Or simply output why a certain test was skipped at the end of the > test suite run (I know it is outputted when the individual test is > skipped, but it would be easier to read that info at the end during > the summary of results). That would be a good idea, I think. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Taking over the Mercurial Migration
/me throws hat into ring. I'm in the middle of migrating fairly large chunks of an overgrown codebase from Subversion to Mercurial, so I might actually have worthwhile input :) -- ~Dan On Wed, Jun 30, 2010 at 10:41:51AM +0200, Georg Brandl wrote: > Am 30.06.2010 07:37, schrieb "Martin v. Löwis": > > It seems that both Dirkjan and Brett are very caught up > > with real life for the coming months. So I suggest that > > some other committer who favors the Mercurial transition > > steps forward and takes over this project. > > > > If nobody volunteers, I propose that we release 3.2 > > from Subversion, and reconsider Mercurial migration > > next year. > > IIUC, Dirkjan is only caught up for another month. I have > no problems with releasing a first 3.2 alpha from SVN and > then switching, so I propose that we target the migration > for August -- I can help in the second half of August if > needed. > > Georg > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/daniel.buch%40gmail.com pgpxNcRxCeJtt.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] How are the bdist_wininst binaries built ?
Hi, I would like to modify the code of the bdist installers, but I don't see any VS project for VS 9.0. How are the wininst-9.0*exe built ? thanks, David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How are the bdist_wininst binaries built ?
> I would like to modify the code of the bdist installers, but I don't > see any VS project for VS 9.0. How are the wininst-9.0*exe built ? See PC/bdist_wininst. HTH, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Taking over the Mercurial Migration
Am 01.07.2010 02:01, schrieb Dan Buch: > /me throws hat into ring. I'm in the middle of migrating fairly > large chunks of an overgrown codebase from Subversion to Mercurial, > so I might actually have worthwhile input :) To all the volunteers: an issue that apparently requires immediate attention is that r21112 fails to convert. For details, see http://mail.python.org/pipermail/python-committers/2010-June/000977.html So somebody please reproduce the problem, propose a patch, and get in contact with Dirkjan to integrate it. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How are the bdist_wininst binaries built ?
On Thu, Jul 1, 2010 at 1:22 PM, "Martin v. Löwis" wrote: >> I would like to modify the code of the bdist installers, but I don't >> see any VS project for VS 9.0. How are the wininst-9.0*exe built ? > > See PC/bdist_wininst. Hm, my question may not have been clear: *how* is the wininst-9.0 built from the bdist_wininst sources ? I see 6, 7.0, 7.1 and 8.0 versions of the visual studio build scripts, but nothing for VS 9.0. cheers, David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How are the bdist_wininst binaries built ?
>> See PC/bdist_wininst. > > Hm, my question may not have been clear: *how* is the wininst-9.0 > built from the bdist_wininst sources ? I see 6, 7.0, 7.1 and 8.0 > versions of the visual studio build scripts, but nothing for VS 9.0. Ah. See PCbuild/bdist_wininst.vcproj. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
[Barry Warsaw] > As an aside, I wonder if it would make sense to create an Ubuntu/Debian > package that essentially just depended on all those -dev packages. That way > you could install this python-uber-dev package and get everything you needed > to build a sumo Python. I don’t know. Any Debian advanced user wanting to build a package can just run “aptitude build-dep python2.6”. Aptitude has the nice property of asking for a confirmation, so it’s even possible to cherry-pick from the lists of deps (press “e” at the confirmation prompt). E.g., I choose readline but not tk. So it seems to me that the information and the command are already here. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
