Re: [Distutils] newbie bootstrap question
David Kim wrote:
I am running the OS X EPD distribution, which uses Enstaller (built on
setuptools). Could this have something to do with it?
Sorry, but yes, this is exactly the problem. EPD comes with Enstaller
instead of setuptools, and thus there is no installed project called
'setuptools', so pkg_resources.require('setuptools') isn't going to find
anything.
As a quick fix, you could re-write the code you have as
pkg_resources.require('enstaller') and be on your way.
I think a better, and longer term, fix is for us to patch the 'require'
method in Enstaller's version of pkg_resources so that it catches a
request for setuptools and maps it to Enstaller. This is probably safe
because you can't have both Enstaller and setuptools installed without
getting errors, plus Enstaller includes a patched version of setuptools.
-- Dave
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] A change to how platform name is generated for binary distributions?
We're starting to try and distribute some pre-built binaries on Solaris
and have come across an issue with how pkg_resources / distutils
generates the platform specification when the distribution includes
Python extensions.
In particular, we'd like to distribute both x86 and amd64 (or x86_64)
binaries for Solaris on Intel since the OS can run in either mode, but
all our egg distributions get the same file name no matter what
architecture they're actually built for. Diving into how the platform
part of a distribution name gets generated, I find that it relies on the
results of 'os.uname' via distutils.util.get_platform().This seems
like it would cause problems on more than just Solaris as it should also
happen anytime someone is using a 32-bit Python on a 64-bit OS/CPU, no?
So it occurs to me that it might be nice to fix up
distutils.util.get_platform() to try and rely on the information that
can be obtained about the sys.executable itself. After all, distutils /
setuptools build extensions using the same compiler and linker flags
that the Python environment was generated with so it seems logical to
tie them together here too. While it may be hard (impossible?) to cover
enough bases of reading compiler flags, many OSes provide tools to
gather metadata about executable files which are fairly easy to use. In
fact, on OSes that use ELF binaries, there should be alot of commonality
in figuring this out I think.
So I'm proposing that a change be made to do the above by wrapping all
places where distutils.util.get_platform returns a value with something
that replaces the last part of the platform spec with the Python
architecture *if* it can safely be determine what that architecture
is. Something like the following pseudo-code wrapper:
def wrapper(platform):
if cached_arch is None:
cached_arch = _get_python_arch()
parts = platform.split('-')
parts[-1] = cached_arch
platform = '-'.join(parts)
return platform
def _get_python_arch():
if sys.platform == 'KNOWN OS':
return method_for_finding_python_arch_on_known_os()
...
else:
return ''
I'm much more familiar with setuptools / pkg_resources than I am with
distutils, so I've actually tried this out with an implementation where
pkg_resources.get_build_platform() is patched and the
_get_python_arch() above only returns something for Solaris. Seems to
work okay for me, but that's likely more because I don't see many
Solaris eggs in the wild than anything I've done explicitly for
backwards compatibility. I understand backwards compatibility would be
a big sticking point here, so I've tried to structure things where
values only change where the implementation is positive it knows what to
do, and what values to return.
Thoughts? If there is agreement, I can submit a patch.
-- Dave
P.S. It is my understanding that there is no equivalent to OS X's 'fat'
dual-binary mode on Solaris for shipping multiple arch binaries. Nor
can I find any difference in filenames or directory names between a
32-bit and 64-bit Python 2.5.4 built from source. Is there some other
alternative to building that would solve the above problem without
patching distutils/pkg_resources?
P.P.S. While you can force the kernel to boot in 32-bit mode,
distutils.util.get_platform() returns the same values because the OS's
uname does. i.e. whether 64-bit or 32-bit Intel, uname returns i86pc
as the machine.
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Solaris and distutils: Need to pass LIBDIR explicitly with -L when building extensions?
Tarek Ziade wrote:
2009/1/31 Floris Bruynooghe floris.bruynoo...@gmail.com:
I'm leaning more and more toward this is actually a bug
with the distutils source on Solaris.
Yes, I agree now that it is a bug in distutils and I agree with your
fix. The if statement should check both that it is SunOS and that it
is using a shared python, just like the linux one. If fact the linux
one could just be modified:
if (sys.paltform.startswith('linux') \
or sys.platform.startswith('gnu') \
or sys.platform.startswith('sunos')) \
and sysconfig.get_config_var('Py_ENABLE_SHARED'):
...
I'll leave the honours of reporting the bug to you if you agree with
this.
Yes thanks to report this, I'll apply your fix,
Regards
Tarek
Hi Tarek,
It isn't clear to me if you fixed it already or were waiting for my
patch, but I just added issue 5132 to the Python issue tracker to
document the issue and provide a patch file.
-- Dave
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Solaris and distutils: Need to pass LIBDIR explicitly with -L when building extensions?
Tarek Ziadé wrote: On Mon, Feb 2, 2009 at 7:29 PM, Dave Peterson dpeter...@enthought.com wrote: Hi Tarek, It isn't clear to me if you fixed it already or were waiting for my patch, but I just added issue 5132 to the Python issue tracker to document the issue and provide a patch file. I was waiting for that, thanks :) Tarek I noticed you removed the Python 2.5 version specification. I wasn't aware there were no further planned maintenance releases for the 2.5 branch. A few of the libs we use still seem to have problems building on 2.6. Guess it is just a matter of time. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Solaris and distutils: Need to pass LIBDIR explicitly with -L when building extensions?
Floris Bruynooghe wrote:
Hi Dave
On Mon, Jan 26, 2009 at 06:48:06PM -0600, Dave Peterson wrote:
I am trying to build a number of projects that use Python extensions on
Solaris 10 and I've discovered that nothing with extensions will link
unless I explicitly pass in a '-L/path/to/python/lib/dir' because
libpython2.5.so is not otherwise found when the '-lpython2.5' argument
is specified during linking.
[...]
If that all seems correct, then it appears the issue is the
finalize_options() source in lib/distutils/commands/build_ext.py. There
are a number of if blocks that explicitly append the value of
distutils.sysconfig.get_config_vars('LIBDIR') to the list of
library_dirs used to link built extensions with. However, there doesn't
seem to be one of these for Solaris / sunos.
Could you point to one of the projects you're having trouble with? I
haven't had any such problems with Python 2.5 on Solaris 10, distutils
always finds the right things when I'm building extension modules.
Pretty much everything I've tried that uses an extension has this
problem. Cython, Numpy, Traits, etc. As Robert Kern pointed out, I'm
using a custom built Python (Python 2.5.4) built and installed into a
custom location via '--prefix'. What Python are you using?
Just for grins, I've tried building Python with a number of other
compilers, but all with a --prefix setting, and they've all exhibited
this problem. I'm leaning more and more toward this is actually a bug
with the distutils source on Solaris.
-- Dave
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Python Package Management Sucks
Josselin Mouette wrote: Le mercredi 01 octobre 2008 à 14:39 -0400, Phillip J. Eby a écrit : To be clear, I mean here that a file (as opposed to a resource) is something that the user is expected to be able to read or copy, or modify. (Whereas a resource is something that is entirely internal to a library, and metadata is information *about* the library itself.) It’s not as simple as that. Python is not the only thing out there, and there are many times where your resources need to be shipped in existing formats, in files that land at specific places. For example icons go in /usr/share/icons, locale files in .mo format in /usr/share/locale, etc. Perhaps I'm putting words into PJE's mouth but it seems to me that if distributions want to put things in widely differing places from where the developer had them (in a single tree), then we're going to need a Python standard library API (implemented per platform/OS that defines those places!) for the code to find these resources/files. Certainly the expectation shouldn't be on developers to have to handle all the possible different locations OSes are going to put things? -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [setuptools] svn'ed file being missed with include_package_data=True
Chris Withers wrote: [1] Anyone know how to get .tgz source builds on windows rather than .zips? Yes, do 'python setup.py sdist --formats=gztar'. You can find this yourself by doing 'python setup.py sdist -h' which will show (among other things): Options for 'sdist' command: --formats formats for source distribution (comma-separated list) --keep-temp (-k) keep the distribution tree around after creating archive file(s) --dist-dir (-d) directory to put the source distribution archive(s) in [default: dist] --help-formatslist available distribution formats And then a 'python setup.py sdist --help-formats' results in: List of available source distribution formats: --formats=bztar bzip2'ed tar-file --formats=gztar gzip'ed tar-file --formats=taruncompressed tar file --formats=zipZIP file --formats=ztar compressed tar file -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [tg-trunk] Re: [ANN] EggFreezer
Ian Bicking wrote: Alberto Valverde wrote: Alberto Valverde wrote: [...] Reading a comment on the philikon article (http://philikon.wordpress.com/2008/06/26/is-there-a-point-to-distributing-egg-files/#comment-47), I also notice that Enthought has done some work on this, it seems by fixing up the binary packages at install time. This seems to be related to an entirely different issue of the location of libraries and binary incompatibilities, which I only slightly understand. Very interesting... some code doing this is not available anywhere for me to study/steal, right? :) Probably, but you'd have to ask the Enthought guys. Sorry for the long delay in responding, the SciPy conference and EPD/ETS releases have kept me quite busy and I hadn't read this mailing list in weeks. :-( Enthought established a post install pattern in some of our egg's EGG_INFO dirs so that we could run a tool derived from easy_install that would automatically run scripts when the egg was installed or uninstalled. These scripts then run, among other things, chrpath on linux, macholib on OS/X, etc to fix up paths to non-pytho libs given the user's install location for the egg. The trouble here is that the users of the egg have to use the derived tool rather than the standard one. This is fine if you have a closed system, but difficult if you're trying to publish eggs to PyPi, etc. for use by the world. The tool's source is in our OSS repo here: https://svn.enthought.com/svn/enthought/Enstaller/trunk I'd recommend asking questions on enthought-dev if you have any questions about that code. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Enthought-dev] Strange naming problem trying to build Enable rpm
Stanley A. Klein wrote: I was able to build rpms for Traits, Envisage, and related packages (e.g., TraitsGUI) for Fedora 7. However, when I tried to package Enable I ran into the following problem: The tar.gz gets built as Enable-3.0.0.dev-r21092.tar.gz, but when the rpm packager gets called it looks for Enable-3.0.0.tar.gz, doesn't find it, and errors out. I've copied a relevant part of the output below. I'm not sure what's happening here since the setup.cfg's for the ETS projects should all be pretty close to the same on this issue. Are you doing a setup.py release bdist_rpm? i.e. using the release alias? If not, that's the source of the problem. BTW: Are you checking in your changes to setup.cfg's? If you don't have privileges, I can make these changes so just send me a patch, set of modified setup.cfgs, or instructions on what to do to reproduce your changes. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Enthought-dev Digest, Vol 48, Issue 49
Hi Stan, Stanley A. Klein wrote: Dave - I wasn't using the release alias. From what you say, I suppose I should. If you're trying to build rpms for the equivalent of the released sdist's and eggs, then yes, you definitely should. If you look at the setup.cfg files in all ETS projects, they all append build tags of '.dev' and force the svn revision to be tagged on, UNLESS you trigger the release alias or explicitly specify egg_info -RDb. I think the Enable project might be somewhat different from the others because it uses both distutils and setuptools. Nope, not at all. We have a number of other projects using numpy.distutils and setuptools together in ETS. I will send you off line a program I used that fixes up the setup.cfg. Excellent. Thanks! -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Bug in setuptools version parsing when appending '.dev'?
Phillip J. Eby wrote:
At 12:02 PM 8/4/2008 -0500, Dave Peterson wrote:
Phillip J. Eby wrote:
At 07:05 PM 7/30/2008 -0500, Dave Peterson wrote:
Am I missing something or is the following a bug whereby adding the
'.dev' tag is doing something weird?
from pkg_resources import parse_requirement as pv
pv('1.0a1.dev') pv('1.0a1')
True
pv('1.0a1') pv('1.0')
True
pv('1.0a1.dev') pv('1.0.dev')
False
pv('1.0a1') pv('1.0.dev')
False
import setuptools
setuptools.__version__
'0.6c8'
This is mainly causing us problems when projects try to track alpha
and beta level bumps in dependencies, such as when project Foo
requires project Bar version 3.0b1 via a requirement like 'Bar =
3.0b1' (which means we don't want the development prereleases of
Bar's first beta release, but anything after that should be okay.)
But then when we actually want to release Bar 3.0 and change the
version number to just '3.0', suddenly installs fail while we try
to run the last set of tests because '3.0.dev' is older than '3.0b1'.
If it is not a bug, how do you handle situations where you want to
run that last round of testing prior to tagging and building
releases? I'd rather do that AFTER making all source changes, and
not have to change the version number after the testing.
This is what 'rc' tags are for, actually. You can put your version
in the source, and use the -b option to egg_info while doing builds
and testing to tack on an 'rc' tag, possibly with a subversion
number as well.
Perhaps I'm missing something, but that doesn't seem like it scales
to a community -- not everyone is going to remember to use a '-b'
option when building and that is going to cause endless support
problems.
Only the creator of a release uses that; the community should never be
building release versions.
Right, and that's the exact problem! They're grabbing the source via
svn checkout from the release tag and doing their own build which ends
up getting labeled as '3.0.dev-r1234' (which seems to be exactly the
right thing to say to me -- it is an svn build of the 3.0 release) but
this does not satisfy dependencies that are in other projects that were
put out requiring Traits = 3.0b1. Sure I can tell people not to do
that when they say it doesn't work, but I think it should work because
the pattern works everywhere else (see below.)
We do put the following in our setup.cfg so that people don't
mistakenly build release versions when not intending to do so:
[egg_info]
tag_build = .dev
tag_svn_revision = 1
Are you staying the standard process for setuptools is to delete /
modify this when tagging a release in the repo?
I personally don't tag releases in the repository; a release is not a
tag. I have an alias called 'release' that maps to 'egg_info -Rdb
', and I use it when issuing release builds.
Great to hear about what you do, but we frequently have to support
customers who need minor updates and having a tag that makes it easy to
branch from, or re-create, a release is a very useful thing. We have to
many developers who need to restart at a release point to be able to
track the code state at release points using any other method we've seen.
However, there's nothing stopping you from creating a utility that
copies the trunk to a tag, checks out the tag, removes the options,
and checks in the tag, if that would be your preferred approach.
Hmm, and I thought I was clear saying that this is exactly what I do not
want to do. :-) Ideally, we don't want to modify tags in anyway.
Like you pointed out above, I don't want people accidentally building
'release' versions except the project maintainers.
If not, how do you avoid the problem of someone building from a
source checkout and finding out that 3.0.dev-r1234 doesn't satisfy
the '= 3.0b1' dependency spec?
I bump the version number in SVN immediately after making a release,
so that the trunk would be '3.1.dev-r1234', not '3.0dev'. If you're
using .dev tags in your egg-info setup, you should bump the version in
setup.py immediately *after* releasing, to avoid just this situation.
We DO do that. That doesn't solve the problem I'm talking about
though. Let me try one more time
There exist people who prefer to build from source they've gotten from
the svn repo. These people don't want to worry about bugs in the trunk
/ development version so they go grab the source from the latest release
tag. They then do a setup.py bdist_egg to build an egg that they
plan to distribute within their company, work group, what have you as
part of their shared project where they defined dependencies as
=3.0b1 While the latest tagged version is 3.0b1, 3.0b2, 3.0c1,
3.0c2, etc. this process works great as all of those are correctly newer
or equal to 3.0b1 even when .dev tags are appended to the version
number. BUT, as soon as we tag and release the final 3.0 release, it
breaks because 3.0.dev 3.0b1
Re: [Distutils] Bug in setuptools version parsing when appending '.dev'?
Phillip J. Eby wrote: At 01:41 PM 8/4/2008 -0400, Alexander Michael wrote: Again, attempting to offer up practical solutions. Edit the setup.cfg's to drop the dev option in the release branches and update the trunk to the next version (i.e. 3.1.dev-rX)? That way, checkouts of the release branches will be 3.0-rX (a post release of 3) and the trunk will be a post of a pre-release that is newer than anything else in the repository. Just a thought... This is basically what I do, except I don't bother having release branches or tags, and instead of editing the setup.cfg, I just use my release alias (which maps to 'egg_info -RDb'). We have that alias too, which is how we build releases but everyone else defaults to a .dev-r1234 type of suffix on their version numbers. So, when I did two back-to-back releases of BytecodeAssembler today (due to finding a bug after the first release), my command sequence was: # start: version in setup.py is 0.4, release on Pypi is 0.3 # do development of version 0.4 w/periodic checkins setup.py wikiup # upload wiki pages setup.py release sdist bdist_egg upload # ... edit version number from 0.4 to 0.5 and check in # ... find bug, fix it, check it in setup.py wikiup # upload wiki pages setup.py release sdist bdist_egg upload # ... edit version number from 0.5 to 0.6 and check in # end: version in setup.py is 0.6, release on Pypi is 0.5 This is pretty much our exact process too! Only difference are that we do alpha and beta versions in the setup.py AND we svn tag the 0.4 and 0.5 versions. The problem I'm having is when someone gets those tagged versions and does a build. Those builds don't satisfy dependencies that the alpha and betas did. :-( -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] [issue30] [PATCH] ability to upload a pre-built egg to PyPi
New submission from Dave Peterson [EMAIL PROTECTED]:
It would be convenient to be able to upload pre-built eggs using the 'python
setup.py upload' command as the owner/author of a project may not have machines
available to cover the platforms for which pre-built binaries are desired.
Having this feature would allow the project owner / maintainer to accept
pre-built eggs from others in their project community and get them onto PyPi.
I've put together and attached a patch that adds a new '--bdist-path' (or '-b')
option to the upload command that allows calls like 'python setup.py upload -b
dist/foo-1.0-py2.5-win32.egg' to work.
The patch is actually for the distutils.command.upload module as I've found that
in setuptools-0.6c8, even though there is a setuptools/command/upload module, it
isn't used at all. I'm assuming this is because there isn't much difference
between the setuptools and distutils versions of this command but that something
was intended eventually. I'm making this a setuptools ticket because I'd like
to see this improvement in setuptools :-)
--
files: upload.py.patch
messages: 77
nosy: dpeterson
priority: feature
status: unread
title: [PATCH] ability to upload a pre-built egg to PyPi
Added file: http://bugs.python.org/setuptools/file11/upload.py.patch
___
Setuptools tracker [EMAIL PROTECTED]
http://bugs.python.org/setuptools/issue30
___--- /command/upload.py.orig 2006-09-19 01:53:05.0 -0500
+++ /command/upload.py 2008-08-05 18:09:12.0 -0500
@@ -30,6 +30,7 @@
('sign', 's',
'sign files to upload using gpg'),
('identity=', 'i', 'GPG identity used to sign files'),
+('bdist-path=', 'b', 'path to bdist to be uploaded'),
]
boolean_options = ['show-response', 'sign']
@@ -40,6 +41,7 @@
self.show_response = 0
self.sign = False
self.identity = None
+self.bdist_path = None
def finalize_options(self):
if self.identity and not self.sign:
@@ -64,6 +66,22 @@
if not self.repository:
self.repository = self.DEFAULT_REPOSITORY
+if self.bdist_path:
+# 'dist_files' is the list of (command, pyversion, file) that
+# have been created by any dist commands run so far. This is
+# filled regardless of whether the run is dry or not. pyversion
+# gives sysconfig.get_python_version() if the dist file is
+# specific to a Python version, 'any' if it is good for all
+# Python versions on the target platform, and '' for a source
+# file. pyversion should not be used to specify minimum or
+# maximum required Python versions; use the metainfo for that
+# instead.
+from distutils import sysconfig
+self.distribution.dist_files.append(
+('bdist_egg', sysconfig.get_python_version(), self.bdist_path)
+)
+
+
def run(self):
if not self.distribution.dist_files:
raise DistutilsOptionError(No dist file created in earlier command)
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Bug in setuptools version parsing when appending '.dev'?
Am I missing something or is the following a bug whereby adding the
'.dev' tag is doing something weird?
from pkg_resources import parse_requirement as pv
pv('1.0a1.dev') pv('1.0a1')
True
pv('1.0a1') pv('1.0')
True
pv('1.0a1.dev') pv('1.0.dev')
False
pv('1.0a1') pv('1.0.dev')
False
import setuptools
setuptools.__version__
'0.6c8'
This is mainly causing us problems when projects try to track alpha and
beta level bumps in dependencies, such as when project Foo requires
project Bar version 3.0b1 via a requirement like 'Bar = 3.0b1' (which
means we don't want the development prereleases of Bar's first beta
release, but anything after that should be okay.) But then when we
actually want to release Bar 3.0 and change the version number to just
'3.0', suddenly installs fail while we try to run the last set of tests
because '3.0.dev' is older than '3.0b1'.
If it is not a bug, how do you handle situations where you want to run
that last round of testing prior to tagging and building releases? I'd
rather do that AFTER making all source changes, and not have to change
the version number after the testing.
-- Dave
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [issue24] Rename easy_install
Phillip J. Eby wrote: At 06:27 PM 6/16/2008 -0500, Dave Peterson wrote: Phillip J. Eby wrote: At 12:57 PM 6/16/2008 +1200, Greg Ewing wrote: Any discussion of that sort seems to get hopelessly bogged down, so bikeshedding is all that's left for people to do. What about testing patches? Writing test code? Writing more docs? There's *plenty* of less controversial work to go around. The only reason most of the outstanding patches I'm aware of haven't been applied yet is because they haven't been tested. (Or more precisely, because setuptools hasn't been thoroughly tested with the patches applied.) I can help test some of the patches Don't test patches - test setuptools with the patches. :) More precisely, make sure you test things beyond what the patch is supposed to do, to make sure that other things aren't affected. This is particularly important for patches to easy_install, which is ridiculously complicated due to all the obscure edge cases it has to be able to handle. Right. That is what I meant but worded poorly. :-) I've seen people posting to the tracker, but I can't commit to svn. Beyond running the setuptools test suite The test suite is pretty useless for most of these kinds of patches. It essentially only exercises various internals of pkg_resources and a few other things that are almost never touched. I'm talking testing as in actually install some packages in a few different kinds of install targets, using a few different options. I don't have a rigorous process for that, as I tend to pick things on the basis of the code paths to be exercised. But that might not be an option for casual testers. Still, I'd run it anyway. :-) I definitely don't have the depth of experience to know what features being exercised hit what code paths. But I do use setuptools for both building and installing on Windows XP, Mac OS X, and various Linux flavors. We heavily use eggs here at Enthought. :-) If I had it all to do over -- and I didn't simply run screaming from attempting the task in the first place -- I would write a full functional test suite, including chrooting tests if necessary. In the long run, it would have saved enormous amounts of manual test time, and we could have had more people involved in development a long time ago. That, by the way, is why writing test code is on the list above. and verifying that things seems to work for me and my environment, is there anything else that will help get some of the patches into svn? BTW, most of those things you mention all effectively boil down to writing patches in one way or another. :-) How do we make sure that after they get some review they get checked in when it seems so few people have check-in privileges? Phillip, you already mentioned that you're short on time and no one else has responded to a plea for finding out who has check-in privileges. Jim Fulton has previously been blessed by me to apply non-controversial patches to setuptools after giving me a heads-up. (But note that he's probably busier than I am, and unlikely to have bandwidth for stuff that doesn't affect zc.buildout or Zope in some way.) If you want to expand the available development pool for setuptools, I would strongly suggest focusing development efforts on creating a regression test suite emphasizing end-to-end functional testing of the current functionality. Such tests would ideally be factored for narrative clarity and compact expressiveness, rather like Jim Fulton's doctests for easy_install's .exe wrappers, and the doctests for zc.buildout. (Because if they're too complicated for me to read, they'll take too long for me to review.) Okay, we'll see what we can do about that here at Enthought. So far we'd been focusing on bugs / new features that we thought needed to be addressed but that effort can be redirected a bit to helping write tests. I think Tarek suggested a sprint this weekend but I'm not sure if any of our guys will be available that soon. I'll ask around. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [issue24] Rename easy_install
Phillip J. Eby wrote: At 12:57 PM 6/16/2008 +1200, Greg Ewing wrote: Any discussion of that sort seems to get hopelessly bogged down, so bikeshedding is all that's left for people to do. What about testing patches? Writing test code? Writing more docs? There's *plenty* of less controversial work to go around. The only reason most of the outstanding patches I'm aware of haven't been applied yet is because they haven't been tested. (Or more precisely, because setuptools hasn't been thoroughly tested with the patches applied.) I can help test some of the patches I've seen people posting to the tracker, but I can't commit to svn. Beyond running the setuptools test suite and verifying that things seems to work for me and my environment, is there anything else that will help get some of the patches into svn? BTW, most of those things you mention all effectively boil down to writing patches in one way or another. :-) How do we make sure that after they get some review they get checked in when it seems so few people have check-in privileges? Phillip, you already mentioned that you're short on time and no one else has responded to a plea for finding out who has check-in privileges. IIRC, you had previously talked about coming up with a process to bless people but I don't think that went very far. I'm interested in participating but unsure of how to go about the process. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] shebang line modified by setuptools
Gael Varoquaux wrote: On Mon, Apr 14, 2008 at 01:30:14PM +1200, Greg Ewing wrote: Gael Varoquaux wrote: a second Python needs to be installed on top of the system Python to add modules to it. Maybe the system should come with two pythons installed, one for use by the system and the other for users to add things to. Or at least be set up so that it appears that way -- they might share files under the hood. I am quite wary about these proposals, as well as the one environment per application ones. In my experience, it is a fairly standard practice across all platforms to install independent application environments for any large, complex applications that come with a vendor support agreement. Admittedly, that isn't alot of Python applications, especially of the kind you're referring to where users are expected to add, remove, and intermix modules. However, that doesn't mean there aren't any Python apps that do fit the bill -- in fact, Enthought does alot of that. We want our apps to work even if the sysadmin upgrades the system libraries, Python, or what have you, or if the user installs multiple applications that have competing library requirements. What you propose resembles very much to what MacOSX does, and MacOSX seems just so broken for Python. I don't use it, but I see on the different scientific-Python-related mailing lists how users have difficulties with MacOSX, and I have heard many people complain about this feature. As a per-application environment, I think it is bad, because it limits reuse. As I see things, applications should be able to have access to all the Python modules installed, to be able to use them, if they need. I get definitely see applications having more feature if some modules are installed (eg. Sphinx, which does syntax highlighting if pygments is installed). I think this discussion is really going on because Python does not have good library-versioning support. What it needs is to get this, and not to get complete isolation, apart for some system-critical tasks, and only for these. AFAIK this is one of the problems setuptools is trying to solve. Not everybody is happy with the technical solution setuptools has come up, this only means it has to be improved. IMO, this sort of multi-version thing only works if *everything* in the stack participates fully. I think the reason more things don't do this is because of the maintenance load it puts on developers / release managers to bump and synchronize versions apporpriately. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] how to easily consume just the parts of eggs that are good for you
Phillip J. Eby wrote: At 03:48 PM 4/10/2008 -0500, Dave Peterson wrote: Stanley A. Klein wrote: On Wed, 2008-04-09 at 18:17 -0500, Dave Peterson wrote: I think I can sum up any further points by simply asking: Should it be safe to assume I can distribute my application via eggs / easy_install just because it is written in Python? I think that based on this discussion the bottom line answer to this question is No. I agree that it seems like that's where things are headed in the discussion. But the discussion doesn't always coincide with the reality, right? I guess I'm trolling more for a statement from the setuptools maintainer here. Particularly since I'm looking for an answer to my question about should Enthought continue to invest time into a setuptools patch that lets developers include docs, config files, etc. in eggs for installation in a FHS-approved location at install time? I think it's more than reasonable to define a standard for including such data. I'm somewhat more skeptical about doing that installation automatically within easy_install. Likewise, I'm skeptical about doing other sorts of non-package, non-script installation. I'd like to see proposals that show due care to cross-platformness, uninstallability, etc. In other words, when it comes to a patch -- the documentation is going to count for a lot more than the code, and I'd rather see a concrete proposal well in advance of the patch. Sooner would be better than later, too, because it's likely that the plan for non-egg installs is going to be affected by the plan as well. I believe I understand, and agree, with your concerns. Let me be clear on the status of where we are in our work: we've internally talked through a number of design possibilities, and are now trying out (via hacking on setuptools) how the one we thought was best worked out. In particular, we're concerned about the difficulty of use in terms of even just the use-cases we have for ETS projects. Also, since we do a bit of cross-platform deployment, we're also investigating those effects on the design as well. That being said, I don't think we're ready to put forward a proposal that would withstand too much public scrutiny quite yet - at least if the resulting discussion implied a significant time or effort commitment on our part to carry the conversation forward. If it sounded like we'd already figured it all out, I apologize for getting people's hopes up! I just wanted to make sure further pursuit in this direction on our part wasn't completely wasted. The above not withstanding, if anyone is interested in talking about it / helping us, we certainly wouldn't ignore you. I just can't promise immediate responses due to pretty pressing customer commitments on our part. Regarding the mention of 'uninstallability' above, I assume it would be sufficient if the installed files were simply included in the generated list of files provided by the --record option since there is currently no uninstall command at all? Or is there something else you'd like to see as an intermediate measure? I'd love to add an uninstall option to easy_install as well (it's something we get hit about alot by our user community) but there's only so many hours in a day. ;-) -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Setting up a setuptools community
Jeff Rush wrote: Dave Peterson wrote: Phillip J. Eby wrote: At 11:40 PM 4/1/2008 -0500, Dave Peterson wrote: I've previously volunteered Enthought to do about the same thing (except to include svn with the Trac instance) but there wasn't much response beyond Jeff Rush saying he was already setting one up, and the response by Phillip to use the python dev bug-tracker and svn instead. ... Phillip, I think everyone is waiting for you to provide direction so can you make some decisions about how setuptools evolves to a larger community effort? As I believe you just pointed out, I already did that. :) Egggcellent. Doing a search for 'setuptools' in the issue tracker only returns two open issues currently. I'm guessing I'm not catching all the issues as a result.It would probably help if we had a clear component to use (i.e. Setuptools). Anyone have any idea who to contact to get that done? I've so far had no luck reading the info on www.python.org/dev looking for this sort of thing. Hi Dave, At PyCon I asked the core team about using the Python tracker for setuptools but Brett Cannon said that no software outside of the Python core is currently using it and it wouldn't be appropriate for setuptools to do so. They were willing to give us our own instance of a tracker, similar to how they have for Jython. Martin v. Löwis has been in the process since PyCon of doing so, but is busy and apparently the Roundup software used is tricky to configure for this and,f for the email address, requires ISP intervention. The plan when this is done is to have a tracker at http://bugs.python.org/setuptools/ with an email address for submission/commenting of [EMAIL PROTECTED] Sorry for the delay - I didn't realize this would be such a big deal to get started. We could also start using Launchpad instantly, but I figured reaching concensus to do so, with Phillip and others more familiar with the Python tracker, would be difficult. While I'm glad to hear there has been progress, I'm concerned we risk losing momentum by waiting. Can we at least start by using their wiki to capture some thoughts on priorities and roadmap? Or does that need to be separate too? Or does Phillip already have that somewhere I haven't seen? -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Setting up a setuptools community
Phillip J. Eby wrote: At 11:40 PM 4/1/2008 -0500, Dave Peterson wrote: I've previously volunteered Enthought to do about the same thing (except to include svn with the Trac instance) but there wasn't much response beyond Jeff Rush saying he was already setting one up, and the response by Phillip to use the python dev bug-tracker and svn instead. ... Phillip, I think everyone is waiting for you to provide direction so can you make some decisions about how setuptools evolves to a larger community effort? As I believe you just pointed out, I already did that. :) Egggcellent. Doing a search for 'setuptools' in the issue tracker only returns two open issues currently. I'm guessing I'm not catching all the issues as a result.It would probably help if we had a clear component to use (i.e. Setuptools). Anyone have any idea who to contact to get that done? I've so far had no luck reading the info on www.python.org/dev looking for this sort of thing. Phillip, I don't believe you responded to either my, nor Jeff Rush's, volunteering for being patch reviewers. At least I haven't seen any response -- perhaps that's you trying to tell me something? :-) What's the process you want to follow on this? Ideally things should be opened up where more people are 'blessed' to commit directly, but I'm not sure how you want to do the 'bless'ing part. Finally, what is the build and test process you've been following? Is there a buildbot that's building and publishing results somewhere or does that also need to be set up? -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Hi Mark, Shortcuts don't, but file associations to those shortcuts do (at least to the best of my understanding,) adding paths does, etc. I agree that a library (extensions or no) doesn't need to do these things. But anything targeted at an end-user on Windows has a reasonable chance of wanting them. Here I thought I was being pretty clear that I was only requesting that things not be done to explicitly make it hard (or impossible) for someone else to extend setuptools (or whatever replaces / refactors it) to do these things. -- Dave Mark Hammond wrote: Even installing shortcuts doesn't need to munge the registry! But regardless, you seem to be arguing that setuptools should morph into a full-blown, general purpose installer for python build apps, capable of doing all kinds of platform specific things which any app may desire -- and while I don't agree with that, it wasn't what was being discussed at all. Writing and installing a Python extension does not need to munge the registry. Installing a general purpose application, written in Python or anything else, will require, in the general case, anything you could possible imagine. Mark *From:* Dave Peterson [mailto:[EMAIL PROTECTED] *Sent:* Wednesday, 2 April 2008 3:30 PM *Cc:* Mark Hammond; distutils-sig *Subject:* Re: [Distutils] [Python-Dev] How we can get rid of eggs for 2.6 and beyond Mark Hammond wrote: (Note: I'm aware that people believe it to be necessary to munge the Windows registry when installing Python packages; I just don't agree with the practice, and don't think we should distort Python's process to coddle it). Whoever thinks it necessary is misguided. Installing a package doesn't require munging the registry and none of the popular installation techniques do. Installing Python itself does, and some packages have special requirements (eg, pywin32 registering COM objects), but in general, Python packages on Windows can ignore the registry. I guess I'll step up and volunteer myself as 'misguided' then. :-) I would very much like to be able to package up my application or library such that it created desktop/start menu/quicklaunch icons for various things like running the app, running examples, opening docs for browsing, etc. I'd also like to be able to associate certain file types with executables/scripts/shortcuts provided by the installation of my project. After all, Windows users in general are not as technically adapt when it comes to command line tools so setting up these nice GUI ways of doing things adds usability significantly. You could argue (like Phillip has) that these operations should be done via an explicitly user-invoked script, and I can buy that for the standard version of the tool. You could also argue that installing applications (which is generally where these kinds of desires come into play) is not the job of the tools we're discussing. But it seems to me that the existing capabilities of setuptools are 80% (or more) of the effort in creating a tool that would allow installation and efficient maintenance of large Python-based applications, such as what my employer delivers to customers. All that being said, I'm fine with the idea that the standard library version of the tool does not enable this. Just so long as nothing is done to actively prevent extensions of that tool to do this sort of thing for those who need or want it. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Python-Dev] How we can get rid of eggs for 2.6 and beyond
Mark Hammond wrote: (Note: I'm aware that people believe it to be necessary to munge the Windows registry when installing Python packages; I just don't agree with the practice, and don't think we should distort Python's process to coddle it). Whoever thinks it necessary is misguided. Installing a package doesn't require munging the registry and none of the popular installation techniques do. Installing Python itself does, and some packages have special requirements (eg, pywin32 registering COM objects), but in general, Python packages on Windows can ignore the registry. I guess I'll step up and volunteer myself as 'misguided' then. :-) I would very much like to be able to package up my application or library such that it created desktop/start menu/quicklaunch icons for various things like running the app, running examples, opening docs for browsing, etc. I'd also like to be able to associate certain file types with executables/scripts/shortcuts provided by the installation of my project. After all, Windows users in general are not as technically adapt when it comes to command line tools so setting up these nice GUI ways of doing things adds usability significantly. You could argue (like Phillip has) that these operations should be done via an explicitly user-invoked script, and I can buy that for the standard version of the tool. You could also argue that installing applications (which is generally where these kinds of desires come into play) is not the job of the tools we're discussing. But it seems to me that the existing capabilities of setuptools are 80% (or more) of the effort in creating a tool that would allow installation and efficient maintenance of large Python-based applications, such as what my employer delivers to customers. All that being said, I'm fine with the idea that the standard library version of the tool does not enable this. Just so long as nothing is done to actively prevent extensions of that tool to do this sort of thing for those who need or want it. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] setuptools bug report: _get_unpatched() is way too clever (and also returns the wrong module)
zooko wrote: I would be willing to host a trac instance to serve as a wiki and issue tracker for setuptools, and a buildbot. (But I don't want to have anything to do with subversion -- I much prefer darcs.) I've previously volunteered Enthought to do about the same thing (except to include svn with the Trac instance) but there wasn't much response beyond Jeff Rush saying he was already setting one up, and the response by Phillip to use the python dev bug-tracker and svn instead. (No one previously talked about buildbot but that makes great sense.) I haven't seen any follow-up on those discussions though. Did I miss it? ;-) I'd really like to get this resolved, along with the issues of who can commit to the codebase, so that more people can start working on some of this stuff being discussed. Phillip, I think everyone is waiting for you to provide direction so can you make some decisions about how setuptools evolves to a larger community effort? -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Request for Input re Packaging
Jeff Rush wrote: 2. If we moved PyPI to serve exclusively over https, for integrity reasons, would this have a major negative impact? Given that urllib2 doesn't support https through a proxy, it would probably cause a problem for easy_install, etc. :-) We had to create a custom urllib2 Handler for a couple of our applications. I just looked to see if I should pursue re-working our solution as a patch and saw that Christopher Li has already posted a patch. http://bugs.python.org/issue1424152 And there is also this outstanding ticket: http://bugs.python.org/issue1448934. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns
Jeff Rush wrote: Dave Peterson wrote: I agree. Let's get that setuptools wiki started and start documenting some of these ideas as a roadmap so that anyone who wants to help out has an idea of what to work on, or factor into what they're currently working on. Anyway, since Enthought is already scratching, I'm fine with the idea of building a standard way to do it that is driven by human-readable data. We just need to setup the process to allow that to happen. So far I haven't seen any responses from you in regards to the setup of an issue/patch tracker, wiki, process to open up the number of commiters, etc. that gives me any confidence I'm not heading off down the wrong path somehow. Perhaps I'm too cautious? Dave, I'm in the process of getting a tracker for setuptools, and I'll work on the wiki shortly, although we have the PackagingBOF wiki for idea collection at the moment. Give me a couple of days, including travel from PyCon. I'm fired up to make this happen. Awesome. Travis O came back into the office from being at PyCon today and I've finally got the full set of e-mail addys from the people who attended the PackagingBOF meetings. I'd like to e-mail everyone with info about the tracker and wiki, and possibly a new dev type mailing list seeded with these names. Anything I can send out now? :-) Actually, I wonder if instead of trying to enhance setuptools for post-install, if maybe we should be looking at buildout recipes and maybe having a way for setuptools dependencies to point to buildout specs. IIRC, buildout specs can be remotely retrieved from a single URL, too. I'll need to read up more on buildout to understand this, but my understanding was that buildout was not something a user ran to install an app, but rather something the developer ran to build and publish an app. The end result of a 'production' buildout is to generate a large tarball or rpm that included everything, right? If so, this goes directly against what Enthought was aiming for, which was to allow delivery of bug-fixes and minor updates in a large app by downloading only smaller units instead of a huge monolithic re-install of everything. Your view of a fine-grained application bundle with the ability to dynamically download updated eggs without re-pulling the entire thing is an interesting contrast to Paul's view of a more monolithic application for easier add/remove/uninstall completeness. Supporting both usage models is going to be a challenge but I think is feasible with some thought. Yes, that will be interesting indeed. I'm not sure we need anything more to support what we'd like to do than to resolve some dependency lookup issues that I've already talked about on this list and at the PackagingBOF. -- Dave -Jeff ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Consistent platform name for 64bit windows
Christian Heimes wrote: [EMAIL PROTECTED] schrieb: So, at the risk of painting a bike-shed, I'd like to propose that we adopt 'AMD64' in distutils (needs a change), platform.py (needs a change to use sys.getwindowsversion() in preference to pywin32, if possible, anyway), and the Python banner (which already uses AMD64). +1 for AMD64 I'm also +1 for AMD64 (or amd64). BTW, the wikipedia entry for AMD64 points out that FreeBSD, NetBSD, OpenBSD, Debian, Ubuntu, Solaris, and the Java Development Kit all use amd64. Though it claims that Microsoft and Sun both use 'x64' in their marketing material. If we ever need names for Itanium and i386 compatible arch I propose IA64 and X86. IA64 seems pretty standard so +1 on that. But I'm -1 on the X86. It's too easy to get confused with what that means versus terms like x86-16, x86-32, and x86-64. Admittedly the first two are little used, but I have seen them once or twice. What's wrong with using i386 itself? -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Capsule Summary of Some Packaging/Deployment Technology Concerns
I've added your comments to a wiki page (http://wiki.python.org/moin/PackagingBOF) I was working on to summarize some of what went on during these BoF meeting, at least from the Enthought point-of-view. Unfortunately, I wasn't at the first night's event and don't yet have Travis Oliphant's notes on it here in front of me (he's still sprinting) so I only added some more detail to your comments, and also noted some previous issues we'd briefly discussed via e-mail with Phillip. It was great to see so many people interested in sharing their experiences and wanting to help things get better! As you can probably guess as a result of this being a two-night meeting, there wasn't enough time to discuss everything that needed to be brought up. It was suggested that a wiki page be created (see above) and that a new mailing list be setup for those who wanted to discuss further. (Some didn't feel the existing distutils-sig was appropriate.) I'll try to get the latter done shortly. -- Dave Jeff Rush wrote: I was in a Packaging BoF yesterday and, although not very relevant to the packager bootstrap thread, Guido has asked me to post some of the concerns. The BoF drew about 15 people... snipped ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns
Phillip J. Eby wrote: At 05:10 PM 3/17/2008 -0500, Jeff Rush wrote: 1. Many felt the existing dependency resolver was not correct. They wanted a full tree traversal resulting in an intersection of all restrictions, instead of a first-acceptable-solution approach taking now, which can result in top-level dependencies not being enforced upon lower levels. The latter is faster however. One solution would be to make the resolver pluggable. Patches welcome, on both counts. Personally, Bob and I originally wanted a full-tree intersection, too, but it turned out to be hairier to implement than it seems at first. My guess is that none of the people who want it, have actually tried to implement it without a factorial or exponential O(). But that doesn't mean I'll be unhappy if somebody succeeds. :) I think we'd make significant progress by just intersecting the dependencies we know about as we progress through the dependency tree. For example, if A requires B==2 and C==3, and if B requires C=2,=4, then at the time we install A we'd pick C==3 and also at the time we install B we'd pick C==3. As opposed to the current scheme that would choose C==4 for the latter case. This would allow dependent projects (think applications here) to better control the versions of the full set of libraries they use. Things would still fail (like they do now) if you ran across dependencies that had no intersection or if you encountered a new requirement after the target projected was already installed. If you really wanted to do a full-tree intersection, it seems to me that the problem is detecting all the dependencies without having to spend significant time downloading/building in order to find them out. This could be solved by simply extending the cheeseshop interface to export the set of requirements outside of the egg / tarball / etc. We've done this for our own egg repository by extracting the appropriate meta-data files out of EGG-INFO and putting it into a separate file. This info is also useful for users as it gives them an idea of how much *new* stuff is going to be installed (a la yum, apt-get, etc.) In other words, we attempt to achieve heuristically what's being proposed to do algorithmically. And my guess is that whatever cases the heuristic is failing at, would probably not be helped by an algorithmic approach either. But I would welcome some actual data, either way. With our ETS projects, we've run into problems with the current heuristic. Perhaps we just don't know how to make it work like we want? We have a set of projects that we want to be individually installable (to the extent that we limit cross-project dependencies) but we also want to make it easy to install the complete set. We use a meta-egg for the latter. It's purpose is only to specify the exact versions of each project that have been explicitly tested to work together -- you could almost think of it as a source control system tag. Whereas on the individual projects, we explicitly want to ensure that people get the latest possible release of each required API so the version requirements are wider here. This setup causes problems whenever we release new versions of projects because it seems easy_install ignores the meta-egg exact versions when it gets down into a project and comes across a wider cross-project dependency. We ended up having to give up on the ranges in the cross-project dependencies and synchronize them to the same values in the meta-egg dependencies. There are numerous side-effects of this that we don't like but we haven't found a way around it. Again, though, patches are welcome. :) (Specifically, for the trunk; I don't see a resolver overhaul as being suitable for the 0.6 stable branch.) We're planning to pursue this (for the above mentioned strategy) as soon as we work ourselves out of a bit of a backlog of other things to do. 2. People want a solution for the handling of documentation. The distutils module has had commented out sections related to this for several years. As with so many other things, this gets tossed around the distutils-sig every now and then. A couple of times I've thrown out some options for how this might be done, but then the conversation peters out around the time anybody would have to actually do some work on it. (Me included, since I don't have an itch that needs scratching in this area.) In particular, if somebody wants to come up with a metadata standard for including documentation in eggs, we've got a boatload of hooks by which it could be done. Nothing's stopping anybody from proposing a standard and building a tool, here. (e.g. using the setuptools command hook, .egg-info writer hook, etc.) Enthought has started an effort (it's currently one of two things in our ETSProjectTools project at
Re: [Distutils] *ugh* - failure to easy_install numpy
[EMAIL PROTECTED] wrote: Any idea what the problem might be? I'm using easy_install 0.6c7 with Python 2.6 (Subversion checkout). I don't know if 0.6c7 is the most recent version or not. Sorry, I've got no idea on the problem but I can answer your other question. There is at least a 0.6c8 that has been released. I'm pretty sure you'll always be able to check for the latest version here: http://pypi.python.org/pypi/setuptools -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] wanted: a catalog of setuptools plugins
zooko wrote: Following up to my own post: On Jan 27, 2008, at 9:17 PM, zooko wrote: One idea that I had was to run a query on the Python Package Index database to find all packages whose setup.py's define entry points in the distutils.commands group name. Another was to put the word out (i.e. in the setuptools documentation) that if your package is a setuptools plugin, that you should name your package setuptools_blah_blah and register it on the Python Package Index. And this morning I just noticed the keywords field in the metadata. Perhaps the setuptools docs could encourage authors of setuptools plugins to put the keyword plugin into their keywords? I think something along the latter lines is better than imposing a specific name format on projects. For example, we have some projects that include entry points for 'distutils.commands' but they also do other things so we wouldn't want to imply they only do setuptools by having a name like 'setuptools_blah_blah'. I'd also point out that there are other types of plugins (Trac, Enthought's Envisage, etc.) so perhaps the keyword should be 'distutils_plugin' or 'setuptools_plugin'? -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] installing scripts into the user's PATH
Phillip J. Eby wrote: At 12:03 PM 1/3/2008 -0600, Carl Karsten wrote: Scripts are files containing Python source code, intended to be started from the command line. http://docs.python.org/dist/node11.html I want to do that. I am hoping that once it is done, it will be in a dir that is in the user's OS PATH. (if that sounds restrictive, I didn't say it right. at this point, I don't care what dir it is in, as long as the user doesn't have to type it. like /usr/bin on linux. not sure where the right place would be on mac/win.) Is this supported? The default locations on everything but Windows will generally have this work. For Windows, there isn't much you can do except tell people to make sure the Python Scripts directory is added to their PATH. Just curious, but is there a reason setuptools uses 'Scripts' on Windows instead of the 'Tools' dir? Unless I'm mistaken, a a new install of Python 2.5 puts scripts in 'Tools'. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] installing scripts into the user's PATH
Phillip J. Eby wrote: At 05:09 PM 1/3/2008 -0200, phil jones wrote: I take it there's nothing in distutils that can actually update the user's OS PATH? Nope. This may not help in the general case, but if you have a more specific, say customer delivery, issue then you might look at Enthought's open-source Enstaller app -- which is a wrapper around easy_install. It does support running of post-install scripts that do things like this, and goes even further on Windows where it can add start menu icons and registry entries. Unfortunately, 'Enstaller' is still somewhat immature and the current version is way, way too bloated but we at Enthought are continuing to invest in it and we do use it already. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] installing scripts into the user's PATH
Phillip J. Eby wrote: At 05:00 PM 1/3/2008 -0600, Dave Peterson wrote: Just curious, but is there a reason setuptools uses 'Scripts' on Windows instead of the 'Tools' dir? Unless I'm mistaken, a a new install of Python 2.5 puts scripts in 'Tools'. It puts Python-supplied scripts there, yes, but the convention of installing user scripts in the Scripts subdirectory goes back at least a decade. Which doesn't mean it's right, but it's certainly not *new*. :) (Oh, and it's *distutils* that chooses that location, not setuptools.) Right, sorry for the confusion on which package was doing it. I just wish either distutils or the .msi installer was changed because it is confusing to describe what to do to users (especially those new to Python and/or Windows) when the Python .msi on windows doesn't even create the Scripts sub-dir, much less add it to a path environment variable for you. :-( -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Sandbox violation help
Robert Kern wrote: Dave Peterson wrote: Most of us here are not seeing this error so I'm thinking it isn't something obviously wrong with the setup.py. You can look at the setup.py at : https://svn.enthought.com/enthought/browser/tags/enthought.interpolate_2.0.0b3/setup.py Note that this project does use numpy.distutils' setup function. Perhaps the error has something to do with that? enthought.interpolate uses weave to built its extension module. weave uses ~/.python24_compiled as its location for the temporaries. We need to fix weave to not do this, or rewrite the extension not to use weave. The latter is probably best. I'm probably just being slow here, but how come the build of enthought.interpolate works for some (I'll assert most here) people but not others? -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Sandbox violation help
Dave Peterson wrote:
Phillip J. Eby wrote:
At 02:15 PM 12/6/2007 -0600, Dave Peterson wrote:
PPS: I do believe other users have reported the same issues on other
systems but we've never quite figured out what the cause is. IIRC,
telling them to manually download the source tarball and do a 'python
setup.py install' seems to work just fine.
Processing enthought.traits-2.0.1b1.tar.gz
Running enthought.traits-2.0.1b1/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-Edu8TK/enthought.traits-2.0.1b1/egg-dist-tmp-IIITHW
error: Setup script exited with error: SandboxViolation:
open('/tmp/easy_install-WZrKOz/enthought.tvtk-2.0.0b2/temp/tmpTTJH2x/zZe82A
', 131266, 384) {}
This looks like a bug that was fixed about a year ago: the sandboxing
for os.open() (which is what the above appears to be calling) wouldn't
allow you to write files *anywhere*, even if it was under a safe
directory location.
That would explain why you don't get the problem, but the user does.
Check their setuptools version, using:
python -c from pkg_resources import require; print
require('setuptools')
The bug was fixed in 0.6c5, so anything older will have this problem
with anything that uses os.open() to write files.
Thanks!
Well, turns out I was premature on assuming this fixed the problem. The
user was originally using 0.6c3 but has since upgraded. Here's the
current output from your request as sent back by the user:
In [3]: pkg_resources.require(setuptools)
Out[3]: [setuptools 0.6c7 (/usr/lib/python2.4/site-packages)]
The original user is still reproducibly getting the following error --
it appears to have moved to a different project, but otherwise seems the
same to me. In addition, one of our internal developers said he was
getting this on one of his machines earlier as well but he was able to
work around it by manually building the project in question and then
going back to using easy_install to install the rest of the dependencies
of the originally requested project. Most of us here are not seeing this
error so I'm thinking it isn't something obviously wrong with the
setup.py. You can look at the setup.py at :
https://svn.enthought.com/enthought/browser/tags/enthought.interpolate_2.0.0b3/setup.py
Note that this project does use numpy.distutils' setup function. Perhaps
the error has something to do with that?
Does anyone have any tips on how to go about debugging what might be
wrong? In particular, is there any way to get an idea of what the
particular file being opened actually represents?
-- Dave
Searching for enthought.interpolate
Reading http://code.enthought.com/enstaller/eggs/source
Best match: enthought.interpolate 2.0.0b3
Downloading
http://code.enthought.com/enstaller/eggs/source/enthought.interpolate-2.0.0
b3.tar.gz Processing enthought.interpolate-2.0.0b3.tar.gz
Running enthought.interpolate-2.0.0b3/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-jA35_C/enthought.interpolate-2.0.0b3/egg-dist-tmp-m08e-2
error: Setup script exited with error: SandboxViolation:
open('/home/hans/.python24_compiled/dummy_TuxOnStage_5383_gODMbl', 131266,
384) {}
The package setup script has attempted [...]
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Sandbox violation help
A user is trying to install Enthought's ETS projects on a Gentoo system
and getting the following error message. I'm not seeing it on my
systems, but then I'm not running Gentoo. Can anyone give me a clue as
to what might be happening or how to go about determining what is
happening? The error message has me really confused since
'temp/tmpTTJH2x/zZe82A' doesn't really give me much to go on as to what
is being attempted outside of the sandbox. It doesn't help that one
line makes it clear something is going on with a tmp directory for
enthought.traits but then after that is the error about a tmp dir for
enthought.tvtk. How'd we get back to enthought.tvtk in the middle of
the enthought.traits install?
-- Dave
PS: 'enthought.traits' is one of the install_requires dependencies of
'enthought.tvtk' so easy_install probably just chained to building and
installing it after enthought.tvtk, right?
PPS: I do believe other users have reported the same issues on other
systems but we've never quite figured out what the cause is. IIRC,
telling them to manually download the source tarball and do a 'python
setup.py install' seems to work just fine.
Processing enthought.traits-2.0.1b1.tar.gz
Running enthought.traits-2.0.1b1/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-Edu8TK/enthought.traits-2.0.1b1/egg-dist-tmp-IIITHW
error: Setup script exited with error: SandboxViolation:
open('/tmp/easy_install-WZrKOz/enthought.tvtk-2.0.0b2/temp/tmpTTJH2x/zZe82A
', 131266, 384) {}
The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.
This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand. Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Sandbox violation help
Phillip J. Eby wrote:
At 02:15 PM 12/6/2007 -0600, Dave Peterson wrote:
PPS: I do believe other users have reported the same issues on other
systems but we've never quite figured out what the cause is. IIRC,
telling them to manually download the source tarball and do a 'python
setup.py install' seems to work just fine.
Processing enthought.traits-2.0.1b1.tar.gz
Running enthought.traits-2.0.1b1/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-Edu8TK/enthought.traits-2.0.1b1/egg-dist-tmp-IIITHW
error: Setup script exited with error: SandboxViolation:
open('/tmp/easy_install-WZrKOz/enthought.tvtk-2.0.0b2/temp/tmpTTJH2x/zZe82A
', 131266, 384) {}
This looks like a bug that was fixed about a year ago: the sandboxing
for os.open() (which is what the above appears to be calling) wouldn't
allow you to write files *anywhere*, even if it was under a safe
directory location.
That would explain why you don't get the problem, but the user does.
Check their setuptools version, using:
python -c from pkg_resources import require; print
require('setuptools')
The bug was fixed in 0.6c5, so anything older will have this problem
with anything that uses os.open() to write files.
Thanks!
-- Dave
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Enthought-dev] Building Fedora RPMs
Phillip J. Eby wrote: At 10:37 AM 10/23/2007 -0400, Stanley A. Klein wrote: On Mon, 2007-10-22 at 18:15 -0500, Dave Peterson wrote: Stanley A. Klein wrote: Other failures included: endo - unpackaged pyc and pyo files (looks like a setup.py issue) I get rpms built for this just fine. Are you sure you have no local changes? The problem with this one may be a setuptools issue. The failure in doing the bdist_rpm is on a file endo.py, which is identified to setuptools in the setup.py as a script. The file is originally in enthought/endo/scripts. The pyc and pyo files get built there properly because of the [install] optimize=1 option in setup.cfg. However, the INSTALLED_FILES also lists a /usr/bin/endo.py without the pyc and pyo files. The Fedora rpm system causes those optimization files to be built unbeknownst to setuptools, which then causes the installed but unpackaged files error. I tried declaring the pyc and pyo files in the script statement, but at that point they don't exist and I got an error to that effect. There is probably a workaround, like commenting out the scripts statement in the setup.py and providing rpm a post-install script to copy endo.* into /usr/bin. However, it would create separate setup.py files for rpm and non-rpm packaging. Better still, take off the '.py' on the 'endo' script, and/or use setuptools' script generation support. I'll take a stab at switching it to use the setuptools script support. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Enthought-dev] Building Fedora RPMs
Phillip J. Eby wrote: At 10:37 AM 10/23/2007 -0400, Stanley A. Klein wrote: On Mon, 2007-10-22 at 18:15 -0500, Dave Peterson wrote: Stanley A. Klein wrote: Other failures included: endo - unpackaged pyc and pyo files (looks like a setup.py issue) I get rpms built for this just fine. Are you sure you have no local changes? The problem with this one may be a setuptools issue. The failure in doing the bdist_rpm is on a file endo.py, which is identified to setuptools in the setup.py as a script. The file is originally in enthought/endo/scripts. The pyc and pyo files get built there properly because of the [install] optimize=1 option in setup.cfg. However, the INSTALLED_FILES also lists a /usr/bin/endo.py without the pyc and pyo files. The Fedora rpm system causes those optimization files to be built unbeknownst to setuptools, which then causes the installed but unpackaged files error. I tried declaring the pyc and pyo files in the script statement, but at that point they don't exist and I got an error to that effect. There is probably a workaround, like commenting out the scripts statement in the setup.py and providing rpm a post-install script to copy endo.* into /usr/bin. However, it would create separate setup.py files for rpm and non-rpm packaging. Better still, take off the '.py' on the 'endo' script, and/or use setuptools' script generation support. Thanks for pointing that out! I've just gotten the time to try modifing the endo project to use the setuptools' script generation -- went pretty easily btw. I can now successfully do a bdist_rpm on my CentOS5 system. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Where is easy_install finding this dependency from?
We've gotten a number of problem reports from people trying to install
our egg releases where there doesn't seem to be enough information being
output to the console to really debug the problem. Here's an
illustrating log taken from a user's report:
===
C:\Documents and Settings\Garyeasy_install -f
http://code.enthought.com/enstaller/eggs/windows/xp ets
Searching for ets
Reading http://code.enthought.com/enstaller/eggs/windows/xp
Best match: ets 2.5b2
Downloading
http://code.enthought.com/enstaller/eggs/windows/xp/ets-2.5b2-py2.5.egg
Processing ets-2.5b2-py2.5.egg
Moving ets-2.5b2-py2.5.egg to c:\python25\lib\site-packages
Adding ets 2.5b2 to easy-install.pth file
Installed c:\python25\lib\site-packages\ets-2.5b2-py2.5.egg
Reading http://code.enthought.com/enstaller/eggs/source
Reading http://code.enthought.com/enstaller/eggs/source/unstable
Processing dependencies for ets
Searching for wx==2.6
Reading http://cheeseshop.python.org/pypi/wx/
Couldn't find index page for 'wx' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://cheeseshop.python.org/pypi/
No local packages or download links found for wx==2.6
error: Could not find suitable distribution for Requirement.parse
('wx==2.6')
===
I've verified that our 'ets' egg does not even mention 'wx' in its
setup.py so something else on this user's system must be requiring it,
right? But there's no information here about where the requirement came
from.
In fact the situation is even worse than that because easy_install
outputs the line about processing dependencies for ets, but the next
step is about some other egg's dependencies and not one of the 20+
dependencies actually listed in the 'ets' egg.
Can easy_install be updated to inform the user of which egg, or the
first one if there are many of them, that generated the requirement on
each thing it is trying to install? I'm thinking of a change in the
line that looks like:
Searching for wx==2.6
to something like
Searching for wx==2.6 (required by foo.bar-1.0)
It also seems like the error line should indicate which egg generated
the requirement for 'wx==2.6', right?
-- Dave
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Finding eggs when dependency_links and user specified URLs both given to easy_install?
What happens if a user specifies a URL via a '-f' option on the easy_install command line but our published eggs specify dependency_links in the setup.py? Will easy_install search the user's specified location first, last, or will it not search them at all? (I'm hoping for the first!) I'm asking because we want to provide binaries (eggs) of our projects for various platforms (RHEL3, RHEL4, RHEL5, Ubuntu 7.04, etc.) all of which generate the same egg name, but our projects include C /C++ extensions that require compilation for the specific platform, so we've structured an egg repository that allows us to separate these equivalently named eggs by the actual target platform. We also want users to be able to install from source tar/zip balls for platforms we haven't built binaries for yet, so our eggs all include dependency_links to point at our egg repo. Due to issues with build management, we aren't currently listing the target platform URLs in each egg's setup.py's dependency_links when it gets built on that target platform. It just has the source tar/zip ball repository's URL. But early indications are that when a user installs by doing something like: easy_install -f enthought platform specific repo url ets==2.5b2 then the first egg gets found properly in the platform specific repo, but all of its dependencies are only being searched for in the source repo. Is this just a misinterpretation of the output generated by easy_install? BTW, it would be nice if there was an option to name your generated eggs based on a full specification of OS and version, hardware arch, etc. And of course, the whole setuptools universe of tools would need to know to search for eggs with that level of accuracy before dropping down to more generic specs. Another similar use case: Some of our users want to locally cache eggs for distribution within their corporate environment for various reasons (eggs tested and approved to work with their corporate desktop standard, etc.) Those users would also be specifying a '-f' option to easy_install to point at their corporate cache. However, all of our eggs will have dependency_links specified. Will these corporate users be able to install and pull things from their local repository, or will easy_install always look at our dependency_links' specified locations first? It doesn't seem convenient to force them to have to remember to provide a '-H' setting, though I guess their IT group could create an alias or batch script that does this for them. I also guess that their IT group could generate their own version of easy_install that customizes / hard-codes the URLs that it would look at, but that seems like kind of a long way to go to solve this problem. -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] [Enthought-dev] Namespace packages and egg-info confusion when using bdist_rpm
Ryan May wrote:
Ryan May wrote:
On Thu, July 26, 2007 2:01 pm, Phillip J. Eby wrote:
At 01:37 PM 7/26/2007 -0400, Stanley A. Klein wrote:
I had the same error. Should I have put in an import enthought in all
the others?
No, apparently the manual import doesn't help. Presumably the ones
you changed need something more like this:
import enthought; enthought.traits = sys.modules['enthought.traits']
import enthought.traits; enthought.traits.ui =
sys.modules['enthought.traits.ui']
I thought the import would be sufficient, but apparently it's not. :(
Again, let me know if it works so I can change setuptools accordingly.
This solution worked for me initially, but now I'm getting warnings when
I run any python script I get:
warning: Not importing directory
'/usr/lib64/python2.5/site-packages/enthought/traits': missing __init__.py
warning: Not importing directory
'/usr/lib64/python2.5/site-packages/enthought/traits/ui': missing
__init__.py
This is when I added the above lines to the .pth files for both traits
and traits.ui. What am I doing wrong?
Turns out that another package (kiva) decided it would be a good idea to
put a __init__.py in the enthought directory, which made python angry.
Sorry for the noise, distutils.
Enthought guys, anybody wanna take a crack at why kiva (and only kiva)
insisted on adding a __init__.py to enthought?
Well, I just checked the source and 'enthought' is declared as a
namespace package in setup.py, and the enthought/__init__.py contains
nothing but:
try:
__import__('pkg_resources').declare_namespace(__name__)
except:
pass
which is what our other projects have. So, since I'm not a bdist_rpm
expert, I'm at a loss of even where to begin on this one.
-- Dave
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Best practices on distributing documentation and examples with eggs?
Over on the enthought-dev mailing list we're having a bit of a discussion on what the best way to distribute documenation and examples is for projects that we distribute binary eggs for. The general consensus is that it would be very nice indeed if there was a way to generate a tarball, or platform install, of just documentation and examples so that people wouldn't need to download a full, presumably significantly larger, source tarball. Another option would be that eggs included the documentation and examples and that, during the installation of the egg, those docs and examples were relocated to a common location (outside of the zip) to make access by users more convenient. This latter idea is similar to how Ruby Gems deal with docs. I don't claim to be a distutils or setuptools guru, so it shouldn't be too surprising that I can't seem to find anything about a setuptools or distutils command to do either of these. Am I missing something? If not, does it seem like something that might be worthy of putting into setuptools? -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Best practices on distributing documentation and examples with eggs?
Stanley A. Klein wrote: On Thu, July 26, 2007 12:46 pm, Phillip J. Eby wrote: At 03:02 AM 7/26/2007 -0500, Dave Peterson wrote: Over on the enthought-dev mailing list we're having a bit of a discussion on what the best way to distribute documenation and examples is for projects that we distribute binary eggs for. The general consensus is that it would be very nice indeed if there was a way to generate a tarball, or platform install, of just documentation and examples so that people wouldn't need to download a full, presumably significantly larger, source tarball. Another option would be that eggs included the documentation and examples and that, during the installation of the egg, those docs and examples were relocated to a common location (outside of the zip) to make access by users more convenient. This latter idea is similar to how Ruby Gems deal with docs. I don't claim to be a distutils or setuptools guru, so it shouldn't be too surprising that I can't seem to find anything about a setuptools or distutils command to do either of these. Am I missing something? There are a few different ways you could do this. The easiest would be to put a docs subdirectory either inside one of your packages or inside your .egg-info directory, then use the pkg_resources resource or metadata APIs to list and extract them. One advantage to using something like .egg-info/docs would be that this could perhaps be recognized by some sort of standard tools in the future. If not, does it seem like something that might be worthy of putting into setuptools? It might be worth establishing convention(s) for, yes. Tools could then be developed around them, including perhaps changes to easy_install. This relates to a question I asked this list earlier this month (but didn't get a response). For Linux systems the Linux Standards Base references the Unix Filesystem Hierarchy Standard (that applies to all Unix systems as well). The FHS specifies that documentation files (other than specially formatted items like man pages) go into /usr/share/doc/package_name_and_version These sometimes include examples, demos, and similar files. For example, the docs on my FC5 system for inkscape go in /usr/share/doc/inkscape-0.45.1. In that case the doc files are a typical minimal set: /usr/share/doc/inkscape-0.45.1/AUTHORS /usr/share/doc/inkscape-0.45.1/COPYING /usr/share/doc/inkscape-0.45.1/ChangeLog /usr/share/doc/inkscape-0.45.1/NEWS /usr/share/doc/inkscape-0.45.1/README and the examples, tutorials, clipart, and many miscellaneous files are in /usr/share/inkscape. The actual executables are in /usr/bin. In some cases the documentation is created as a separate package. For example, Python does this for its html-based docs and on my FC5 system, the python html docs are in /usr/share/doc/python-docs-2.4.3/. Similar considerations go to configuration files that are supposed to go into /etc. There are a number of other rules, and they are generally observed by systems that use rpm and deb packaging such as Fedora and Ubuntu. I couldn't figure out how to make this happen when using bdist_rpm, which is why I asked the earlier question. It seems to me that the only way using current Python packaging features would be to put the docs somewhere in the Python site-packages hierarchy and do a post-install scripted move to where they belong under the LSB/FHS rules. It would be preferable to get them to go where they belong without the need for post-install scripting. In further discussions at Enthought, we're now thinking that creating a 'docs' dir as a top-level under python itself makes sense, and then the installation of eggs could copy docs and/or examples there in a manner similar to how it handles scripts into the platforms scripts dir. It would make sense that they should be put in dirs named with a format like project name-version so that the user could differentiate the docs for each project they installed. If this was done, it seems like it wouldn't be too hard for Stan, and other rpm builders, to either create symbolic links or copy these to /usr/share/doc in order to maintain compatibility with the LSB. One other thought we've been throwing around is an idea of adding something to easy-install so that users can still get a minimal binary distribution if they want, and then, if they want docs, run a command like: easy_install --get-docs enthought.traits where easy_install would then introspect the enthought.traits egg and figure out how to retrieve the docs and install them into ../python/docs. If we found time to work on implementing something, is there a preference for which path we should pursue? -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] distutils.util.get_platform() for Windows
Mark Hammond wrote: A good value for 'architecture' isn't clear. Either 'AMD' or 'Itanium' appeals at first glance, but I'm a little concerned that (say) a casual user with a new Intel Core Duo processor will not know they should use something labelled as AMD (eg, I explicitly asked for an Intel chip, not an AMD one). An alternative could be be 'x64' or 'i64', but I'm not sure that casual user would be any better off, and with only a single letter distinguishing them, there is scope for confusion. On my final (mutant) hand, it seems that Itanium will be a historical footnote and demand for Itanium 64bit packages will be tiny (I've had a reasonable number of requests for x64 versions of pywin32, but zero for i64), so I doubt many packages will bother with Itanium. So, I'm leaning towards 'win64-x64' and 'win64-i64', with the expectation that the 'i64' variant will be rarely seen in the wild. Alternatively, 'win-x64' and 'win-i64' appear reasonable - it doesn't seem necessary that '64' appear twice in the name. I could be mistaken, but I believe the standard abbreviations for these architectures are 'x86_64' (intel/amd chipset) and 'ia64' (itanium) -- see the first two paragraphs of the wikipedia article on Itanium for an example: http://en.wikipedia.org/wiki/Itanium So how about something like: win-x86_64 win-ia64 -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Are there best practices on creating/using egg extras?
Hello, I'm not sure if I've painted Enthought into a corner or not, but I can't figure out if there is a way to help users of a library, delivered as an egg, to know and/or maintain dependencies on extras declared in that egg. By which I mean that if my library, called X, declares extras 1, 2, and 3, and the source for X includes all the code that implements the features in extras 1, 2, and 3, and the extras document the external dependencies component X has to get those extras to work, how does someone importing from the API in X know whether their dependency on it should be 'X', 'X[1]', X[1,2]', etc. ? After all, all the API methods are already there even if they didn't install the extras. So far the only mechanisms I can see are (a) a manual one which depends on people (everyone using X!) knowing the internals of X such that they can tell that if they import symbols a, b, or c, then that means they need extra 1, etc., and (b) trial and error iteration driven by unit / integration tests. Am I just misusing or misunderstanding extras here? BTW, what I'm trying to do is convert Enthought's old monolithic distribution of ETS into components that could be installed individually. The first step of this was to package each component as an egg; the second was to ensure that cross-component dependencies were put in place (simply declaring A requires B if code in component A imported a symbol from packages in component B); the third was to try to minimize these cross-component dependencies by minor refactoring and introduction of extras to represent non-core functionality; and the fourth is to ensure that cross-component dependencies properly include extras as needed. It's this last step that has got me thinking about this problem. I've been able to mostly automate the previous steps, or at least write tools to help me, but for this fourth step, I can't figure out how to do it consistently. It is quite possible that I've missed some best practice of when to create or use extras. Any advice would be greatly appreciated! -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Are there best practices on creating/using egg extras?
Phillip J. Eby wrote: At 01:50 PM 7/20/2007 -0500, Dave Peterson wrote: Hello, I'm not sure if I've painted Enthought into a corner or not, but I can't figure out if there is a way to help users of a library, delivered as an egg, to know and/or maintain dependencies on extras declared in that egg. By which I mean that if my library, called X, declares extras 1, 2, and 3, and the source for X includes all the code that implements the features in extras 1, 2, and 3, and the extras document the external dependencies component X has to get those extras to work, how does someone importing from the API in X know whether their dependency on it should be 'X', 'X[1]', X[1,2]', etc. ? After all, all the API methods are already there even if they didn't install the extras. So far the only mechanisms I can see are (a) a manual one which depends on people (everyone using X!) knowing the internals of X such that they can tell that if they import symbols a, b, or c, then that means they need extra 1, etc., and (b) trial and error iteration driven by unit / integration tests. Am I just misusing or misunderstanding extras here? Note that if you have code that imports from your extras, you can always include: pkg_resources.require(X[1]) right before the imports. Then, instead of an obscure import error, users will get an obscure DistributionNotFound error that at least tells them one of the packages they'll need to install. ;-) Right. Always better to have an obscure error that is more informative about how to solve it. :-) But someone still needs to know when to insert pkg_resources.require(X[1]) in their code. Which means knowing which symbols in X's API need a given extra. So I'm back at my original problem, no? -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Is there an official stable release of setuptools?
Phillip J. Eby wrote: At 03:44 PM 7/18/2007 -0500, Dave Peterson wrote: Hello, Is there a blessed-as-stable, official, release of setuptools? Perhaps it's just me but a version number of the form '0.6c6' or '0.7a1' just doesn't seem like the developers think it is stable yet, even if it is being widely used. :-) It depends on your definition of stable, of course. :) There will be an 0.6c7, fixing a couple of outstanding issues. I will then at some point simply bless an 0.6 final, regardless of whether there are any more bugs found, because having an 0.6c8 is just ridiculous. ;-) We're in the same boat with some of our components. ;-) 0.6c6 is nonetheless an official *release* of setuptools. We do not have a stable/unstable distinction, only a release/development distinction. Okay. That's good enough for me. Sounds like we can effectively think of any non-dev release of setuptools as what I was terming a 'stable' release. Perhaps in general we should stop using the term 'stable' and change it to 'released'? I'll use that in the rest of this e-mail. 0.7a1 has not been released, but 0.6c6 has. Releases are uploaded to the CheeseShop, and do not have 'dev-r' tags in their version. However, there are others who think 'stable' just means that we've found, in our own testing, that things generally work as advertised and that really, our build process is building correctly. (I may be paraphrasing incorrectly, but since other Enthought-ers read this list, I'll trust them to correct me!) This would mean that we could put a binary of setuptools 0.7a1 up in our stable repo. Since 0.7a1 does not exist yet (only unreleased 0.7a1dev-r versions), you won't be able to do that just yet. :) Yeah, here is where it gets tricky. Apparently some of us got overzealous (my term, not theirs) and modified code in *our* component source (that we now want to 'release') such that it depends on the API in 0.7a1dev-r. We are currently building dev builds of your dev source such that they have a version specified like '0.7a1dev-r.devMMDDHHMM' which, beyond being quite a mouthful, distinguishes any of our numerous attempts at building your dev source from one another. They each get tagged with the date of the build since they may be built against the same svn revision. (Not such a big deal with setuptools, but the principal is meant for other components where getting the build right may take several attempts over days or weeks.) Anyway, when we want to 'release' our component, we want a corresponding release of the dependency -- but you're not there yet -- so we're talking about putting out a version called '0.7a1dev-r' in our 'releases' dir. This seems to be the only solution given where we are (besides backing out the implementation that relies on the 0.7a1 features.) Since people can already get an install of 0.7a1dev-r this probably isn't the end of the world, but we are now calling it a release when you're not -- which bothers me. :-) I've never had to think like a 'packager' before trying to set this up. (Thanks for that term by the way Rob!) Perhaps its time for more googling of how they do it. Thanks for your responses! -- Dave ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
