[Distutils] RFC: Standard Declaration of tests in eggs
Here is a rough draft proposal for declaring tests in eggs: Introduction Software packages should have automated tests. Consumers of packages will often want to run these tests. Tools should be able to do this automatically. This proposal seeks to provide a way for automated tools to discover tests in distributions, including eggs, so that tests can be run or so that test runners can be automatically created to run the tests. Proposal This proposal aims to be extremely simple. It has 2 parts: 1. A 'test_suite' entry point is defined. An egg can provide zero or more test_suite entry points. These entry points will define callable objects that can be called without arguments and that return unittest test suites. 2. An optional 'tests' extra is defined. When creating test runners or dynamically loading distributions to load tests, any distributions listed in extra requires for the 'tests' extra shall be included in the working set for the test runner. Thoughts? Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Skipping namespace file?
Thanks for the reply, but I don't really understand your response. I don't
know why multiple packages would install the same __init__.py file. I think
that would be an error to do so on my part as a packager. Maybe I am
missing the point.
Here is some more info. Maybe this enough to show what I am doing wrong.
This install does not work (e.g. import dap fails):
%p/bin/python%type_raw[python] setup.py install --root=%d
--single-version-externally-managed
I do NOT get this file installed.
/sw/lib/python2.5/site-packages/dap/__init__.py
But this does work (as in the dap module works import dap and import
dap.client but succeed and function correctly):
%p/bin/python%type_raw[python] setup.py install --root=%d
--single-version-externally-managed
cp dap/__init__.py* %i/lib/python%type_raw[python]/site-packages/dap/
I then end up with
/sw/lib/python2.5/site-packages/dap/__init__.py
Here is what the pth looks like. Is there something wrong with it then?
cat /sw/lib/python2.5/site-packages/dap-2.2.5.7-py2.5-nspkg.pth
import sys,new,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'],
*('dap', 'plugins')); ie = os.path.exists(os.path.join(p,'__init__.py')); m
= not ie and sys.modules.setdefault('dap.plugins',new.module('dap.plugins'));
mp = (m or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and
mp.append(p)
import sys,new,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'],
*('dap', 'responses')); ie = os.path.exists(os.path.join(p,'__init__.py'));
m = not ie and sys.modules.setdefault('dap.responses',new.module('
dap.responses')); mp = (m or []) and m.__dict__.setdefault('__path__',[]);
(p not in mp) and mp.append(p)
If I read that right, it is not letting python know about the dap module, so
that I can't do something like import dap.client. Does this mean that the
setup.py is not configured correctly to include a dap?
Thanks!
-kurt
On 1/4/07, Phillip J. Eby [EMAIL PROTECTED] wrote:
At 10:47 AM 1/3/2007 -0500, Kurt Schwehr wrote:
Hi All,
I am working on packaging pydap and am getting some strange behavior.
Here is the install phase...
/sw/bin/python2.4 setup.py install
--root=/sw/src/fink.build/root-dap-py24-2.2.5.7-1
--single-version-externally-managed
And when it gets to the __init__.py file for dap, it skips it. This
causes the package to not work.
It should also be installing a .pth file that makes it work, but if your
final installation destination is not a 'site' directory (e.g. Python's
site-packages directory), this will not work.
Anyone have an idea why it is doing this?
Because system packaging tools like RPM et al do not like it when multiple
packages install the same file. dap.plugins and dap.responses are
namespace packages, which means that other projects can install modules
in them. Those modules mustn't install an __init__.py, as it would
overwrite the one supplied by pydap. The setuptools solution to this
problem is to never install an __init__.py at all for such packages, when
used with a packaging tool (which is implied by the use of --root).
Instead, setuptools generates a uniquely-named .pth file for each project,
that sets up the namespace package at runtime. Look for a .pth file being
placed in your
/sw/src/fink.build/root-dap-py24-2.2.5.7-1
/sw/lib/python2.4/site-packages/
directory. The problem is that if you then install the package somewhere
other than /sw/lib/python2.4/site-packages, or use the -S option to Python
at runtime, it may not work.
This install proceedure works well with other packages in mac
osx/fink. I
also see the behavior without the --single-version-externally-managed
flag.
That's because --root implies --single-version-externally-managed.
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Skipping namespace file?
At 09:05 AM 1/5/2007 -0500, Kurt Schwehr wrote:
Here is what the pth looks like. Is there something wrong with it then?
cat /sw/lib/python2.5/site-packages/dap-2.2.5.7-py2.5-nspkg.pth
import sys,new,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'],
*('dap', 'plugins')); ie = os.path.exists(os.path.join (p,'__init__.py'));
m = not ie and
sys.modules.setdefault('dap.plugins',new.module('dap.plugins')); mp = (m
or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p)
import sys,new,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'],
*('dap', 'responses')); ie =
os.path.exists(os.path.join(p,'__init__.py')); m = not ie and
sys.modules.setdefault(' dap.responses',new.module('dap.responses')); mp =
(m or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and
mp.append(p)
If I read that right, it is not letting python know about the dap module,
so that I can't do something like import dap.client. Does this mean
that the setup.py is not configured correctly to include a dap?
Yes, you're right, that is exactly the problem. There are two possible fixes:
1. add 'dap' to the namespace_packages setting in the setup.py
2. change setuptools to figure out that this is needed
Since setuptools already knows that it shouldn't include the __init__ for
dap, it can reasonably be considered a setuptools bug that it doesnt'
generate the .pth correctly for that case, so I will fix that. In the
meantime, as a workaround, adding 'dap' to the namespace_packages setting
will allow you to proceed.
Thanks for your patience and help in identifying the actual bug.
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Standard Declaration of tests in eggs
David Fraser wrote: Jim Fulton wrote: Here is a rough draft proposal for declaring tests in eggs: Introduction Software packages should have automated tests. Consumers of packages will often want to run these tests. Tools should be able to do this automatically. This proposal seeks to provide a way for automated tools to discover tests in distributions, including eggs, so that tests can be run or so that test runners can be automatically created to run the tests. Proposal This proposal aims to be extremely simple. It has 2 parts: 1. A 'test_suite' entry point is defined. An egg can provide zero or more test_suite entry points. These entry points will define callable objects that can be called without arguments and that return unittest test suites. How would this work if for example, you're using an alternative testing framework (like py.test) for your test? I'm not familiar with py.test. I guess it's not based on unittest. Why? Couldn't it at least have a unittest wrapper, like the one I wrote for doctest? Even though I use doctest almost exclusively, I view unittest as a common API that various testing frameworks can and should play with. I certainly think there should be some common API like that and see unittest as the incumbant. It would be nice to be able to bootstrap it :-) What do you mean by that? 2. An optional 'tests' extra is defined. When creating test runners or dynamically loading distributions to load tests, any distributions listed in extra requires for the 'tests' extra shall be included in the working set for the test runner. Great, so at least the testing framework could be declared as a dependency My assumption (and my use case) is that you'd run whatever test runner you want and tell it to run tests from some given eggs. So the test runner would already be loaded. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Standard Declaration of tests in eggs
At 08:46 AM 1/5/2007 -0500, Jim Fulton wrote: Here is a rough draft proposal for declaring tests in eggs: Introduction Software packages should have automated tests. Consumers of packages will often want to run these tests. Tools should be able to do this automatically. This proposal seeks to provide a way for automated tools to discover tests in distributions, including eggs, so that tests can be run or so that test runners can be automatically created to run the tests. Proposal This proposal aims to be extremely simple. It has 2 parts: 1. A 'test_suite' entry point is defined. An egg can provide zero or more test_suite entry points. These entry points will define callable objects that can be called without arguments and that return unittest test suites. 2. An optional 'tests' extra is defined. When creating test runners or dynamically loading distributions to load tests, any distributions listed in extra requires for the 'tests' extra shall be included in the working set for the test runner. Thoughts? Point #2 is unnecessary, since individual entry points can list extras in square brackets following the module/attribute information. When loading an entry point, these extras automatically get require()'d. Conversely, if the test runner wants to manage the loading process, it can simply inspect the entry point object to determine the names of the extras. Regarding point #1, I'm not sure this is enought to define what's necessary. For example, it should perhaps be stated that the entry point must be in code that will be installed by either the distribution itself, or that is included in the code provided by the entry point's extras. Also, an egg can't provide more than one entry point with the same name, so rather than a 'test_suite' entry point, we probably want an entry point *group*, perhaps something like 'installed.test_suites'. Next, there needs to be reasonable support for dynamic test discovery, such as setuptools' own ScanningLoader. I would suggest that there be an optional entry point for the test loader class to be used to process the other entry points' target objects. The standard unittest protocol for loadTestsFromName() takes two arguments: a name and an object. Passing an empty string for the name, and the object loaded by the test suite entry point, suffices to enable normal behavior for loaders such as ScanningLoader, and I believe 'nose' as well as any other well-behaved unittest extensions. However, I think the spec should try to define what well-behaved means in terms of I/O, result reporting, etc. (Obviously, one requirement is that the test loader must be able to take an empty name string and an object in its loadTestsFromName() method, and return a test suite.) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] RFC: Standard Declaration of tests in eggs
At 01:11 PM 1/5/2007 -0500, Jim Fulton wrote: Phillip J. Eby wrote: ... Why can't an entry point invoke a test loader itself? This seems much simpler and more straightforward to me. Because that requires you to write code for something that can adequately be expressed through an existing configuration mechanism. *And* you have to write that code in every project. It's probably a couple of lines. Import a loader and call it. That doesn't seem like a big deal to me. Nice and explicit too. And it's enormously repetitive if you have a lot of projects. Besides, I expect most projects will have a single test suite (function) that could just be named directly. You expect wrongly. :) 'nose' and 'py.test' are popular *precisely because* they do NOT require this. Setuptools changed to emulate them by adding ScanningLoader, even though it originally did the test suite function thing you're proposing. See, I originally wrote all my test suites the way you're proposing, but I changed because this is one of those convention beats configuration situations. It's easier to just conform to a policy and have a tool that applies the policy. Different projects may have different policies, so providing an option for the loader, allows them to follow it. (And it made my life easier, too, as I began doing more small projects instead of a few large ones.) And, if you want your approach to be widely adopted, it would be best to allow projects to follow their own policies without requiring them to add any code, even if it's a one-liner. So, *technically*, there may be no reason to do this, but from a usability, friendliness, compatiblity, marketing, adoption, etc. format I don't think the standard will be successful without allowing this out. People often have heavy psychological investment in existing tools, and it is hundreds of times easier to rationalize adding a line of configuration to setup() than it is to rationalize adding two or three lines of code. One is merely a packaging change, the other is a programming change. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Skipping namespace file?
I've now fixed the bug below in both the 0.6 branch and the development trunk. You can get them by easy_installing either 'setuptools==dev06' or 'setuptools==dev', respectively. At 11:43 AM 1/5/2007 -0500, Phillip J. Eby wrote: At 09:05 AM 1/5/2007 -0500, Kurt Schwehr wrote: If I read that right, it is not letting python know about the dap module, so that I can't do something like import dap.client. Does this mean that the setup.py is not configured correctly to include a dap? Yes, you're right, that is exactly the problem. There are two possible fixes: 1. add 'dap' to the namespace_packages setting in the setup.py 2. change setuptools to figure out that this is needed Since setuptools already knows that it shouldn't include the __init__ for dap, it can reasonably be considered a setuptools bug that it doesnt' generate the .pth correctly for that case, so I will fix that. In the meantime, as a workaround, adding 'dap' to the namespace_packages setting will allow you to proceed. Thanks for your patience and help in identifying the actual bug. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Adding a setuptool command that runs when setup.py build is invoked
I'm trying to create a package that provides a setuptool command that will compile idl files when you run python setup.py build or python setup.pyinstall. I've figured out how to add an additional command (build_omniidl) which I can run with: python setup.py build_omniidl But I'm not sure how to wire it up so that python setup.py build automatically invokes build_omniidl for me. Basically I want build_omniidl to act just like build_python. Is this even possible with setuptools? If so anyone know where I can see an example? ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
