[Zope-dev] Egg install bot results
I've created a bot process (see http://svn.repoze.org/playground/trunk/chris/zopesvnchecker/checker.py) that: - checks out the trunk subdir of each top-level directory in svn.zope.org - if the resulting directory does not have a setup.py, we skip the directory - if the resulting directory does have a setup.py, we create a temporary virtualenv with setuptools installed and run the virtualenv's bin/python passing it setup.py install. - We capture the results of the install, and print a summary. The summary results are at http://www.plope.com/static/misc/checker_results.txt . The details pickle generated by the script is at http://www.plope.com/static/misc/results.pickle . My analysis of the results are these: - A version conflict exists in a low-level dependency between a requirement for zope.traversing 3.4.0 and a requirement for zope.traversing=3.5.0a1.dev-r78730 when installing many zope.* eggs using setup.py install (and easy_install), making it impractical for non-Zope people to actually install most of the interesting and released zope.* modules. This conflict needs to get fixed. Additionally, some modules (like zope.pagetemplate) should not fail with this dependency error in the first place; instead their dependencies need to become less conservative. - The only 10 packages (out of 80) in the zope.app namespace can be easy_installed. All the others fail with the above dependency error. Suggestions: - Find and fix the zope.traversing easy_install conflict. I'll try to debug this. - Institute a policy that all distributions that are released to the cheeseshop should be installable via easy_install. IMO, if they are not installable this way, they should not be released to the cheeseshop, given the larger Python community's expectations. - Figure out why buildout can (apparently) qsuccesfully install dependencies of currently failing zope.* eggs while easy_install can't. I probably won't be able to do this. - An automated test should ensure that easy_install does something reasonable with the latest release of each distribution. If the test fails, a new release should be made, or the dependency bug tracked down and fixed. My checker script could be made to do this, and I'd provide the work if someone told me which distributions to test, and how to maintain the list of distributions to test over time. - Ditch the idea of releasing separate distributions for each package in zope.app. The individual eggs typically can't be used outside of a zope appserver installation (and if they can, they probably shouldn't be in zope.app, they should be in zope or they should be their own top-level package), and the namespaciness of zope.app is suspect when it's unlikely that anyone who is not a Zope committer will release a distribution which makes use of that namespace package. Their current overgranularity makes distributing them as separate eggs and releasing them to the cheeseshop a form of cheeseshop pollution, especially given that so few of them can actually currently be installed using easy_install or setup.py install. If cheeseshop is going to continue to be used as the index, I'd suggest creating a zope.app top-level svn module with a single setup.py in containing all the packages that are meant to go into zope.app. Version the resulting zope.app distribution as necessary instead of versioning many more granular zope.app.* distros. It's OK if some people don't use some of the functionality in the resulting egg, just toss everything in. There is precedent here in the Paste distribution. It has many submodules and does many things, but it comes in the form of a single egg. Yes, you lose the ability to make a bugfix in one subpackage and release it, but IIRC the intent is to trim zope.app down anyway, pushing libraryish things out to top-level or zope.* packages. Issues: - Who's in charge? Whomever you might be, to what extent do you agree/disagree with the above suggestions? If you agree with any, how can I help fix things? - I'm unsure how anybody is able to install Zope 3 right now using eggs, unless there's some fundamental difference in the way easy_install resolves dependencies vs. buildout. I have not looked at how Stephan's KGS works yet, though, so I'm sure I'm just missing some magic. - We should consider fixing setuptools install to detect conflicts before attempting to install anything. The current regime of find conflicts halfway through an install is IMO untenable in the long term for using eggs as a distribution mechanism. This may mean a very invasive change to both the package index and the client software, though. ___ Zope-Dev maillist - Zope-Dev@zope.org
Re: [Zope-dev] Egg install bot results
I've created a variant of the script I used to generate the results referenced in my original message within this thread. http://svn.repoze.org/playground/trunk/chris/zopesvnchecker/releasechecker.py The new variant attempts to easy_install each cheeseshop project that has 'zope' in its author metadata (found via the cheeseshop's search XML-RPC interface) into a pristine virtualenv.This mimics what end-users would see if they attempted to install Zope-related packages from the cheeseshop without using zc.buildout. I'll set this up to run every few days. I'll attempt to send its output to zope-coders. - C On Nov 13, 2007, at 4:29 PM, Chris McDonough wrote: I've created a bot process (see http://svn.repoze.org/playground/trunk/chris/zopesvnchecker/ checker.py) that: - checks out the trunk subdir of each top-level directory in svn.zope.org - if the resulting directory does not have a setup.py, we skip the directory - if the resulting directory does have a setup.py, we create a temporary virtualenv with setuptools installed and run the virtualenv's bin/python passing it setup.py install. - We capture the results of the install, and print a summary. The summary results are at http://www.plope.com/static/misc/checker_results.txt . The details pickle generated by the script is at http://www.plope.com/static/misc/results.pickle . My analysis of the results are these: - A version conflict exists in a low-level dependency between a requirement for zope.traversing 3.4.0 and a requirement for zope.traversing=3.5.0a1.dev-r78730 when installing many zope.* eggs using setup.py install (and easy_install), making it impractical for non-Zope people to actually install most of the interesting and released zope.* modules. This conflict needs to get fixed. Additionally, some modules (like zope.pagetemplate) should not fail with this dependency error in the first place; instead their dependencies need to become less conservative. - The only 10 packages (out of 80) in the zope.app namespace can be easy_installed. All the others fail with the above dependency error. Suggestions: - Find and fix the zope.traversing easy_install conflict. I'll try to debug this. - Institute a policy that all distributions that are released to the cheeseshop should be installable via easy_install. IMO, if they are not installable this way, they should not be released to the cheeseshop, given the larger Python community's expectations. - Figure out why buildout can (apparently) qsuccesfully install dependencies of currently failing zope.* eggs while easy_install can't. I probably won't be able to do this. - An automated test should ensure that easy_install does something reasonable with the latest release of each distribution. If the test fails, a new release should be made, or the dependency bug tracked down and fixed. My checker script could be made to do this, and I'd provide the work if someone told me which distributions to test, and how to maintain the list of distributions to test over time. - Ditch the idea of releasing separate distributions for each package in zope.app. The individual eggs typically can't be used outside of a zope appserver installation (and if they can, they probably shouldn't be in zope.app, they should be in zope or they should be their own top-level package), and the namespaciness of zope.app is suspect when it's unlikely that anyone who is not a Zope committer will release a distribution which makes use of that namespace package. Their current overgranularity makes distributing them as separate eggs and releasing them to the cheeseshop a form of cheeseshop pollution, especially given that so few of them can actually currently be installed using easy_install or setup.py install. If cheeseshop is going to continue to be used as the index, I'd suggest creating a zope.app top-level svn module with a single setup.py in containing all the packages that are meant to go into zope.app. Version the resulting zope.app distribution as necessary instead of versioning many more granular zope.app.* distros. It's OK if some people don't use some of the functionality in the resulting egg, just toss everything in. There is precedent here in the Paste distribution. It has many submodules and does many things, but it comes in the form of a single egg. Yes, you lose the ability to make a bugfix in one subpackage and release it, but IIRC the intent is to trim zope.app down anyway, pushing libraryish things out to top-level or zope.* packages. Issues: - Who's in charge? Whomever you might be, to what extent do you agree/disagree with the above suggestions? If you agree with any, how can I help fix things? - I'm unsure how anybody is able to install Zope 3 right now using eggs, unless there's some fundamental difference in