Re: [Zope3-dev] Let's move over to zope-dev
On Oct 11, 2007, at 12:34 PM, Marius Gedminas wrote: On Thu, Oct 11, 2007 at 10:17:13AM -0400, Jim Fulton wrote: I think we have agreement (or an absense of anyone who is willing to publicly admit to disagree) to retire this list and move over to zope-dev. Let's do so. I'll wait a little while before trying to actively prevent posts here. I suggest we let any existing threads run out and start new threads on zope-dev. Will someboy with mailman admin access move over the subscriber list, or do we have to get off our lazy asses and resubscribe ourselves? If someone wants to do this and needs access, I'm happy to set them up. Not it! Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Let's move over to zope-dev
I think we have agreement (or an absense of anyone who is willing to publicly admit to disagree) to retire this list and move over to zope- dev. Let's do so. I'll wait a little while before trying to actively prevent posts here. I suggest we let any existing threads run out and start new threads on zope-dev. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] eggsplosion and tests
On Oct 10, 2007, at 9:45 AM, Martijn Faassen wrote: Hi there, I just tried something Stephan also tried, but deserves another topic. I just wrote a simple script that takes all the eggs grok uses and tries to run their tests. This is to see whether our pile of eggs isn't broken in some way. I get a ton of errors. Mostly a ton of import errors, but other things as well, probably mostly due to version skew where one release assumes it can do things with APIs in another release that another release doesn't allow yet or not anymore. And then the whole testrunner bails out somewhere in the middle: Traceback (most recent call last): File "/home/faassen/buildout/grok-0.10/testbuildout/bin/test", line 115, in ? zope.testing.testrunner.run((['--tests-pattern', '^f?tests$', '- v']) + [ File "/home/faassen/buildout-eggs/tmpRHS1Xv/zope.testing-3.4- py2.4.egg/zope/testing/testrunner.py", line 271, in run File "/home/faassen/buildout-eggs/tmpRHS1Xv/zope.testing-3.4- py2.4.egg/zope/testing/testrunner.py", line 448, in run_with_options File "/home/faassen/buildout-eggs/tmpRHS1Xv/zope.testing-3.4- py2.4.egg/zope/testing/testrunner.py", line 661, in resume_tests zope.testing.testrunner.SubprocessError We'll have a lot of work cut out for us to fix this. This is one point where I can see an evolving index will help. The trouble is we ended up starting with such a messed up state in the first place. :( 2 minor notes: - zc.buildout and some recipes tests don't run properly outside of their buildouts. This is because the test setup leverages aspects iof the buildout, especially the presense of the source to build develop eggs needed for the tests. This almost certainly can and should be fixed, but I consider it a low priority in relation to other priorities and limited resources. - ZODB packaging, which is waay too complicated for hysterical reasons, fails to include some files needed for testing. This makes it impossible to run the tests outside of it;s development environment. This too can and should be fixed. I consider this a higher priority but won't have time to deal with it for a while. I think ZODB's setup should be rewritten from scratch. Much of the complexity arises from working around limitations in distutils, some of which have been fixed. Other complexity probably arises from a stubborn desire not to require setuptools for ZODB. Anyway, if someone wants to ehlp with the setup script, it would be appreciated, Otherwise, I'll get to it eventually. In the mean time, I suggest not running the ZODB tests. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Zope 3 releases?
I'm not sure that "library" or "collection of libraries" is the right term for what we want to be. I think we've been using it because it stands in sharp contrast to "application", which, BTW, isn't exactly what Zope 2 is. I think these terms were useful to make some points, but neither is accurate. FWIW, I have a fairly open mind on this topic. Lots of good points are being made. :) Jim On Oct 7, 2007, at 5:13 PM, Martijn Faassen wrote: Jim Fulton wrote: On Oct 7, 2007, at 6:25 AM, Lennart Regebro wrote: [snip] - We need a *realistic* (especially wrt available resources) process for managing releases. There are 2 aspects of this. We shouldn't make plans for which there aren't enough resources. We also shouldn't plan significant tasks that people won't care enough to work on. I think the classic Zope 3.4 release is a good example of a large effort that really wouldn't benefit many people, if any. Do you have a sort explanation on what is the missing resource? Is it, as it was for 3.3, lack of people-hours with knowledge in fixing the last bugs? I'm not entirely sure. I just observe that this doesn't seem to be making much progress. I think it's one of the drawbacks of taking an ecosystem/libraries approach instead of a application/framework style approach. An application or framework typically is an integrated whole that has a single version number. An ecosystem or set of libraries can be integrated (which Zope 3 is) but everything evolves at different rates and there's no single thing to install or talk about. I'm not saying an ecosystem approach is bad, if that's what Zope 3 wants to be. I do think that such an approach needs to be supplemented by a framework approach (and I've been putting work into one way to do that). If Zope 3 is an ecosystem, a "release" of Zope 3 the ecosystem doesn't really make much sense. To follow the comparison with Linux distribution, it's more like a "distribution" of an ecosystem. I'd therefore suggest that the release of Zope 3.4, if it ever actually happens, will be the last release of Zope 3 the application server framework. I hope that besides Grok, some community will stand up that takes a less radical approach to building an application server on top of the Zope 3 ecosystem. People having existing applications in Zope 3 to maintain (like myself with the Document Library) will have a need for it, and Grok doesn't suit everyone's tastes anyway, especially people comfortable with existing Zope 3 practices. As I said elsewhere, I suggest we call this project not "Zope 3" but something else, to avoid confusion with the Zope ecosystem (and also to avoid implying it's the clear successor to Zope 2. I think we can safely say by now that's not how history went). Regards, Martijn ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Known-good-sets problem
On Oct 7, 2007, at 7:33 AM, Lennart Regebro wrote: On 10/6/07, Jim Fulton <[EMAIL PROTECTED]> wrote: Only people who ask for updates. Which by default is every time somebody runs a buildout, right? Yes I think this depends on the use cases. I think most people would like to get bug fixes. The intent of such a 3.4 index would be to give people the latest 3.4 updates which should give no new features. This is intended to mimick what happens with linux package respositories. Ah, good point. But I think what Martijn is saying is that when we come with new *features* this should be made into a new page. Which would then be Zope 3.5, or something. Or did I misunderstand him? I don't know if you did or not. I'll just note that different indexes might have different update policies. A Zope 3.4 index should only get bug fixes. But you are right, small bugfixes, and in particular security fixes, should indeed go into the already existing page. Which means all you would need to get a security fix would be to run buildout, possibly forcing it to check for new versions if that is set to false by default. That would indeed be a very nice feature. I think so. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Zope 3 releases?
On Oct 7, 2007, at 6:25 AM, Lennart Regebro wrote: On 10/6/07, Jim Fulton <[EMAIL PROTECTED]> wrote: - We need to decide what a Zope 3 release is (or maybe multiple flavors). I favor copying the linux experiences, but have an open mind. I'm not sure what you mean with that, I mean we need to decide what a release is. but for me, a known good set, or what you call it, plus a buildout that uses this, would be enough for a Zope 3.4 release. If we easily can build the old standard downloadable tgz as well, then fine, but it's not necessary. That is my opinion too. - We need a *realistic* (especially wrt available resources) process for managing releases. There are 2 aspects of this. We shouldn't make plans for which there aren't enough resources. We also shouldn't plan significant tasks that people won't care enough to work on. I think the classic Zope 3.4 release is a good example of a large effort that really wouldn't benefit many people, if any. Do you have a sort explanation on what is the missing resource? Is it, as it was for 3.3, lack of people-hours with knowledge in fixing the last bugs? I'm not entirely sure. I just observe that this doesn't seem to be making much progress. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Zope 3 releases?
IMO, the Python standard library and "batteries included" is a mess. Despite being a Python contributor, I've very unmotivated to be a contributor because the time lag between contributing and deriving benefit from the contributions is too long. People had similar complaints about Zope releases. I'll note that the problem is exacerbated by the incompatibilities that (to some degree inevitably) often occur in Python releases. Big-bang releases don't prevent update problems. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Known-good-sets problem
On Oct 6, 2007, at 6:07 PM, Martijn Faassen wrote: Jim Fulton wrote: On Oct 6, 2007, at 5:24 AM, Martijn Faassen wrote: [snip] What I really don't like about this proposal is that it talks about updating index pages. If I understand this right, an updated index page will force everybody that uses this index page into an update. Only people who ask for updates. Yes, but it's important in releases. If I release my application to someone else, they will be asking for an update while I might never had. I cannot test this as the updated versions might've been released *after* I make my application release. Of course, each update to an index is, in essense a new release of the index. ... I don't think this is acceptable. I think this depends on the use cases. I think most people would like to get bug fixes. The intent of such a 3.4 index would be to give people the latest 3.4 updates which should give no new features. This is intended to mimick what happens with linux package respositories. Yes, bugfixes is less of a problem than feature releases, if this is well managed. Since packages evolve at different rates, what is done for feature releases of packages? Do they end up in the same index or separate indexes? How to determine when to make a new index? Those are all good questions that need to be answered. Of the top of my head, :) - I imagine there would be an index for each feature release that gets bug -fix updates. - There would be an index for the next feature release that includes stable packages with new features. There would be some requirements for packages to be deamed ready for the next release. Let me make the case for bug-for-bug compatibility: I assume by this you mean setting fixed versions. The problems during release as I sketched out above still exist. Granted for bugfix releases the chances are lower that something breaks than it is now, but we all know people sometimes disagree about what actually constitutes a bugfix (or larger change), and people also sometimes have workarounds or code assumptions that depend on bugs. Bugfixes can break working code. I'm not suggesting that setting fixed versions is a bad idea. I think some projects may choose to go this way. This is a valid policy decision. Other developers would, IMO, benefit from having a relatively stable index as a source of new packages. I don't think this is an either-or decision. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: [Zope3-Users] Re: Known-good-sets problem
On Oct 6, 2007, at 5:28 AM, Martijn Faassen wrote: Benji York wrote: Stephan Richter wrote: 2. How many packages should be controlled in this index? I think we should definitely add packages from z3c and the zc namespace. What is the motivation to include non-controlled packages? I suppose it is to let people use those packages with (in this case) Zope 3.4. What if someone wants Zope 3.4 and Twisted version X and Plone version Y (just making those up). Perhaps we need a way to refer to several KGS when constructing an application. Or is one KGS supposed to define a "universe" of packages known to work together. If so, I would think there would be no place for non- controlled packages. A feature to combine lists of versions is definitely needed. I may be a developer of an application that uses both Grok and KSS, for instance. This can be solved by me maintaining my own list, but I'd prefer to rely on two lists that are already there. That's the application developer's case. The framework developer's case requires me to publish such combined lists. If I have my own framework on top of Grok and KSS, I'd like to publish a list that's a combination of those without having to copy them. As I mentioned in my response to Benji, it's tricky to get the semantics right. IMO, you want a particular KGS to be authoritative. There isn't anything in setuptools to support this semantic. You can use the mirroring approach (maybe with some minor tweaking) to combine KGSs on the server. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Known-good-sets problem
On Oct 5, 2007, at 10:45 PM, Benji York wrote: Stephan Richter wrote: 2. How many packages should be controlled in this index? I think we should definitely add packages from z3c and the zc namespace. What is the motivation to include non-controlled packages? I suppose it is to let people use those packages with (in this case) Zope 3.4. Yes What if someone wants Zope 3.4 and Twisted version X and Plone version Y (just making those up). That's why the KGS index is a PyPI mirror of all uncontrolled packages. Perhaps we need a way to refer to several KGS when constructing an application. Or is one KGS supposed to define a "universe" of packages known to work together. If so, I would think there would be no place for non-controlled packages. The semantics I think we want are kind of tricky. A KGS index needs to be authoritative for the projects it controls. If we looked in multiple KGSs, there would need to be semantics for deciding which was authoritative. setuptools lets you define a single index and multiple find-link servers. The highest version found on any server is authoritative. I think supporting multiple KGSs with the right semantics would be useful, but there isn't a way to do it in setuptools. We can achieve the same effect on the server. For example, with this software, you could chain several KGSs together. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Zope 3 releases?
Recent discussions make it obvious to me that we need to come to terms with Zope 3 releases. I have some ideas and observations. I don't have decisions at this point. We need to make some decisions as a community, with me hopefully helping at times to get us unstuck. :) - The classic 3.4 release seems to be stalled. This was to be a release modeled on previous Zope 3 releases. It provides a monolithic Zope 3 application. I expect that it is stalled because the person who volunteered to see it through overcommitted and, more importantly, it isn't of much practical use, except as a testing platform or as some sort of known good configuration. The original rational was that we didn't want to be too disruptive in the next release. I think we've been overtaken by events and a lack of interested resources. - There is a desperate need for a known good configuration of components that work well together and that people can build on without having to think too hard about the underlying platform. - There is a need for a mechanism for testing "core" components to make sure they don't cause breakage. - We need to decide what Zope 3 is. Zope 3 is *not* an application. I think there is fairly broad agreement on that. I can think of several words that could be used to describe what it *is*, like "platform", "environment", "ecosystem". I think it's time we came up with the elevator speech for what Zope 3 is. It should be a few sentences at most. It need not be carved in stone forever, but it should be agreed on and used for tactical planning. This should probably go hand in hand with the bigger definition of what Zope is along the lines of the ideas that Tres expressed at the DZUG in Potsdam. - We need to decide what a Zope 3 release is (or maybe multiple flavors). I favor copying the linux experiences, but have an open mind. - We need a *realistic* (especially wrt available resources) process for managing releases. There are 2 aspects of this. We shouldn't make plans for which there aren't enough resources. We also shouldn't plan significant tasks that people won't care enough to work on. I think the classic Zope 3.4 release is a good example of a large effort that really wouldn't benefit many people, if any. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases
ver the broader Zope 3 community. Why did this have to happen within the Grok project? I have the impression that too much, people in the Zope 3 community think that as soon as they themselves and the other expert developers know what to do (assimilated on the mailing list and online), this is the end of the problem. We don't have a single Zope 3 release anymore, but we do want people to use our code and report bugs on it. We were supposed to. I wonder what happened to that. As far as I can see this *requires* a list of versions somewhere that is shared between people and that can be communicated about easily. It requires a *release procedure* around this list of versions. As I tried to point out before long ago, the Zope 3 project is not somehow so special it doesn't need releases. I think the Zope 3 project is somewhat special, which it may need some kind of releases. :) After all, we have releases of individual eggs. I don't think it makes much sense to release the traditional Zope 3 application except perhaps as a testing platform. I do think we need something like what linux distros provide. We need to establish a better understanding of this. as we have fixed the problem with Grok. If non-Grok users are interested in our fixes, please let us know though. We've just made this massive investment. I'd suggest people to switch to Grok anyway, as we actually think about this stuff and care about having our house in order. Are you suggesting that other people aren't thinking about this and don't care? Fred doesn't appear to care to think about this much, certainly, and is liking it that way. So based on a snarky comment by one person, you tar the entire community. I'll note, BYW, that if you had provided more details, as Philipp eventually did, I think Fred's response would have been more sympathetic. (Just guessing) I hope that the proposal for setting up a known-good-set index will be helpful. I'm sure Stephan would like to know the versions you chose. I imagine he'll be able to get those by looking at Grok. After 3 days of hard work by 2 people, we're still not done with the Grok eggs. We get weird effects like final releases of packages *suddenly* needing an entirely new package, between the beta and the final. I'm quite bothered by this. Me too. We still have quite a few dev-r515135 style eggs in the mix as well. That's bad. A single known-good index, by the way, doesn't solve all problems. It will evolve forward as people release newer known-good versions of eggs. This means that nobody can really rely on such an index, as suddenly they might be starting to use 3.6 eggs where they were using 3.5 eggs. Even bugfix releases are currently hardly safe: using 3.4.1 for some Zope 3 package might mean that suddenly it requires an entirely new package altogether, introducing new deprecation warnings and so on (as in this thread). This is all a matter of the rules for maintaining the index and the care put into it. If it is well run, a big if, then it should be as reliable as, say, a linux package repository. This doesn't guarantee that there won't be problems. With our Grok versions list, we publish a list *per* version of Grok. I hope this is the intent for the known-good index as well. If someone says they use Grok 0.11, Is Grok 0.11 a feature release? Would you expect the version list for Grok .11 to change as bug fixes are made? we want to know *exactly* which versions they are using without requiring them to send us a long list too. One version number should be enough. Our installation tools support this, and our documentation documents this. I think this is the only maintainable way to actually have a community developing and using a common code base. My suggestion is that there should be stable indexes for each "feature release". Changes to these indexes should be made to fix bugs, not add new features. Of course, there could be other less stable indexes for people who want them. If you want to nail all of the versions for a specific release of grok, then it seems to me that versions in a meta egg or a buildout configuration should work well. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases
On Oct 6, 2007, at 5:06 AM, Martijn Faassen wrote: Jim Fulton wrote: On Oct 5, 2007, at 1:59 PM, Martijn Faassen wrote: [snip] but moved to a new place. zope.app.error, is, I understand, gone now. I have no idea about this specific move. If there was a zope.app.error before, then distributions of it should still exist and new distributions should be backward compatible. After we fixed a bunch of errors in them, they're backward compatible, yes. With deprecation warnings. The code moved to zope.error so zope.app.error is now empty. I really don't think going from zope.app.applicationcontrol 3.4 beta whatever, going to final 3.4.1 suddenly means a whole new package should appear with new deprecation warnings. Agreed. I thought that *never* in the 3.4 line of eggs should they *suddenly* start relying on 3.5 eggs. That's nothing to do with the notion of a 3.4 release, but with the notion that during the stabilization phase, or with minor bugfix releases, you don't suddenly start relying on a new feature release of something else (or in this case, an entirely new release). I think I agree with the spirit of the above, but not the specifics. You restate the specifics below in a way I whole-heartedly agree with. There isn't a 3.4 "line" of eggs. There could be a set of projects versions associated with a "3.4 release of Zope3", but the individual version number could be almost anything. Anyway, I think the rule should be: "When you do a final or bugfix release of a package, you can't start requiring a new feature release of another package." +1 Translated to version numbers: "If X.Y.x has been relying on A.B.x, X.Y.x + 1 cannot start relying on A.B + n, only on new A.B.x + n releases, where x is one of (b0, b1, 1, 2, 3, ...) and n is one of (1, 2, 3 ...)" Exactly. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Known-good-sets problem
On Oct 6, 2007, at 5:24 AM, Martijn Faassen wrote: Hey, I'm glad some steps are taken! What I really don't like about this proposal is that it talks about updating index pages. If I understand this right, an updated index page will force everybody that uses this index page into an update. Only people who ask for updates. I don't think this is acceptable. I think this depends on the use cases. I think most people would like to get bug fixes. The intent of such a 3.4 index would be to give people the latest 3.4 updates which should give no new features. This is intended to mimick what happens with linux package respositories. Instead I'd suggest you institute a rule that index pages should never be changed after initial publication, just like eggs shouldn't be changed after publication either. Instead, a new index page should be published for each update. People can then choose to switch whenever they like. You could manage an index like that. I don't think that would be very useful. IMO, if you want to really nail version numbers, then it would be better to just use meta eggs or buildout version specifications. The goal of the index is to give people a stable source of distributions that are known to work together. My main suggestion is to not forget that this needs to be clearly documented and the document describing which steps to take should be published. Absolutely. The index is just a mechanism that will enable processes that actually address the problems we're struggling with, My main worry is that I don't like maintaining dependency lists on remote servers somewhere. My intuition is that we should maintain these lists close to the actual software. With grok we're managing this list now as a versions.cfg in subversion. Anyway, this is just a worry, and we'll have to see how this pans out. That's a valid approach, at least for some problems. If the software is an application, the best approach IMO is for each release to specify precisely the versions it's using. If software is a platform, then more flexibility may be needed, This index idea seeks to learn from experience with linux distributions. We've gone another direction with the Grok project this week, where we publish a versions.cfg on the web that gets included in a buildout.cfg. This approach was based on suggestions by yourself to me, and had been discussed by Philipp on the mailing list earlier, but apparently the choice was made to go for index pages instead. Yup. Of course, it only works with buildout. At the time, I remember getting push back from someone that they wanted a solution that didn't require buildout. A custom index is more open both wrt packaging technology and to use by different applications. I think Zope 3 would benefit from a stable repository of eggs that can be used even if people aren't using Grok or buildout. Of course, the index technology isn't worth much without solid processes behind it. So, it's a bit of a shame this extensive work will now be made mostly redundant by these new efforts. I don't think so. Certainly, the detective work you've done figuring out what's compatibly should be used by whoever assembles a 3.4 index. Anyway, we'll be using this for a while at least and we'll wait and see how this new works pans out. I think the index complements techniques like meta eggs or buildout.cfgs with nailed versions. Also, keep in mind that this effort results from a desire to help and because people care. Be careful about accusing people of not caring and then dismissing their efforts to help. I really appreciate the efforts of Philipp and Baiju to help us make incremental progress on the release process for individual projects. I think we need to start working (thinking & talking) about the release process for the larger Zope 3 ecosystem. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases
On Oct 6, 2007, at 4:14 AM, Fred Drake wrote: On 10/6/07, Philipp von Weitershausen <[EMAIL PROTECTED]> wrote: Yet zope.app.error was split up into zope.error and zope.app.error without releasing a zope.app.error 3.4.0 final first. The split up should have been done entirely in the 3.5.x series, *after* producing stable 3.4.0 releases. It's all there in Subversion. Someone who wants a final 3.4.0 release is welcome to make one. But existing stable versions shouldn't have been updated (and presumably changed) to depend on the new unstable packages. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases
On Oct 6, 2007, at 3:52 AM, Philipp von Weitershausen wrote: Jim Fulton wrote: I have no idea about this specific move. If there was a zope.app.error before, then distributions of it should still exist and new distributions should be backward compatible. zope.error 3.5 is needed by: required by zope.app.applicationcontrol 3.4.1. required by zope.app.appsetup 3.4.1. required by zope.app.publication 3.4.2. OK. I'm not sure what the issue is here. The issue is that the packages Martijn mentioned above as well as zope.app.error were in 3.4.x beta mode. That means they were stabilizing. Yet zope.app.error was split up into zope.error and zope.app.error without releasing a zope.app.error 3.4.0 final first. The split up should have been done entirely in the 3.5.x series, *after* producing stable 3.4.0 releases. Yup. I see. And the sin was compounded by making stable packages depend on the new unstable packages. That is fairly key. Thanks for explaining this. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases
On Oct 5, 2007, at 1:55 PM, Martijn Faassen wrote: ... Grok will pick up the balls Zope 3 dropped here. Hm. I didn't think Zope 3 was animate. Who are you referring to? We actually care about a Grok version as it's the main way to get people to actually use Zope 3 stuff. We noticed this while we were going through egg dependencies by the way, not "using a Zope 3 version". I don't know what you are trying to say. ... Anyway, I personally don't care much that Zope 3.4 is unusuable without a massive investment in time sorting through eggs, It's a shame that you are taking this view. as we have fixed the problem with Grok. If non-Grok users are interested in our fixes, please let us know though. We've just made this massive investment. I'd suggest people to switch to Grok anyway, as we actually think about this stuff and care about having our house in order. Are you suggesting that other people aren't thinking about this and don't care? I hope that the proposal for setting up a known-good-set index will be helpful. I'm sure Stephan would like to know the versions you chose. I imagine he'll be able to get those by looking at Grok. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Known-good-sets problem
I discussed this a bit this afternoon with Stephan and we came up with an idea that we think might help. Stephan is going to try to prototype it. I'll try to explain it. The basic idea is to provide a custom index. There will be such an index for each "known good set" (KGS). An example of a KGS would be the KGS corresponding to the Zope 3.4 release. A KGS would have a set of controlled projects. A KGS index will have manually-managed project pages for all controlled projects. For other projects, it will mirror PyPI. The prototype will build on my cheeseshop mirroring software. We will add a controlled projects list as configuration data. The mirroring software will ignore updates for controlled projects. This is a very small change to existing simple software. (The controlled project directories could be managed by either editing index pages or by placing approved distros into a server directory.) The custom index will be a static web site. With this in place, we can establish KGSs and, for controlled projects, the index pages will only be updated when a distribution for a known project has been carefully vetted. Users can set up buildout or easy_install to use the specific KGS as their index server. Each KGS will have a release manager who will be responsible for maintaining the KGS. We will also create a buildout that tests packages in the KGS. When one wants to test a change to a core package, they would: - check out the buildout - maybe change the index option to point to a particular KGS - check out the project(s) they want to test and configure them as develop eggs - run the buildout test script. Hopefully, this will give us much greater stability than we've had up to now. -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases
On Oct 5, 2007, at 1:59 PM, Martijn Faassen wrote: Jim Fulton wrote: On Oct 5, 2007, at 12:07 PM, Martijn Faassen wrote: Hi there, zope.error is a 3.5 egg, but is needed by 3.4.x releases. I guess this also happened because large package refactorings happened and were released as 3.4.x releases. It's pretty bizarre to run into, though. The satellite projects have version #s that are independent from the Zope version #s. Any similarity is a hysterical accident. :) I don't understand. Probably because you didn't provide specifics so My answer is to a different question than you may have been asking. :) zope.error is zope.app.error, I don't know what this means. but moved to a new place. zope.app.error, is, I understand, gone now. I have no idea about this specific move. If there was a zope.app.error before, then distributions of it should still exist and new distributions should be backward compatible. zope.error 3.5 is needed by: required by zope.app.applicationcontrol 3.4.1. required by zope.app.appsetup 3.4.1. required by zope.app.publication 3.4.2. OK. I'm not sure what the issue is here. Are these "satellite projects"? What is a satellite project? Yes. These are satellite projects. We use the word "satallite project" for the new individual egg projects to distinguish them from the classic Zope 3 tree. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.error is a 3.5 egg, but is needed by 3.4.x releases
On Oct 5, 2007, at 12:07 PM, Martijn Faassen wrote: Hi there, zope.error is a 3.5 egg, but is needed by 3.4.x releases. I guess this also happened because large package refactorings happened and were released as 3.4.x releases. It's pretty bizarre to run into, though. The satellite projects have version #s that are independent from the Zope version #s. Any similarity is a hysterical accident. :) Jim P.S. This has nothing to do with whether the Zope 3.4 release is important. -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: [Zope-dev] I'd love to merge the zope3-dev and zope-dev lists
On Oct 4, 2007, at 10:02 AM, Baiju M wrote: Jim Fulton wrote: Any objections? This would basically involve retiring the zope3-dev list and moving zope3 developers to the zope-dev list. +1 What about retiring #zope3-dev IRC channel and only using #zope ? I though of that. Historically, the #zope channel was much chattier, which makes it hard for me to deal with. When #zope3-dev was set up, I asked that people keep the noise level down, which has made it much more useful, imo. Maybe we should keep these issues separate for now. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] WSGI2
There's work going on to create a second version of WSGI. Last time, we didn't pay much attention until WSGI was a done deal. This time, I think it would be better if we were involved earlier. Unfortunately, I don't have time to pay attention. Does anyone else? Jim P.S. I'd lobe to avoid typos. Especially in subject lines. :/ -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] I'd lobe to merge the zope3-dev and zope-dev lists
Any objections? This would basically involve retiring the zope3-dev list and moving zope3 developers to the zope-dev list. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Release process closure
On Oct 4, 2007, at 9:25 AM, Baiju M wrote: Jim Fulton wrote: On Oct 4, 2007, at 6:51 AM, Philipp von Weitershausen wrote: > On 4 Oct 2007, at 00:59 , Jim Fulton wrote: >> On Oct 3, 2007, at 3:44 PM, Philipp von Weitershausen wrote: >> >>> Jim Fulton wrote: >>>> I'd really like to get to closure on the current approved >>>> release process. Philipp, would you mind separating the >>>> release process into a separate file? Or do you mind if I do >>>> it? >>> >>> Done: >>> http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/ releasing-software.txt >>> >> >> Cool. I think you can delete 5b. You already update the date, >> as you should, on the trunk or branch. You want the actual >> release date to be part of the change log, so it has to be >> entered before making the tag. > > Done. > >> I think we need to split "d" into: >> >> d) "Create a source release" >> >> e) Test the source release. At a minimum, rerun the package tests >> using the source release. (I really need to add a buildout option >> to help with this.) > > So how would I do this? This feels a bit complicated: > > 1. Create a source distribution with:: > > $ python setup.py sdist > > 2. Extract the tarball:: > > $ tar xzf dist/foo.package-X.Y.tgz > > 3. Edit buildout.cfg to make the result of the tarball a develop > egg *instead* of the stuff in 'src':: > > [buildout] develop = foo.package-X.Y > > 4. Rerun the buildout:: > > $ bin/buildout > > 5. Run the tests:: > > $ bin/test No. :) Currently, you could: - Create the source distro. (Note that I always use sparkling clean Pythons, so the command you give doesn't work for me as setuptools isn't importable. I always use: bin/buildout setup . sdist Why you cannot install setuptools ? It's not that I cannot. I *will* not. I like my Python to be shiny sparkly clean. My Python is what I get after running configure, make, make-install. Period. My site- packages is empty. IMO, this is the only sane way to develop. Also, I get very very very cranky when someone wastes my time with a problem that is ultimately traced to crap installed in their Python that isn't part of a standard install. Very cranky. I'm feeling cranky just thinking about it. :) Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Release process closure
On Oct 4, 2007, at 6:51 AM, Philipp von Weitershausen wrote: On 4 Oct 2007, at 00:59 , Jim Fulton wrote: On Oct 3, 2007, at 3:44 PM, Philipp von Weitershausen wrote: Jim Fulton wrote: I'd really like to get to closure on the current approved release process. Philipp, would you mind separating the release process into a separate file? Or do you mind if I do it? Done: http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/ releasing-software.txt Cool. I think you can delete 5b. You already update the date, as you should, on the trunk or branch. You want the actual release date to be part of the change log, so it has to be entered before making the tag. Done. I think we need to split "d" into: d) "Create a source release" e) Test the source release. At a minimum, rerun the package tests using the source release. (I really need to add a buildout option to help with this.) So how would I do this? This feels a bit complicated: 1. Create a source distribution with:: $ python setup.py sdist 2. Extract the tarball:: $ tar xzf dist/foo.package-X.Y.tgz 3. Edit buildout.cfg to make the result of the tarball a develop egg *instead* of the stuff in 'src':: [buildout] develop = foo.package-X.Y 4. Rerun the buildout:: $ bin/buildout 5. Run the tests:: $ bin/test No. :) Currently, you could: - Create the source distro. (Note that I always use sparkling clean Pythons, so the command you give doesn't work for me as setuptools isn't importable. I always use: bin/buildout setup . sdist - Add your dist directory to the list of find links - Specify the new version in your requirements - remove the develop entry - run the buildout - run the tests As I said, buildout could automate this in the future. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Release process closure
On Oct 3, 2007, at 3:44 PM, Philipp von Weitershausen wrote: Jim Fulton wrote: I'd really like to get to closure on the current approved release process. Philipp, would you mind separating the release process into a separate file? Or do you mind if I do it? Done: http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/ releasing-software.txt Cool. I think you can delete 5b. You already update the date, as you should, on the trunk or branch. You want the actual release date to be part of the change log, so it has to be entered before making the tag. I think we need to split "d" into: d) "Create a source release" e) Test the source release. At a minimum, rerun the package tests using the source release. (I really need to add a buildout option to help with this.) f) If the package has extensions, make and test a windows egg. (You may need to get someone with the needed Windows tools to help you with this. :) f) Upload the release(s) to PyPI Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.dottedname doesn't have a CHANGES.txt (again?)
On Oct 2, 2007, at 6:52 PM, Stephan Richter wrote: On Tuesday 02 October 2007 17:14, Jim Fulton wrote: One hole I see is giving people guidance on what needs to be tested (and how) before a release is made. My preference would be to rely heavily on judgement with a few checks so as not to make things too heavy. This might rule out some releasers. This is odd text to quote, considering what follows. Do you really think that an automated tool is going to be able to determine on a case by case basis what needs to be tested? Heck, I find it hard to judge what is best to test. I think we need to think about what the process needs to be. It's not at all clear to me and yet you are ready to automate it. I can only hope that this wasn't the text you intended to quote. I want tools! Actually, I just want one tool. You want a silver bullet. 70% of the release process is repetition and that needs to be factored into a tool. I think that is overly optimistic, but I have an open mind. Frankly though, your desire for an automated tool frightens me. This tool should have been written before the Zope trunk was blown into pieces, but it wasn't. :-( I'm skeptical that such a tool can do that much. Certainly, when we broke the trunk up, we didn't know what all the issues would be. We couldn't automate a process we didn't yet fully understand. My only real regret is that we broke things too far too fast, but that is water under the bridge. Some of the fouls this week had nothing to do with the release process. The breakage you (I assume) and Roger caused by removing needed files would have caused just as much havoc in the old days. You seem to think that the trunk is everything, so making changes there and adjusting all of the effected software *on the trunk* is enough. This kind of breakage used to occur before and caused much pain for 3rd-party consumers of the Zope 3 tree. Other breakage occurred after people tried to give you advice on how to do things more reliably. Saying you need a tool is not an excuse for ignoring advice. There is no way that we will be able to support this many packages in the future, if we keep doing this manually. You should know that we already *are* maintaining this many packages. People have been able to figure out how to release packages in a fairly stable way. We're learning from our mistakes and creating processes to make things better. I have already spent days on doing eggs, when I really just wanted to code. :-( Then stop. The current process is too manual and requires too much judgement for you. OK, then stop updating core packages. We all make mistakes. Even with the best tools, processes and intentions, bad things will happen. No one will hold a grudge as long as people are being responsible and trying to do the right thing. OTOH, while wishing for a tool and or for better processes is fine, blaming the current processes for your mistakes is getting tiresome. jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Release process closure
On Oct 3, 2007, at 11:46 AM, Benji York wrote: Jim Fulton wrote: For example, a packaging of MochiKit 1.3.1 might have a version # like 1.3.1-1, or, if it is exciting enough: 1.3.1-1.1 I assume setuptools interprets version numbers like this in a reasonable way. AFAIK. :) Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Release process closure
On Oct 3, 2007, at 11:12 AM, Benji York wrote: Jim Fulton wrote: I'd really like to get to closure on the current approved release process. +1 There are other improvements that might be made, but let's not let the perfect be the enemy of the good. This isn't perfect, so I get to suggest it. I propose that we codify the practice of release tags having their x.y.z version number as their name, and in the setup.py. Which implies that we codify 3-number version numbers, that is: major.minor.bugfix with optional modifiers of the form dev, aN, bN, or cN. I also suggest a special version numbering scheme when we package something else. For example, if we package Twisted ro package some external js library in a Python package. In this case, I suggest we use the version number of the thing being packages with the addition of a post-release addition. For example, a packaging of MochiKit 1.3.1 might have a version # like 1.3.1-1, or, if it is exciting enough: 1.3.1-1.1 Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Release process closure
I'd really like to get to closure on the current approved release process. Philipp, would you mind separating the release process into a separate file? Or do you mind if I do it? Some comments on the current draft at: http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/ maintaining-software.txt WRT version numbers in setup.py. I'm inclined to endorse Philipp's recommendation for now. If there was a way to specify a version number on the command line (or in a buildout.cfg) when creating develop eggs, then I'd have a different position, but given current technology, I think Philipp's recommendation, as I understand it, is best. I think there are some details missing. I think the intend is that the version number in the setup.py on the trunk and on release branches should have the "dev" suffix. I think this is good as a reminder and a flag that accidentilly released dev eggs are suspect. I think the dance should be that, to make a release, you make a tag and then edit the version number on the tag. I think this sort of editing on a tag is reasonable and it is fortuitous that svn allows it. Also, by default, the "next" version on the trunk should be a release with the second number incremented. The next version on a release branch should have the 3rd number incremented. This is a minor detail, especially since I think we can avoid release branches for most projects and I think that release branches shouldn't be created until they are needed. Of course, tags should always be created. There are other improvements that might be made, but let's not let the perfect be the enemy of the good. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.dottedname doesn't have a CHANGES.txt (again?)
On Oct 2, 2007, at 5:05 PM, Philipp von Weitershausen wrote: Martijn Faassen wrote: A release of zope.dottedname was made apparently today that refers to a CHANGES.txt but doesn't have one. Probable scenario: someone forgot to svn add a CHANGES.txt, then didn't check out before the tag before releasing... Right. This is actually a *common* mistake. Not everybody believed me when I sad that it was. I think we've got proof of this once more now. A couple of days ago I added some very explicit instructions on how you should make a release to my guide [1]. Basically it says you should *first* make a tag and only create the release *from the tag*. I hope that people will take this more seriously now. I do, It's not that I consider my guide to be a papal edict (I'm not Jim, after all), :) It would help if you would split out the release making part so we can focus on that. but it *is* based on experience and best practices. Having made releases for many Zope eggs (if you want proof, just look at [2]), I think I can safely say that I know my way around the pitfalls now. Agreed. That said, I welcome any suggestion that other people have while creating releases. We need to get better at this stuff! Yup. See above about separating the release instructions. One hole I see is giving people guidance on what needs to be tested (and how) before a release is made. My preference would be to rely heavily on judgement with a few checks so as not to make things too heavy. This might rule out some releasers. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 26, 2007, at 6:14 PM, Jim Fulton wrote: ... I agree that the develop egg case is problematic. I'd like to think about that. Well, I thought and I thought I could probably arrange that develop eggs created by buildout got their versions from the buildout spec. Something like: [buildout] develop = . ==1.4 foo ==2.1 This would override whatever version was given in setup.py. This wouldn't help people who don't develop with buildout though. Alternatively, we could add some API that would allow our setup files to take a --version argument. So our setup files on the trunk or branches might have something like: setup( version = comman_line_version() or something. I'm not an expert on extending setuptools. I'm somewhat pessimistic about getting new features in setuptools as 0.7 seems to have fairly grandiose objectives. There don't seem to be incremental feature releases planned. :( I have a feeling that we/I should learn how to extend setuptools to get the features we need. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 27, 2007, at 8:00 AM, Philipp von Weitershausen wrote: ... buildout setup? I've never seen that. I can't find any documentation on it in zc.buildout. G. Sure enough, the documentation doesn't mention it. That's an annoying documentation bug, I'm pretty sure it was documented at one time. Also, some examples in the documentation use it. How does it work? bin/buildout setup setup arguments ... Does it call setup.py for every development egg in buildout.cfg? No. Why not just call python setup.py? Because: - The setup you are invoking uses setuptools and you are using a clean Python, or - The the setup.py you're using doesn't use setuptools but you want to use setuptools commands. Basically, the setup command is a convenience that gets setuptools into the path and imported before invoking the given setup. I find this quite useful. One annoyance is that, although there are workarounds, it generally wants there to be a buildout.cfg file around and I often want to use it to run a setup.py that has nothing to do with a buildout. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 27, 2007, at 7:37 AM, Philipp von Weitershausen wrote: On 27 Sep 2007, at 12:20 , Stephan Richter wrote: egg_info does not validate the trove classifiers, for example. I tried this last night before writing this mail. Well, to be honest, I wonder how you can mess up with the classifiers. I just always copy them from http://pypi.python.org/ pypi?%3Aaction=list_classifiers. Anything else is just insane... The classifiers are a PITA. I suspect we should be very restrictive in using them. It's especially annoying that release status can generally be inferred from the version number. I'm beginning to wonder if we should have some setuptools extensions that automate some of these things, Another idea I've been thinking about is a custom index that is used rather than PyPI. The main motivation was to automate management of non-public distributions, but such a front end could also automate some aspects of releases that setuptools doesn't handle. But, if you wish for such a tool, let your wish be my command. With the attached verifyclassifiers.py script, you may do so using the following command: python setup.py --classifiers | python verifyclassifiers.py Cool. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 27, 2007, at 7:18 AM, Philipp von Weitershausen wrote: On 27 Sep 2007, at 13:07 , Martijn Faassen wrote: On 9/27/07, Tres Seaver <[EMAIL PROTECTED]> wrote: [snip] Further, anybody who finds the effort of creating a "fresh' checkout bevore making a release too burdensome should consider themselves self-selected out of the "release manager" pool. I'm *not* kidding about that: taking shortcuts durng the release process transfers pain / cost to everyone downstream. While I fully agree that releases should be done with care and attention, and that doing bad releases creates pain/cost for everybody, this line of argumentation can be used to back up any form of complication to the release process. :) Let's focus on the reasons for each step and keep the discussion at that level, please? I think it's useful if people ask "is that really necessary?" for steps in the release process. Just weigh the pros and cons. I'd like to understand more about the trouble Philipp ran into doing releases from the original checkout and then tagging. I've summarized this in another thread [1], but I'll be happy to elaborate here: * People actually forget to make the tag. This happened to Roger the other day. It happened to Jim before with zc.buildout. I'm not mentioning their names to point fingers. I'm just pointing out that people who've been with us for a long time forget. If the process said they should actually check out the tag, they'll hardly forget. Having a script we have to mindlessly follow will help. * People create the wrong tags or tags in the wrong places. This happened to Jim the other day with ZODB 3.7.1. Again, I'm not blaming him. But I believe if he had to check out the tag to make the release, he would've caught his mistake. I doubt it. I would have probably just rereleased 3.7.1, possibly with 3.7.2 uselessly included. * People forget to clean up the 'build' directory. This happened to me the other day. Let me elaborate: I have a trunk checkout of zopeproject that I use to develop it. Whenever I wanted to make a release, I'd just prepare it, tag it and generate the distributino from the trunk checkout. The problem was that I had moved stuff around, but the old stuff was still making it to the egg because it still sat around in the 'build' direcdtory that distutils leaves. I know it sucks that distutils doesn't clean up after itself, but that's just the way it is. If you're forced to use a clean checkout, this can't happen. The build directory is really annoying. * People forget to check in important files. This happened to Baiju the other day. He created a CHANGES.txt file and referred to it from setup.py. The only problem is that he forgot to check it in. It can happen, I'm not blaming him. The problem is that setuptools looks at which files are in svn in order to create the archive manifest. So CHANGES.txt wasn't in the tarball and the egg was completely uninstallable (because setup.py couldn't be executed since CHANGES.txt was missing). These are four separate cases where I've actually witnessed myself or other people mess up. We're forgetful, we can't do anything about that. We can, however, force us to catch our mistakes. I believe that if we made everybody create the tarballs from the tag, it would improve the situation a lot. I'm beginning to see a consensus that we should make it impossible to create distributions from the trunk or a release branch. Yup. BTW, I find it helpful to test source releases by unpacking them and seeing if I can build eggs from the unpacked source. Of course, it would be even better to run the tests. I suspect that a buildout feature could automate this. Something along the lines of a buildout command that: creates distros from the listed develop directories and then installs those distros rather than the develop eggs so that tests could be run against the distros. This would probably be something good to do when I have some time to get back to buildout. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 27, 2007, at 4:45 AM, Philipp von Weitershausen wrote: On 27 Sep 2007, at 02:29 , Tres Seaver wrote: Further, anybody who finds the effort of creating a "fresh' checkout bevore making a release too burdensome should consider themselves self-selected out of the "release manager" pool. I'm *not* kidding about that: taking shortcuts durng the release process transfers pain / cost to everyone downstream. Amen, brother. Well said. +1 -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: PLEASE don't remove eggs [was Re: [Zope3-dev] faulty releases and pypi access [update]]
On Sep 26, 2007, at 10:35 AM, Jim Fulton wrote: This is also a technical issue: As long as zc.buildout and setuptools foolishly accept dependency links from an egg, it'll be painful to detect accidental reliance on external repositories. That's a good point. I wouldn't go so far as to say "foolishly", but I would say that this is a policy that should be overrideable. Checkins to buildout (with tests, of course) accepted. Sort of that, a feature request in launchpad would be helpful so this idea doesn't get forgotten. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 28, 2007, at 1:36 PM, Dieter Maurer wrote: ... Your example: You have release 1.1. You start working on the next release: 1.2dev-r#. Someone fixes a bug in 1.1 and releases 1.1.1. With quite high probability, the bug fix should go as well into the 1.2 development. Thus, the problem you describe (1.1.1 is newer than 1.2dev-r; but it treated as older then 1.2dev by 'setuptools') has an effect only for a very limited time -- the time until the fix (1.1 --> 1.1.1) went into the 1.2dev release, which would probably become some 1.2dev-r!. Sure, but that time could stretch out quite a bit. I suppose that this problem somewhat inherent in having multiple release branches. For example, whatever we do, there will always be the possibility that an x.y.z release could have fixes in it that aren't in an x.y+1 release yet, especially if x.y is a non-final release. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: AW: AW: [Zope3-dev] zope.app.securitypolicy deferred imports errors.
On Oct 2, 2007, at 9:24 AM, Roger Ineichen wrote: Hi Jim, Betreff: Re: AW: [Zope3-dev] zope.app.securitypolicy deferred imports errors. [...] Yes, please do. It's up to people making changes to use some judgement. I mentioned in that thread that when I make changes to "core" components, I often do test against the old trunk tree. Despite all of your complaints about the need to do this, you didn't. That's not true. I did test the package against the trunk. But the problem was, I also adjusted the imports in the packages. hehe. This sort of practice makes any arguments for testing against the "trunk" kind of meaningless. What I was missing is to run the tests against a older version of the trunk. So perhaps now you want people to test against old versions of the trunk? How many old versions? A better approach would have been to add backward compatibility tests for each backward-compatibility support change you made, as Lennart suggested. Even then, to be more sure that you caught everything, you should have made the changes in two stages: 1. Refactor the package in question providing backward compatibility support, including tests of that support. Test against other unchanged packages. 2. Then change the other packages is you must. ... I know there is no difference between the old trunk and the new egg development testing. This error whould also happen to me if I where doing it in the old trunk setup. My problem was not test my work against a older version of the trunk. Or like Lennart told, the missing tests for integration (import changes). The latter. The 2-step process that we've used in the past would also have helped. I personaly think, less test -- more errors. So test! What's stopping you? The old tree is still available! The crapy "do it by your self test setup" is stopping me really hard or at least slowing me down right now more then the it should. Maybe it should slow you down more. It's not easy to follow all the ideas behind egg, setuptools, easy_install, buildout in such a speed like we changed to this patterns. I agree "Speed kills", but I think switch to eggs force us to "take it all or nothing". We probably do need more formalized testing/release methodologies. As I'm sure Tres and others will rightly point out, there's probably a lot we can learn in this area from linux distributors. In the mean time, changing packages that lots of other people depend on requires judgement and care. If you don't think you're up to it, then don't try it. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: AW: [Zope3-dev] zope.app.securitypolicy deferred imports errors.
On Oct 2, 2007, at 6:03 AM, Roger Ineichen wrote: Hi Lennart Betreff: [Zope3-dev] zope.app.securitypolicy deferred imports errors. Roger Inechens split of zope.app.securitypolicy into zope.securitypolicy cause loads of problems, because many of the deferred imports were incorrect. I saw Stefan Richter fixed some, and I have fixed some more. Yes you are right First somebody that can make releases (which may or may not include me, I honestly don't know who can make releases of eggs) needs to make a new one. But we also need to avoid this stuff in the future. Yes, we always have to avoid errors, but sometimes it happens. And since we dont test against the trunk, we don't see this kind of errors anymore. This means we test against custom projects. I didn't fully recognize this and was trusting the tests. But I was wrong. We don't have tests for this anymore. We probably didn't have test for my error at all. In other words; Right now we only test if a egg works within their dependency. But we don't test other eggs if they work with the egg we develop. See all my different mails about this topic! Yes, please do. It's up to people making changes to use some judgement. I mentioned in that thread that when I make changes to "core" components, I often do test against the old trunk tree. Despite all of your complaints about the need to do this, you didn't. How is the recommended process to solve this? Some sort of unittest to make sure the deferred impoirt work? Is there an example of this somewhere? I'm proposing to test against a set of possible breaking stuff. This means we need a kind of zope3 trunk egg which our test will deoend on. As I mentioned in the thread you want us to pay attention to, this isn't necessary. In fact, I doubt it would be useful. I think we need a Zope 3 application buildout that builds the traditional Zope 3 app. You can then introduce a develop egg for the package you are changing there. In the mean time, all you need is a trunk checkout and, possibly, to adjust an external. (I don't remember how the externals are set up there.) See the mails form Stpehan Richter about that. Jim also agrees on that but is thinking this should be optional as far as I understand the mails between Stephan and Jim. We defently lost the overall tests we had in the trunk setup. Most people aren't changing these components. Nevertheless, we still have the old tree available for testing. We didn't lose anything. And this is a very bad idea. If we don't recommend to run all tests again form a single egg refactoring, we will see more errors like this. Don't you think it's a little hypocritical bemoaning that we're not doing this when you don't do it yourself? ... I personaly think, less test -- more errors. So test! What's stopping you? The old tree is still available! If someone wants to make this better, a working Zope 3 application buildout would help a lot. I doubt it would be that hard to finish at this point. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] major packaging reorganization happening in 3.4 releases?
On Oct 2, 2007, at 5:25 AM, Martijn Faassen wrote: Hi there, Besides causing us a lot of problems here at the Grok sprint, I also wonder why in the world are we doing major packaging reorganizations and releasing them as minor 3.4.x releases? You'd think such a reorganization would warrant a 3.5 release! Agreed. Someone needs to sloow down. "Speed kills." deserves to be added to the Zen of Python. (Actually, ZoP does have a variation of that.) Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: New package zc.configure provides an exclude directive for excluding zcml files
On Oct 2, 2007, at 3:29 AM, Martijn Faassen wrote: Jim Fulton wrote: [snip] Maybe grok was already trimmed down. In my case, I basically eliminated all ZMI support (since I didn't need it). I got about 40%, Grok is trimmed down in the sense that it doesn't depend on all Zope 3 packages, though due to the interesting dependency structure it still relies on about 100 eggs. We didn't do any trimming down of ZMI support. Note that it wasn't my goal to trim the number of eggs I got, although trimming that, or, in theory, using zipped eggs would speed startup as well. I'm using the zope.app.zcmlfiles egg as my base. Would it be possible to get a list of exclude statements that you use to eliminate ZMI support in your project? I imagine our list is far from complete. Here is the zcml file I'm including rather than incliding zope.app.zcmlfiles/configure.zcml: http://namespaces.zope.org/zope"; xmlns:i18n="http://namespaces.zope.org/i18n"; i18n_domain="zope" package="zope.app.zcmlfiles" > file="session.zcml" /> file="ftpplugins.zcml" /> file="httpplugins.zcml" /> file="groupfolder.zcml" /> file="password.zcml" /> file="principalfolder.zcml" /> Note that this was made by just copying the configure.zcml from zope.app.zcmlfiles and commenting out some things and adding excludes. I basically kept commenting things or excluding things until my tests failed. :) I could probably go a little further if I worked a lot harder, so of course, I stopped. :) I'll also probably have to add some things back later when I pay attention to i18n. (This app uses extjs for it's UI and I haven't figured out how I'm going to approach i18n for that. extjs rocks btw.) Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] New package zc.configure provides an exclude directive for excluding zcml files
On Oct 1, 2007, at 11:17 AM, Jim Fulton wrote: On Oct 1, 2007, at 11:09 AM, Lennart Regebro wrote: On 9/29/07, Jim Fulton <[EMAIL PROTECTED]> wrote: This helps in cases where you want the configuration for a package, but you don't want some of the configuration that it includes. This allowed me to trim quite a bit off of the startup time, and, more importantly, the test setup time, for a project I'm working on. Works like a charm. We tried it here at the grok sprint. Although we were only able to speed up Grok startup with 10%. Maybe we can get to 20-30% if we work more on it, but we're not sure it's worth it. http://regebro.wordpress.com/2007/10/01/neanderthal-sprint- speeding-up-the-grok-startup/ Maybe grok was already trimmed down. In my case, I basically eliminated all ZMI support (since I didn't need it). I got about 40%, Oh BTW, I ran Zope with debug logging. That way I could see what was being included. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] New package zc.configure provides an exclude directive for excluding zcml files
On Oct 1, 2007, at 11:09 AM, Lennart Regebro wrote: On 9/29/07, Jim Fulton <[EMAIL PROTECTED]> wrote: This helps in cases where you want the configuration for a package, but you don't want some of the configuration that it includes. This allowed me to trim quite a bit off of the startup time, and, more importantly, the test setup time, for a project I'm working on. Works like a charm. We tried it here at the grok sprint. Although we were only able to speed up Grok startup with 10%. Maybe we can get to 20-30% if we work more on it, but we're not sure it's worth it. http://regebro.wordpress.com/2007/10/01/neanderthal-sprint-speeding- up-the-grok-startup/ Maybe grok was already trimmed down. In my case, I basically eliminated all ZMI support (since I didn't need it). I got about 40%, Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] New package zc.configure provides an exclude directive for excluding zcml files
This helps in cases where you want the configuration for a package, but you don't want some of the configuration that it includes. This allowed me to trim quite a bit off of the startup time, and, more importantly, the test setup time, for a project I'm working on. See: http://pypi.python.org/pypi/zc.configuration Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: reasonable syntax for multi-adaptation
On Sep 28, 2007, at 11:14 AM, Martijn Faassen wrote: ... On 9/28/07, Jim Fulton <[EMAIL PROTECTED]> wrote: [snip] we could investigate whether we can't come up with something that: * doesn't break the existing notation. The cleanest way to support such non-interference seems to be to do this using an extra .adapt method. This is unfortunate, as I at least consider it prettier if it used the IFoo() syntax (even though I proposed .adapt). I agree. Hm, you agree with my evaluation, but would this mean you'd suggest we use 'adapt' or extend the __call__ syntax? In the short-to-medium term the adapt method is my preference. ... After thinking about it, I think I can avoid the extra hook. Don't worry about the details. :) Cool! The person that tries to implement this will have to worry about them, though, so we'll get back to you on that, I'm sure. Yup. ... Do you suggest we extend zope.component with support for this, or would you suggest we indeed create a new package? You'll need to modify InterfaceClass to add the new adapt method and __call__ semantics. The hook installed by zope.component will need to be updated as well. I see no new package being warrented. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: reasonable syntax for multi-adaptation
On Sep 28, 2007, at 10:31 AM, Martijn Faassen wrote: Dominik Huber wrote: It would be great, if we could handle other adaption-derivations by the proposed unified, reasonable adaption-api too. [snip interesting proposal] Okay, so there seems to be quite a bit of consensus for *some* form of support for this. I've seen a number of proposals which to me look quite reasonable. Since Jim doesn't like this much but there seems to be a lot of support from others, And am willing, therefore, to compromise we could investigate whether we can't come up with something that: * doesn't break the existing notation. The cleanest way to support such non-interference seems to be to do this using an extra .adapt method. This is unfortunate, as I at least consider it prettier if it used the IFoo() syntax (even though I proposed .adapt). I agree. * creates a small a new hook inside zope.interface to support this After thinking about it, I think I can avoid the extra hook. Don't worry about the details. :) * can be plugged in by an optional extension, like zope.interfacextended See above. * implements the proposed new features. I like the idea of also allowing utility lookup through the same mechanism. I don't, but I'm willing to give in to popular demand. BTW, we *could* use interface __call__ for utility lookup, as there would be no conflict with current notation *if* we specified a default using a keyword argument. I would far prefer using IFoo() or IFoo(default=None) to look up a utility over: IFoo.adapt() or IFoo.adapt(default=None) for, I hope, obvious reasons. I'd also like to support specifying a default for adapter lookup via a keyword argument. In fact, I'd like to deprecate passing the default positionally. This would open the door, sometime in the future, to allowing multi-adaptation via: IFoo(ob1, ob2). So, any volunteers to try to implement this? I insist on reviewing any change before checking it in. This will create an unavoidable bottle neck. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 27, 2007, at 1:42 PM, Dieter Maurer wrote: Jim Fulton wrote at 2007-9-26 18:14 -0400: ... We've just released 1.1. We guess the next release is 1.2. We change things and release, 1.2dev-r#. Someone fixes a bug and releases 1.1.1. Now there's a dev release of 1.2 that is actually older than the 1.1.1 release but that setuptools considers to be newer. I think this is fairly problematic. Then again, I think that dev releases are problematic in a lot of ways. In your example, it is likely, that the fix will also go into the 1.2 development branch. I.e. you will get an 1.2dev-r! with "! > #". I'm not sure what you mean by "my example" or why what you said would be so. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Why do we restrict our egg testing?
On Sep 27, 2007, at 10:00 AM, Stephan Richter wrote: On Thursday 27 September 2007 09:35, Jim Fulton wrote: On Sep 26, 2007, at 7:57 PM, Roger Ineichen wrote: Hi Can anybody tell me why we restrict our test setup in zope eggs and only use a subset of package for our test setup? It isn't practical, during development, to test all of the eggs that might be affected by a change, which is, BTW a *much* larger set than the old Zope 3 tree. I am not saying that testing *all* packages is necessary, but a healthy set of them. We did not consider this impractical when developing on the entire trunk. Back then, it was part of our development responsibility. We always ensured that some set of Zope 3 was always stable together. But we were developing the entire trunk as a whole. We were developing a monolith. For example, it was acceptable, when changing a particular Python package to "fix" the tests in other packages that broke by changing the tests (or the code they test) to reflect the change in the original component. This provided less than no protection to 3rd-party packages. Recently, y'all removed some files from zope.app.security. I bet you adjusted other files in the Zope 3 tree to reflect that change, but that didn't help the many more 3rd-party packages and applications that were broken. This is completely gone now. I want a solution; buildbot is okay, Good. Now we need someone to implement it. ... Depends on what "this" is. I think running all of the tests in the old Zope 3 tree whenever we change a component is a waste of time. I beg to differ. It has something to do with responsibility to the community. I am not saying you have to test all of Zope 3 when changing zc.* for example. But I think if a package in a defined "core" set is changed, requiring to run all the tests of the "core" set should be required. A really good example is ``zope.component``. Not testing it across several packages is very dangerous for our stability. That's a good example. The last time I changed zope.component, I *did* test it with the tree. It would be useful to be able to do that conveniently in the future. What I really want is a buildout for a big Zope 3 application, similar to what we've released in the past. Then, I will often choose to test a change in something as core as cope.component as a develop egg in that buildout. I would definitely do that. (I don't want a meta egg.) I'll note that I don't work on "core" components very often so testing against the Zope 3 app wouldn't be something that I would do often, but I'll concede that on those rare occasions that I do, having something like this would be useful. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 26, 2007, at 8:26 PM, Tres Seaver wrote: ... If we are going to have a change log, which we should, I would prefer it to be included in source distributions. I want them present in the *installed* egg, not just in the source distribution: setuptools doesn't preserve source distributions after creating / installing the '.egg' version. Agreed, but Including a file other that README in the root requires extra effort that I don't want to require -- writing setup.py files is hard enough as it is. Put the "real" README.txt and CHANGES.txt in the actual package: the version which is a peer of 'setup.py' should be a stub which points to the "real" versions. I can live with this, but I don't like it. It feels more complicated to me. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Known working sets II [was: Eggification redux]
On Sep 26, 2007, at 8:22 PM, Tres Seaver wrote: ... I think that replacing 'index_url' with a "gated community" of packages is the only path to sanity here: the contract of the Cheeseshop (share new releases of all packages with everyone ASAP") is incompatible with our goals ("ensure that users can install a given package and its dependencies, and have them work"). Regretfully, I agree. People here at ZC are already moving in this direction internally. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Why do we restrict our egg testing?
On Sep 26, 2007, at 7:57 PM, Roger Ineichen wrote: Hi Can anybody tell me why we restrict our test setup in zope eggs and only use a subset of package for our test setup? It isn't practical, during development, to test all of the eggs that might be affected by a change, which is, BTW a *much* larger set than the old Zope 3 tree. For unit testing of the egg under development, there isn't much point in running other tests. For functional testing, you want to test the package's layer and, for an application a set of components that may be configured quite differently than the Zope tree. I think there is value in testing some universe in an off-line way using something like buildbot, Why do we not use a Zope3 meta egg which contains all our zope packages as a test base. This whould allow us to test the same we have in the zope3 trunk and let us run *buildout/test -s zope* from within each egg. I personally don't see much value in that, especially considering the effort involved. what is the builbot doing right now? Does the builbot still runs test on the trunk? Or does the buildbot test the eggs? Unfortunately, buildbot is a bit of a pita. :( It seems to be high maintenance. If someone wants buildbots to run, they should help or make it happen. Christian Theune has a buildbot setup that ran tests for all of the projects in the repo. I think this was *very* promising, but I don't know what the current status is. Is this a bad idea? Depends on what "this" is. I think running all of the tests in the old Zope 3 tree whenever we change a component is a waste of time. If "this" is having an automated system for testing a wide collection of packages to check for dependency breakage, then "this" is a great idea, but potentially a lot of work. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 26, 2007, at 6:23 PM, Philipp von Weitershausen wrote: On 27 Sep 2007, at 00:14 , Jim Fulton wrote: Let's say X depends on Y and I'm developing them simultaneously as development eggs in one sandbox (e.g. buildout). I add a feature to Y that I'd like to use in X. That means I'll have to change X's version dependency regarding Y so that it now depends on Y>a.b where a.b is the latest release that didn't have the feature I added. Will Y with the setup.py stating version='unreleased' satisfy the Y>a.b requirement that X (rightfully) has? If not, then I think we have a problem. If yes, then I find that very confusing. Leaving aside for the moment the fact that "develop" eggs are used for system egg installs :(, I think if a buildout uses a develop egg, there should be a very strong presumption that it it should be used. OTOH, if you have a part that depends on a previous major version, then you don't really want to use a develop egg for the later release. This is a bit of an edge case though. It is as long as you ignore the fact that all major Linux distros install develop eggs into site-packages. I already left that aside. :) Buildout should be able to detect this case and deal with it, which is why I left it aside. I still see the development egg case the best argument for putting the next anticipated version number into setup.py. I think the compromise of version number + 'dev' marker would work best. I agree that the develop egg case is problematic. I'd like to think about that. If you ever actually make dev releases, then guessing wrong about the next version could cause real problems. Oh, I'm not actually thinking about dev releases. I too think they're very problematic. I just suggested using the 'dev' marker as a way to tell branches or the trunk apart from tags. But people will make these releases and leaving the version out will mean that these releases will be harmless. On the branches and trunk, setup.py's version is the next anticipated release + 'dev', while on tags it's just release version. That way people won't accidentally make releases from branches or the trunk (because they'd get an egg with a 'dev' in the version). They'll have to use a tag. You think getting an egg with dev in the version will stop them? Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re-normalizers instead of ellipses in doctests [was: some checkin]
On Sep 26, 2007, at 5:52 PM, Philipp von Weitershausen wrote: Marius Gedminas wrote: Log message for revision 80145: Make the test pass on both Python 2.4 and 2.5. ... -=- Modified: z3c.form/trunk/src/z3c/form/action.txt === --- z3c.form/trunk/src/z3c/form/action.txt 2007-09-26 21:26:25 UTC (rev 80144) +++ z3c.form/trunk/src/z3c/form/action.txt 2007-09-26 21:49:09 UTC (rev 80145) @@ -236,4 +236,5 @@ > >>> eventlog[-1].error - > + + It's too bad that Exception's __repr__ changed in Python 2.5. The reason is that it's a new-style class now, which is actually a benefit at the same time. Anyway, these ellipses are very ugly and confusing. I think they're okay-ish when you know exactly what they're supposed to stand for. In this case we can definitely do better. They also have the disadvantage that they can swallow unexpected output. This is a fairly serious downside. It would be nice if doctest provided a variation on ellipsis that didn't consume newlines. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 26, 2007, at 5:37 PM, Philipp von Weitershausen wrote: On 26 Sep 2007, at 23:18 , Jim Fulton wrote: On Sep 26, 2007, at 4:34 PM, Benji York wrote: Philipp von Weitershausen wrote: Stephan Richter wrote: Because I disagree with that, since you cannot know the next version. You can always know the minimum version. If you just released 3.4.2, I think it's sensible to point the next release to 3.4.3. If you later decide that you really need a feature release, you can always bump to 3.5.0a1 (which would be the first release in the 3.5.x series). Gary and Benji made me pay attention to this point. :) I *really* don't like setting the version to the number for the next release, since one often doesn't know what it is. Maybe. However, when you do the actual release then, you're still going to have to find out which version number to use. On release branches this is a trivial step, of course: You just look at the latest tagged version and increment accordingly. On the trunk it might be trickier. After 1.0, do we have 1.1? Or 2.0? Maye this aspect isn't such a big deal, though. You can always look at the previous tags or at CHANGES.txt. Why not leave the version totally out of the setup.py in the trunk? Benji and Gary won me over to this point of view. :) There's a *really* good reason for putting the upcoming version number in setup.py, though: development eggs. Good point. Let's say X depends on Y and I'm developing them simultaneously as development eggs in one sandbox (e.g. buildout). I add a feature to Y that I'd like to use in X. That means I'll have to change X's version dependency regarding Y so that it now depends on Y>a.b where a.b is the latest release that didn't have the feature I added. Will Y with the setup.py stating version='unreleased' satisfy the Y>a.b requirement that X (rightfully) has? If not, then I think we have a problem. If yes, then I find that very confusing. Leaving aside for the moment the fact that "develop" eggs are used for system egg installs :(, I think if a buildout uses a develop egg, there should be a very strong presumption that it it should be used. OTOH, if you have a part that depends on a previous major version, then you don't really want to use a develop egg for the later release. This is a bit of an edge case though. I know that if Y's setup.py stated version='a.b-dev', it will work. This what my guide currently suggests (including taking it out just for release tags). A problem with this is that someone else might make a release of Y, either to fix a bug or to add a feature while your working on the new feature in Y. After branching for a release we can set the version (e.g., 1.2), make a release, and tag the branch. We should not require branches. I would only bother creating maintenance branches when they are needed. +1 Z, B, G, and I propose the following: Who's Z? :) - Leave the version # out of setup.py on the trunk and on branches. When it is time to make a release, either from the trunk or from a maint branch: - Update changes.txt, adding a heading for the new # and date - Create a tag - check out or switch to the tag - Set the version in setup.py on the tag. Check it in. - Make the release from the tag. I could live with that, even with the fact that you'd be modifying a tag, as long as it's done in this exact order and the only modificdations to a tag woudl be setting setup.py. I still see the development egg case the best argument for putting the next anticipated version number into setup.py. I think the compromise of version number + 'dev' marker would work best. I agree that the develop egg case is problematic. I'd like to think about that. If you ever actually make dev releases, then guessing wrong about the next version could cause real problems. Consider the following case: We've just released 1.1. We guess the next release is 1.2. We change things and release, 1.2dev-r#. Someone fixes a bug and releases 1.1.1. Now there's a dev release of 1.2 that is actually older than the 1.1.1 release but that setuptools considers to be newer. I think this is fairly problematic. Then again, I think that dev releases are problematic in a lot of ways. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 26, 2007, at 5:28 PM, Stephan Richter wrote: On Wednesday 26 September 2007 17:18, Jim Fulton wrote: - Update changes.txt, adding a heading for the new # and date - Create a tag - check out or switch to the tag - Set the version in setup.py on the tag. Check it in. - Make the release from the tag. Changing tags is not that good. I'd rather check in a aversion number then. ;-) This is exactly what we've been doing for Zope 3 releases -- for the same reason. I think it is the one acceptable and reasonable change to a tag. The benefit of forcing us to release from a tag is, imo, significant. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 26, 2007, at 4:34 PM, Benji York wrote: Philipp von Weitershausen wrote: Stephan Richter wrote: Because I disagree with that, since you cannot know the next version. You can always know the minimum version. If you just released 3.4.2, I think it's sensible to point the next release to 3.4.3. If you later decide that you really need a feature release, you can always bump to 3.5.0a1 (which would be the first release in the 3.5.x series). Gary and Benji made me pay attention to this point. :) I *really* don't like setting the version to the number for the next release, since one often doesn't know what it is. Why not leave the version totally out of the setup.py in the trunk? Benji and Gary won me over to this point of view. :) After branching for a release we can set the version (e.g., 1.2), make a release, and tag the branch. We should not require branches. I would only bother creating maintenance branches when they are needed. Z, B, G, and I propose the following: - Leave the version # out of setup.py on the trunk and on branches. When it is time to make a release, either from the trunk or from a maint branch: - Update changes.txt, adding a heading for the new # and date - Create a tag - check out or switch to the tag - Set the version in setup.py on the tag. Check it in. - Make the release from the tag. (BTW, when creating maint branches after the fact, the branch should be made by copying the trunk at the revision that the first release branch for that tag was made, so as not to accidentally get a version #.) I'd prefer not to make an edict, so Benji and Gary will now persuade you that this is the best way to go. ;) Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: reasonable syntax for multi-adaptation
On Sep 26, 2007, at 3:37 PM, Dieter Maurer wrote: Jim Fulton wrote at 2007-9-26 15:10 -0400: ... Jim Fulton wrote at 2007-9-26 11:29 -0400: ... IFoo(x) IBar(multi=(x,y)) Actually, that is not the case. If x already provides IFoo, then in the first case, the existing object is retuned. Nothing is instantiated. OTOH, in: getMultiAdapter([x], IFoo) or getAdapter(x, IFoo) either there is an error or some factory will be called. x won't be returned unless the factory happens to return it. Is this not an irrelevant implementation detail? No, the specified behavior is different. Hm. But "getAdapter" and "getMultiAdapter" may return "x" as well (when the factory decides to do this). Thus, why is it relevant? Because they don't take into account what x already provides. They will always call some factory. Also, they never call __conforms__. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 26, 2007, at 3:32 PM, Philipp von Weitershausen wrote: Jim Fulton wrote: On Sep 26, 2007, at 1:18 PM, Fred Drake wrote: On 9/26/07, Martijn Faassen <[EMAIL PROTECTED]> wrote: What does one need to tell setup.py to make sure CHANGES.txt is available? I understand it isn't by default, then? Hm, it does appear to be there by default. I checked grok 0.10's tgz and it's there, and we didn't do anything special. Do you mean available in the unpacked sdist, or as part of the installation? I don't think any of README, README.txt, CHANGES, CHANGES.txt (from the root of the distribution, not from inside the package) are actually installed. If they are, I'd love to know where. By default READM(.txt) is installed in a source distribution. Anything else in the root (aside from setup.py of course and source files themselves) aren't without extra setup chants. (These chants can be figured out with some effort. I never remember them, so, if I want to do something like this, I have to figure them out again or try to find an example with them.) I was wrong. Dang. I think I understand setuptools/distutils and I'm proven wrong again. Sheesh. Actually, no package data is ever included unless you're either * working from an svn checkout, in which case setuptools will use the list of which files are in svn and which aren't as a hint of what to include and what not Certainly, I expect CHANGES.txt to be in svn. I've just doeble checked that files from the root of the package that are checked into svn *are* included in sdists. I retract my objection to CHANGES.txt and offer my apologies. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: reasonable syntax for multi-adaptation
On Sep 26, 2007, at 3:00 PM, Dieter Maurer wrote: Jim Fulton wrote at 2007-9-26 11:29 -0400: ... IFoo(x) IBar(multi=(x,y)) Actually, that is not the case. If x already provides IFoo, then in the first case, the existing object is retuned. Nothing is instantiated. OTOH, in: getMultiAdapter([x], IFoo) or getAdapter(x, IFoo) either there is an error or some factory will be called. x won't be returned unless the factory happens to return it. Is this not an irrelevant implementation detail? No, the specified behavior is different. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 26, 2007, at 1:18 PM, Fred Drake wrote: On 9/26/07, Martijn Faassen <[EMAIL PROTECTED]> wrote: What does one need to tell setup.py to make sure CHANGES.txt is available? I understand it isn't by default, then? Hm, it does appear to be there by default. I checked grok 0.10's tgz and it's there, and we didn't do anything special. Do you mean available in the unpacked sdist, or as part of the installation? I don't think any of README, README.txt, CHANGES, CHANGES.txt (from the root of the distribution, not from inside the package) are actually installed. If they are, I'd love to know where. By default READM(.txt) is installed in a source distribution. Anything else in the root (aside from setup.py of course and source files themselves) aren't without extra setup chants. (These chants can be figured out with some effort. I never remember them, so, if I want to do something like this, I have to figure them out again or try to find an example with them.) If we are going to have a change log, which we should, I would prefer it to be included in source distributions. Including a file other that README in the root requires extra effort that I don't want to require -- writing setup.py files is hard enough as it is. I'm not crazy about Tres' idea of putting these files in the Python packages themselves, but it does have the advantage that it causes the files to be included in eggs as well as source distributions. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Automated egg releases
On Sep 26, 2007, at 11:19 AM, Martijn Faassen wrote: Stephan Richter wrote: [snip] Doing another checkout of the tag will create a significant overhead to the release process of a package. I'd like to highlight this. We need to be careful we don't increase release overhead too much, otherwise it won't happen/people will make mistakes. +10 Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: reasonable syntax for multi-adaptation
On Sep 26, 2007, at 11:08 AM, Brandon Craig Rhodes wrote: Jim Fulton <[EMAIL PROTECTED]> writes: On Sep 26, 2007, at 10:04 AM, Brandon Craig Rhodes wrote: Instead, multi-adaption should look like this: IFoo(multi=(obj1, obj2)) IFoo(multi=(obj1, obj2), name='site_foo') Ah, using keyword arguments to get around limitations (especially backward compatibility issues) with the current API is a neat idea. If we were going to do this though, I think a method syntax would be cleaner. As in: IFoo.adapt([ob1, ob2], 'site_foo', None) -1. Unfortunately the singular verb "adapt" makes it look like normal adaptation is what is being called for - it looks here like you are trying to adapt a list to the IFoo interface. Maybe ".multiadapt()"? Note that IFoo(ob) has some special semantics that don't apply to the multi- or named-adapter case. Agreed! The semantics are different. But mightn't we simply document this difference between single- and multi-adaptation everywhere we need to, rather than letting it force us into splitting the adapter syntax into two unwieldy pieces? I don't consider either unwieldy. ... So, I am not sure that I see yet the problem with "mixing APIs". The semantics of the call would change in fundamental ways based on the arguments passed. I think this is very bad. If you disagree, sorry. :) For me, the essential issue is that in both single- and multi- adaptation you are returned an instance of an adapter that has been instantiated with the objects you are adapting. Both of these syntaxes: IFoo(x) IBar(multi=(x,y)) Actually, that is not the case. If x already provides IFoo, then in the first case, the existing object is retuned. Nothing is instantiated. OTOH, in: getMultiAdapter([x], IFoo) or getAdapter(x, IFoo) either there is an error or some factory will be called. x won't be returned unless the factory happens to return it. ... An added complication is that interfaces don't provide adaption directly, but via a hook. The existing hook api wouldn't work for mult or named adaptation, so a new hook would be needed. I had assumed that IBar(multi=(obj1, obj2)) would simply dispatch to the existing getMultiAdapter() call; how, exactly, would hooks get involved and complicate things? Feel free to just cite a line number and tell me to go read, all I need is a pointer to get started understanding this. zope.interface does *not* depend on zope.component. It only does adaptation the way it does because it provides a hook that zope.component fills. Other people are using zope.interface with different adaptation schemes. Adding multi- or named- adaption support would require a similar hook. I don't really have time to continue this discussion. You made a reasonable proposal, but for various reasons I've tried to explain, I don't support it. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 26, 2007, at 10:42 AM, Stephan Richter wrote: On Wednesday 26 September 2007 10:40, Jim Fulton wrote: What about using CHANGES.txt, which we should be maintaining anyway? I agree with a change log. CHANGES.txt is difficult to get included in distributions. Having one requires a more complex setup.py. I'd rather recommend including a change log in README.txt. -1. I don't like that. We include CHANGES.txt via the long description; that's good enough for me. Do you think that the change log should be included in the distribution? Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access [update]
On Sep 26, 2007, at 10:10 AM, Martijn Faassen wrote: Stephan Richter wrote: On Wednesday 26 September 2007 05:02, Christian Theune wrote: Hmm. While doing that I also noticed that we were at 3.4.0a1 yesterday evening. The stable release was made from that without making a maintenance branch and bumping the trunk to 3.5. There is conflicting information here. :-) Some people say we need branches, others say we don't. I think it'd be good to have a tag in any case. A tag can always be converted into a branch later. Yup. I don't think there is disagreement about tags and we don't really need agreement on branches. And where is an agreed-upon document that you have to list the next version in the setup.py file after the release? Because I disagree with that, since you cannot know the next version. I disagree with too, for the same reason. What about using CHANGES.txt, which we should be maintaining anyway? I agree with a change log. CHANGES.txt is difficult to get included in distributions. Having one requires a more complex setup.py. I'd rather recommend including a change log in README.txt. Before making a release, change CHANGES.txt, and convert the section: unreleased -- Fixed bugs blah blah. To: 3.5.1 (2007-09-26) -- Fixed bugs blah blah. That is, come up with a release number, a release date, update setup.py with the release number, and tag it. Then, after the release, add in a new section: unreleased -- This way checking CHANGES.txt should tell you what's going on with releases. If someone forgot to do the last step, you see immediately something is wrong, as you want to add your change to the 'unreleased' section but there's nothing there yet. You can then reconstruct what happened from the cheeseshop or the tags, and put in a new 'unreleased' section. I agree except for the location of the change log. Jim P.S. ZODB's NEWS.txt/History.txt aren't workable for me and I plan to switch ZODB to using a change log in it's readme. Unfortunately, it's setup is so complex It will take me a largish chunk of time to unwind this. -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: faulty releases and pypi access
Well said. Jim On Sep 26, 2007, at 10:10 AM, Philipp von Weitershausen wrote: Stephan Richter wrote: On Wednesday 26 September 2007 09:36, Martijn Faassen wrote: I think tagging things in svn is a minimal requirement, though. I understood that some of this stuff wasn't tagged? I agree. And Roger simply forgot. As Marius pointed out, as we do more releases, problems like this will occur frequently. Making mistakes is human, so let's develop a tool that does some basic checking. I'm not yet convinced that we will have more releases. We have more releases now because we're trying to get to stable versions. After that, the eggs are on their own and I think we'll make much less releases. But even if we got more releases to do, I don't see the problem. With more releases we'd get more practice and make less mistakes. One of the things we should realize in this process is that -- even though setuptools makes it so damn easy to register and uplaod stuff to PyPI -- releases are actually important business. They can't just be created in a jiffy. They require attention and concentration. And a well-defined process. -- http://worldcookery.com -- Professional Zope documentation and training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: PLEASE don't remove eggs [was Re: [Zope3-dev] faulty releases and pypi access [update]]
On Sep 26, 2007, at 10:10 AM, Fred Drake wrote: On 9/26/07, Gary Poster <[EMAIL PROTECTED]> wrote: I should make our own private copies of the eggs we use, to completely isolate us from these sorts of things, but I have not gotten around to it...nor am I thrilled at the prospect of that overhead. This is also a technical issue: As long as zc.buildout and setuptools foolishly accept dependency links from an egg, it'll be painful to detect accidental reliance on external repositories. That's a good point. I wouldn't go so far as to say "foolishly", but I would say that this is a policy that should be overrideable. Checkins to buildout (with tests, of course) accepted. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] reasonable syntax for multi-adaptation
On Sep 26, 2007, at 10:04 AM, Brandon Craig Rhodes wrote: The current syntax for multi-adaptation makes the interface look like an object of the adaptation, rather than the actor in the operation. Instead, multi-adaption should look like this: IFoo(multi=(obj1, obj2)) or: IFoo(multi=(obj1, obj2), name='site_foo') Ah, using keyword arguments to get around limitations (especially backward compatibility issues) with the current API is a neat idea. If we were going to do this though, I think a method syntax would be cleaner. As in: IFoo.adapt([ob1, ob2], 'site_foo', None) Note that IFoo(ob) has some special semantics that don't apply to the multi- or named-adapter case. In particular: - The object is returned if it already provides the interface, and - The object's __conform__ method is used if it is present. Neither of these make sense in the multi- or named-adapter cases. Given the differences in semantics, I wouldn't want to mix the APIs. An added complication is that interfaces don't provide adaption directly, but via a hook. The existing hook api wouldn't work for mult or named adaptation, so a new hook would be needed. While I can see benefit from having an interface method for doing multi and named adaptation, I don't think the benefit is worth the cost. I'm -1 on your proposal and -0 on my variation. :) Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] faulty releases and pypi access [update]
This would be a good issue to bring up on the distutils-sig list. Jim On Sep 26, 2007, at 9:53 AM, Stephan Richter wrote: On Wednesday 26 September 2007 05:53, Stefan H. Holek wrote: WinZip has the habit of ignoring files it deems "empty" and paths it deems "too long". Best to avoid. The problem is that Roger has installed TAR and GZIP. They are also available in the path. However, for some reason it uses WinZip. Now, he has tested the eggs on his machine and they worked. Oh man, ... Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] faulty releases and pypi access [update]
Sorry, I decided not to reply and hit the wrong button in my mailer. :) On Sep 26, 2007, at 9:54 AM, Jim Fulton wrote: On Sep 26, 2007, at 9:51 AM, Stephan Richter wrote: On Wednesday 26 September 2007 05:02, Christian Theune wrote: Hmm. While doing that I also noticed that we were at 3.4.0a1 yesterday evening. The stable release was made from that without making a maintenance branch and bumping the trunk to 3.5. There is conflicting information here. :-) Some people say we need branches, others say we don't. And where is an agreed-upon document that you have to list the next version in the setup.py file after the release? Because I disagree with that, since you cannot know the next version. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] faulty releases and pypi access [update]
On Sep 26, 2007, at 9:51 AM, Stephan Richter wrote: On Wednesday 26 September 2007 05:02, Christian Theune wrote: Hmm. While doing that I also noticed that we were at 3.4.0a1 yesterday evening. The stable release was made from that without making a maintenance branch and bumping the trunk to 3.5. There is conflicting information here. :-) Some people say we need branches, others say we don't. And where is an agreed-upon document that you have to list the next version in the setup.py file after the release? Because I disagree with that, since you cannot know the next version. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Automated egg releases
I'm not too keen on trying to automate this with a Python script. I suggest we start with a human script. I think Philipp has a start at this. Philipp, could you remind us where this is? I suggest we review it and then post it prominately somewhere that people (I) can easily find it. I, for one, promise to read it every time I make a release until I've memorized it. Jim On Sep 26, 2007, at 9:13 AM, Marius Gedminas wrote: People make mistakes. Can we reduce the number/severity of those mistakes by creating a Python script to automate the release process as much as possible? Things like: - check that the version number in setup.py doesn't match an existing release in cheeseshop - check whether you have modified but not committed (or unversioned) files in the source tree - check that you've created a tag in subversion for this release (or, alternatively, offer to create the tag for you) - build an egg, install it locally in a temporary directory, then run the test suite to check for any missing files or whatever (a somewhat related idea would be to set up a buildbot to check for new zope-related egg releases and run their test suite in a sandbox, to notice breakages and email the guilty parties) - run the correct setup.py command to build/register/upload the egg to cheeseshop and/or www.zope.org I'd be happy to work on such a script during the sprint, if someone could help me figure out what exactly it should to do. Marius Gedminas -- If something has not yet gone wrong then it would ultimately have been beneficial for it to go wrong. ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] faulty releases and pypi access
On Sep 26, 2007, at 4:16 AM, Christian Theune wrote: This is IMHO a good example why we shouldn't go for 'everyone can make a release'. I had the same feeling yesterday. I kept silent because the alternative is that only a few -- including me -- make a release. :) This deserves more thought though. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] AW: Proposal, free views
On Sep 25, 2007, at 12:49 PM, Philipp von Weitershausen wrote: ... Note that I've seen checkins that add deprecation warnings. Even worse, they're talking about removing stuff in the future. I thought we had reached the consensus that we weren't going to remove stuff anymore. In other words, that we were never ever going to break backwards-compatibility again. I think we should just not raise any deprecation warnings at all. Just the imports for BBB and be done with it. We said we would only break backward compatibility under dire circumstances. :) I think that deprecation warnings are fine to let people know "This isn't the way you should be doing this. That other way is better." It helps people use packages according to the state of the art, which will continue to evolve and it will make developers feel better, which is worth something. :) Also, if we ever *do* need to make a backward-incompatible change to a package, we'll have some license to clean up rotten apis. Jim -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Known working sets II [was: Eggification redux]
On Sep 25, 2007, at 11:22 AM, Martijn Faassen wrote: Hey, On 9/25/07, Jim Fulton <[EMAIL PROTECTED]> wrote: [snip] I'm skeptical that you are going to get these changes in setuptools. I'm not crazy about them myself as they make writing setup files even more complicated. I'd much rather have a way for a comsumer to say: "Use version V of project P even though it doesn't satisfy a dependency." Basically, a way to override a dependency. This is something that buildout could be taught to do, although *I* don't have time to do it "today". That solves the problem too. Should we then encourage everyone to hardcode version numbers in their setup.py's dependencies list? If not, framework packages can maintain such lists. No, only in "application" (or very thick framework packages). This should certainly not be done for library packages. OTOH, buildout already provides an alternate solution to this, which isn't good enough because you want something that will work w/o buildout. Oh well. I am not sure what buildout mechanism you're referring to. A buildout configuration can specify a versions section that specified the versions to be used in the buildout. Other buildout configurations can extend the original configuration, specifying different versions as desired., A version list over HTTP? Configurations can be specified as URLs. Anything that urrlib2 understand will work. For example, you can include URLs in a buildout extends option. I also don't understand why you say I'm asking for it to work without buildout. You aren't? Cool. Well, other people have. When this issue first came up, Philipp worked out the buildout solution and others complained they wanted something that would work with workingenv, I'm not saying that this is an invalid desire. I'm just pointing out that there is a solution of you're willing to use buildout. The requirement is that not the application developers but the framework developers are easily able to maintain this list of dependencies. That is: * if someone puts the depencency to framework 1.7 in their dependencies in setup.py, it'll automatically get the pinned dependencies of this framework. Well, the buildout solution won't help that. * if someone tells us, we used framework 1.7, we *know* what dependencies this implied (unless they did a manual override). If reusing the framework includes reusing a buildout configuration that specifies the versions used by the framework, then buildout can help. So, if I write an application that uses framework 1.7, all I should have to do is set this in my dependencies list in setup.py. If I then decide to upgrade to 1.8, that's all I need to adjust. There is no long list of dependencies for me to maintain, as the framework developers do this for me. It'd be nice if I had the ability to override what the framework said, of course. setuptools doesn't let you do this. Maybe you want to lobby for a change on the distutils sig. Buildout could be enhanced to provide this feature, as I mentioned before. I know you referred us maintaining a buildout versions list on some HTTP site. This indeed solves most of the above. I don't think this ideal: * it doesn't work in a train. Of course, egg downloads don't either, but at least one tends to have a well-filled eggs directory already. * it requires the developers to upload a new list to some HTTP site each time they make a framework release. With eggs, it's nice that a few setup commands are all you need to do to make a release. So, I'd prefer to maintain this list in the package the list is talking about, and to only have a single release artifact (the package itself), instead of two (the package and the list). Of course, since it works now, we might want to do this for now anyway. I just don't think it's ideal. Agreed. Would it be possible for buildout to retrieve such a list from an egg if it's maintained as an extra, made-up setup.py variable? I just tried this, but it doesn't seem to be stored in egg-info. In theory, but not today. Your use case would be better served by adding a way to tell buildout to use a package version even if it violates some requirements, Finally, it would be nice to have the ability to maintain multiple lists of dependencies of significant sub-frameworks, and to have the ability to combine them in larger frameworks. That is, someone could be maintaining a Zope3-core package that depends on a whole bunch of Zope 3 packages, and Grok would then depend on this. I'd be nice if Grok wouldn't need to maintain its own list of pinned Zope3-core packages in this case but could rely on the release management and integrated testing procedure of the package it depends on. This is all "would be ni
Re: [Zope3-dev] Re: Known working sets II [was: Eggification redux]
On Sep 25, 2007, at 10:04 AM, Martijn Faassen wrote: Tres Seaver wrote: [snip] This is certainly an interesting approach. I'd be curious how you would garden this known working set. Martijn makes a pretty good case for maintaining such working sets close to the package in question (e.g. the grok egg, the Plone egg, etc.). I would argue that this problem is too big for "developer convenience" to drive it: we need concerted effort from the different "communities of interest" to manage the problem, in much the same way that Debian / Fedora etc. manage their various distribution releases. I want the ability to make releases of Grok so that when someone writes an application against it, it will continue to work, no matter what egg releases follow. That's a community of interest, I guess? I just want the technical ability to do so, and easily. If it's *not* convenient for developers to maintain it, it won't get maintained well by the various communities of interest. I think it's important to maintain these lists of dependencies close to what they talk about, preferably inside the packages themselves. As far as I know what is required is the ability for grok, in its setup.py, to include a list of suggested pinned dependencies (besides, and separate from, the normal dependencies). It should also be easy to configure buildout to inspect this list. What is also required is a way to easily create and maintain this list. Now I think you're talking about ways to maintain, report and test such lists below, but I want to solve my immediate problems first. I also need this solved preferably today. It's the primary hurt of Grok today. Everything else is peanuts. Of course, if we don't need flexibility to allow application developers to diverge from Grok's recommendation, this problem is solved today, except for the bit to actually generate the list of dependencies. We can simply hardcode them as dependencies in Grok's setup.py and tell everybody who wants to use newer versions of eggs for whatever reasons in their applications "too bad, wait for the new release". I'm skeptical that you are going to get these changes in setuptools. I'm not crazy about them myself as they make writing setup files even more complicated. I'd much rather have a way for a comsumer to say: "Use version V of project P even though it doesn't satisfy a dependency." Basically, a way to override a dependency. This is something that buildout could be taught to do, although *I* don't have time to do it "today". OTOH, buildout already provides an alternate solution to this, which isn't good enough because you want something that will work w/o buildout. Oh well. JIm -- Jim Fulton Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Transaction bound cache
No, that would introduce a dependency on zodb. I suggest a separate package. Jim On Sep 17, 2007, at 6:03 AM, Christian Zagrodnick wrote: Hi i've got a very simple transaction bound cache implementation. That is the cache gets invalidated on transaction end. It's used like this: class Foo(object): data = TransactionBoundCache('_v_store_it_here', dict) where `dict` is the cache factory. Shall I add this to zope.cachedescriptors? -- Christian Zagrodnick gocept gmbh & co. kg · forsterstrasse 29 · 06112 halle/saale www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891 ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.testbrowser packaging
extras are a terrible feature. They aren't fully supported by setuptools and they make it more complicated. Did you write tests for every permutation of your extras? Jim On Sep 15, 2007, at 9:44 PM, Stephan Richter wrote: On Saturday 15 September 2007 08:48, Benji York wrote: 1) introduce a "zope" extra that everyone will have to use (basically just rename "test" to "zope"; I prefer this solution. I have done this before for z3c.rml, where I put the page template support into a "pagetemplate" extra declaration. I liked this solution a lot. I would have never considered developing another package for a fairly trivial extension of a few lines. I think eggs have the potential for package proliferation and senseless overhead. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: zope.testbrowser packaging
+1 Also, extras is a miss-feature. Jim On Sep 15, 2007, at 10:43 AM, Philipp von Weitershausen wrote: Benji York wrote: I have a small issue with zope.testbrowser packaging I'd like to get some input on. If I were to have started the project today, it would likely have been zc.testbrowser, which would have no Zope 3 dependencies (or functionality) and zc.testbrowser.zope, which would have, and depended on zc.testbrowser. Well, that didn't happen, but there are parallels to the current situation that might be informative. There is a configuration bug in testbrowser that means that unless you include the "test" extra, you won't get the Zope 3 dependencies. I suspect most people either include that extra, or accidentally include the dependencies through other packages. I have two ideas for fixing this: 1) introduce a "zope" extra that everyone will have to use (basically just rename "test" to "zope"; 2) take a lesson from the fictional zc.testbrowser and introduce another package (zope.testbrowser.zope) that contains the Zope 3 bits and depends on zope.testbrowser. I think this would be very hard if not impossible to do from a packaging perspective (declaring zope.testbrowser a namespace package *and* have it contain things like README, configure.zcml, etc.). I think I prefer the second, despite it's strange appearance. Thoughts? Let's look at this from the beginning. zope.testbrowser contains a) a reusable, completely Zope-independent test browser b) integration with zope.app.testing.functional, in other words a test browser for testing web applications based on zope.publisher. I think in its current use, zope.testbrowser is *mostly* used as b). When used as a), I don't think anybody is bothered by the fact that it might or might not have more dependencies (other than the inconvenience of having to install more stuff than actually necessary). So here's what I suggest: Factor out a) to a new package 'zc.testbrowser' (or whatever) and make 'zope.testbrowser', the remaining b), depend on zc.testbrowser, zope.app.testing and all that other stuff properly. That way - packaging and nomenclature are straight-forward, - we don't have to break backwards compatibility anywhere, - people who have used 'zope.testbrowser' because of a) until now won't experience any problems, even though we should probably tell them to switch to zc.testbrowser. -- http://worldcookery.com -- Professional Zope documentation and training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: What does python 3000 mean for zope?
On Sep 11, 2007, at 10:28 AM, David Pratt wrote: I was hoping Jim might respond to this thread since I am certain there is concern about what this means for the future of Zope. I am hoping that core communities of python framework developers may come together on what is in their best interests. OK, here's my opinion. I am not speaking for ZC. I don't think we should worry until Python 3 is much farther along. I agree with those who think that it will be hard to move to Python 3. Put another way, I think the benefit won't be worth the effort. My suspicion is that this will be true for *lots* of projects. I expect Python 2 to be with us for a long time. This is just a wild guess. A lot can change in a year or 2. I think our official position should be that Zope is a Python 2 application and we'll evaluate opportunities to leverage Python 3 over time as they present themselves. I'm not going to reply to any replies to this post as I don't intend to spend any more time on this issue myself. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] New Contributor References
Lately, I've started being a bit more careful about accepting new contributors. In particular, I'm requesting that new contributors provide the name of an existing contributor who is familiar with them and their work. I (or someone) really need to add this to the contributor agreement, but I'm not sure when I'll have time. Now, It's taking a fair bit of time writing new contributors to ask for references. I encourage folks to request committer access, I just ask that people provide the names of one or more existing committers who can vouch for them. Thanks. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: RFC: Known working sets
On Sep 6, 2007, at 4:02 AM, Chris Withers wrote: Martin Aspeli wrote: Jim Fulton wrote: I'm very much against making setuptools *more* complicated than it already is. Indeed, but surely managing "known good" sets of components comes under its remit of version management? Sure. It does this via requirements. Ok, forgive me for being dumb then, but why are we looking to add similar to zc.buildout? We're talking more about a general pattern in zc.buildout. The deficiencies of using setup.py for this alone are described well in the original proposal. Yup, and this was the reason for my original question to Jim: why do something in zc.buildout rather than fixing the problems with setuptools? Because neither the problem nor the fix are well understood, imo, and setuptools is already too complicated. Perhaps the same could be said about buildout, but no new buildout features are needed to experiment with the issue at this point. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: RFC: Known working sets
On Sep 5, 2007, at 10:48 AM, Chris Withers wrote: Jim Fulton wrote: I'm very much against making setuptools *more* complicated than it already is. Indeed, but surely managing "known good" sets of components comes under its remit of version management? Sure. It does this via requirements. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: RFC: Known working sets
On Sep 3, 2007, at 4:13 PM, Philipp von Weitershausen wrote: Wichert Akkerman wrote: The only problem is that distributing grok-0.11.cfg is a bit tedious. How about if buildout could get it from the web? How about making it available from an egg, through a hook in egg-info perhaps? This is a very good point. So good in fact that I thought of it myself :) I was already writing the email when your message came in. Martijn and I discussed the known working set problem over IRC this afternoon. He brought up a few good points which suggest that the version data could be associated with the egg: The approach that I described in my original posting (and which actually works *today*, as I found out!) has some disadvantages. For instance, the discoverability and release mechanism of the known working set configuration file differs a lot from our normal packages. Releasing a package is a well-known routine by now. How and where would we release the working set configuration file? SVN? Another problem are dependencies and how we'd like to depend on other working sets. Let's say we made a 'Zope' working set that contained the stable versions of the zope.* packages. It would make sense for grok to depend on this information. And packages using grok should depend on that. It'll be complicated to maintain such a chain in separate text files, especially across version updates. It only seems natural to use the egg dependency mechanism for this. So, a possible solution is to associate the known working version info with an egg. More to the point, an egg could -- in addition to simply listing the names of its dependencies -- also specify which versions of those eggs it works best with. Wichert and I suggest that this could be put in a file in the EGG-INFO directory. The only problem is that we usually don't version control egg-info directories. That means the information needs to be contained or at least referenced in setup.py and then written upon "setup.py egg_info". Here's what setup.py *could* look like:: known_working_versions = { 'ZODB3': '3.8.0', 'zope.component': 3.4.0, ... } setup( name='grok', version='0.11', ... install_requires=known_working_versions.keys(), known_working_versions=known_working_versions ) When EGG-INFO is written, a custom writer will then take this information and generate 'known_working_versions.txt' or whatever in EGG-INFO. The only problem is that this custom writer needs to be installed when setup.py is called to create egg, in other words to generate the right kind of eggs you'd need to have something installed first. Not sure if this is better than maintaining .egg- info directories in SVN... How would the "known_working_versions" be used? You haven't specified that. Presumably this would be something that is overridable. How would this be overridden? I'm very much against making setuptools *more* complicated than it already is. Perhaps buildout (and setuptools) should grow a mechanism for being able to override/resolve version conflicts. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: broken zope.lifecycleevent 3.4.0 on cheeseshop?
On Sep 2, 2007, at 8:20 AM, Benji York wrote: Baiju M wrote: BTW, do we need to upload eggs to http://download.zope.org/ distribution/ any more ? If we feel that depending on PyPI to install zope packages is OK, then we can decommission /distribution/. For the commercial projects I'm involved in we make sure we have private copies of everything we depend on, so I don't have a desire to keep / distribution/ around. I like distribution as a place to put packages that aren't ready for PyPI. Unfortunately, we've put a lot of packages in PyPI that aren't ready IMO. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] ZConfig 2.4 final?
I suggest making a 2.5 (final) release and being done with it. My alpha releases were undoubtedly in ignorance of the existing tag. Jim On Aug 30, 2007, at 11:24 AM, Philipp von Weitershausen wrote: ZConfig 2.4 currently seems to be in alpha mode. Are we going to make a ZConfig 2.4 final for the Zope 3.4 release? And what about the "2.4" tag that's in subversion? The name suggests that it's a 2.4 final release, but it can't be because Jim created that tag 11 months ago (before any of the 2.4 alphas were created) with the comment "Make tag for Zope 3.3 release". See for yourselves at http://svn.zope.org/ZConfig/tags/. What to do? -- http://worldcookery.com -- Professional Zope documentation and training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Please don't use svn.zope.org urls in distutils homepage urls
On Aug 25, 2007, at 4:59 AM, Lorenzo Gil Sanchez wrote: Thanks for replying. I meant to make another reply but fogot. Well, probably I was to fast predicting the causes of my problem. They only thing I know is that a couple of months ago I could browse the repository without any problem and suddenly I start getting a default Apache page and a lot of permission denied errors while trying to do the same. I'm sure I was browsing the same URL (http://svn.zope.org) since I had it on my bookmarks. Oddly enough, after this message thread I can browse the repository again from the device and I swear I didn't change any proxy configuration or anything at all. Your client id was being blocked because it was used by a poorly- behaved spider. It was blocked to block the spider. We'ev unblocked it and are hoping that the spider doesn't come back. If it does, we'll find some other way to block it. In any case, it had nothing to do with blocking setuptools. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] RFC: Guide for maintaining software in the Zope repository
On Aug 23, 2007, at 8:37 PM, Philipp von Weitershausen wrote: We have 100+ packages that make up what used to be distributed as "Zope3". We have numerous more packages in svn.zope.org. Most of them are developed, released and distributed individually. We like to think this is a good thing (I certainly do). But currently we have a bit of a chaos [2]. It's not bad, but I fear without some guidance, it'll get worse. Christian Theune recently wrote a document [1] in which he outlined how we should get to a development process and what topics it should touch. This document is very hands-on and describes actions that should be taken to reach these goals. I've taken the liberty to jump ahead and write down some current practices: http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/ maintaining-software.txt This is a great start. Thanks! A few small points: - I'm going to mostly stay out of the style debate except to note that the Zope style guide builds on PEP8. It doesn't disagree with it much accept in the case of some naming, due to the fact that the ZSG made a commitment before PEP8 did. - On doctest, there should be greater emphasis on there being 2 kinds of tests, executable documentation and other tests. I think there is value in executable documentation, but it should be documentation first. A lot of our doctests that we think/wish are documentation are not very good documentation. We need to do a better job of this. I do feel strongly that even non-documentation tests should be written in a fairly literate way with documentation of the test itself, I strongly prefer the doctest format for these tests, but I don't want to be an evil dictator about it. I suggest that classic unit tests can be used for new tests, but *only* if they are well documented. I've never seen a classic unit test that was, but I'm open to the theoretical possibility. :) BTW, I've seen poorly documented doctests too. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Please don't use svn.zope.org urls in distutils homepage urls
On Aug 24, 2007, at 5:07 AM, Lorenzo Gil Sanchez wrote: Aha! After reading this message now I finally understand the reason I can't browse svn.zope.org with my Nokia N800[1] anymore. I don't see what one has to do with the other. This used to be quite useful since I could read lots of *.txt documents from zope3 and related projects just before sleeping. I wish I could do it again. Nothing we've done should get in the way of your Nokia N800. I've setup an Apache server just to see what user-agent my device is giving and this is what came out from the log: 192.168.1.15 - - [24/Aug/2007:11:00:32 +0200] "GET / HTTP/1.1" 403 3926 "-" "Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux armv6l; U) Opera 8.5 [es_ES] Tablet browser 0.0.14 RX-34_2007SE_4.2007.26-8" So please, adjust the Apache configuation so N800 users can browse the repository again. We are only excluding setuptools. Not any other clients. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] __repr__ of ValidationErrors in zope.schema
On Aug 24, 2007, at 4:00 AM, Christian Zagrodnick wrote: Hi I'm implementing the getValidationErrors thingy right now and once again stumbled upon the ValidationErrors. Their __repr__ is all but useful. For instance "TooSmall": TooSmall(8, 10) 8 10 Another sort of related issue is that you only get the __doc__ string when calling the .doc() method. "Value is too small." doesn't help a lot. Something like "The value 8 is too small. At least 10 is required." would be much more informative. What should we do about this? While better __str__s or __reprs__ is often nice, as a general principal, errors should be displayed through adaptation -- basically views. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Leak in zope.component?
On Aug 23, 2007, at 1:53 AM, Christian Zagrodnick wrote: On 2007-08-22 16:12:42 +0200, Jim Fulton <[EMAIL PROTECTED]> said: On Aug 22, 2007, at 8:16 AM, Christian Zagrodnick wrote: Hi, I'm doing things wich z3c.zalchemy and trying to find out why my database connections are kept open even after unregistering everything. Given the following very simple test: import sys import zope.component import zope.interface class IUtil(zope.interface.Interface): ... pass gsm = zope.component.getGlobalSiteManager() utility = object() sys.getrefcount(utility) 2 gsm.registerUtility(utility, IUtil) sys.getrefcount(utility) 6 gsm.unregisterUtility(utility, IUtil) True sys.getrefcount(utility) # this fails 2 -- File "/Users/zagy/development/z3c.zalchemy/src/z3c/zalchemy/ tests/ dispose.txt", line 17, in dispose.txt Failed example: sys.getrefcount(utility) # this fails Expected: 2 Got: 4 So there are now two more references than before register/ unregister. Am I missing something? Or is it leaking somewhere? Here's what I get with the trunk of zope.component: [EMAIL PROTECTED]:~/p/zope/component/trunk$ bin/py >>> import sys, zope.component, zope.interface >>> class IUtil(zope.interface.Interface): pass ... >>> gsm = zope.component.getGlobalSiteManager() >>> utility = object(); sys.getrefcount(utility) 2 >>> gsm.registerUtility(utility, IUtil); sys.getrefcount(utility) 5 >>> gsm.unregisterUtility(utility, IUtil); sys.getrefcount(utility) True 2 So I can't reproduce what you are seeing. Try adding: >>> import gc; gc.collect() before your last getrefcount call. That doesn't change a thing. Could it be the testrunner having an eye on registrations? I don't know what you mean. I ran my example interactively. What happens when you run my example interactively? Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: Backward compatibility and major releases (series) update
On Aug 22, 2007, at 4:15 PM, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jim Fulton wrote: A while ago, we had a discussion about backward compatibility and decided that only major releases should be backward compatible. So, for example, a 1.2 release should be backward compatible with a 1.1 release, but a 2 release could be backward incompatible with a 1 release. We then said we wanted a nice way to spell depending on a major release. (The current way to spell depending on a major release is "foo >=R.dev +1. Cleanliness is not a good enough reason to break a public API, for instance. If necessary, the incompatible stuff might be better off moving to a new package / API name altogether, with the old name left as a pure compatibility shim (perhaps with "evergreen" deprecation warnings). Yup, which is what Gary is doing with zc.relationship. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Backward compatibility and major releases (series) update
A while ago, we had a discussion about backward compatibility and decided that only major releases should be backward compatible. So, for example, a 1.2 release should be backward compatible with a 1.1 release, but a 2 release could be backward incompatible with a 1 release. We then said we wanted a nice way to spell depending on a major release. (The current way to spell depending on a major release is "foo >=R.dev is the major release number.) We recently had an opportunity to experience this with zc.relationship. zc.relationship 2 was released and some packages were updated to require zc.relationship 1. Unfortunately, not all of the packages we use were updated and this caused version conflict errors. (This was partly a result of setuptools weak conflict resolution algorithm.) My initial response was, "oh, we need to update all of the other packages that depend on zc.relationship to require version 1." But then I started wondering how we would migrate to version 2 and realized that it was going to be rather hard. All of the dependent packages would have to move in lock step and we'd be back to a monolith. This was enough to make me think that backward incompatible changes are just untenable. I gave a hint to this in some later email threads. Since then, I've looked at a number of packages that we've split out from Zope that have excessive dependencies. zope.component is a great example. The excessive dependencies (at least the hard ones to deal with) are a result of poor factoring of functionality at a time when dependencies didn't matter. Unfortunately, I think the only way to fix some of these is to split off functionality, which will introduce backward incompatibility. I eventually came to the conclusion that our original conclusion was sound, but that we should only introduce backward incompatibilities when the need is very dire, as it will cause lots of pain. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: We should be able to stop unhiding PyPI releases.
On Aug 22, 2007, at 10:50 AM, Tres Seaver wrote: ... I would argue that hiding old releases is a major misfeature of PyPI, personally: humans probably have as much to gain from seeing the older versions as scripts. Agreed, although the UI degrades quite a bit when there are multiple unhidden releases. PyPI could use quite a bit of improvement. The place to get involved is the catalog sig. :) Even "brown bag" releases should remain available (albeit, with perhaps some way to tag them as problematic). Hidden releases are still available. I certainly think there is value in hiding many old releases, especially non-final ones, as long as the releases are still available for download. Releasing a software package is a contract which implies more than "flavor of the week" status, at least in lots of circles. It is important that releases remain available and finable by setuptools. The simple index achieves that. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] We should be able to stop unhiding PyPI releases.
As of Monday's zc.buildout release, buildout now uses the simple package index: http://cheeseshop.python.org/simple by default. (You can cause it to use another index by setting the buildout index option.) This change should provide significantly better performance and will make it easier to use historical releases, because the simple interfaces shows releases that have been hidden from the human interface. Normally, when new releases are registered with PyPI, old releases are hidden. This has been a problem for buildouts or application packages that fix package versions, because hidden releases weren't visible to setuptools. To get around this, we had to unhide old releases, which was a pain. The simple package index shows all releases, including hidden ones. Now that buildout uses the simple index by default, hidden packages should no longer be a problem, at least for buildout users. People using easy_install can configure it to use the simple index. They might want to lobby to have easy_install use the simple index by default. If there are no strong objections, I plan to stop unhiding old releases. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Leak in zope.component?
On Aug 22, 2007, at 8:16 AM, Christian Zagrodnick wrote: Hi, I'm doing things wich z3c.zalchemy and trying to find out why my database connections are kept open even after unregistering everything. Given the following very simple test: import sys import zope.component import zope.interface class IUtil(zope.interface.Interface): ... pass gsm = zope.component.getGlobalSiteManager() utility = object() sys.getrefcount(utility) 2 gsm.registerUtility(utility, IUtil) sys.getrefcount(utility) 6 gsm.unregisterUtility(utility, IUtil) True sys.getrefcount(utility) # this fails 2 -- File "/Users/zagy/development/z3c.zalchemy/src/z3c/zalchemy/tests/ dispose.txt", line 17, in dispose.txt Failed example: sys.getrefcount(utility) # this fails Expected: 2 Got: 4 So there are now two more references than before register/ unregister. Am I missing something? Or is it leaking somewhere? Here's what I get with the trunk of zope.component: [EMAIL PROTECTED]:~/p/zope/component/trunk$ bin/py >>> import sys, zope.component, zope.interface >>> class IUtil(zope.interface.Interface): pass ... >>> gsm = zope.component.getGlobalSiteManager() >>> utility = object(); sys.getrefcount(utility) 2 >>> gsm.registerUtility(utility, IUtil); sys.getrefcount(utility) 5 >>> gsm.unregisterUtility(utility, IUtil); sys.getrefcount(utility) True 2 So I can't reproduce what you are seeing. Try adding: >>> import gc; gc.collect() before your last getrefcount call. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: SVN: zope.location/trunk/setup.py using pypi as homepage instead of svn.zope.org
On Aug 20, 2007, at 8:22 PM, Stephan Richter wrote: On Saturday 18 August 2007 17:03, Philipp von Weitershausen wrote: * Please also don't forget to add a changelog entry in README.txt, especially if you're adding features. If there's no README.txt yet, this is a good time to give it one. It should start out with a simple paragraph and have at least one section called "Changes". You could then use its contents as the long_description in setup.py. Other packages (e.g. zope.proxy or zope.publisher) can serve as examples. Why do the changes have to be part of the README file? That seems no good. I think a separate file is much better. If you use a separate file, and you are going to create a source distribution, then you need to have extra magic in your setup.py to include CHANGES.txt in the distribution. IMO, setup magic is bad. I'd prefer to see simpler setup.py files and thus I prefer to include the change log in README.txt. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: SVN: zope.location/trunk/setup.py using pypi as homepage instead of svn.zope.org
On Aug 21, 2007, at 2:34 AM, Wichert Akkerman wrote: ... If you really want to you can combine both files in the long_description from your setup.py. That can make your cheesehop package page overly large though. How so? I think as long as a the long description has a table of contents at the top, I don't think that size is an issue. Note that with the latest version of buildout, buildout uses the new simple index, http://cheeseshop.python.org/simple by default, so there is no longer a buildout performance penalty for having large cheeseshop pages. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] browser:page's for="" can be class browser:menuItem can't
On Aug 20, 2007, at 7:57 PM, Stephan Richter wrote: On Monday 20 August 2007 09:45, Christian Zagrodnick wrote: Should we fix that? Yes, or just deprecate the menu stuff. ;-) The default menu framework was insufficient in every real-world project I have worked on. Menu items based on the context just do not work. They don't? I have certainly found them useful in the ZMI. As bad as the ZMI is for many apps, I find it very useful as a default UI and for groveling around in site component registries. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: Heads up: major change in zc.buildout policy for selecting distributions
On Aug 20, 2007, at 1:18 PM, Jim Fulton wrote: I'm about to make a new release of zc.buildout that uses a different policy for selecting distributions. In particular, by default, zc.buildout will now prefer final distributions over non- final ones. If there are final and non-final distributions that satisfy a requirement, buildout will, by default, select a final distribution even if there are non-final distributions that satisfy a requirement. This release will cause many existing buildouts to use older distributions than they do now. You can change this behavior by providing a prefer-final buildout option: [buildout] prefer-final = false I decided, with some prompting from Martin Aspeli, that this would be too disruptive. The latest release of zc.buildout still supports newest releases. To prefer final releases, you have to set the prefer-final option to true. In zc.buildout version 2, final releases will be prefered by default. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com