[Zope3-dev] Let's move over to zope-dev

2007-10-11 Thread Jim Fulton


I think we have agreement (or an absense of anyone who is willing to  
publicly admit to disagree) to retire this list and move over to zope- 
dev.  Let's do so.  I'll wait a little while before trying to  
actively prevent posts here.  I suggest we let any existing threads  
run out and start new threads on zope-dev.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] eggsplosion and tests

2007-10-10 Thread Jim Fulton


On Oct 10, 2007, at 9:45 AM, Martijn Faassen wrote:


Hi there,

I just tried something Stephan also tried, but deserves another  
topic. I just wrote a simple script that takes all the eggs grok  
uses and tries to run their tests. This is to see whether our pile  
of eggs isn't broken in some way.


I get a ton of errors. Mostly a ton of import errors, but other  
things as well, probably mostly due to version skew where one  
release assumes it can do things with APIs in another release that  
another release doesn't allow yet or not anymore.


And then the whole testrunner bails out somewhere in the middle:

Traceback (most recent call last):
  File /home/faassen/buildout/grok-0.10/testbuildout/bin/test,  
line 115, in ?
zope.testing.testrunner.run((['--tests-pattern', '^f?tests$', '- 
v']) + [
  File /home/faassen/buildout-eggs/tmpRHS1Xv/zope.testing-3.4- 
py2.4.egg/zope/testing/testrunner.py, line 271, in run
  File /home/faassen/buildout-eggs/tmpRHS1Xv/zope.testing-3.4- 
py2.4.egg/zope/testing/testrunner.py, line 448, in run_with_options
  File /home/faassen/buildout-eggs/tmpRHS1Xv/zope.testing-3.4- 
py2.4.egg/zope/testing/testrunner.py, line 661, in resume_tests

zope.testing.testrunner.SubprocessError

We'll have a lot of work cut out for us to fix this.

This is one point where I can see an evolving index will help. The  
trouble is we ended up starting with such a messed up state in the  
first place. :(


2 minor notes:

- zc.buildout and some recipes tests don't run properly outside of  
their buildouts.  This is because the test setup leverages aspects  
iof the buildout, especially the presense of the source to build  
develop eggs needed for the tests.  This almost certainly can and  
should be fixed, but I consider it a low priority in relation to  
other priorities and limited resources.


- ZODB packaging, which is waay too complicated for hysterical  
reasons, fails to include some files needed for testing.  This makes  
it impossible to run the tests outside of it;s development  
environment.  This too can and should be fixed.  I consider this a  
higher priority but won't have time to deal with it for a while.  I  
think ZODB's setup should be rewritten from scratch. Much of the  
complexity arises from working around limitations in distutils, some  
of which have been fixed. Other complexity probably arises from a  
stubborn desire not to require setuptools for ZODB.  Anyway, if  
someone wants to ehlp with the setup script, it would be appreciated,  
Otherwise, I'll get to it eventually.  In the mean time, I suggest  
not running the ZODB tests.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Zope 3 releases?

2007-10-08 Thread Jim Fulton


I'm not sure that library or collection of libraries is the right  
term for what we want to be.  I think we've been using it because it  
stands in sharp contrast to application, which, BTW, isn't exactly  
what Zope 2 is.  I think these terms were useful to make some points,  
but neither is accurate. FWIW, I have a fairly open mind on this  
topic. Lots of good points are being made. :)


Jim

On Oct 7, 2007, at 5:13 PM, Martijn Faassen wrote:


Jim Fulton wrote:

On Oct 7, 2007, at 6:25 AM, Lennart Regebro wrote:

[snip]
- We need a *realistic* (especially wrt available resources)  
process

for managing releases.  There are 2 aspects of this.  We shouldn't
make plans for which there aren't enough resources.  We also
shouldn't plan significant tasks that people won't care enough to
work on.  I think the classic Zope 3.4 release is a good example  
of a

large effort that really wouldn't benefit many people, if any.


Do you have a sort explanation on what is the missing resource?  
Is it,

as it was for 3.3, lack of people-hours with knowledge in fixing the
last bugs?
I'm not entirely sure.  I just observe that this doesn't seem to  
be making much progress.


I think it's one of the drawbacks of taking an ecosystem/libraries  
approach instead of a application/framework style approach. An  
application or framework typically is an integrated whole that has  
a single version number. An ecosystem or set of libraries can be  
integrated (which Zope 3 is) but everything evolves at different  
rates and there's no single thing to install or talk about.


I'm not saying an ecosystem approach is bad, if that's what Zope 3  
wants to be. I do think that such an approach needs to be  
supplemented by a framework approach (and I've been putting work  
into one way to do that).


If Zope 3 is an ecosystem, a release of Zope 3 the ecosystem  
doesn't really make much sense. To follow the comparison with Linux  
distribution, it's more like a distribution of an ecosystem. I'd  
therefore suggest that the release of Zope 3.4, if it ever actually  
happens, will be the last release of Zope 3 the application server  
framework.


I hope that besides Grok, some community will stand up that takes a  
less radical approach to building an application server on top of  
the Zope 3 ecosystem. People having existing applications in Zope 3  
to maintain (like myself with the Document Library) will have a  
need for it, and Grok doesn't suit everyone's tastes anyway,  
especially people comfortable with existing Zope 3 practices. As I  
said elsewhere, I suggest we call this project not Zope 3 but  
something else, to avoid confusion with the Zope ecosystem (and  
also to avoid implying it's the clear successor to Zope 2. I think  
we can safely say by now that's not how history went).


Regards,

Martijn

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Known-good-sets problem

2007-10-07 Thread Jim Fulton


On Oct 6, 2007, at 6:07 PM, Martijn Faassen wrote:


Jim Fulton wrote:

On Oct 6, 2007, at 5:24 AM, Martijn Faassen wrote:

[snip]
What I really don't like about this proposal is that it talks  
about updating index pages. If I understand this right, an  
updated index page will force everybody that uses this index page  
into an update.

Only people who ask for updates.


Yes, but it's important in releases. If I release my application to  
someone else, they will be asking for an update while I might never  
had. I cannot test this as the updated versions might've been  
released *after* I make my application release.


Of course, each update to an index is, in essense a new release of  
the index.


...


I don't think this is acceptable.
I think this depends on the use cases.  I think most people would  
like to get bug fixes.  The intent of such a 3.4 index would be to  
give people the latest 3.4 updates which should give no new  
features.  This is intended to mimick what happens with linux  
package respositories.


Yes, bugfixes is less of a problem than feature releases, if this  
is well managed. Since packages evolve at different rates, what is  
done for feature releases of packages? Do they end up in the same  
index or separate indexes? How to determine when to make a new index?


Those are all good questions that need to be answered.  Of the top of  
my head, :)


- I imagine there would be an index for each feature release that  
gets bug -fix updates.


- There would be an index for the next feature release that includes  
stable packages with new features.  There would be some requirements  
for packages to be deamed ready for the next release.



Let me make the case for bug-for-bug compatibility:


I assume by this you mean setting fixed versions.

The problems during release as I sketched out above still exist.  
Granted for bugfix releases the chances are lower that something  
breaks than it is now, but we all know people sometimes disagree  
about what actually constitutes a bugfix (or larger change), and  
people also sometimes have workarounds or code assumptions that  
depend on bugs. Bugfixes can break working code.


I'm not suggesting that setting fixed versions is a bad idea.  I  
think some projects may choose to go this way.  This is a valid  
policy decision.  Other developers would, IMO, benefit from having a  
relatively stable index as a source of new packages.  I don't think  
this is an either-or decision.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Zope 3 releases?

2007-10-07 Thread Jim Fulton
IMO, the Python standard library and batteries included is a mess.   
Despite being a Python contributor, I've very unmotivated to be a  
contributor because the time lag between contributing and deriving  
benefit from the contributions is too long.  People had similar  
complaints about Zope releases.  I'll note that the problem is  
exacerbated by the incompatibilities that (to some degree inevitably)  
often occur in Python releases.  Big-bang releases don't prevent  
update problems.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Zope 3 releases?

2007-10-07 Thread Jim Fulton


On Oct 7, 2007, at 6:25 AM, Lennart Regebro wrote:


On 10/6/07, Jim Fulton [EMAIL PROTECTED] wrote:

- We need to decide what a Zope 3 release is (or maybe multiple
flavors).  I favor copying the linux experiences, but have an open  
mind.


I'm not sure what you mean with that,


I mean we need to decide what a release is.



but for me, a known good set, or
what you call it, plus a buildout that uses this, would be enough for
a Zope 3.4 release.
If we easily can build the old standard downloadable tgz as well, then
fine, but it's not necessary.


That is my opinion too.


- We need a *realistic* (especially wrt available resources) process
for managing releases.  There are 2 aspects of this.  We shouldn't
make plans for which there aren't enough resources.  We also
shouldn't plan significant tasks that people won't care enough to
work on.  I think the classic Zope 3.4 release is a good example of a
large effort that really wouldn't benefit many people, if any.


Do you have a sort explanation on what is the missing resource? Is it,
as it was for 3.3, lack of people-hours with knowledge in fixing the
last bugs?


I'm not entirely sure.  I just observe that this doesn't seem to be  
making much progress.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Known-good-sets problem

2007-10-07 Thread Jim Fulton


On Oct 7, 2007, at 7:33 AM, Lennart Regebro wrote:


On 10/6/07, Jim Fulton [EMAIL PROTECTED] wrote:

Only people who ask for updates.


Which by default is every time somebody runs a buildout, right?


Yes


I think this depends on the use cases.  I think most people would
like to get bug fixes.  The intent of such a 3.4 index would be to
give people the latest 3.4 updates which should give no new
features.  This is intended to mimick what happens with linux package
respositories.


Ah, good point. But I think what Martijn is saying is that when we
come with new *features* this should be made into a new page. Which
would then be Zope 3.5, or something.  Or did I misunderstand him?


I don't know if you did or not. I'll just note that different indexes  
might have different update policies.  A Zope 3.4 index should only  
get bug fixes.



But you are right, small bugfixes, and in particular security fixes,
should indeed go into the already existing page. Which means all you
would need to get a security fix would be to run buildout, possibly
forcing it to check for new versions if that is set to false by
default. That would indeed be a very nice feature.


I think so.

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Known-good-sets problem

2007-10-06 Thread Jim Fulton


On Oct 6, 2007, at 5:24 AM, Martijn Faassen wrote:


Hey,

I'm glad some steps are taken!

What I really don't like about this proposal is that it talks about  
updating index pages. If I understand this right, an updated index  
page will force everybody that uses this index page into an update.


Only people who ask for updates.


I don't think this is acceptable.


I think this depends on the use cases.  I think most people would  
like to get bug fixes.  The intent of such a 3.4 index would be to  
give people the latest 3.4 updates which should give no new  
features.  This is intended to mimick what happens with linux package  
respositories.



Instead I'd suggest you institute a rule that index pages should  
never be changed after initial publication, just like eggs  
shouldn't be changed after publication either. Instead, a new index  
page should be published for each update. People can then choose to  
switch whenever they like.


You could manage an index like that.  I don't think that would be  
very useful.  IMO, if you want to really nail version numbers, then  
it would be better to just use meta eggs or buildout version  
specifications.


The goal of the index is to give people a stable source of  
distributions that are known to work together.


My main suggestion is to not forget that this needs to be clearly  
documented and the document describing which steps to take should  
be published.


Absolutely. The index is just a mechanism that will enable processes  
that actually address the problems we're struggling with,


My main worry is that I don't like maintaining dependency lists on  
remote servers somewhere. My intuition is that we should maintain  
these lists close to the actual software. With grok we're managing  
this list now as a versions.cfg in subversion. Anyway, this is just  
a worry, and we'll have to see how this pans out.


That's a valid approach, at least for some problems.

If the software is an application, the best approach IMO is for each  
release to specify precisely the versions it's using.  If software is  
a platform, then more flexibility may be needed,  This index idea  
seeks to learn from experience with linux distributions.


We've gone another direction with the Grok project this week, where  
we publish a versions.cfg on the web that gets included in a  
buildout.cfg. This approach was based on suggestions by yourself to  
me, and had been discussed by Philipp on the mailing list earlier,  
but apparently the choice was made to go for index pages instead.


Yup.  Of course, it only works with buildout.  At the time, I  
remember getting push back from someone that they wanted a solution  
that didn't require buildout.  A custom index is more open both wrt  
packaging technology and to use by different applications.  I think  
Zope 3 would benefit from a stable repository of eggs that can be  
used even if people aren't using Grok or buildout.


Of course, the index technology isn't worth much without solid  
processes behind it.


So, it's a bit of a shame this extensive work will now be made  
mostly redundant by these new efforts.


I don't think so.  Certainly, the detective work you've done figuring  
out what's compatibly should be used by whoever assembles a 3.4 index.


Anyway, we'll be using this for a while at least and we'll wait and  
see how this new works pans out.


I think the index complements techniques like meta eggs or  
buildout.cfgs with nailed versions.


Also, keep in mind that this effort results from a desire to help and  
because people care.  Be careful about accusing people of not caring  
and then dismissing their efforts to help.


I really appreciate the efforts of Philipp and Baiju to help us make  
incremental progress on the release process for individual projects.  
I think we need to start working (thinking  talking) about the  
release process for the larger Zope 3 ecosystem.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases

2007-10-06 Thread Jim Fulton


On Oct 6, 2007, at 5:06 AM, Martijn Faassen wrote:


Jim Fulton wrote:

On Oct 5, 2007, at 1:59 PM, Martijn Faassen wrote:

[snip]
but moved to a new place. zope.app.error, is, I understand, gone  
now.
I have no idea about this specific move.  If there was a  
zope.app.error before, then distributions of it should still exist  
and new distributions should be backward compatible.


After we fixed a bunch of errors in them, they're backward  
compatible, yes. With deprecation warnings. The code moved to  
zope.error so zope.app.error is now empty.


I really don't think going from zope.app.applicationcontrol 3.4  
beta whatever, going to final 3.4.1 suddenly means a whole new  
package should appear with new deprecation warnings.


Agreed.

I thought that *never* in the 3.4 line of eggs should they  
*suddenly* start relying on 3.5 eggs. That's nothing to do with the  
notion of a 3.4 release, but with the notion that during the  
stabilization phase, or with minor bugfix releases, you don't  
suddenly start relying on a new feature release of something else  
(or in this case, an entirely new release).


I think I agree with the spirit of the above, but not the specifics.   
You restate the specifics below in a way I whole-heartedly agree  
with. There isn't a 3.4 line of eggs.  There could be a set of  
projects versions associated with a 3.4 release of Zope3, but the  
individual version number could be almost anything.



Anyway, I think the rule should be:

When you do a final or bugfix release of a package, you can't  
start requiring a new feature release of another package.


+1


Translated to version numbers:

If X.Y.x has been relying on A.B.x, X.Y.x + 1 cannot start relying  
on A.B + n, only on new A.B.x + n releases, where x is one of (b0,  
b1, 1, 2, 3, ...) and n is one of (1, 2, 3 ...)


Exactly.

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases

2007-10-06 Thread Jim Fulton
 that  
as soon as they themselves and the other expert developers know  
what to do (assimilated on the mailing list and online), this is  
the end of the problem. We don't have a single Zope 3 release  
anymore, but we do want people to use our code and report bugs on it.


We were supposed to. I wonder what happened to that.

As far as I can see this *requires* a list of versions somewhere  
that is shared between people and that can be communicated about  
easily. It requires a *release procedure* around this list of  
versions. As I tried to point out before long ago, the Zope 3  
project is not somehow so special it doesn't need releases.


I think the Zope 3 project is somewhat special, which it may need  
some kind of releases. :) After all, we have releases of individual  
eggs.  I don't think it makes much sense to release the traditional  
Zope 3 application except perhaps as a testing platform.  I do think  
we need something like what linux distros provide.  We need to  
establish a better understanding of this.



as we have fixed the problem with Grok. If non-Grok users are  
interested in our fixes, please let us know though. We've just  
made this massive investment. I'd suggest people to switch to  
Grok anyway, as we actually think about this stuff and care about  
having our house in order.
Are you suggesting that other people aren't thinking about this  
and don't care?


Fred doesn't appear to care to think about this much, certainly,  
and is liking it that way.


So based on a snarky comment by one person, you tar the entire  
community. I'll note, BYW, that if you had provided more details, as  
Philipp eventually did, I think Fred's response would have been more  
sympathetic. (Just guessing)


I hope that the proposal for setting up a known-good-set index  
will be helpful.  I'm sure Stephan would like to know the versions  
you chose. I imagine he'll be able to get those by looking at Grok.


After 3 days of hard work by 2 people, we're still not done with  
the Grok eggs. We get weird effects like final releases of packages  
*suddenly* needing an entirely new package, between the beta and  
the final. I'm quite bothered by this.


Me too.


We still have quite a few dev-r515135 style eggs in the mix as well.


That's bad.

A single known-good index, by the way, doesn't solve all problems.  
It will evolve forward as people release newer known-good versions  
of eggs. This means that nobody can really rely on such an index,  
as suddenly they might be starting to use 3.6 eggs where they were  
using 3.5 eggs. Even bugfix releases are currently hardly safe:  
using 3.4.1 for some Zope 3 package might mean that suddenly it  
requires an entirely new package altogether, introducing new  
deprecation warnings and so on (as in this thread).


This is all a matter of the rules for maintaining the index and the  
care put into it.  If it is well run, a big if, then it should be as  
reliable as, say, a linux package repository. This doesn't guarantee  
that there won't be problems.


With our Grok versions list, we publish a list *per* version of  
Grok. I hope this is the intent for the known-good index as well.  
If someone says they use Grok 0.11,


Is Grok 0.11 a feature release?  Would you expect the version list  
for Grok .11 to change as bug fixes are made?



we want to know *exactly* which versions they are using without  
requiring them to send us a long list too. One version number  
should be enough. Our installation tools support this, and our  
documentation documents this. I think this is the only maintainable  
way to actually have a community developing and using a common code  
base.


My suggestion is that there should be stable indexes for each  
feature release.  Changes to these indexes should be made to fix  
bugs, not add new features.  Of course, there could be other less  
stable indexes for people who want them.  If you want to nail all of  
the versions for a specific release of grok, then it seems to me that  
versions in a meta egg or a buildout configuration should work well.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Zope 3 releases?

2007-10-06 Thread Jim Fulton


Recent discussions make it obvious to me that we need to come to  
terms with Zope 3 releases.


I have some ideas and observations.  I don't have decisions at this  
point.  We need to make some decisions as a community, with me  
hopefully helping at times to get us unstuck. :)


- The classic 3.4 release seems to be stalled.  This was to be a  
release modeled on previous Zope 3 releases.  It provides a  
monolithic Zope 3 application.  I expect that it is stalled because  
the person who volunteered to see it through overcommitted and, more  
importantly, it isn't of much practical use, except as a testing  
platform or as some sort of known good configuration.  The original  
rational was that we didn't want to be too disruptive in the next  
release.  I think we've been overtaken by events and a lack of  
interested resources.


- There is a desperate need for a  known good configuration of  
components that work well together and that people can build on  
without having to think too hard about the underlying platform.


- There is a need for a mechanism for testing core components to  
make sure they don't cause breakage.


- We need to decide what Zope 3 is.  Zope 3 is *not* an application.  
I think there is fairly broad agreement on that.  I can think of  
several words that could be used to describe what it *is*, like  
platform, environment, ecosystem.  I think it's time we came up  
with the elevator speech for what Zope 3 is.  It should be a few  
sentences at most.  It need not be carved in stone forever, but it  
should be agreed on and used for tactical planning.  This should  
probably go hand in hand with the bigger definition of what Zope is  
along the lines of the ideas that Tres expressed at the DZUG in Potsdam.


- We need to decide what a Zope 3 release is (or maybe multiple  
flavors).  I favor copying the linux experiences, but have an open mind.


- We need a *realistic* (especially wrt available resources) process  
for managing releases.  There are 2 aspects of this.  We shouldn't  
make plans for which there aren't enough resources.  We also  
shouldn't plan significant tasks that people won't care enough to  
work on.  I think the classic Zope 3.4 release is a good example of a  
large effort that really wouldn't benefit many people, if any.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Known-good-sets problem

2007-10-06 Thread Jim Fulton


On Oct 5, 2007, at 10:45 PM, Benji York wrote:


Stephan Richter wrote:
2. How many packages should be controlled in this index? I think  
we should definitely add packages from z3c and the zc namespace.


What is the motivation to include non-controlled packages?  I  
suppose it is to let people use those packages with (in this case)  
Zope 3.4.


Yes

  What if someone wants Zope 3.4 and Twisted version X and Plone  
version Y (just making those up).


That's why the KGS index is a PyPI mirror of all uncontrolled packages.

  Perhaps we need a way to refer to several KGS when constructing  
an application. Or is one KGS supposed to define a universe of  
packages known to work together.  If so, I would think there would  
be no place for non-controlled packages.


The semantics I think we want are kind of tricky. A KGS index needs  
to be authoritative for the projects it controls.  If we looked in  
multiple KGSs, there would need to be semantics for deciding which  
was authoritative.  setuptools lets you define a single index and  
multiple find-link servers.  The highest version found on any server  
is authoritative.  I think supporting multiple KGSs with the right  
semantics would be useful, but there isn't a way to do it in  
setuptools. We can achieve the  same effect on the server.  For  
example, with this software, you could chain several KGSs together.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: [Zope3-Users] Re: Known-good-sets problem

2007-10-06 Thread Jim Fulton


On Oct 6, 2007, at 5:28 AM, Martijn Faassen wrote:


Benji York wrote:

Stephan Richter wrote:
2. How many packages should be controlled in this index? I think  
we should definitely add packages from z3c and the zc namespace.
What is the motivation to include non-controlled packages?  I  
suppose it is to let people use those packages with (in this case)  
Zope 3.4.  What if someone wants Zope 3.4 and Twisted version X  
and Plone version Y (just making those up).  Perhaps we need a way  
to refer to several KGS when constructing an application.  Or is  
one KGS supposed to define a universe of packages known to work  
together.  If so, I would think there would be no place for non- 
controlled packages.


A feature to combine lists of versions is definitely needed. I may  
be a developer of an application that uses both Grok and KSS, for  
instance. This can be solved by me maintaining my own list, but I'd  
prefer to rely on two lists that are already there.


That's the application developer's case. The framework developer's  
case requires me to publish such combined lists. If I have my own  
framework on top of Grok and KSS, I'd like to publish a list that's  
a combination of those without having to copy them.


As I mentioned in my response to Benji, it's tricky to get the  
semantics right.  IMO, you want a particular KGS to be  
authoritative.  There isn't anything in setuptools to support this  
semantic.  You can use the mirroring approach (maybe with some minor  
tweaking) to combine KGSs on the server.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] zope.error is a 3.5 egg, but is needed by 3.4.x releases

2007-10-05 Thread Jim Fulton


On Oct 5, 2007, at 12:07 PM, Martijn Faassen wrote:


Hi there,

zope.error is a 3.5 egg, but is needed by 3.4.x releases. I guess  
this also happened because large package refactorings happened and  
were released as 3.4.x releases. It's pretty bizarre to run into,  
though.


The satellite projects have version #s that are independent from the  
Zope version #s. Any similarity is a hysterical accident. :)


Jim

P.S. This has nothing to do with whether the Zope 3.4 release is  
important.


--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases

2007-10-05 Thread Jim Fulton


On Oct 5, 2007, at 1:59 PM, Martijn Faassen wrote:


Jim Fulton wrote:

On Oct 5, 2007, at 12:07 PM, Martijn Faassen wrote:

Hi there,

zope.error is a 3.5 egg, but is needed by 3.4.x releases. I guess  
this also happened because large package refactorings happened  
and were released as 3.4.x releases. It's pretty bizarre to run  
into, though.
The satellite projects have version #s that are independent from  
the Zope version #s. Any similarity is a hysterical accident. :)


I don't understand.


Probably because you didn't provide specifics so My answer is to a  
different question than you may have been asking. :)



zope.error is zope.app.error,


I don't know what this means.


but moved to a new place. zope.app.error, is, I understand, gone now.


I have no idea about this specific move.  If there was a  
zope.app.error before, then distributions of it should still exist  
and new distributions should be backward compatible.



zope.error 3.5 is needed by:



  required by zope.app.applicationcontrol 3.4.1.
  required by zope.app.appsetup 3.4.1.
  required by zope.app.publication 3.4.2.


OK. I'm not sure what the issue is here.


Are these satellite projects? What is a satellite project?


Yes. These are satellite projects.  We use the word satallite  
project for the new individual egg projects to distinguish them from  
the classic Zope 3 tree.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Known-good-sets problem

2007-10-05 Thread Jim Fulton


I discussed this a bit this afternoon with Stephan and we came up  
with an idea that we think might help.  Stephan is going to try to  
prototype it.  I'll try to explain it.


The basic idea is to provide a custom index.  There will be such an  
index for each known good set (KGS).  An example of a KGS would be  
the KGS corresponding to the Zope 3.4 release.  A KGS would have a  
set of controlled projects.  A KGS index will have manually-managed  
project pages for all controlled projects. For other projects, it  
will mirror PyPI.


The prototype will build on my cheeseshop mirroring software.  We  
will add a controlled projects list as configuration data. The  
mirroring software will ignore updates for controlled projects. This  
is a very small change to existing simple software. (The controlled  
project directories could be managed by either editing index pages or  
by placing approved distros into a server directory.) The custom  
index will be a static web site.


With this in place, we can establish KGSs and, for controlled  
projects, the index pages will only be updated when a distribution  
for a known project has been carefully vetted.


Users can set up buildout or easy_install to use the specific KGS as  
their index server.  Each KGS will have a release manager who will be  
responsible for maintaining the KGS.


We will also create a buildout that tests packages in the KGS.  When  
one wants to test a change to a core package, they would:


- check out the buildout
- maybe change the index option to point to a particular KGS
- check out the project(s) they want to test and configure them as  
develop eggs

- run the buildout test script.

Hopefully, this will give us much greater stability than we've had up  
to now.


--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: zope.error is a 3.5 egg, but is needed by 3.4.x releases

2007-10-05 Thread Jim Fulton


On Oct 5, 2007, at 1:55 PM, Martijn Faassen wrote:
...

Grok will pick up the balls Zope 3 dropped here.


Hm. I didn't think Zope 3 was animate.  Who are you referring to?

We actually care about a Grok version as it's the main way to get  
people to actually use Zope 3 stuff.


We noticed this while we were going through egg dependencies by the  
way, not using a Zope 3 version.


I don't know what you are trying to say.

...

Anyway, I personally don't care much that Zope 3.4 is unusuable  
without a massive investment in time sorting through eggs,


It's a shame that you are taking this view.

as we have fixed the problem with Grok. If non-Grok users are  
interested in our fixes, please let us know though. We've just made  
this massive investment. I'd suggest people to switch to Grok  
anyway, as we actually think about this stuff and care about having  
our house in order.


Are you suggesting that other people aren't thinking about this and  
don't care?


I hope that the proposal for setting up a known-good-set index will  
be helpful.  I'm sure Stephan would like to know the versions you  
chose. I imagine he'll be able to get those by looking at Grok.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Release process closure

2007-10-04 Thread Jim Fulton


On Oct 4, 2007, at 9:25 AM, Baiju M wrote:


Jim Fulton wrote:


 On Oct 4, 2007, at 6:51 AM, Philipp von Weitershausen wrote:

 On 4 Oct 2007, at 00:59 , Jim Fulton wrote:
 On Oct 3, 2007, at 3:44 PM, Philipp von Weitershausen wrote:

 Jim Fulton wrote:
 I'd really like to get to closure on the current approved
 release process. Philipp, would you mind separating the
 release process into a separate file?  Or do you mind if I do
 it?

 Done:

http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/ 
releasing-software.txt



 Cool.  I think you can delete 5b.  You already update the date,
 as you should, on the trunk or branch.  You want the actual
 release date to be part of the change log, so it has to be
 entered before making the tag.

 Done.

 I think we need to split d into:

 d) Create a source release

 e) Test the source release. At a minimum, rerun the package tests
 using the source release. (I really need to add a buildout option
 to help with this.)

 So how would I do this? This feels a bit complicated:

 1. Create a source distribution with::

 $ python setup.py sdist

 2. Extract the tarball::

 $ tar xzf dist/foo.package-X.Y.tgz

 3. Edit buildout.cfg to make the result of the tarball a develop
 egg *instead* of the stuff in 'src'::

 [buildout] develop = foo.package-X.Y

 4. Rerun the buildout::

 $ bin/buildout

 5. Run the tests::

 $ bin/test

 No. :)

 Currently, you could:

 - Create the source distro. (Note that I always use sparkling clean
 Pythons, so the command you give doesn't work for me as setuptools
 isn't importable. I always use: bin/buildout setup . sdist



Why you cannot install setuptools ?


It's not that I cannot. I *will* not.

I like my Python to be shiny sparkly clean.  My Python is what I get  
after running configure, make, make-install. Period.  My site- 
packages is empty.  IMO, this is the only sane way to develop.


Also, I get very very very cranky when someone wastes my time with a  
problem that is ultimately traced to crap installed in their Python  
that isn't part of a standard install. Very cranky.  I'm feeling  
cranky just thinking about it. :)


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] I'd lobe to merge the zope3-dev and zope-dev lists

2007-10-04 Thread Jim Fulton


Any objections?

This would basically involve retiring the zope3-dev list and moving  
zope3 developers to the zope-dev list.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] WSGI2

2007-10-04 Thread Jim Fulton


There's work going on to create a second version of WSGI.  Last time,  
we didn't pay much attention until WSGI was a done deal.  This time,  
I think it would be better if we were involved earlier.   
Unfortunately, I don't have time to pay attention. Does anyone else?


Jim

P.S.  I'd lobe to avoid typos.  Especially in subject lines. :/

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: [Zope-dev] I'd love to merge the zope3-dev and zope-dev lists

2007-10-04 Thread Jim Fulton


On Oct 4, 2007, at 10:02 AM, Baiju M wrote:


Jim Fulton wrote:


 Any objections?

 This would basically involve retiring the zope3-dev list and moving
 zope3 developers to the zope-dev list.


+1

What about retiring #zope3-dev IRC channel and only using #zope ?


I though of that.  Historically, the #zope channel was much chattier,  
which makes it hard for me to deal with. When #zope3-dev was set up,  
I asked that people keep the noise level down, which has made it much  
more useful, imo.


Maybe we should keep these issues separate for now.

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Release process closure

2007-10-03 Thread Jim Fulton


I'd really like to get to closure on the current approved release  
process. Philipp, would you mind separating the release process into  
a separate file?  Or do you mind if I do it?


Some comments on the current draft at:

  http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/ 
maintaining-software.txt


WRT version numbers in setup.py.  I'm inclined to endorse Philipp's  
recommendation for now.  If there was a way to specify a version  
number on the command line (or in a buildout.cfg) when creating  
develop eggs, then I'd have a different position, but given current  
technology, I think Philipp's recommendation, as I understand it, is  
best.


I think there are some details missing.  I think the intend is that  
the version number in the setup.py on the trunk and on release  
branches should have the dev suffix.  I think this is good as a  
reminder and a flag that accidentilly released dev eggs are suspect.


I think the dance should be that, to make a release, you make a tag  
and then edit the version number on the tag.  I think this sort of  
editing on a tag is reasonable and it is fortuitous that svn allows it.


Also, by default, the next version on the trunk should be a release  
with the second number incremented.  The next version on a release  
branch should have the 3rd number incremented. This is a minor  
detail, especially since I think we can avoid release branches for  
most projects and I think that release branches shouldn't be created  
until they are needed.  Of course, tags should always be created.


There are other improvements that might be made, but let's not let  
the perfect be the enemy of the good.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Release process closure

2007-10-03 Thread Jim Fulton


On Oct 3, 2007, at 11:12 AM, Benji York wrote:


Jim Fulton wrote:
I'd really like to get to closure on the current approved release   
process.


+1

There are other improvements that might be made, but let's not  
let  the perfect be the enemy of the good.


This isn't perfect, so I get to suggest it. wink

I propose that we codify the practice of release tags having their  
x.y.z version number as their name, and in the setup.py.


Which implies that we codify 3-number version numbers, that is:

  major.minor.bugfix

with optional modifiers of the form dev, aN, bN, or cN.

I also suggest a special version numbering scheme when we package  
something else.  For example, if we package Twisted ro package some  
external js library in a Python package.  In this case, I suggest  
we   use the version number of the thing being packages with the  
addition of a post-release addition.  For example, a packaging of  
MochiKit 1.3.1 might have a version # like 1.3.1-1, or, if it is  
exciting enough: 1.3.1-1.1


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Release process closure

2007-10-03 Thread Jim Fulton


On Oct 3, 2007, at 11:46 AM, Benji York wrote:


Jim Fulton wrote:
For example, a packaging of  MochiKit 1.3.1 might have a version #  
like 1.3.1-1, or, if it is  exciting enough: 1.3.1-1.1


I assume setuptools interprets version numbers like this in a  
reasonable way.


AFAIK. :)

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: zope.dottedname doesn't have a CHANGES.txt (again?)

2007-10-03 Thread Jim Fulton


On Oct 2, 2007, at 6:52 PM, Stephan Richter wrote:


On Tuesday 02 October 2007 17:14, Jim Fulton wrote:

One hole I see is giving people guidance on what needs to be tested
(and how) before a release is made.  My preference would be to rely
heavily on judgement with a few checks so as not to make things too
heavy.  This might rule out some releasers.


This is odd text to quote, considering what follows. Do you really  
think that an automated tool is going to be able to determine on a  
case by case basis what needs to be tested?  Heck, I find it hard to  
judge what is best to test.  I think we need to think about what the  
process needs to be. It's not at all clear to me and yet you are  
ready to automate it.   I can only hope that this wasn't the text you  
intended to quote.



I want tools! Actually, I just want one tool.


You want a silver bullet.


70% of the release process is
repetition and that needs to be factored into a tool.


I think that is overly optimistic, but I have an open mind. Frankly  
though, your desire for an automated tool frightens me.



This tool should have
been written before the Zope trunk was blown into pieces, but it  
wasn't. :-(


I'm skeptical that such a tool can do that much. Certainly, when we  
broke the trunk up, we didn't know what all the issues would be.  We  
couldn't automate a process we didn't yet fully understand. My only  
real regret is that we broke things too far too fast, but that is  
water under the bridge.


Some of the fouls this week had nothing to do with the release  
process.  The breakage you (I assume) and Roger caused by removing  
needed files would have caused just as much havoc in the old days.   
You seem to think that the trunk is everything, so making changes  
there and adjusting all of the effected software *on the trunk* is  
enough.  This kind of breakage used to occur before and caused much  
pain for 3rd-party consumers of the Zope 3 tree.


Other breakage occurred after people tried to give you advice on how  
to do things more reliably.  Saying you need a tool is not an excuse  
for ignoring advice.


There is no way that we will be able to support this many packages  
in the

future, if we keep doing this manually.


You should know that we already *are* maintaining this many packages.  
People have been able to figure out how to release packages in a  
fairly stable way.  We're learning from our mistakes and creating  
processes to make things better.



I have already spent days on doing
eggs, when I really just wanted to code. :-(


Then stop.  The current process is too manual and requires too much  
judgement for you.  OK, then stop updating core packages.


We all make mistakes.  Even with the best tools, processes and  
intentions, bad things will happen.  No one will hold a grudge as  
long as people are being responsible and trying to do the right  
thing.  OTOH, while wishing for a tool and or for better processes is  
fine, blaming the current processes for your mistakes is getting  
tiresome.


jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Release process closure

2007-10-03 Thread Jim Fulton


On Oct 3, 2007, at 3:44 PM, Philipp von Weitershausen wrote:


Jim Fulton wrote:
I'd really like to get to closure on the current approved release  
process. Philipp, would you mind separating the release process  
into a separate file?  Or do you mind if I do it?


Done: http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/ 
releasing-software.txt


Cool.  I think you can delete 5b.  You already update the date, as  
you should, on the trunk or branch.  You want the actual release date  
to be part of the change log, so it has to be entered before making  
the tag.


I think we need to split d into:

 d) Create a source release

 e) Test the source release. At a minimum, rerun the package tests  
using the source release.

  (I really need to add a buildout option to help with this.)

  f) If the package has extensions, make and test a windows egg.  
(You may need to get someone with

 the needed Windows tools to help you with this. :)

  f) Upload the release(s) to PyPI

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: New package zc.configure provides an exclude directive for excluding zcml files

2007-10-02 Thread Jim Fulton


On Oct 2, 2007, at 3:29 AM, Martijn Faassen wrote:


Jim Fulton wrote:

[snip]
Maybe grok was already trimmed down.  In my case, I basically  
eliminated all ZMI support (since I didn't need it).  I got about  
40%,


Grok is trimmed down in the sense that it doesn't depend on all  
Zope 3 packages, though due to the interesting dependency structure  
it still relies on about 100 eggs. We didn't do any trimming down  
of ZMI support.


Note that it wasn't my goal to trim the number of eggs I got,  
although trimming that, or, in theory, using zipped eggs would speed  
startup as well.  I'm using the zope.app.zcmlfiles egg as my base.


Would it be possible to get a list of exclude statements that you  
use to eliminate ZMI support in your project? I imagine our list is  
far from complete.


Here is the zcml file I'm including rather than incliding  
zope.app.zcmlfiles/configure.zcml:


  configure
 xmlns=http://namespaces.zope.org/zope;
 xmlns:i18n=http://namespaces.zope.org/i18n;
 i18n_domain=zope
 package=zope.app.zcmlfiles
 

exclude package=zope.app.catalog.browser /
exclude package=zope.app.container.browser /
exclude package=zope.app.form.browser /
exclude package=zope.app.generations /
exclude package=zope.app.intid.browser /
exclude package=zope.app.session file=browser.zcml /
exclude package=zope.app.security.browser /
exclude package=zope.app.securitypolicy.browser /

exclude package=zope.app.authentication.browser /


exclude package=zope.app.authentication.browser  
file=session.zcml /
exclude package=zope.app.authentication  
file=ftpplugins.zcml /
exclude package=zope.app.authentication  
file=httpplugins.zcml /
exclude package=zope.app.authentication  
file=groupfolder.zcml /
exclude package=zope.app.authentication.browser  
file=password.zcml /
exclude package=zope.app.authentication  
file=principalfolder.zcml /


exclude package=zc.userfolder.browser /

!-- This was copied from zcmlfiles.  Bits are commented out to
 speed startup. --

!-- note that we need to do this early, as later startup
 subscribers may break without fixups --

exclude package=zope.app.component.browser /
include package=zope.app.component /

include package=zope.app.generations file=subscriber.zcml /

!-- Ordinary Application (non-view) configuration) --
  !--   include package=zope.app.interface / --
include package=zope.app.security /
include package=zope.component /
include package=zope.annotation /
  !--   include package=zope.app.dependable / --
include package=zope.app.content /
include package=zope.publisher /

  !--   include file=menus.zcml / --

  !--   include package=zope.copypastemove / --
  !--   include package=zope.size / --
include package=zope.location /
include package=zope.app.container /

exclude package=zope.app.publisher.xmlrpc /
include package=zope.app.publisher /

include package=zope.app.publication file=meta.zcml /
include package=zope.app.publication /


include package=zope.traversing /
include package=zope.app.pagetemplate /
  !--   include package=zope.app.zapi / --

!-- Views --
  !--   include package=zope.app.http / --

!-- Translations --
  !--   configure package=zope.app.locales --
  !-- i18n:registerTranslations directory=. / --
  !--   /configure --

exclude package=zope.app.i18n.xmlrpc /
exclude package=zope.app.i18n.browser /
include package=zope.app.i18n /


!-- Database boostrapping and maintanance --
include package=zope.app.appsetup /
include package=zope.app.zopeappgenerations /

!-- Services --
include package=zope.app.principalannotation /

!-- Utilities --
  !--   include package=zope.app.error / --

!-- Broken-object support --
  !--   include package=zope.app.broken / --

!-- Skins --

  !--   include package=zope.app.basicskin / --
include package=zope.app.rotterdam /

!-- Additional packages --

  !--   include package=zope.app.applicationcontrol / --
  !--   include package=zope.dublincore / --
include package=zope.app.wsgi /


!-- Content types --
  !--   include package=zope.app.folder / --

!-- browser Configurations --
include file=browser.zcml /

  /configure

Note that this was made by just copying the configure.zcml from  
zope.app.zcmlfiles and commenting out some things and adding  
excludes. I basically kept commenting things or excluding things  
until my tests failed. :) I could probably go a little further if I  
worked a lot harder, so of course, I stopped. :) I'll also probably  
have to add some things back later when I pay attention to i18n.  
(This app uses extjs for it's UI and I haven't figured out how I'm  
going to approach i18n for that. extjs rocks btw.)


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org

Re: [Zope3-dev] major packaging reorganization happening in 3.4 releases?

2007-10-02 Thread Jim Fulton


On Oct 2, 2007, at 5:25 AM, Martijn Faassen wrote:


Hi there,

Besides causing us a lot of problems here at the Grok sprint, I  
also wonder why in the world are we doing major packaging  
reorganizations and releasing them as minor 3.4.x releases? You'd  
think such a reorganization would warrant a 3.5 release!


Agreed. Someone needs to sloow down.  Speed kills. deserves to  
be added to the Zen of Python. (Actually, ZoP does have a variation  
of that.)


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: AW: [Zope3-dev] zope.app.securitypolicy deferred imports errors.

2007-10-02 Thread Jim Fulton


On Oct 2, 2007, at 6:03 AM, Roger Ineichen wrote:


Hi Lennart


Betreff: [Zope3-dev] zope.app.securitypolicy deferred imports errors.

Roger Inechens split of zope.app.securitypolicy into
zope.securitypolicy cause loads of problems, because many of
the deferred imports were incorrect. I saw Stefan Richter
fixed some, and I have fixed some more.


Yes you are right


First somebody that can make releases (which may or may not
include me, I honestly don't know who can make releases of
eggs) needs to make a new one. But we also need to avoid this
stuff in the future.


Yes, we always have to avoid errors, but sometimes it happens.
And since we dont test against the trunk, we don't see this
kind of errors anymore. This means we test against custom
projects. I didn't fully recognize this and was trusting the
tests. But I was wrong. We don't have tests for this anymore.
We probably didn't have test for my error at all.

In other words; Right now we only test if a egg works
within their dependency. But we don't test other eggs
if they work with the egg we develop.


See all my different mails about this topic!


Yes, please do.  It's up to people making changes to use some  
judgement.  I mentioned in that thread that when I make changes to  
core components, I often do test against the old trunk tree.   
Despite all of your complaints about the need to do this, you didn't.





How is the recommended process to solve this? Some sort of
unittest to make sure the deferred impoirt work? Is there an
example of this somewhere?


I'm proposing to test against a set of possible breaking
stuff. This means we need a kind of zope3 trunk egg which
our test will deoend on.


As I mentioned in the thread you want us to pay attention to, this  
isn't necessary.  In fact, I doubt it would be useful.  I think we  
need a Zope 3 application buildout that builds the traditional Zope 3  
app. You can then introduce a develop egg for the package you are  
changing there.  In the mean time, all you need is a trunk checkout  
and, possibly, to adjust an external. (I don't remember how the  
externals are set up there.)



See the mails form Stpehan Richter about that. Jim also
agrees on that but is thinking this should be optional
as far as I understand the mails between Stephan and Jim.

We defently lost the overall tests we had in the trunk setup.


Most people aren't changing these components. Nevertheless, we still  
have the old tree available for testing. We didn't lose anything.



And this is a very bad idea. If we don't recommend to run
all tests again form a single egg refactoring, we will
see more errors like this.


Don't you think it's a little hypocritical bemoaning that we're not  
doing this when you don't do it yourself?


...


I personaly think, less test -- more errors.


So test! What's stopping you?  The old tree is still available!

If someone wants to make this better, a working Zope 3 application  
buildout would help a lot.  I doubt it would be that hard to finish  
at this point.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: AW: AW: [Zope3-dev] zope.app.securitypolicy deferred imports errors.

2007-10-02 Thread Jim Fulton


On Oct 2, 2007, at 9:24 AM, Roger Ineichen wrote:


Hi Jim,


Betreff: Re: AW: [Zope3-dev] zope.app.securitypolicy deferred
imports errors.


[...]


Yes, please do.  It's up to people making changes to use some
judgement.  I mentioned in that thread that when I make changes to
core components, I often do test against the old trunk tree.
Despite all of your complaints about the need to do this, you didn't.


That's not true. I did test the package against the trunk.
But the problem was, I also adjusted the imports in the packages.


hehe.  This sort of practice makes any arguments for testing against  
the trunk kind of meaningless.




What I was missing is to run the tests against a older version of
the trunk.


So perhaps now you want people to test against old versions of the  
trunk?  How many old versions?


A better approach would have been to add backward compatibility tests  
for each backward-compatibility support change you made, as Lennart  
suggested.  Even then, to be more sure that you caught everything,  
you should have made the changes in two stages:


1. Refactor the package in question providing backward compatibility  
support, including tests of that support.  Test against other  
unchanged packages.


2. Then change the other packages is you must.

...


I know there is no difference between the old trunk and the new egg
development testing. This error whould also happen to me if I where
doing it in the old trunk setup. My problem was not test my work
against a older version of the trunk. Or like Lennart told,
the missing tests for integration (import changes).


The latter.  The 2-step process that we've used in the past would  
also have helped.





I personaly think, less test -- more errors.


So test! What's stopping you?  The old tree is still available!


The crapy do it by your self test setup is stopping me really
hard or at least slowing me down right now more then the it should.


Maybe it should slow you down more.


It's not easy to follow all the ideas behind egg, setuptools,
easy_install, buildout in such a speed like we changed to this
patterns. I agree Speed kills, but I think switch to eggs
force us to take it all or nothing.


We probably do need more formalized testing/release methodologies. As  
I'm sure Tres and others will rightly point out, there's probably a  
lot we can learn in this area from linux distributors.  In the mean  
time, changing packages that lots of other people depend on requires  
judgement and care.  If you don't think you're up to it, then don't  
try it.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access [update]

2007-10-02 Thread Jim Fulton


On Sep 28, 2007, at 1:36 PM, Dieter Maurer wrote:
...

Your example:

 You have release 1.1.

 You start working on the next release: 1.2dev-r#.

 Someone fixes a bug in 1.1 and releases 1.1.1.

With quite high probability, the bug fix should go as well into
the 1.2 development.

Thus, the problem you describe (1.1.1 is newer than 1.2dev-r;
but it treated as older then 1.2dev by 'setuptools')
has an effect only for a very limited time -- the time until
the fix (1.1 -- 1.1.1) went into the 1.2dev release, which
would probably become some 1.2dev-r!.


Sure, but that time could stretch out quite a bit.  I suppose that  
this problem somewhat inherent in having multiple release branches.   
For example, whatever we do, there will always be the possibility  
that an x.y.z release could have fixes in it that aren't in an x.y+1  
release yet, especially if x.y is a non-final release.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: PLEASE don't remove eggs [was Re: [Zope3-dev] faulty releases and pypi access [update]]

2007-10-02 Thread Jim Fulton


On Sep 26, 2007, at 10:35 AM, Jim Fulton wrote:
This is also a technical issue:  As long as zc.buildout and  
setuptools

foolishly accept dependency links from an egg, it'll be painful to
detect accidental reliance on external repositories.


That's a good point.  I wouldn't go so far as to say foolishly,  
but I would say that this is a policy that should be overrideable.   
Checkins to buildout (with tests, of course) accepted.


Sort of that, a feature request in launchpad would be helpful so this  
idea doesn't get forgotten.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: faulty releases and pypi access [update]

2007-10-02 Thread Jim Fulton


On Sep 27, 2007, at 4:45 AM, Philipp von Weitershausen wrote:


On 27 Sep 2007, at 02:29 , Tres Seaver wrote:

Further, anybody who finds the effort of creating a fresh' checkout
bevore making a release too burdensome should consider themselves
self-selected out of the release manager pool.

I'm *not* kidding about that:  taking shortcuts durng the release
process transfers pain / cost to everyone downstream.


Amen, brother. Well said.


+1

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: faulty releases and pypi access [update]

2007-10-02 Thread Jim Fulton


On Sep 27, 2007, at 7:18 AM, Philipp von Weitershausen wrote:


On 27 Sep 2007, at 13:07 , Martijn Faassen wrote:

On 9/27/07, Tres Seaver [EMAIL PROTECTED] wrote:
[snip]

Further, anybody who finds the effort of creating a fresh' checkout
bevore making a release too burdensome should consider themselves
self-selected out of the release manager pool.

I'm *not* kidding about that:  taking shortcuts durng the release
process transfers pain / cost to everyone downstream.


While I fully agree that releases should be done with care and
attention, and that doing bad releases creates pain/cost for
everybody, this line of argumentation can be used to back up any form
of complication to the release process. :)

Let's focus on the reasons for each step and keep the discussion at
that level, please? I think it's useful if people ask is that really
necessary? for steps in the release process. Just weigh the pros and
cons. I'd like to understand more about the trouble Philipp ran into
doing releases from the original checkout and then tagging.


I've summarized this in another thread [1], but I'll be happy to  
elaborate here:


* People actually forget to make the tag. This happened to Roger  
the other day. It happened to Jim before with zc.buildout. I'm not  
mentioning their names to point fingers. I'm just pointing out that  
people who've been with us for a long time forget. If the process  
said they should actually check out the tag, they'll hardly forget.


Having a script we have to mindlessly follow will help.

* People create the wrong tags or tags in the wrong places. This  
happened to Jim the other day with ZODB 3.7.1. Again, I'm not  
blaming him. But I believe if he had to check out the tag to make  
the release, he would've caught his mistake.


I doubt it.  I would have probably just rereleased 3.7.1, possibly  
with 3.7.2 uselessly included.


* People forget to clean up the 'build' directory. This happened to  
me the other day. Let me elaborate: I have a trunk checkout of  
zopeproject that I use to develop it. Whenever I wanted to make a  
release, I'd just prepare it, tag it and generate the distributino  
from the trunk checkout. The problem was that I had moved stuff  
around, but the old stuff was still making it to the egg because it  
still sat around in the 'build' direcdtory that distutils leaves. I  
know it sucks that distutils doesn't clean up after itself, but  
that's just the way it is. If you're forced to use a clean  
checkout, this can't happen.


The build directory is really annoying.

* People forget to check in important files. This happened to Baiju  
the other day. He created a CHANGES.txt file and referred to it  
from setup.py. The only problem is that he forgot to check it in.  
It can happen, I'm not blaming him. The problem is that setuptools  
looks at which files are in svn in order to create the archive  
manifest. So CHANGES.txt wasn't in the tarball and the egg was  
completely uninstallable (because setup.py couldn't be executed  
since CHANGES.txt was missing).


These are four separate cases where I've actually witnessed myself  
or other people mess up. We're forgetful, we can't do anything  
about that. We can, however, force us to catch our mistakes. I  
believe that if we made everybody create the tarballs from the tag,  
it would improve the situation a lot.


I'm beginning to see a consensus that we should make it impossible  
to create distributions from the trunk or a release branch.


Yup. BTW, I find it helpful to test source releases by unpacking them  
and seeing if I can build eggs from the unpacked source.  Of course,  
it would be even better to run the tests.  I suspect that a buildout  
feature could automate this.  Something along the lines of a buildout  
command that: creates distros from the listed develop directories and  
then installs those distros rather than the develop eggs so that  
tests could be run against the distros.  This would probably be  
something good to do when I have some time to get back to buildout.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: faulty releases and pypi access [update]

2007-10-02 Thread Jim Fulton


On Sep 27, 2007, at 7:37 AM, Philipp von Weitershausen wrote:


On 27 Sep 2007, at 12:20 , Stephan Richter wrote:
egg_info does not validate the trove classifiers, for example. I  
tried this

last night before writing this mail.


Well, to be honest, I wonder how you can mess up with the  
classifiers. I just always copy them from http://pypi.python.org/ 
pypi?%3Aaction=list_classifiers. Anything else is just insane...


The classifiers are a PITA.  I suspect we should be very restrictive  
in using them.


It's especially annoying that release status can generally be  
inferred from the version number.


I'm beginning to wonder if we should have some setuptools extensions  
that automate some of these things,


Another idea I've been thinking about is a custom index that is used  
rather than PyPI.  The main motivation was to automate management of  
non-public distributions, but such a front end could also automate  
some aspects of releases that setuptools doesn't handle.


But, if you wish for such a tool, let your wish be my command. With  
the attached verifyclassifiers.py script, you may do so using the  
following command:


  python setup.py --classifiers | python verifyclassifiers.py


Cool.

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access [update]

2007-10-02 Thread Jim Fulton


On Sep 26, 2007, at 6:14 PM, Jim Fulton wrote:
...
I agree that the develop egg case is problematic.  I'd like to  
think about that.


Well, I thought and I thought 

I could probably arrange that develop eggs created by buildout got  
their versions from the buildout spec. Something like:


  [buildout]
  develop =
 . ==1.4
 foo ==2.1

This would override whatever version was given in setup.py.

This wouldn't help people who don't develop with buildout though.

Alternatively, we could add some API that would allow our setup files  
to take a --version argument.


So our setup files on the trunk or branches might have something like:

  setup(
 version = comman_line_version()

or something.  I'm not an expert on extending setuptools.

I'm somewhat pessimistic about getting new features in setuptools as  
0.7 seems to have fairly grandiose objectives.  There don't seem to  
be incremental feature releases planned. :(


I have a feeling that we/I should learn how to extend setuptools to  
get the features we need.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] New package zc.configure provides an exclude directive for excluding zcml files

2007-10-01 Thread Jim Fulton


On Oct 1, 2007, at 11:17 AM, Jim Fulton wrote:



On Oct 1, 2007, at 11:09 AM, Lennart Regebro wrote:


On 9/29/07, Jim Fulton [EMAIL PROTECTED] wrote:


This helps in cases where you want the configuration for a package,
but you don't want some of the configuration that it includes.  This
allowed me to trim quite a bit off of the startup time, and, more
importantly, the test setup time, for a project I'm working on.


Works like a charm. We tried it here at the grok sprint. Although we
were only able to speed up Grok startup with 10%. Maybe we can get to
20-30% if we work more on it, but we're not sure it's worth it.

http://regebro.wordpress.com/2007/10/01/neanderthal-sprint- 
speeding-up-the-grok-startup/


Maybe grok was already trimmed down.  In my case, I basically  
eliminated all ZMI support (since I didn't need it).  I got about 40%,


Oh BTW, I ran Zope with debug logging. That way I could see what was  
being included.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: reasonable syntax for multi-adaptation

2007-09-28 Thread Jim Fulton


On Sep 28, 2007, at 11:14 AM, Martijn Faassen wrote:
...

On 9/28/07, Jim Fulton [EMAIL PROTECTED] wrote:
[snip]

we could investigate whether we can't come up with something that:

* doesn't break the existing notation. The cleanest way to support
such non-interference seems to be to do this using an extra .adapt
method. This is unfortunate, as I at least consider it prettier if
it used the IFoo() syntax (even though I proposed .adapt).


I agree.


Hm, you agree with my evaluation, but would this mean you'd suggest we
use 'adapt' or extend the __call__ syntax?


In the short-to-medium term the adapt method is my preference.

...


After thinking about it, I think I can avoid the extra hook.  Don't
worry about the details. :)


Cool! The person that tries to implement this will have to worry about
them, though, so we'll get back to you on that, I'm sure.


Yup.

...


Do you suggest we extend zope.component with support for this, or
would you suggest we indeed create a new package?


You'll need to modify InterfaceClass to add the new adapt method and  
__call__ semantics. The hook installed by zope.component will need to  
be updated as well.  I see no new package being warrented.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: reasonable syntax for multi-adaptation

2007-09-28 Thread Jim Fulton


On Sep 28, 2007, at 10:31 AM, Martijn Faassen wrote:


Dominik Huber wrote:
It would be great, if we could handle other adaption-derivations  
by the proposed unified, reasonable adaption-api too.

[snip interesting proposal]

Okay, so there seems to be quite a bit of consensus for *some* form  
of support for this. I've seen a number of proposals which to me  
look quite reasonable.


Since Jim doesn't like this much but there seems to be a lot of  
support from others,


And am willing, therefore, to compromise


we could investigate whether we can't come up with something that:

* doesn't break the existing notation. The cleanest way to support  
such non-interference seems to be to do this using an extra .adapt  
method. This is unfortunate, as I at least consider it prettier if  
it used the IFoo() syntax (even though I proposed .adapt).


I agree.



* creates a small a new hook inside zope.interface to support this


After thinking about it, I think I can avoid the extra hook.  Don't  
worry about the details. :)




* can be plugged in by an optional extension, like  
zope.interfacextended


See above.

* implements the proposed new features. I like the idea of also  
allowing utility lookup through the same mechanism.


I don't, but I'm willing to give in to popular demand.

BTW, we *could* use interface __call__ for utility lookup, as there  
would be no conflict with current notation *if* we specified a  
default using a keyword argument.

I would far prefer using

  IFoo() or IFoo(default=None)

to look up a utility over:

  IFoo.adapt() or IFoo.adapt(default=None)

for, I hope, obvious reasons.

I'd also like to support specifying a default for adapter lookup via  
a keyword argument.  In fact, I'd like to deprecate passing the  
default positionally.  This would open the door, sometime in the  
future, to allowing multi-adaptation via: IFoo(ob1, ob2).



So, any volunteers to try to implement this?


I insist on reviewing any change before checking it in.  This will  
create an unavoidable bottle neck.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Why do we restrict our egg testing?

2007-09-27 Thread Jim Fulton


On Sep 26, 2007, at 7:57 PM, Roger Ineichen wrote:


Hi

Can anybody tell me why we restrict our test setup
in zope eggs and only use a subset of package for
our test setup?


It isn't practical, during development, to test all of the eggs that  
might be affected by a change, which is, BTW a *much* larger set than  
the old Zope 3 tree.


For unit testing of the egg under development, there isn't much point  
in running other tests.  For functional testing, you want to test the  
package's layer and, for an application a set of components that may  
be configured quite differently than the Zope tree.


I think there is value in testing some universe in an off-line way  
using something like buildbot,



Why do we not use a Zope3 meta egg which contains all
our zope packages as a test base. This whould allow
us to test the same we have in the zope3 trunk and let
us run *buildout/test -s zope* from within each egg.


I personally don't see much value in that, especially considering the  
effort involved.



what is the builbot doing right now? Does the builbot
still runs test on the trunk? Or does the buildbot test
the eggs?


Unfortunately, buildbot is a bit of a pita.  :(  It seems to be high  
maintenance.  If someone wants buildbots to run, they should help or  
make it happen.  Christian Theune has a buildbot setup that ran tests  
for all of the projects in the repo.  I think this was *very*  
promising, but I don't know what the current status is.



Is this a bad idea?


Depends on what this is.  I think running all of the tests in the  
old Zope 3 tree whenever we change a component is a waste of time.


If this is having an automated system for testing a wide collection  
of packages to check for dependency breakage, then this is a great  
idea, but potentially a lot of work.


Jim


--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: faulty releases and pypi access [update]

2007-09-27 Thread Jim Fulton


On Sep 26, 2007, at 8:26 PM, Tres Seaver wrote:
...

If we are going to have a change log, which we should, I would prefer
it to be included in source distributions.


I want them present in the *installed* egg, not just in the source
distribution:  setuptools doesn't preserve source distributions after
creating / installing the '.egg' version.


Agreed, but


 Including a file other
that README in the root requires extra effort that I don't want to
require -- writing setup.py files is hard enough as it is.


Put the real README.txt and CHANGES.txt in the actual package:  the
version which is a peer of 'setup.py' should be a stub which points to
the real versions.


I can live with this, but I don't like it.  It feels more complicated  
to me.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Why do we restrict our egg testing?

2007-09-27 Thread Jim Fulton


On Sep 27, 2007, at 10:00 AM, Stephan Richter wrote:


On Thursday 27 September 2007 09:35, Jim Fulton wrote:

On Sep 26, 2007, at 7:57 PM, Roger Ineichen wrote:

Hi

Can anybody tell me why we restrict our test setup
in zope eggs and only use a subset of package for
our test setup?


It isn't practical, during development, to test all of the eggs that
might be affected by a change, which is, BTW a *much* larger set than
the old Zope 3 tree.


I am not saying that testing *all* packages is necessary, but a  
healthy set of
them. We did not consider this impractical when developing on the  
entire
trunk. Back then, it was part of our development responsibility. We  
always

ensured that some set of Zope 3 was always stable together.


But we were developing the entire trunk as a whole.  We were  
developing a monolith. For example, it was acceptable, when changing  
a particular Python package to fix the tests in other packages that  
broke by changing the tests (or the code they test) to reflect the  
change in the original component.  This provided less than no  
protection to 3rd-party packages.


Recently, y'all removed some files from zope.app.security.  I bet you  
adjusted other files in the Zope 3 tree to reflect that change, but  
that didn't help the many more 3rd-party packages and applications  
that were broken.




This is
completely gone now. I want a solution; buildbot is okay,


Good. Now we need someone to implement it.

...


Depends on what this is.  I think running all of the tests in the
old Zope 3 tree whenever we change a component is a waste of time.


I beg to differ. It has something to do with responsibility to the  
community.

I am not saying you have to test all of Zope 3 when changing zc.* for
example. But I think if a package in a defined core set is changed,
requiring to run all the tests of the core set should be  
required. A really
good example is ``zope.component``. Not testing it across several  
packages is

very dangerous for our stability.


That's a good example.  The last time I changed zope.component, I  
*did* test it with the tree.  It would be useful to be able to do  
that conveniently in the future.


What I really want is a buildout for a big Zope 3 application,  
similar to what we've released in the past.  Then, I will often  
choose to test a change in something as core as cope.component as a  
develop egg in that buildout.  I would definitely do that.

(I don't want a meta egg.)

I'll note that I don't work on core components very often so  
testing against the Zope 3 app wouldn't be something that I would do  
often, but I'll concede that on those rare occasions that I do,  
having something like this would be useful.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access [update]

2007-09-27 Thread Jim Fulton


On Sep 27, 2007, at 1:42 PM, Dieter Maurer wrote:


Jim Fulton wrote at 2007-9-26 18:14 -0400:

...

We've just released 1.1. We guess the next release is 1.2.  We change
things and release, 1.2dev-r#. Someone fixes a bug and releases
1.1.1.  Now there's a dev release of 1.2 that is actually older than
the 1.1.1 release but that setuptools considers to be newer.  I think
this is fairly problematic.  Then again, I think that dev releases
are problematic in a lot of ways.


In your example, it is likely, that the fix will also go into the
1.2 development branch. I.e. you will get an 1.2dev-r!
with !  #.


I'm not sure what you mean by my example or why what you said would  
be so.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] faulty releases and pypi access

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 4:16 AM, Christian Theune wrote:
This is IMHO a good example why we shouldn't go for 'everyone can  
make a

release'.


I had the same feeling yesterday. I kept silent because the  
alternative is that only a few -- including me -- make a release. :)   
This deserves more thought though.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Automated egg releases

2007-09-26 Thread Jim Fulton
I'm not too keen on trying to automate this with a Python script.  I  
suggest we start with a human script.  I think Philipp has a start at  
this.  Philipp, could you remind
us where this is?  I suggest we review it and then post it  
prominately somewhere that people (I) can easily find it.  I, for  
one, promise to read it every time I make a release until I've  
memorized it.


Jim

On Sep 26, 2007, at 9:13 AM, Marius Gedminas wrote:


People make mistakes.  Can we reduce the number/severity of those
mistakes by creating a Python script to automate the release  
process as

much as possible?  Things like:

  - check that the version number in setup.py doesn't match an  
existing

release in cheeseshop

  - check whether you have modified but not committed (or unversioned)
files in the source tree

  - check that you've created a tag in subversion for this release  
(or,

alternatively, offer to create the tag for you)

  - build an egg, install it locally in a temporary directory, then  
run

the test suite to check for any missing files or whatever

(a somewhat related idea would be to set up a buildbot to  
check
for new zope-related egg releases and run their test suite  
in a

sandbox, to notice breakages and email the guilty parties)

  - run the correct setup.py command to build/register/upload the  
egg to

cheeseshop and/or www.zope.org

I'd be happy to work on such a script during the sprint, if someone
could help me figure out what exactly it should to do.

Marius Gedminas
--
If something has not yet gone wrong then it would ultimately have been
beneficial for it to go wrong.
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 9:51 AM, Stephan Richter wrote:


On Wednesday 26 September 2007 05:02, Christian Theune wrote:
Hmm. While doing that I also noticed that we were at 3.4.0a1  
yesterday

evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.


There is conflicting information here. :-) Some people say we need  
branches,
others say we don't. And where is an agreed-upon document that you  
have to
list the next version in the setup.py file after the release?  
Because I

disagree with that, since you cannot know the next version.

Regards,
Stephan
--
Stephan Richter
CBU Physics  Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton

Sorry, I decided not to reply and hit the wrong button in my mailer. :)

On Sep 26, 2007, at 9:54 AM, Jim Fulton wrote:



On Sep 26, 2007, at 9:51 AM, Stephan Richter wrote:


On Wednesday 26 September 2007 05:02, Christian Theune wrote:
Hmm. While doing that I also noticed that we were at 3.4.0a1  
yesterday

evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.


There is conflicting information here. :-) Some people say we need  
branches,
others say we don't. And where is an agreed-upon document that you  
have to
list the next version in the setup.py file after the release?  
Because I

disagree with that, since you cannot know the next version.

Regards,
Stephan
--
Stephan Richter
CBU Physics  Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton

This would be a good issue to bring up on the distutils-sig list.

Jim

On Sep 26, 2007, at 9:53 AM, Stephan Richter wrote:


On Wednesday 26 September 2007 05:53, Stefan H. Holek wrote:

WinZip has the habit of ignoring files it deems empty and paths it
deems too long. Best to avoid.


The problem is that Roger has installed TAR and GZIP. They are also  
available
in the path. However, for some reason it uses WinZip. Now, he has  
tested the

eggs on his machine and they worked. Oh man, ...

Regards,
Stephan
--
Stephan Richter
CBU Physics  Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: PLEASE don't remove eggs [was Re: [Zope3-dev] faulty releases and pypi access [update]]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 10:10 AM, Fred Drake wrote:


On 9/26/07, Gary Poster [EMAIL PROTECTED] wrote:

I should make our own private copies of the eggs we use, to
completely isolate us from these sorts of things, but I have not
gotten around to it...nor am I thrilled at the prospect of that
overhead.


This is also a technical issue:  As long as zc.buildout and setuptools
foolishly accept dependency links from an egg, it'll be painful to
detect accidental reliance on external repositories.


That's a good point.  I wouldn't go so far as to say foolishly, but  
I would say that this is a policy that should be overrideable.   
Checkins to buildout (with tests, of course) accepted.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access

2007-09-26 Thread Jim Fulton

Well said.

Jim

On Sep 26, 2007, at 10:10 AM, Philipp von Weitershausen wrote:


Stephan Richter wrote:

On Wednesday 26 September 2007 09:36, Martijn Faassen wrote:

I think tagging things in svn is a minimal requirement, though. I
understood that some of this stuff wasn't tagged?
I agree. And Roger simply forgot. As Marius pointed out, as we do  
more releases, problems like this will occur frequently. Making  
mistakes is human, so let's develop a tool that does some basic  
checking.


I'm not yet convinced that we will have more releases. We have more  
releases now because we're trying to get to stable versions. After  
that, the eggs are on their own and I think we'll make much less  
releases.


But even if we got more releases to do, I don't see the problem.  
With more releases we'd get more practice and make less mistakes.


One of the things we should realize in this process is that -- even  
though setuptools makes it so damn easy to register and uplaod  
stuff to PyPI -- releases are actually important business. They  
can't just be created in a jiffy. They require attention and  
concentration. And a well-defined process.



--
http://worldcookery.com -- Professional Zope documentation and  
training

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 10:10 AM, Martijn Faassen wrote:


Stephan Richter wrote:

On Wednesday 26 September 2007 05:02, Christian Theune wrote:
Hmm. While doing that I also noticed that we were at 3.4.0a1  
yesterday

evening. The stable release was made from that without making a
maintenance branch and bumping the trunk to 3.5.
There is conflicting information here. :-) Some people say we need  
branches, others say we don't.


I think it'd be good to have a tag in any case. A tag can always be  
converted into a branch later.


Yup. I don't think there is disagreement about tags and we don't  
really need agreement on branches.




And where is an agreed-upon document that you have to list the  
next version in the setup.py file after the release? Because I  
disagree with that, since you cannot know the next version.


I disagree with too, for the same reason.

What about using CHANGES.txt, which we should be maintaining anyway?


I agree with a change log.  CHANGES.txt is difficult to get included  
in distributions.  Having one requires a more complex setup.py.  I'd  
rather recommend including a change log in README.txt.





Before making a release, change CHANGES.txt, and convert the section:

unreleased
--

Fixed bugs blah blah.

To:

3.5.1 (2007-09-26)
--

Fixed bugs blah blah.

That is, come up with a release number, a release date, update  
setup.py with the release number, and tag it.


Then, after the release, add in a new section:

unreleased
--

This way checking CHANGES.txt should tell you what's going on with  
releases. If someone forgot to do the last step, you see  
immediately something is wrong, as you want to add your change to  
the 'unreleased' section but there's nothing there yet. You can  
then reconstruct what happened from the cheeseshop or the tags, and  
put in a new 'unreleased' section.


I agree except for the location of the change log.

Jim

P.S. ZODB's NEWS.txt/History.txt aren't workable for me and I plan to  
switch ZODB to using a change log in it's readme.  Unfortunately,  
it's setup is so complex It will take me a largish chunk of time to  
unwind this.


--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 10:42 AM, Stephan Richter wrote:


On Wednesday 26 September 2007 10:40, Jim Fulton wrote:

What about using CHANGES.txt, which we should be maintaining anyway?


I agree with a change log.  CHANGES.txt is difficult to get included
in distributions.  Having one requires a more complex setup.py.  I'd
rather recommend including a change log in README.txt.


-1. I don't like that. We include CHANGES.txt via the long  
description; that's

good enough for me.


Do you think that the change log should be included in the distribution?

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] reasonable syntax for multi-adaptation

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 10:04 AM, Brandon Craig Rhodes wrote:


The current syntax for multi-adaptation makes the interface look like
an object of the adaptation, rather than the actor in the operation.
Instead, multi-adaption should look like this:

  IFoo(multi=(obj1, obj2))

or:

  IFoo(multi=(obj1, obj2), name='site_foo')


Ah, using keyword arguments to get around limitations (especially  
backward compatibility issues) with the current API is a neat idea.


If we were going to do this though, I think a method syntax would be  
cleaner.  As in:


  IFoo.adapt([ob1, ob2], 'site_foo', None)

Note that IFoo(ob) has some special semantics that don't apply to the  
multi- or named-adapter case. In particular:


- The object is returned if it already provides the interface, and

- The object's __conform__ method is used if it is present.

Neither of these make sense in the multi- or named-adapter cases.   
Given the differences in semantics, I wouldn't want to mix the APIs.


An added complication is that interfaces don't provide adaption  
directly, but via a hook. The existing hook api wouldn't work for  
mult or named adaptation, so a new hook would be needed.


While I can see benefit from having an interface method for doing  
multi and named adaptation, I don't think the benefit is worth the cost.


I'm -1 on your proposal and -0 on my variation. :)

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: reasonable syntax for multi-adaptation

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 11:08 AM, Brandon Craig Rhodes wrote:


Jim Fulton [EMAIL PROTECTED] writes:


On Sep 26, 2007, at 10:04 AM, Brandon Craig Rhodes wrote:


Instead, multi-adaption should look like this:

  IFoo(multi=(obj1, obj2))
  IFoo(multi=(obj1, obj2), name='site_foo')


Ah, using keyword arguments to get around limitations (especially
backward compatibility issues) with the current API is a neat idea.
If we were going to do this though, I think a method syntax would be
cleaner.  As in:

  IFoo.adapt([ob1, ob2], 'site_foo', None)


-1.

Unfortunately the singular verb adapt makes it look like normal
adaptation is what is being called for - it looks here like you are
trying to adapt a list to the IFoo interface.

Maybe .multiadapt()?


Note that IFoo(ob) has some special semantics that don't apply to
the multi- or named-adapter case.


Agreed!  The semantics are different.  But mightn't we simply document
this difference between single- and multi-adaptation everywhere we
need to, rather than letting it force us into splitting the adapter
syntax into two unwieldy pieces?


I don't consider either unwieldy.

...


So, I am not sure that I see yet the problem with mixing APIs.


The semantics of the call would change in fundamental ways based on  
the arguments passed. I think this is very bad.  If you disagree,  
sorry. :)




For me, the essential issue is that in both single- and multi-
adaptation you are returned an instance of an adapter that has been
instantiated with the objects you are adapting.  Both of these
syntaxes:

IFoo(x)
IBar(multi=(x,y))


Actually, that is not the case.  If x already provides IFoo, then in  
the first case, the existing object is retuned. Nothing is  
instantiated.  OTOH, in:


  getMultiAdapter([x], IFoo)

or
  getAdapter(x, IFoo)

either there is an error or some factory will be called.  x won't be  
returned unless the factory happens to return it.


...


An added complication is that interfaces don't provide adaption
directly, but via a hook. The existing hook api wouldn't work for
mult or named adaptation, so a new hook would be needed.


I had assumed that IBar(multi=(obj1, obj2)) would simply dispatch to
the existing getMultiAdapter() call; how, exactly, would hooks get
involved and complicate things?  Feel free to just cite a line number
and tell me to go read, all I need is a pointer to get started
understanding this.


zope.interface does *not* depend on zope.component.  It only does  
adaptation the way it does because it provides a hook that  
zope.component fills.  Other people are using zope.interface with  
different adaptation schemes.


Adding multi- or named- adaption support would require a similar hook.

I don't really have time to continue this discussion.  You made a  
reasonable proposal, but for various reasons I've tried to explain, I  
don't support it.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Automated egg releases

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 11:19 AM, Martijn Faassen wrote:


Stephan Richter wrote:
[snip]
Doing another checkout of the tag will create a significant  
overhead to the release process of a package.


I'd like to highlight this. We need to be careful we don't increase  
release overhead too much, otherwise it won't happen/people will  
make mistakes.


+10

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 1:18 PM, Fred Drake wrote:


On 9/26/07, Martijn Faassen [EMAIL PROTECTED] wrote:

What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does  
appear to

be there by default. I checked grok 0.10's tgz and it's there, and we
didn't do anything special.


Do you mean available in the unpacked sdist, or as part of the
installation?  I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed.  If they are, I'd love to know where.


By default READM(.txt) is installed in a source distribution.  
Anything else in the root (aside from setup.py of course and source  
files themselves) aren't without extra setup chants. (These chants  
can be figured out with some effort.  I never remember them, so, if I  
want to do something like this, I have to figure them out again or  
try to find an example with them.)


If we are going to have a change log, which we should, I would prefer  
it to be included in source distributions.  Including a file other  
that README in the root requires extra effort that I don't want to  
require -- writing setup.py files is hard enough as it is.


I'm not crazy about Tres' idea of putting these files in the Python  
packages themselves, but it does have the advantage that it causes  
the files to be included in eggs as well as source distributions.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: reasonable syntax for multi-adaptation

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 3:00 PM, Dieter Maurer wrote:


Jim Fulton wrote at 2007-9-26 11:29 -0400:

...

IFoo(x)
IBar(multi=(x,y))


Actually, that is not the case.  If x already provides IFoo, then in
the first case, the existing object is retuned. Nothing is
instantiated.  OTOH, in:

  getMultiAdapter([x], IFoo)

or
  getAdapter(x, IFoo)

either there is an error or some factory will be called.  x won't be
returned unless the factory happens to return it.


Is this not an irrelevant implementation detail?


No, the specified behavior is different.

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 3:32 PM, Philipp von Weitershausen wrote:


Jim Fulton wrote:

On Sep 26, 2007, at 1:18 PM, Fred Drake wrote:

On 9/26/07, Martijn Faassen [EMAIL PROTECTED] wrote:

What does one need to tell setup.py to make sure CHANGES.txt is
available? I understand it isn't by default, then? Hm, it does  
appear to
be there by default. I checked grok 0.10's tgz and it's there,  
and we

didn't do anything special.


Do you mean available in the unpacked sdist, or as part of the
installation?  I don't think any of README, README.txt, CHANGES,
CHANGES.txt (from the root of the distribution, not from inside the
package) are actually installed.  If they are, I'd love to know  
where.
By default READM(.txt) is installed in a source distribution.  
Anything else in the root (aside from setup.py of course and  
source files themselves) aren't without extra setup chants. (These  
chants can be figured out with some effort.  I never remember  
them, so, if I want to do something like this, I have to figure  
them out again or try to find an example with them.)


I was wrong.  Dang.  I think I understand setuptools/distutils and  
I'm proven wrong again. Sheesh.




Actually, no package data is ever included unless you're either

* working from an svn checkout, in which case setuptools will use  
the list of which files are in svn and which aren't as a hint of  
what to include and what not


Certainly, I expect CHANGES.txt to be in svn.  I've just doeble  
checked that files from the root of the package that are checked into  
svn *are* included in sdists.


I retract my objection to CHANGES.txt and offer my apologies.

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: reasonable syntax for multi-adaptation

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 3:37 PM, Dieter Maurer wrote:


Jim Fulton wrote at 2007-9-26 15:10 -0400:

...

Jim Fulton wrote at 2007-9-26 11:29 -0400:

...

IFoo(x)
IBar(multi=(x,y))


Actually, that is not the case.  If x already provides IFoo,  
then in

the first case, the existing object is retuned. Nothing is
instantiated.  OTOH, in:

  getMultiAdapter([x], IFoo)

or
  getAdapter(x, IFoo)

either there is an error or some factory will be called.  x  
won't be

returned unless the factory happens to return it.


Is this not an irrelevant implementation detail?


No, the specified behavior is different.


Hm. But getAdapter and getMultiAdapter may return x as well
(when the factory decides to do this).

Thus, why is it relevant?


Because they don't take into account what x already provides.  They  
will always call some factory.   Also, they never call __conforms__.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 4:34 PM, Benji York wrote:


Philipp von Weitershausen wrote:

Stephan Richter wrote:
Because I disagree with that, since you cannot know the next  
version.
You can always know the minimum version. If you just released  
3.4.2, I think it's sensible to point the next release to 3.4.3.  
If you later decide that you really need a feature release, you  
can always bump to 3.5.0a1 (which would be the first release in  
the 3.5.x series).


Gary and Benji made me pay attention to this point. :)

I *really* don't like setting the version to the number for the next  
release, since one often doesn't know what it is.



Why not leave the version totally out of the setup.py in the trunk?


Benji and Gary won me over to this point of view. :)

After branching for a release we can set the version (e.g., 1.2),  
make a release, and tag the branch.


We should not require branches.  I would only bother creating  
maintenance branches when they are needed.


Z, B, G, and I propose the following:

- Leave the version # out of setup.py on the trunk and on branches.

When it is time to make a release, either from the trunk or from a  
maint branch:


- Update changes.txt, adding a heading for the new # and date

- Create a tag

- check out or switch to the tag

- Set the version in setup.py on the tag. Check it in.

- Make the release from the tag.

(BTW, when creating maint branches after the fact, the branch should  
be made by copying the trunk at the revision that the first release  
branch for that tag was made, so as not to accidentally get a version  
#.)


I'd prefer not to make an edict, so Benji and Gary will now persuade  
you that this is the best way to go. ;)


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 5:28 PM, Stephan Richter wrote:


On Wednesday 26 September 2007 17:18, Jim Fulton wrote:

- Update changes.txt, adding a heading for the new # and date

- Create a tag

- check out or switch to the tag

- Set the version in setup.py on the tag. Check it in.

- Make the release from the tag.


Changing tags is not that good. I'd rather check in a aversion number
then. ;-)


This is exactly what we've been doing for Zope 3 releases -- for the  
same reason.  I think it is the one acceptable and reasonable change  
to a tag.  The benefit of forcing us to release from a tag is, imo,  
significant.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 5:37 PM, Philipp von Weitershausen wrote:


On 26 Sep 2007, at 23:18 , Jim Fulton wrote:

On Sep 26, 2007, at 4:34 PM, Benji York wrote:


Philipp von Weitershausen wrote:

Stephan Richter wrote:
Because I disagree with that, since you cannot know the next  
version.
You can always know the minimum version. If you just released  
3.4.2, I think it's sensible to point the next release to 3.4.3.  
If you later decide that you really need a feature release, you  
can always bump to 3.5.0a1 (which would be the first release in  
the 3.5.x series).


Gary and Benji made me pay attention to this point. :)

I *really* don't like setting the version to the number for the  
next release, since one often doesn't know what it is.


Maybe. However, when you do the actual release then, you're still  
going to have to find out which version number to use. On release  
branches this is a trivial step, of course: You just look at the  
latest tagged version and increment accordingly. On the trunk it  
might be trickier. After 1.0, do we have 1.1? Or 2.0?


Maye this aspect isn't such a big deal, though.


You can always look at the previous tags or at CHANGES.txt.




Why not leave the version totally out of the setup.py in the trunk?


Benji and Gary won me over to this point of view. :)


There's a *really* good reason for putting the upcoming version  
number in setup.py, though: development eggs.


Good point.

Let's say X depends on Y and I'm developing them simultaneously as  
development eggs in one sandbox (e.g. buildout). I add a feature to  
Y that I'd like to use in X. That means I'll have to change X's  
version dependency regarding Y so that it now depends on Ya.b  
where a.b is the latest release that didn't have the feature I added.


Will Y with the setup.py stating version='unreleased' satisfy the  
Ya.b requirement that X (rightfully) has? If not, then I think we  
have a problem. If yes, then I find that very confusing.


Leaving aside for the moment the fact that develop eggs are used  
for system egg installs :(,


I think if a buildout uses a develop egg, there should be a very  
strong presumption that it it should be used. OTOH, if you have a  
part that depends on a previous major version, then you don't really  
want to use a develop egg for the later release.  This is a bit of an  
edge case though.


I know that if Y's setup.py stated version='a.b-dev', it will work.  
This what my guide currently suggests (including taking it out just  
for release tags).


A problem with this is that someone else might make a release of Y,  
either to fix a bug or to add a feature while your working on the new  
feature in Y.


After branching for a release we can set the version (e.g., 1.2),  
make a release, and tag the branch.


We should not require branches.  I would only bother creating  
maintenance branches when they are needed.


+1


Z, B, G, and I propose the following:


Who's Z? :)


- Leave the version # out of setup.py on the trunk and on branches.

When it is time to make a release, either from the trunk or from a  
maint branch:


- Update changes.txt, adding a heading for the new # and date

- Create a tag

- check out or switch to the tag

- Set the version in setup.py on the tag. Check it in.

- Make the release from the tag.


I could live with that, even with the fact that you'd be modifying  
a tag, as long as it's done in this exact order and the only  
modificdations to a tag woudl be setting setup.py.


I still see the development egg case the best argument for putting  
the next anticipated version number into setup.py. I think the  
compromise of version number + 'dev' marker would work best.


I agree that the develop egg case is problematic.  I'd like to think  
about that.


If you ever actually make dev releases, then guessing wrong about the  
next version could cause real problems.  Consider the following case:


We've just released 1.1. We guess the next release is 1.2.  We change  
things and release, 1.2dev-r#. Someone fixes a bug and releases  
1.1.1.  Now there's a dev release of 1.2 that is actually older than  
the 1.1.1 release but that setuptools considers to be newer.  I think  
this is fairly problematic.  Then again, I think that dev releases  
are problematic in a lot of ways.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re-normalizers instead of ellipses in doctests [was: some checkin]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 5:52 PM, Philipp von Weitershausen wrote:


Marius Gedminas wrote:

Log message for revision 80145:
  Make the test pass on both Python 2.4 and 2.5.


...

-=-
Modified: z3c.form/trunk/src/z3c/form/action.txt
===
--- z3c.form/trunk/src/z3c/form/action.txt	2007-09-26 21:26:25 UTC  
(rev 80144)
+++ z3c.form/trunk/src/z3c/form/action.txt	2007-09-26 21:49:09 UTC  
(rev 80145)

@@ -236,4 +236,5 @@
   ActionErrorOccurred for Action 'cancel' u'Cancel'
 eventlog[-1].error
-  ActionExecutionError wrapping  
zope.interface.exceptions.Invalid ...

+  ActionExecutionError wrapping ...Invalid...
+


It's too bad that Exception's __repr__ changed in Python 2.5. The  
reason is that it's a new-style class now, which is actually a  
benefit at the same time.


Anyway, these ellipses are very ugly and confusing. I think they're  
okay-ish when you know exactly what they're supposed to stand for.  
In this case we can definitely do better.


They also have the disadvantage that they can swallow unexpected  
output.  This is a fairly serious downside.  It would be nice if  
doctest provided a variation on ellipsis that didn't consume newlines.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: faulty releases and pypi access [update]

2007-09-26 Thread Jim Fulton


On Sep 26, 2007, at 6:23 PM, Philipp von Weitershausen wrote:


On 27 Sep 2007, at 00:14 , Jim Fulton wrote:
Let's say X depends on Y and I'm developing them simultaneously  
as development eggs in one sandbox (e.g. buildout). I add a  
feature to Y that I'd like to use in X. That means I'll have to  
change X's version dependency regarding Y so that it now depends  
on Ya.b where a.b is the latest release that didn't have the  
feature I added.


Will Y with the setup.py stating version='unreleased' satisfy the  
Ya.b requirement that X (rightfully) has? If not, then I think  
we have a problem. If yes, then I find that very confusing.


Leaving aside for the moment the fact that develop eggs are used  
for system egg installs :(,


I think if a buildout uses a develop egg, there should be a very  
strong presumption that it it should be used. OTOH, if you have a  
part that depends on a previous major version, then you don't  
really want to use a develop egg for the later release.  This is a  
bit of an edge case though.


It is as long as you ignore the fact that all major Linux distros  
install develop eggs into site-packages.


I already left that aside. :)

Buildout should be able to detect this case and deal with it, which  
is why I left it aside.




I still see the development egg case the best argument for  
putting the next anticipated version number into setup.py. I  
think the compromise of version number + 'dev' marker would work  
best.


I agree that the develop egg case is problematic.  I'd like to  
think about that.


If you ever actually make dev releases, then guessing wrong about  
the next version could cause real problems.


Oh, I'm not actually thinking about dev releases. I too think  
they're very problematic. I just suggested using the 'dev' marker  
as a way to tell branches or the trunk apart from tags.


But people will make these releases and leaving the version out will  
mean that these releases will be harmless.


On the branches and trunk, setup.py's version is the next  
anticipated release + 'dev', while on tags it's just release  
version. That way people won't accidentally make releases from  
branches or the trunk (because they'd get an egg with a 'dev' in  
the version). They'll have to use a tag.


You think getting an egg with dev in the version will stop them?

Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Known working sets II [was: Eggification redux]

2007-09-25 Thread Jim Fulton


On Sep 25, 2007, at 10:04 AM, Martijn Faassen wrote:


Tres Seaver wrote:
[snip]
This is certainly an interesting approach. I'd be curious how you  
would garden this known working set. Martijn makes a pretty good  
case for maintaining such working sets close to the package in  
question (e.g. the grok egg, the Plone egg, etc.).
I would argue that this problem is too big for developer  
convenience
to drive it:  we need concerted effort from the different  
communities
of interest to manage the problem, in much the same way that  
Debian /

Fedora etc. manage their various distribution releases.


I want the ability to make releases of Grok so that when someone  
writes an application against it, it will continue to work, no  
matter what egg releases follow. That's a community of interest, I  
guess? I just want the technical ability to do so, and easily. If  
it's *not* convenient for developers to maintain it, it won't get  
maintained well by the various communities of interest. I think  
it's important to maintain these lists of dependencies close to  
what they talk about, preferably inside the packages themselves.


As far as I know what is required is the ability for grok, in its  
setup.py, to include a list of suggested pinned dependencies  
(besides, and separate from, the normal dependencies). It should  
also be easy to configure buildout to inspect this list. What is  
also required is a way to easily create and maintain this list.


Now I think you're talking about ways to maintain, report and test  
such lists below, but I want to solve my immediate problems first.


I also need this solved preferably today. It's the primary hurt of  
Grok today. Everything else is peanuts.


Of course, if we don't need flexibility to allow application  
developers to diverge from Grok's recommendation, this problem is  
solved today, except for the bit to actually generate the list of  
dependencies. We can simply hardcode them as dependencies in Grok's  
setup.py and tell everybody who wants to use newer versions of eggs  
for whatever reasons in their applications too bad, wait for the  
new release.


I'm skeptical that you are going to get these changes in setuptools.   
I'm not crazy about them myself as they make writing setup files even  
more complicated.  I'd much rather have a way for a comsumer to say:  
Use version V of project P even though it doesn't satisfy a  
dependency.  Basically, a way to override a dependency.  This is  
something that buildout could be taught to do, although *I* don't  
have time to do it today.


OTOH, buildout already provides an alternate solution to this, which  
isn't good enough because you want something that will work w/o  
buildout.  Oh well.


JIm

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Known working sets II [was: Eggification redux]

2007-09-25 Thread Jim Fulton


On Sep 25, 2007, at 11:22 AM, Martijn Faassen wrote:


Hey,

On 9/25/07, Jim Fulton [EMAIL PROTECTED] wrote:
[snip]

I'm skeptical that you are going to get these changes in setuptools.
I'm not crazy about them myself as they make writing setup files even
more complicated.  I'd much rather have a way for a comsumer to say:
Use version V of project P even though it doesn't satisfy a
dependency.  Basically, a way to override a dependency.  This is
something that buildout could be taught to do, although *I* don't
have time to do it today.


That solves the problem too.

Should we then encourage everyone to hardcode version numbers in their
setup.py's dependencies list? If not, framework packages can maintain
such lists.


No, only in application (or very thick framework packages).

This should certainly not be done for library packages.



OTOH, buildout already provides an alternate solution to this, which
isn't good enough because you want something that will work w/o
buildout.  Oh well.


I am not sure what buildout mechanism you're referring to.


A buildout configuration can specify a versions section that  
specified the versions to be used in the buildout.  Other buildout  
configurations can extend the original configuration, specifying  
different versions as desired.,




A version
list over HTTP?


Configurations can be specified as URLs.  Anything that urrlib2  
understand will work. For example, you can include URLs in a buildout  
extends option.


I also don't understand why you say I'm asking for it to work  
without buildout.


You aren't?  Cool. Well, other people have.  When this issue first  
came up, Philipp worked out the buildout solution and others  
complained they wanted something that would work with workingenv,   
I'm not saying that this is an invalid desire.  I'm just pointing out  
that there is a solution of you're willing to use buildout.




The requirement is that not the application developers but the
framework developers are easily able to maintain this list of
dependencies. That is:

* if someone puts the depencency to framework 1.7 in their
dependencies in setup.py, it'll automatically get the pinned
dependencies of this framework.


Well, the buildout solution won't help that.



* if someone tells us, we used framework 1.7, we *know* what
dependencies this implied (unless they did a manual override).


If reusing the framework includes reusing a buildout configuration  
that specifies the versions used by the framework, then buildout can  
help.



So, if I write an application that uses framework 1.7, all I should
have to do is set this in my dependencies list in setup.py. If I then
decide to upgrade to 1.8, that's all I need to adjust. There is no
long list of dependencies for me to maintain, as the framework
developers do this for me. It'd be nice if I had the ability to
override what the framework said, of course.


setuptools doesn't let you do this. Maybe you want to lobby for a  
change on the distutils sig.


Buildout could be enhanced to provide this feature, as I mentioned  
before.




I know you referred us maintaining a buildout versions list on some
HTTP site. This indeed solves most of the above. I don't think this
ideal:

* it doesn't work in a train. Of course, egg downloads don't either,
but at least one tends to have a well-filled eggs directory already.

* it requires the developers to upload a new list to some HTTP site
each time they make a framework release. With eggs, it's nice that a
few setup commands are all you need to do to make a release. So, I'd
prefer to maintain this list in the package the list is talking about,
and to only have a single release artifact (the package itself),
instead of two (the package and the list).

Of course, since it works now, we might want to do this for now
anyway. I just don't think it's ideal.


Agreed.


Would it be possible for buildout to retrieve such a list from an egg
if it's maintained as an extra, made-up setup.py variable? I just
tried this, but it doesn't seem to be stored in egg-info.


In theory, but not today.  Your use case would be better served by  
adding a way to tell buildout to use a package version even if it  
violates some requirements,



Finally, it would be nice to have the ability to maintain multiple
lists of dependencies of significant sub-frameworks, and to have the
ability to combine them in larger frameworks. That is, someone could
be maintaining a Zope3-core package that depends on a whole bunch of
Zope 3 packages, and Grok would then depend on this. I'd be nice if
Grok wouldn't need to maintain its own list of pinned Zope3-core
packages in this case but could rely on the release management and
integrated testing procedure of the package it depends on. This is all
would be nice right now, though.


I think there are lots of potential complexity that all this  
flexibility could add.  I'm not sure how nice that would be.


In any case, you should probably raise this issue

Re: [Zope3-dev] AW: Proposal, free views

2007-09-25 Thread Jim Fulton


On Sep 25, 2007, at 12:49 PM, Philipp von Weitershausen wrote:
...
Note that I've seen checkins that add deprecation warnings. Even  
worse, they're talking about removing stuff in the future. I  
thought we had reached the consensus that we weren't going to  
remove stuff anymore. In other words, that we were never ever going  
to break backwards-compatibility again.


I think we should just not raise any deprecation warnings at all.  
Just the imports for BBB and be done with it.


We said we would only break backward compatibility under dire  
circumstances. :)


I think that deprecation warnings are fine to let people know This  
isn't the way you should be doing this. That other way is better.   
It helps people use packages according to the state of the art, which  
will continue to evolve and it will make developers feel better,  
which is worth something. :)


Also, if we ever *do* need to make a backward-incompatible change to  
a package, we'll have some license to clean up rotten apis.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: zope.testbrowser packaging

2007-09-17 Thread Jim Fulton

+1

Also, extras is a miss-feature.

Jim

On Sep 15, 2007, at 10:43 AM, Philipp von Weitershausen wrote:


Benji York wrote:
I have a small issue with zope.testbrowser packaging I'd like to  
get some input on.  If I were to have started the project today,  
it would likely have been zc.testbrowser, which would have no Zope  
3 dependencies (or functionality) and zc.testbrowser.zope, which  
would have, and depended on zc.testbrowser.  Well, that didn't  
happen, but there are parallels to the current situation that  
might be informative.
There is a configuration bug in testbrowser that means that unless  
you include the test extra, you won't get the Zope 3  
dependencies.  I suspect most people either include that extra, or  
accidentally include the dependencies through other packages.  I  
have two ideas for fixing this:
1) introduce a zope extra that everyone will have to use  
(basically just rename test to zope;
2) take a lesson from the fictional zc.testbrowser and introduce  
another package (zope.testbrowser.zope) that contains the Zope 3  
bits and depends on zope.testbrowser.


I think this would be very hard if not impossible to do from a  
packaging perspective (declaring zope.testbrowser a namespace  
package *and* have it contain things like README, configure.zcml,  
etc.).


I think I prefer the second, despite it's strange appearance.   
Thoughts?


Let's look at this from the beginning. zope.testbrowser contains

a) a reusable, completely Zope-independent test browser

b) integration with zope.app.testing.functional, in other words a test
   browser for testing web applications based on zope.publisher.

I think in its current use, zope.testbrowser is *mostly* used as  
b). When used as a), I don't think anybody is bothered by the fact  
that it might or might not have more dependencies (other than the  
inconvenience of having to install more stuff than actually  
necessary).


So here's what I suggest: Factor out a) to a new package  
'zc.testbrowser' (or whatever) and make 'zope.testbrowser', the  
remaining b), depend on zc.testbrowser, zope.app.testing and all  
that other stuff properly.


That way

- packaging and nomenclature are straight-forward,

- we don't have to break backwards compatibility anywhere,

- people who have used 'zope.testbrowser' because of a) until now  
won't experience any problems, even though we should probably tell  
them to switch to zc.testbrowser.




--
http://worldcookery.com -- Professional Zope documentation and  
training

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] zope.testbrowser packaging

2007-09-17 Thread Jim Fulton
extras are a terrible feature.  They aren't fully supported by  
setuptools and they make it more complicated.  Did you write tests  
for every permutation of your extras?


Jim

On Sep 15, 2007, at 9:44 PM, Stephan Richter wrote:


On Saturday 15 September 2007 08:48, Benji York wrote:

1) introduce a zope extra that everyone will have to use (basically
just rename test to zope;


I prefer this solution. I have done this before for z3c.rml, where  
I put the
page template support into a pagetemplate extra declaration. I  
liked this
solution a lot. I would have never considered developing another  
package for
a fairly trivial extension of a few lines. I think eggs have the  
potential

for package proliferation and senseless overhead.

Regards,
Stephan
--
Stephan Richter
CBU Physics  Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Transaction bound cache

2007-09-17 Thread Jim Fulton
No, that would introduce a dependency on zodb.  I suggest a separate  
package.


Jim

On Sep 17, 2007, at 6:03 AM, Christian Zagrodnick wrote:


Hi

i've got a very simple transaction bound cache implementation. That  
is the cache gets invalidated on transaction end.


It's used like this:

class Foo(object):

  data = TransactionBoundCache('_v_store_it_here', dict)

where `dict` is the cache factory.


Shall I add this to zope.cachedescriptors?

--
Christian Zagrodnick

gocept gmbh  co. kg  ·  forsterstrasse 29 · 06112 halle/saale
www.gocept.com · fon. +49 345 12298894 · fax. +49 345 12298891



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: What does python 3000 mean for zope?

2007-09-13 Thread Jim Fulton


On Sep 11, 2007, at 10:28 AM, David Pratt wrote:

I was hoping Jim might respond to this thread since I am certain  
there is concern about what this means for the future of Zope. I am  
hoping that core communities of python framework developers may  
come together on what is in their best interests.


OK, here's my opinion.  I am not speaking for ZC.  I don't think we  
should worry until Python 3 is much farther along.  I agree with  
those who think that it will be hard to move to Python 3.  Put  
another way, I think the benefit won't be worth the effort.  My  
suspicion is that this will be true for *lots* of projects.  I expect  
Python 2 to be with us for a long time. This is just a wild guess.  A  
lot can change in a year or 2.  I think our official position should  
be that Zope is a Python 2 application and we'll evaluate  
opportunities to leverage Python 3 over time as they present themselves.


I'm not going to reply to any replies to this post as I don't intend  
to spend any more time on this issue myself.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] New Contributor References

2007-09-12 Thread Jim Fulton


Lately, I've started being a bit more careful about accepting new  
contributors.  In particular, I'm requesting that new contributors  
provide the name of an existing contributor who is familiar with them  
and their work.  I (or someone) really need to add this to the  
contributor agreement, but I'm not sure when I'll have time.  Now,  
It's taking a fair bit of time writing new contributors to ask for  
references. I encourage folks to request committer access, I just ask  
that people provide the names of one or more existing committers who  
can vouch for them.


Thanks.

Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: RFC: Known working sets

2007-09-06 Thread Jim Fulton


On Sep 6, 2007, at 4:02 AM, Chris Withers wrote:


Martin Aspeli wrote:

Jim Fulton wrote:
I'm very much against making setuptools *more* complicated  
than it already is.
Indeed, but surely managing known good sets of components  
comes under its remit of version management?

Sure.  It does this via requirements.


Ok, forgive me for being dumb then, but why are we looking to add  
similar to zc.buildout?
We're talking more about a general pattern in zc.buildout. The  
deficiencies of using setup.py for this alone are described well  
in the original proposal.


Yup, and this was the reason for my original question to Jim: why  
do something in zc.buildout rather than fixing the problems with  
setuptools?


Because neither the problem nor the fix are well understood, imo, and  
setuptools is already too complicated.


Perhaps the same could be said about buildout, but no new buildout  
features are needed to experiment with the issue at this point.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: RFC: Known working sets

2007-09-05 Thread Jim Fulton


On Sep 5, 2007, at 10:48 AM, Chris Withers wrote:


Jim Fulton wrote:
I'm very much against making setuptools *more* complicated than it  
already is.


Indeed, but surely managing known good sets of components comes  
under its remit of version management?


Sure.  It does this via requirements.

Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: broken zope.lifecycleevent 3.4.0 on cheeseshop?

2007-09-04 Thread Jim Fulton


On Sep 2, 2007, at 8:20 AM, Benji York wrote:


Baiju M wrote:
BTW, do we need to upload eggs to http://download.zope.org/ 
distribution/

any more ?


If we feel that depending on PyPI to install zope packages is OK,  
then we can decommission /distribution/.  For the commercial  
projects I'm involved in we make sure we have private copies of  
everything we depend on, so I don't have a desire to keep / 
distribution/ around.


I like distribution as a place to put packages that aren't ready for  
PyPI. Unfortunately, we've put a lot of packages in PyPI that aren't  
ready IMO.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: RFC: Known working sets

2007-09-04 Thread Jim Fulton


On Sep 3, 2007, at 4:13 PM, Philipp von Weitershausen wrote:


Wichert Akkerman wrote:
The only problem is that distributing grok-0.11.cfg is a bit  
tedious. How about if buildout could get it from the web?

How about making it available from an egg, through a hook in egg-info
perhaps?


This is a very good point. So good in fact that I thought of it  
myself :) I was already writing the email when your message came in.



Martijn and I discussed the known working set problem over IRC this  
afternoon. He brought up a few good points which suggest that the  
version data could be associated with the egg:


The approach that I described in my original posting (and which  
actually works *today*, as I found out!) has some disadvantages.  
For instance, the discoverability and release mechanism of the  
known working set configuration file differs a lot from our normal  
packages. Releasing a package is a well-known routine by now. How  
and where would we release the working set configuration file? SVN?


Another problem are dependencies and how we'd like to depend on  
other working sets. Let's say we made a 'Zope' working set that  
contained the stable versions of the zope.* packages. It would make  
sense for grok to depend on this information. And packages using  
grok should depend on that. It'll be complicated to maintain such a  
chain in separate text files, especially across version updates. It  
only seems natural to use the egg dependency mechanism for this.


So, a possible solution is to associate the known working version  
info with an egg. More to the point, an egg could -- in addition to  
simply listing the names of its dependencies -- also specify which  
versions of those eggs it works best with. Wichert and I suggest  
that this could be put in a file in the EGG-INFO directory.


The only problem is that we usually don't version control egg-info  
directories. That means the information needs to be contained or at  
least referenced in setup.py and then written upon setup.py  
egg_info. Here's what setup.py *could* look like::


  known_working_versions = {
'ZODB3': '3.8.0',
'zope.component': 3.4.0,
...
  }

  setup(
  name='grok',
  version='0.11',
  ...
  install_requires=known_working_versions.keys(),
  known_working_versions=known_working_versions
  )

When EGG-INFO is written, a custom writer will then take this  
information and generate 'known_working_versions.txt' or whatever  
in EGG-INFO. The only problem is that this custom writer needs to  
be installed when setup.py is called to create egg, in other words  
to generate the right kind of eggs you'd need to have something  
installed first. Not sure if this is better than maintaining .egg- 
info directories in SVN...


How would the known_working_versions be used?  You haven't  
specified that.  Presumably this would be something that is  
overridable. How would this be overridden?


I'm very much against making setuptools *more* complicated than it  
already is.


Perhaps buildout (and setuptools) should grow a mechanism for being  
able to override/resolve version conflicts.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] ZConfig 2.4 final?

2007-08-30 Thread Jim Fulton
I suggest making a 2.5 (final) release and being done with it.  My  
alpha releases were undoubtedly in ignorance of the existing tag.


Jim

On Aug 30, 2007, at 11:24 AM, Philipp von Weitershausen wrote:

ZConfig 2.4 currently seems to be in alpha mode. Are we going to  
make a ZConfig 2.4 final for the Zope 3.4 release?


And what about the 2.4 tag that's in subversion? The name  
suggests that it's a 2.4 final release, but it can't be because Jim  
created that tag 11 months ago (before any of the 2.4 alphas were  
created) with the comment Make tag for Zope 3.3 release. See  for  
yourselves at http://svn.zope.org/ZConfig/tags/. What to do?



--
http://worldcookery.com -- Professional Zope documentation and  
training

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Guide for maintaining software in the Zope repository

2007-08-27 Thread Jim Fulton


On Aug 23, 2007, at 8:37 PM, Philipp von Weitershausen wrote:

We have 100+ packages that make up what used to be distributed as  
Zope3. We have numerous more packages in svn.zope.org. Most of  
them are developed, released and distributed individually. We like  
to think this is a good thing (I certainly do). But currently we  
have a bit of a chaos [2]. It's not bad, but I fear without some  
guidance, it'll get worse.


Christian Theune recently wrote a document [1] in which he outlined  
how we should get to a development process and what topics it  
should touch.  This document is very hands-on and describes actions  
that should be taken to reach these goals. I've taken the liberty  
to jump ahead and write down some current practices:


http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/ 
maintaining-software.txt


This is a great start. Thanks!

A few small points:

- I'm going to mostly stay out of the style debate except to note  
that the Zope style guide builds on PEP8.  It doesn't disagree with  
it much accept in the case of some naming, due to the fact that the  
ZSG made a commitment before PEP8 did.


- On doctest, there should be greater emphasis on there being 2 kinds  
of tests, executable documentation and other tests.  I think there is  
value in executable documentation, but it should be documentation  
first.  A lot of our doctests that we think/wish are documentation  
are not very good documentation.  We need to do a better job of this.


I do feel strongly that even non-documentation tests should be  
written in a fairly literate way with documentation of the test  
itself,  I strongly prefer the doctest format for these tests, but I  
don't want to be an evil dictator about it. I suggest that classic  
unit tests can be used for new tests, but *only* if they are well  
documented.  I've never seen a classic unit test that was, but I'm  
open to the theoretical possibility. :)

BTW, I've seen poorly documented doctests too.

Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Please don't use svn.zope.org urls in distutils homepage urls

2007-08-27 Thread Jim Fulton


On Aug 25, 2007, at 4:59 AM, Lorenzo Gil Sanchez wrote:

Thanks for replying.  I meant to make another reply but fogot.


Well, probably I was to fast predicting the causes of my problem. They
only thing I know is that a couple of months ago I could browse the
repository without any problem and suddenly I start getting a default
Apache page and a lot of permission denied errors while trying to  
do the
same. I'm sure I was browsing the same URL (http://svn.zope.org)  
since I

had it on my bookmarks.

Oddly enough, after this message thread I can browse the repository
again from the device and I swear I didn't change any proxy
configuration or anything at all.


Your client id was being blocked because it was used by a poorly- 
behaved spider. It was blocked to block the spider.  We'ev unblocked  
it and are hoping that the spider doesn't come back.  If it does,  
we'll find some other way to block it.  In any case, it had nothing  
to do with blocking setuptools.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] __repr__ of ValidationErrors in zope.schema

2007-08-24 Thread Jim Fulton


On Aug 24, 2007, at 4:00 AM, Christian Zagrodnick wrote:


Hi

I'm implementing the getValidationErrors thingy right now and once  
again stumbled upon the ValidationErrors. Their __repr__ is all but  
useful.


For instance TooSmall:


TooSmall(8, 10)

8 10

Another sort of related issue is that you only get the __doc__  
string when calling the .doc() method. Value is too small.  
doesn't help a lot.


Something like The value 8 is too small. At least 10 is required.  
would be much more informative.


What should we do about this?


While better __str__s or __reprs__ is often nice, as a general  
principal, errors should be displayed through adaptation -- basically  
views.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Please don't use svn.zope.org urls in distutils homepage urls

2007-08-24 Thread Jim Fulton


On Aug 24, 2007, at 5:07 AM, Lorenzo Gil Sanchez wrote:


Aha! After reading this message now I finally understand the reason I
can't browse svn.zope.org with my Nokia N800[1] anymore.


I don't see what one has to do with the other.

This used to be quite useful since I could read lots of *.txt  
documents
from zope3 and related projects just before sleeping. I wish I  
could do

it again.


Nothing we've done should get in the way of your Nokia N800.


I've setup an Apache server just to see what user-agent my device is
giving and this is what came out from the log:

192.168.1.15 - - [24/Aug/2007:11:00:32 +0200] GET / HTTP/1.1 403  
3926
- Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux armv6l; U) Opera  
8.5

[es_ES] Tablet browser 0.0.14 RX-34_2007SE_4.2007.26-8

So please, adjust the Apache configuation so N800 users can browse the
repository again.


We are only excluding setuptools. Not any other clients.

Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Leak in zope.component?

2007-08-23 Thread Jim Fulton


On Aug 23, 2007, at 1:53 AM, Christian Zagrodnick wrote:


On 2007-08-22 16:12:42 +0200, Jim Fulton [EMAIL PROTECTED] said:


On Aug 22, 2007, at 8:16 AM, Christian Zagrodnick wrote:

Hi,
I'm doing things wich z3c.zalchemy and trying to find out why my   
database connections are kept open  even after unregistering   
everything.

Given the following very simple test:

import sys
import zope.component
import zope.interface
class IUtil(zope.interface.Interface):

... pass

gsm = zope.component.getGlobalSiteManager()
utility = object()
sys.getrefcount(utility)

2

gsm.registerUtility(utility, IUtil)
sys.getrefcount(utility)

6

gsm.unregisterUtility(utility, IUtil)

True

sys.getrefcount(utility)  # this fails

2
 
--
File /Users/zagy/development/z3c.zalchemy/src/z3c/zalchemy/ 
tests/ dispose.txt, line 17, in dispose.txt

Failed example:
   sys.getrefcount(utility)  # this fails
Expected:
   2
Got:
   4
So there are now two more references than before register/  
unregister. Am I missing something? Or is it leaking somewhere?

Here's what I get with the trunk of zope.component:
[EMAIL PROTECTED]:~/p/zope/component/trunk$ bin/py
  import sys, zope.component, zope.interface
  class IUtil(zope.interface.Interface): pass
...
  gsm = zope.component.getGlobalSiteManager()
  utility = object(); sys.getrefcount(utility)
2
  gsm.registerUtility(utility, IUtil); sys.getrefcount(utility)
5
  gsm.unregisterUtility(utility, IUtil); sys.getrefcount(utility)
True
2
So I can't reproduce what you are seeing.
Try adding:
 import gc; gc.collect()
before your last getrefcount call.


That doesn't change a thing. Could it be the testrunner having an  
eye on registrations?


I don't know what you mean.  I ran my example interactively.  What  
happens when you run my example interactively?


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: SVN: zope.location/trunk/setup.py using pypi as homepage instead of svn.zope.org

2007-08-22 Thread Jim Fulton


On Aug 20, 2007, at 8:22 PM, Stephan Richter wrote:


On Saturday 18 August 2007 17:03, Philipp von Weitershausen wrote:

* Please also don't forget to add a changelog entry in README.txt,
especially if you're adding features. If there's no README.txt  
yet, this

is a good time to give it one. It should start out with a simple
paragraph and have at least one section called Changes. You  
could then

use its contents as the long_description in setup.py. Other packages
(e.g. zope.proxy or zope.publisher) can serve as examples.


Why do the changes have to be part of the README file? That seems  
no good. I

think a separate file is much better.


If you use a separate file, and you are going to create a source  
distribution, then you need to have extra magic in your setup.py to  
include CHANGES.txt in the distribution.  IMO, setup magic is bad.   
I'd prefer to see simpler setup.py files and thus I prefer to include  
the change log in README.txt.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Leak in zope.component?

2007-08-22 Thread Jim Fulton


On Aug 22, 2007, at 8:16 AM, Christian Zagrodnick wrote:


Hi,

I'm doing things wich z3c.zalchemy and trying to find out why my  
database connections are kept open  even after unregistering  
everything.


Given the following very simple test:


import sys
import zope.component
import zope.interface
class IUtil(zope.interface.Interface):

... pass


gsm = zope.component.getGlobalSiteManager()
utility = object()
sys.getrefcount(utility)

2

gsm.registerUtility(utility, IUtil)
sys.getrefcount(utility)

6


gsm.unregisterUtility(utility, IUtil)

True

sys.getrefcount(utility)  # this fails

2

--
File /Users/zagy/development/z3c.zalchemy/src/z3c/zalchemy/tests/ 
dispose.txt, line 17, in dispose.txt

Failed example:
   sys.getrefcount(utility)  # this fails
Expected:
   2
Got:
   4



So there are now two more references than before register/ 
unregister. Am I missing something? Or is it leaking somewhere?


Here's what I get with the trunk of zope.component:

[EMAIL PROTECTED]:~/p/zope/component/trunk$ bin/py

 import sys, zope.component, zope.interface
 class IUtil(zope.interface.Interface): pass
...
 gsm = zope.component.getGlobalSiteManager()
 utility = object(); sys.getrefcount(utility)
2
 gsm.registerUtility(utility, IUtil); sys.getrefcount(utility)
5
 gsm.unregisterUtility(utility, IUtil); sys.getrefcount(utility)
True
2

So I can't reproduce what you are seeing.

Try adding:

import gc; gc.collect()

before your last getrefcount call.

Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] We should be able to stop unhiding PyPI releases.

2007-08-22 Thread Jim Fulton


As of Monday's zc.buildout release, buildout now uses the simple  
package index:


  http://cheeseshop.python.org/simple

by default.  (You can cause it to use another index by setting the  
buildout index option.)


This change should provide significantly better performance and will  
make it easier to use historical releases, because the simple  
interfaces shows releases that have been hidden from the human  
interface.


Normally, when new releases are registered with PyPI, old releases  
are hidden.  This has been a problem for buildouts or application  
packages that fix package versions, because hidden releases weren't  
visible to setuptools.  To get around this, we had to unhide old  
releases, which was a pain.  The simple package index shows all  
releases, including hidden ones.  Now that buildout uses the simple  
index by default, hidden packages should no longer be a problem, at  
least for buildout users.  People using easy_install can configure it  
to use the simple index.  They might want to lobby to have  
easy_install use the simple index by default.


If there are no strong objections, I plan to stop unhiding old releases.

Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: We should be able to stop unhiding PyPI releases.

2007-08-22 Thread Jim Fulton


On Aug 22, 2007, at 10:50 AM, Tres Seaver wrote:
...

I would argue that hiding old releases is a major misfeature of PyPI,
personally:  humans probably have as much to gain from seeing the  
older

versions as scripts.


Agreed, although the UI degrades quite a bit when there are multiple  
unhidden releases.  PyPI could use quite a bit of improvement.  The  
place to get involved is the catalog sig. :)




  Even brown bag releases should remain available
(albeit, with perhaps some way to tag them as problematic).


Hidden releases are still available.

I certainly think there is value in hiding many old releases,  
especially non-final ones, as long as the releases are still  
available for download.



Releasing a software package is a contract which implies more than
flavor of the week status, at least in lots of circles.


It is important that releases remain available and finable by  
setuptools.  The simple index achieves that.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Backward compatibility and major releases (series) update

2007-08-22 Thread Jim Fulton


A while ago, we had a discussion about backward compatibility and  
decided that only major releases should be backward compatible.  So,  
for example, a 1.2 release should be backward compatible with a 1.1  
release, but a 2 release could be backward incompatible with a 1  
release.  We then said we wanted a nice way to spell depending on a  
major release.  (The current way to spell depending on a major  
release is foo =R.dev R.dev, where foo is the project name and R  
is the major release number.)


We recently had an opportunity to experience this with  
zc.relationship. zc.relationship 2 was released and some packages  
were updated to require zc.relationship 1.  Unfortunately, not all of  
the packages we use were updated and this caused version conflict  
errors.  (This was partly a result of setuptools weak conflict  
resolution algorithm.) My initial response was, oh, we need to  
update all of the other packages that depend on zc.relationship to  
require version 1.  But then I started wondering how we would  
migrate to version 2 and realized that it was going to be rather  
hard.  All of the dependent packages would have to move in lock step  
and we'd be back to a monolith.  This was enough to make me think  
that backward incompatible changes are just untenable.  I gave a hint  
to this in some later email threads.


Since then, I've looked at a number of packages that we've split out  
from Zope that have excessive dependencies.  zope.component is a  
great example.  The excessive dependencies (at least the hard ones to  
deal with) are a result of poor factoring of functionality at a time  
when dependencies didn't matter.  Unfortunately, I think the only way  
to fix some of these is to split off functionality, which will  
introduce backward incompatibility.


I eventually came to the conclusion that our original conclusion was  
sound, but that we should only introduce backward incompatibilities  
when the need is very dire, as it will cause lots of pain.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: Backward compatibility and major releases (series) update

2007-08-22 Thread Jim Fulton


On Aug 22, 2007, at 4:15 PM, Tres Seaver wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Jim Fulton wrote:

A while ago, we had a discussion about backward compatibility and
decided that only major releases should be backward compatible.  So,
for example, a 1.2 release should be backward compatible with a 1.1
release, but a 2 release could be backward incompatible with a 1
release.  We then said we wanted a nice way to spell depending on a
major release.  (The current way to spell depending on a major
release is foo =R.dev R.dev, where foo is the project name and R
is the major release number.)

We recently had an opportunity to experience this with
zc.relationship. zc.relationship 2 was released and some packages
were updated to require zc.relationship 1.  Unfortunately, not all of
the packages we use were updated and this caused version conflict
errors.  (This was partly a result of setuptools weak conflict
resolution algorithm.) My initial response was, oh, we need to
update all of the other packages that depend on zc.relationship to
require version 1.  But then I started wondering how we would
migrate to version 2 and realized that it was going to be rather
hard.  All of the dependent packages would have to move in lock step
and we'd be back to a monolith.  This was enough to make me think
that backward incompatible changes are just untenable.  I gave a hint
to this in some later email threads.

Since then, I've looked at a number of packages that we've split out
from Zope that have excessive dependencies.  zope.component is a
great example.  The excessive dependencies (at least the hard ones to
deal with) are a result of poor factoring of functionality at a time
when dependencies didn't matter.  Unfortunately, I think the only way
to fix some of these is to split off functionality, which will
introduce backward incompatibility.

I eventually came to the conclusion that our original conclusion was
sound, but that we should only introduce backward incompatibilities
when the need is very dire, as it will cause lots of pain.


+1.  Cleanliness is not a good enough reason to break a public API,
for instance.  If necessary, the incompatible stuff might be better
off moving to a new package / API name altogether, with the old name
left as a pure compatibility shim (perhaps with evergreen  
deprecation

warnings).


Yup, which is what Gary is doing with zc.relationship.

Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: SVN: zope.location/trunk/setup.py using pypi as homepage instead of svn.zope.org

2007-08-21 Thread Jim Fulton


On Aug 21, 2007, at 2:34 AM, Wichert Akkerman wrote:
...
If you really want to you can combine both files in the  
long_description

from your setup.py. That can make your cheesehop package page overly
large though.


How so?  I think as long as a the long description has a table of  
contents at the top, I don't think that size is an issue.


Note that with the latest version of buildout, buildout uses the new  
simple index, http://cheeseshop.python.org/simple by default, so  
there is no longer a buildout performance penalty for having large  
cheeseshop pages.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Add function for schema validation in zope.schema

2007-08-20 Thread Jim Fulton


On Aug 20, 2007, at 7:48 AM, Christian Zagrodnick wrote:


Hi,

zope.schema defines how a schema can be validated, defines errors  
and provides a way to validate single fields.


I think we should add a function to validate a given schema on a  
given class. This should include constraints and invariants:


validateSchema(IMySchema, myobject)  [or alike]

I'm not sure about return values or exceptions. There are different  
use cases:


1. Is the object valid? - True/False
2. What's wrong with the object - {attribute/field-name: what's  
wrong}, what's wrong with invariants.


Comments?


I remember discussing this a year or more ago.  I don't remember where.

I'd like to see this fit into the zope.interface.verify framework.   
I'd like to see the verify module refactored to be pluggable and I'd  
like to see schema versification plug into this.  This would *also*  
motivate (or at least enable) people to do better method verification  
and possibly provide other kinds of verification.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] browser:page's for= can be class browser:menuItem can't

2007-08-20 Thread Jim Fulton


On Aug 20, 2007, at 9:45 AM, Christian Zagrodnick wrote:


Hoi

I just figured, that the browser:menuItem cannot be registered for  
classes. But you can with browser:page. I think if a browser:page  
can be registered for a class a menuItem should support this, too.


Also a browser:page for=someclass menu=... / will break on  
getting the menu items because the implementation doesn't support  
classes.


Should we fix that?


Yes. Any time we have a for, we should allow interfaces or classes.

BTW, you may soon discover what I discovered the other day, which is  
that the tiny menu framework used for zmi_views doesn't work with  
menu items registered for classes.  Someone should fix that too. :)


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Heads up: major change in zc.buildout policy for selecting distributions

2007-08-20 Thread Jim Fulton


I'm about to make a new release of zc.buildout that uses a different  
policy for selecting distributions.  In particular, by default,  
zc.buildout will now prefer final distributions over non-final ones.   
If there are final and non-final distributions that satisfy a  
requirement, buildout will, by default, select a final distribution  
even if there are non-final distributions that satisfy a requirement.


This release will cause many existing buildouts to use older  
distributions than they do now.


You can change this behavior by providing a prefer-final buildout  
option:


  [buildout]
  prefer-final = false

Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Heads up: major change in zc.buildout policy for selecting distributions

2007-08-20 Thread Jim Fulton


On Aug 20, 2007, at 2:02 PM, Martin Aspeli wrote:


Jim Fulton wrote:
I'm about to make a new release of zc.buildout that uses a  
different  policy for selecting distributions.  In particular, by  
default,  zc.buildout will now prefer final distributions over non- 
final ones.   If there are final and non-final distributions that  
satisfy a  requirement, buildout will, by default, select a final  
distribution  even if there are non-final distributions that  
satisfy a requirement.
This release will cause many existing buildouts to use older   
distributions than they do now.
You can change this behavior by providing a prefer-final buildout   
option:

   [buildout]
   prefer-final = false


How will this be versioned?


It is 1.0.0b29.

(The buildout version numbers are a bit of a mess, as I was  
overoptimistic when I made the initial beta  release.  Hm, maybe I  
should start using es.  These could be eek release. ;)


It'd be helpful if it were a different major (or at least minor)  
version than the current one, to make it easier for people to stick  
with a version that has the policy they normally use.


Maybe I should have, but the release is out of the barn.

If this ends up causing too much pain, I could back this out by  
making a b30 release with the old policy, although, I'd rather not.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: Heads up: major change in zc.buildout policy for selecting distributions

2007-08-20 Thread Jim Fulton


On Aug 20, 2007, at 2:18 PM, Jim Fulton wrote:



On Aug 20, 2007, at 2:02 PM, Martin Aspeli wrote:


Jim Fulton wrote:
I'm about to make a new release of zc.buildout that uses a  
different  policy for selecting distributions.  In particular, by  
default,  zc.buildout will now prefer final distributions over  
non-final ones.   If there are final and non-final distributions  
that satisfy a  requirement, buildout will, by default, select a  
final distribution  even if there are non-final distributions  
that satisfy a requirement.
This release will cause many existing buildouts to use older   
distributions than they do now.
You can change this behavior by providing a prefer-final  
buildout  option:

   [buildout]
   prefer-final = false


How will this be versioned?


It is 1.0.0b29.

(The buildout version numbers are a bit of a mess, as I was  
overoptimistic when I made the initial beta  release.  Hm, maybe  
I should start using es.  These could be eek release. ;)


It'd be helpful if it were a different major (or at least minor)  
version than the current one, to make it easier for people to  
stick with a version that has the policy they normally use.


Maybe I should have, but the release is out of the barn.

If this ends up causing too much pain, I could back this out by  
making a b30 release with the old policy, although, I'd rather not.


No, I'm wrong, I'm going to back these changes out with a b30 release  
and then release them with a 2x release.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: Heads up: major change in zc.buildout policy for selecting distributions

2007-08-20 Thread Jim Fulton


On Aug 20, 2007, at 1:18 PM, Jim Fulton wrote:
I'm about to make a new release of zc.buildout that uses a  
different policy for selecting distributions.  In particular, by  
default, zc.buildout will now prefer final distributions over non- 
final ones.  If there are final and non-final distributions that  
satisfy a requirement, buildout will, by default, select a final  
distribution even if there are non-final distributions that satisfy  
a requirement.


This release will cause many existing buildouts to use older  
distributions than they do now.


You can change this behavior by providing a prefer-final buildout  
option:


  [buildout]
  prefer-final = false


I decided, with some prompting from Martin Aspeli, that this would be  
too disruptive.  The latest release of zc.buildout still supports  
newest releases.  To prefer final releases, you have to set the  
prefer-final option to true.  In zc.buildout version 2,  final  
releases will be prefered by default.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] 'thread_local' AttributeError in zope.security.checker

2007-08-15 Thread Jim Fulton


On Aug 14, 2007, at 7:32 PM, Paul Carduner wrote:


Hello zope3-dev,

I came across an error I have not seen before.  Just for reference,
here is the last snippet from the traceback:

File /home/pcardune/Work/ZContact/zcontact-lp/eggs/tmpgwuq6O/ 
zope.security-3.4.0b4-py2.4-linux-i686.egg/zope/security/checker.py,

line 420, in ?
AttributeError: 'module' object has no attribute 'thread_local'


I wish you had included the whole traceback.



The interesting part to me is that it was so easy to fix.  Here is a
snippet of the relevant area of failing code from
zope/security/checker.py

# Get optimized versions
try:
import zope.security._zope_security_checker
except ImportError:
pass
else:
from zope.security._zope_security_checker import _checkers,  
selectChecker

from zope.security._zope_security_checker import NoProxy, Checker
from zope.security._zope_security_checker import _defaultChecker
from zope.security._zope_security_checker import  
_available_by_default

zope.interface.classImplements(Checker, INameBasedChecker)

The very first import statement is throwing the error, but alas, it is
not an ImportError.  If I patch this code by catching the
AttributeError in the same way the ImportError is caught, then
everything works fine.


Except you don't get the C optimizations. IOW, you would just be  
covering up a real error.




I tried looking into the c code, but after
years of python development, the c code just frightens me (a bit).
That said, I would really appreciate any pointers people might be able
to give me on what thread_local is for and why I (don't) need it.


I rearranged the imports in zope.security yesterday.  I apparently  
broke your usage, but I can't tell what your usage is.  I'll need to  
see the whole traceback.  In particular, I'll need to see the import  
order.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



[Zope3-dev] Re: 'thread_local' AttributeError in zope.security.checker

2007-08-15 Thread Jim Fulton


On Aug 15, 2007, at 10:34 AM, Tres Seaver wrote:
...
We badly need to get a handle on how we expect people to fetch eggs  
for
packages:  having such a refactoring affect an indirect client (one  
not

expecting to track day-to-day development) is a disaster.


I'm not sure disaster is the word I'd use, but in general, I agree.


Dumping revision-stamped (unreleased) eggs into a directory used by
unsuspecting users is a recipe for bringing *all* development to a
screeching halt.


Yeah, I need to release the buildout changes that makes buildout  
prefer released eggs.


Note that in this particular case, the real problem was circular  
imports.  Circular imports create bugs that are hard or impossible to  
test for.  I'd love to outlaw them.  I wish we at least had a tool to  
warn about circular imports when testing.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] 'thread_local' AttributeError in zope.security.checker

2007-08-15 Thread Jim Fulton

I've just released a b5 egg which should correct the problem.

Jim

On Aug 15, 2007, at 9:51 AM, Wichert Akkerman wrote:


Previously Jim Fulton wrote:


On Aug 14, 2007, at 7:32 PM, Paul Carduner wrote:


Hello zope3-dev,

I came across an error I have not seen before.  Just for reference,
here is the last snippet from the traceback:

File /home/pcardune/Work/ZContact/zcontact-lp/eggs/tmpgwuq6O/
zope.security-3.4.0b4-py2.4-linux-i686.egg/zope/security/ 
checker.py,

line 420, in ?
AttributeError: 'module' object has no attribute 'thread_local'


I wish you had included the whole traceback.


Here you are:

[snow;/tmp/test]-119 bin/zopectl fg
/tmp/test/parts/app/runzope -C /tmp/test/parts/zopectl/zope.conf
Traceback (most recent call last):
  File /tmp/test/parts/app/runzope, line 97, in ?
import zope.app.twisted.main
  File /home/wichert/buildout-eggs/tmp-GMMRj/ 
zope.app.twisted-3.4.0b1_r76119-py2.4.egg/zope/app/twisted/ 
main.py, line 32, in ?
  File /usr/lib/python2.4/site-packages/PIL/__init__.py, line 22,  
in ?


  File /home/wichert/buildout-eggs/tmpzGONPg/ 
zope.app.appsetup-3.4.0a1-py2.4.egg/zope/app/appsetup/appsetup.py,  
line 25, in ?
  File /home/wichert/buildout-eggs/tmpG_NQ4q/zope.security-3.4.0b3- 
py2.4-linux-i686.egg/zope/security/management.py, line 25, in ?
  File /home/wichert/buildout-eggs/tmpG_NQ4q/zope.security-3.4.0b3- 
py2.4-linux-i686.egg/zope/security/checker.py, line 420, in ?

AttributeError: 'module' object has no attribute 'thread_local'

Wichert.

--
Wichert Akkerman [EMAIL PROTECTED]It is simple to make things.
http://www.wiggy.net/   It is hard to make things  
simple.

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com



--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: zope.security problems related to Python 2.5 update? (Was: Removed zope.security 3.4b4)

2007-08-15 Thread Jim Fulton


On Aug 15, 2007, at 11:52 AM, Martijn Faassen wrote:


Wichert Akkerman wrote:

Previously Christian Theune wrote:

Am Mittwoch, den 15.08.2007, 14:15 +0200 schrieb Christian Theune:

Hi,

I saw the error reports about zope.security and got talked to  
from a few
people that they had severe problems managing around this. I  
removed the

3.4b4 distribution from download.zope.org to allow them to continue
using zopeproject and grokproject until we find a fix.

I wasn't sure about that, so if someone has even worse problems  
now,
please speak up and I'll put the package back into place again.  
I hope I

didn't cause any more problems than exist already.
This is deeper than I thought. b3 doesn't work either. It looks  
like the

merge from the branch with support for Python 2.5 is causing the
problems.

I can confirm that the last release before the python 2.5 support was
added (3.4.0-b2) works correctly.


Just ran into this one. I think removing it is the right decision.  
b2 did work for me, but b4 just broke on me.


Try b5.

Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] Re: zope.security problems related to Python 2.5 update? (Was: Removed zope.security 3.4b4)

2007-08-15 Thread Jim Fulton


On Aug 15, 2007, at 11:56 AM, Christian Theune wrote:

I'm not sure that we should have to rely on removing packages like
those.


I'm fairly sure that removing them is *not* the right solution.  For  
example, that might have broken other applications.  (After all, I  
released b4 for a reason.)


The right short-term fix in this case was to use b2 in grok until b5  
was released.



As Tres pointed out we should just avoid that everybody gets
those immediately anyway, but we should keep them as historic releases
in general.


Only getting released versions would probably have just delayed the  
problem until it would have been harder to solve. I was able to  
address this quickly because the changes that caused it were fresh in  
my mind.  It would have been harder if the problem were reported much  
later.


This particular problem was due to a circular import, which is a very  
hard thing to test for sadly.  We need to fix this.


Jim

--
Jim Fulton  mailto:[EMAIL PROTECTED]Python 
Powered!
CTO (540) 361-1714  
http://www.python.org
Zope Corporationhttp://www.zope.com http://www.zope.org



___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



  1   2   3   4   5   6   7   8   >