Chris McDonough wrote: [snip extending configuration patterns instead of replacing wholesale] > Often this code makes the subframework tremendously complex, and the > subframework grows inappropriate dependencies along the way. *Sometimes* the > situation gets so confusing for a new user, they just quit and go use > something > else.
I am curious about your use of the word "often". I think what you describe has happened for the example you give, namely the publication/traversal stack. I think it also has happened to a certain extent for the user authentication story. I'd like to identify other such nasty grown subframeworks and you seem to know about a few more. I think in part this happened because there wasn't enough documentation about wholesale replacement patterns (and not enough documentation period), and in part it happened because the APIs or component relationships were just not designed as well as we'd like (as you remark below), and in part it happened because people often don't *want* to do a wholesale replacement but exactly the kind of "only reconfigure this bit" you describe > This is a pattern that happens over and over again in Zope development. In > my > personal opinion, the original error was trying to make the subframework > configurable at all. Instead, the subframework should be replaceable easily, > but it should itself be relatively rigid. At very least, for subframeworks > that > really do require extra configuration (should be very few), this > configuration > should be done via highly specialized ZCML directives (or grokkers), as > opposed > to some very general adapter registration that can't be easily discerned from > other adapter registrations by a newbie. I agree that specialized ZCML directives and grokkers are a good thing. I think we had too much of a tendency to get away from those and instead generalize everything. I think we should define clear Python APIs for particular configuration concepts, not just ZCML directives or grokkers. In turn the ZCML/grokker implementations can use this Python API and be really minimal in themselves. That said, I do think there's implementation value in a general registry. It's just not a great API in most cases. > If the subframeworks were more rigid (but replaceable), the *intent* of the > subframework author could be more easily discerned, and fewer people would > fall > into the trap of adding more configuration code to a subframework instead of > just replacing it entirely. And fewer people would just walk away in > frustration. While I understand the sentiment and I agree with the general advice, I again must note that sometimes it's useful to be able to configure a subframework. "everything as normal with just a few tweaks" is after all a good way forward in many cases. The idea you are promoting is "clearly focused frameworks". They do one thing well, and are configurable in a few ways and clearly replaceable too. > What does this have to do with packaging? Well, currently, there's a > dizzying > number of packages that make up "the ZTK" (nee "Zope 3"). Each of these > packages is a pure peer of all others in a PyPI listing with no real way to > get > a sense of their relative importance other than performing a linear audit. One way to get some hints is to look at a dependency diagram and see which ones are lower down in the stack. > Even > if a user *does* do a linear search of all of them, it's still awful hard to > discern for some new user which ones are "important", and which ones just > happen > to exist by some inequity of history without trying to install it. The user > needs to gain some holistic knowledge of the system in order to discern the > important bits from these historical inequities. > > Most new users understandably just walk away from *all* Zope packages before > they gain this knowledge; it's just too hard for them to tell the difference > between the truly important and reusable bits and the stuff that just happens > to > be packaged up and released but which is useless outside of some highly > specific > context. In effect, we just don't communicate *intent* very effectively in > our > current packaging structure. Sure, agreed. > In my opinion, this is why a lot of Python developers who are otherwise very > smart have given up on trying to use Zope packages. The time required to > figure > out which ones are useful and which ones aren't is just too high. It's way > "easier" for them to write them all off wholesale and just write what they > need > from scratch or use somebody else's software. Good developers tend to like > small, useful libraries and frameworks. Agreed again. > We can ameliorate the situation in a few ways: > > - We can reduce the number of distributions. > > - We can make each current Zope package distribution independently useful. > > My suggestion is that we do the former first in order to communicate *current > intent* (the "state of reality right now"). We can do the latter after this > in > pieces, hopefully aided by some new developers we've picked up by making it > easier to find useful stuff. In order for your plan to work I think you'd need two things ideally: * remove a lot of the released packages from pypi. This is a no-go. People depend on these in a massive way. * at least move around the existing packages in svn.zope.org to some kind of "random crap" area so people aren't confronted with them. > Once we deflate our current set of packages down to a reasonable number, the > packages listed in PyPI will immediately start to reflect "the state of > reality > right now". As a result, we'll hopefully be able to get some new blood in the > form of new developers that use the smaller bits outside Zope to help us > tease > the truly independent pieces out of the larger pile. If we do this, at no > time > after the deflation will PyPI listings ever as badly advertise "the state of > reality" as it is advertised right now, and the community will hopefully > again > start to grow. How do you propose we reduce the PyPI listing without breaking a lot of code? The other practical issue is community-oriented: we have been able to break so many dependencies and have an awareness of these issues precisely because packages *were* split up. Meanwhile we've invested quite a bit of effort in making existing Zope packages at least *depend* more or less logically on other packages. That doesn't mean they're always, or even typically, independently useful in the sense you want: they're useful only if you buy into a lot of the Zope story. I hope we can reduce the buy-in more once we explode the publisher, but it'll always be that way to a certain extent. A package that depends on 10 other packages might not be considered "independently useful" by many, as it assumes quite a bit of the framework to be present. But that doesn't mean it's useless as an independent package. Regards, Martijn _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )