Re: [Zope-dev] Change in repository approach to software sharing
Tim Peters wrote: [Jim Fulton] The Zope project includes a number of interrelated subprojects, such as: - Zope 2 - Zope 3 - ZODB - ZConfig Software from the ZODB and ZConfig projects are shared by Zope 2 and Zope 3. Note that ZODB also depends on ZConfig. Yup and zdaemon and zLOG We want this sharing to be very convenient for people working on Zope 2 and Zope 3. We don't want users of the Zope 2 and Zope 3 repository to have to do separate checkouts for ZODB and ZConfig. CVS supports such software sharing through it's "module" system. The module system has some flaws, so we use symbolic links instead. In a response to: http://dev.zope.org/Zope3/MovingSCMToSubversion John Belmonte has suggested that Zope 2 and Zope 3 should depend on specific versions of shared packages, rather than on the head. I'm inclined to agree. Didn't we endure a lot of pain over the last year to get away from that model? At one point there were 7 lines of ZODB I was wrestling with every day. For example, there was the ZODB 3.1 branch, which "in theory" corresponded to the Zope 2.6 branch, but in practice never matched it. Life got much better (for both Zope 2.6 maintenance and ZODB 3.1 maintenance) when we abandoned the ZODB3-3_1-branch branch of ZODB3 and did all ZODB 3.1 maintenance directly on the Zope-2_6-branch branch of Zope. Likewise for ZODB3-3_2-branch versus Zope-2_7-branch: the former was abandoned, and life got better for everyone because of that. Currently the HEAD of the ZODB module is the same as the HEAD view of ZODB when viewed from the Zope and Zope3 modules, which isn't a pure win, but isn't a pure loss either. I think that subversion will make it easier to manage the changes. Periodically, changes will be merged into Zope 2 and Zope 3. This should be straightforward. People working on ZODB and ZConfig have to test their changes against both Zope 2 and Zope 3 to avoid breakage. I don't see that this part can change, no matter how names are shuffled. ZODB needs to be tested with Zope 2 and Zope 3 whenever updates are merged into those projects. Of course, you'll probably want to test more frequently, but you don't have to test every change you make. This is very burdensome That part either -- all the tests in all contexts still need to be run. But not for every change. and causes much pain when they get it wrong. Ditto, although you're (re)introducing mechanism that alleviates this at the cost of increasing the number of ZODB snapshots in active use. Fortunately, subversion provides a mechanism for sharing specific revisions. We'll be able to have the convenience of getting ZODB and ZConfig (and other shared software) when we do a checkout *and* we'll be able to control what parts we get. To see how this will work, we'll look at ZConfig as an example (because it has a single package) of reusable software that we will include in Zope 3. In the repository, we'll have a top-level Zope3 project directory, with the standard svn subdirectories "trunk", "branches", and "tags". We'll also have a top-level ZConfig project directory. The "trunk" of the ZConfig Python package will be in ZConfig/trunk/src/ZConfig. If we create a tag T1 of ZConfig, then the Python package for that tag will be in ZConfig/tags/T1/src/ZConfig. Now, when we set up the Zope 3 repository, we will create the ZConfig package in Zope 3 by copying a *tag* from the ZConfig project: svn copy svn+ssh://svn.zope.org/repos/ZConfig/tags/T1/src/ZConfig \ svn+ssh://svn.zope.org/repos/Zope3/trunk/src/ZConfig -m 'Bring ZConfig T1 into main branch' Then, whenever someone checks out the Z3 trunk, they'll get the ZConfig from T1. I don't yet know enough about svn to guess how the next step works: since ZODB all by itself depends on ZConfig, presumably similar stitching of ZConfig will take place in the top-level ZODB project. Yup > Now when we go on to stitch ZODB into Z3, how will ZODB's attempt to stitch in its own version of ZConfig play with Z3's attempt to stitch in T1 above? I'm not picturing how this works. Maybe it's simple! You won't stich in ZODB's copy of ZConfig. You'll need to work with the one used in Zope. If we need to, we can even make Zope3-local changes to ZConfig. That would bloat ZConfig maintenance headaches exponentially. I'm confident of that, because of ZODB experience with (at least) 7 concurrently active ZODB snapshots -- it's simply intractable to keep straight what should and shouldn't get cross-ported across this set; mistakes and oversights are as much the norm as the exception then; the non-ZODB people doing stitching of ZODB into their projects won't be confident about exactly which tag they should stitch in; expediency will cause them to stitch in "whatever seems to work" at the time; and then the latter becomes an active snapshot that needs to be maintained too. Don't make Zope3-local changes to ZConfig or ZODB, though, and at least that nest of rats
RE: [Zope-dev] Change in repository approach to software sharing
[Jim Fulton] > The Zope project includes a number of interrelated subprojects, > such as: > >- Zope 2 > >- Zope 3 > >- ZODB > >- ZConfig > > Software from the ZODB and ZConfig projects are shared by Zope 2 > and Zope 3. Note that ZODB also depends on ZConfig. > We want this sharing to be very convenient for people working on Zope 2 > and Zope 3. We don't want users of the Zope 2 and Zope 3 repository to > have to do separate checkouts for ZODB and ZConfig. CVS supports such > software sharing through it's "module" system. The module system has > some flaws, so we use symbolic links instead. > > In a response to: > >http://dev.zope.org/Zope3/MovingSCMToSubversion > > John Belmonte has suggested that Zope 2 and Zope 3 should depend > on specific versions of shared packages, rather than on the head. > I'm inclined to agree. Didn't we endure a lot of pain over the last year to get away from that model? At one point there were 7 lines of ZODB I was wrestling with every day. For example, there was the ZODB 3.1 branch, which "in theory" corresponded to the Zope 2.6 branch, but in practice never matched it. Life got much better (for both Zope 2.6 maintenance and ZODB 3.1 maintenance) when we abandoned the ZODB3-3_1-branch branch of ZODB3 and did all ZODB 3.1 maintenance directly on the Zope-2_6-branch branch of Zope. Likewise for ZODB3-3_2-branch versus Zope-2_7-branch: the former was abandoned, and life got better for everyone because of that. Currently the HEAD of the ZODB module is the same as the HEAD view of ZODB when viewed from the Zope and Zope3 modules, which isn't a pure win, but isn't a pure loss either. > People working on ZODB and ZConfig have to test their changes against > both Zope 2 and Zope 3 to avoid breakage. I don't see that this part can change, no matter how names are shuffled. > This is very burdensome That part either -- all the tests in all contexts still need to be run. > and causes much pain when they get it wrong. Ditto, although you're (re)introducing mechanism that alleviates this at the cost of increasing the number of ZODB snapshots in active use. > Fortunately, subversion provides a mechanism for sharing specific > revisions. We'll be able to have the convenience of getting > ZODB and ZConfig (and other shared software) when we do a checkout > *and* we'll be able to control what parts we get. > > To see how this will work, we'll look at ZConfig as an > example (because it has a single package) of reusable software > that we will include in Zope 3. > > In the repository, we'll have a top-level Zope3 project directory, > with the standard svn subdirectories "trunk", "branches", and "tags". > > We'll also have a top-level ZConfig project directory. The "trunk" > of the ZConfig Python package will be in ZConfig/trunk/src/ZConfig. > If we create a tag T1 of ZConfig, then the Python package for that tag > will be in ZConfig/tags/T1/src/ZConfig. > > Now, when we set up the Zope 3 repository, we will create the ZConfig > package in Zope 3 by copying a *tag* from the ZConfig project: > >svn copy svn+ssh://svn.zope.org/repos/ZConfig/tags/T1/src/ZConfig \ > svn+ssh://svn.zope.org/repos/Zope3/trunk/src/ZConfig > -m 'Bring ZConfig T1 into main branch' > > Then, whenever someone checks out the Z3 trunk, they'll get > the ZConfig from T1. I don't yet know enough about svn to guess how the next step works: since ZODB all by itself depends on ZConfig, presumably similar stitching of ZConfig will take place in the top-level ZODB project. Now when we go on to stitch ZODB into Z3, how will ZODB's attempt to stitch in its own version of ZConfig play with Z3's attempt to stitch in T1 above? I'm not picturing how this works. Maybe it's simple! > If we need to, we can even make Zope3-local changes to ZConfig. That would bloat ZConfig maintenance headaches exponentially. I'm confident of that, because of ZODB experience with (at least) 7 concurrently active ZODB snapshots -- it's simply intractable to keep straight what should and shouldn't get cross-ported across this set; mistakes and oversights are as much the norm as the exception then; the non-ZODB people doing stitching of ZODB into their projects won't be confident about exactly which tag they should stitch in; expediency will cause them to stitch in "whatever seems to work" at the time; and then the latter becomes an active snapshot that needs to be maintained too. Don't make Zope3-local changes to ZConfig or ZODB, though, and at least that nest of rats can be sidestepped. > Later, we may decide to upgrade the Zope 3 head to use ZConfig tag 3. Ya, and the Zope head may decide to use ZConfig tag 44, while the Zope 2.8 maintenance branch decides to use ZConfig tag 37 despite that ZConfig 3.3.1 (which is supposed to go out with the next Zope 2.8 release) is actually released at tag 39. The dark side of "flexibility" is that it creates that many more ways to get out of syn
