Mark, I feel like you and I are just coming at the same problem from different directions. I think we do have common goals here (asynchronous releases, simplifying of 'Trunk'), but we picture the end result slightly differently. So, it might be worth teasing out more of your reasoning for *why* you are suggesting some of these changes...
Hopefully, I can ask a few questions here to lead to that better understanding. On 4/1/2011 5:08 AM, Mark Diggory wrote: > Tim et. al., > > The http://scm.dspace.org/svn/repo/modules directory is intended by me > to be the "archetype" for how and where we should be maintaining > literally all the modules of DSpace including portions of DSpace API. I > understand at the moment it looks a little bit chaotic, but what needs > to be understood is (as Tim has suggested) that we can organize things > within this space to assure we are appropriately grouping the modules. > But what is most critical is the support for individual svn project > spaces and release histories for each module. > > I argued extensively during my redesign of dspace to use maven in 2006, > that many of the individual dspace-xxx modules should have had their own > svn project spaces (trunk/branches/tags) so that they could have > separate releasing, but the group pushed back excessively on creating > more than on trunk at that time. This was not only disappointment to > me, but most of our forward motion ceased after the excessive debate and > breaking dspace-api up further was never brought to fruition. It took > my next initiative to move the SVN repository off of Sourceforge to OSS > hosting that finally got a top level directory in place and allowed us > to start creating individual separate projects. And, now we have a > "modules" and "sandbox" space within the repository. This is where the > experimental growth and creation of new addons for DSpace is happening, > if your not understanding what is going on in these directories, ask > questions and we will do our best to describe why and how specific > projects exist. Yes, I think we all understand 'modules' and 'sandbox' in SVN, and their usefulness in creating separate individual projects & allowing for a place to experiment & create new addons. There is a concern is that there's not enough differentiation between modules which are *stable*, and those which are *prototypes* or still a *work in progress*. The wiki Modules documentation (see below) helps with that understanding. But I, for one, still think some SVN reorg may also help. In other words, do we really need to have nearly 40 various modules of varying degrees of stability all under SVN "modules"? Maybe we need some sub-categorization here, or some sort of general 'groupings' to make things more manageable? Otherwise the list will just continue to grow into the 100s and get more confusing / difficult to manage. https://wiki.duraspace.org/display/DSPACE/Modules > I have to be critical that DSpace Services have no reason to be in the > main DSpace trunk. It is a standalone tool with no dependency on > dspace-api, this was how it was designed to be. Placing it under trunk > actually defeats all the benefits of releasing separately. And to be > honest, none one should be locally altering the implementation of the > Service Manager codebase in the customization of your DSpace instance. > This doesn't mean other committers in the group can't alter the code > and contribute improvements to it, everyone is certainly welcome to > contribute. Its very serious, you shouldn't be touching this code if you > are just customizing your local DSpace instance, this is why its source > is separate from the distribution and has separate release cycles. The question in my mind is this: Why does DSpace Services need to be released separately from a formal DSpace release (or 'dspace-api')? What are the benefits you see in doing so? Is it really just a matter of the lack of dependency between 'services' and 'dspace-api' that makes you want to separate them out? This really isn't clear to me. So, if you could clarify, maybe this would help me to understand the direction you suggest. > Based upon the alternate proposal, I feel that something is being > misunderstood about the goals of the Domain Model and > original Asynchronous work. The Domain > Model > (https://wiki.duraspace.org/display/DSPACE/Refactoring+the+DSpace+Domain+Model) > being created as a Addon Project is specifically designed to > allow asynchronous release of addons that may be later released as part > of the formal DSpace release. The Domain Model, Its corresponding > Services and DAO, need to be maintained quite separately from the > release trunk to facilitate the asynchronous release process. This is > because versioning these API outside of the trunk will allow the > following features: To be clear, I'm in *full* support of refactoring the Domain Model (not sure if this was misunderstood or not). However, I was just trying to tease out whether *some* of that Core API could still reside in Trunk, or if refactoring would *require* us to somehow pull nearly everything out of Trunk. This seems to be the suggestion of your original Asynchronous Release proposal, where Trunk no longer has any Java code within it (rather, I believe your SVN reorg suggestion is that Trunk becomes just Maven poms and the DSpace installation/config files, and nearly all Java source code is stored external to Trunk, in 'modules' or similar) > a.) Because the Core API (which really have not changed since DSpace has > been initially released) doesn't actually need to be rereleased every > single version of DSpace, it can be depended on as more stable than the > underlying implementation(s), which are currently the target of > considerable criticism and interest in reimplementation. Likewise, > because it is a "Contract", we can institute policies of maintaining > backward and/or forwards compatibility across new DSpace releases. My alternative proposal comes at this at a different direction. Namely, I consider the "core API" to be the thing that *does* require a new formal release to occur. In fact, in my proposal, I'm in agreement that the Core API changes less than the Web UIs/Services -- that's specifically why I suggested that the Web UIs/Services will need to support asynchronous releases. I feel that *because* the Core API changes less frequently, updates to it can be released synchronously (e.g. in the next major version of DSpace). Therefore, we end up with a similar concept of asynchronous releases, but the "Core APIs" would be released synchronously so that they can be thoroughly vetted, tested, etc. (via a Testathon & similar). Are we referring to the same thing when we say "core API"? To me, the "Core API" is a *set* of core modules -- primarily 'dspace-api', but I'd also include other underlying "core" modules like 'dspace-servies' here. Am I missing something here, Mark? (If I am, please clarify!) > > b.) Because of this "Contract", users of the Domain Model wil be > insulated from changes to the internal DSpace implementation, their work > will be more stable and require less source code merging than the > current approach of direct alteration of the codebase or direct > overriding of core implementations. > > c.) Because the Core API would reside outside of the trunk, Addons can > be released using it as a dependency and then depended on in the trunk > without having to be released / maintained within the trunk. This allows > for 3rd parties to have an easier time releasing their addons to DSpace. > Likewise, this allows addons maintained in separate organizations to > be contributed and released with DSpace without having to drag them into > the trunk. > > Gaining this capability is the absolute reason we are > currently striving to complete the work. To give you an example, in 1.6 > we were forced to copy dspace-statistics into the trunk because it > needed to depend on dspace-api and be depended on by other trunk modules > like the dspace-xmlui-api. By eliminating the need to depend on > dspace-api and instead on an external release of dspace-core-api, we > will be able to move dspace-statistics back out of the trunk and release > it on its own version, and include that version into the > dspace-xmlui-api/webapp etc. By creating dspace-core-api outside of the > Trunk, we are creating a whole new way to interact with the DSpace > Domain Model and its underlying implementation. Developer working on > DSpace in this new situation only need to checkout the parts of DSpace > that they are interested in. This might be the "meat" of the issue here. Your view seems to be that we need everything to be asynchronously released because it simplifies dependencies between our various modules. As you stated, things have had to be moved to Trunk because of this. I agree, we need to simplify these dependencies, and avoid an 'overcrowded trunk". I agree completely with those goals. But I still have a few questions about the approach you suggest. Namely, your example to me seems like it could be solved in a different way in the future. You said, the dependency looked like this: dspace-api <-- dspace-stats <-- dspace-xmlui-api (I'm using the <-- to signify "depends on") Obviously, this meant in 1.6 we had to move 'dspace-stats' to Trunk, because *both* dspace-api and dspace-xmlui-api were in Trunk. But, what if *only* dspace-api was in Trunk? (My alternative proposal suggests that 'dspace-xmlui-api' actually moves external to trunk, alongside other XMLUI modules) If only 'dspace-api' was in Trunk, then couldn't 'dspace-stats' still reside outside of Trunk, and we would be able to achieve the same dependency structure mentioned above? (This is an honest question -- trying to work this out in my head, and making sure I understand the logic here.) > I am very confident about contributing the Core Domain Model and > Services. I have been working on Asyncronous release for the last year > and this work is not as much"coding" as it is the culmination of a great > deal of code analysis and research. I will be consolidating the > proposals to reflect that they are, in fact, one in the same. I fear > that because many do understand the original intentions, they are not > understanding the necessity of the work. I will also argue that the > changes are not unduly complex and are meant to actually improve the > organization of the codebase and make it easier to maintain your > customizations, addons, etc. Likewise the work gives the community a > template for how the service manager should be used in the DSpace > application. > > For the minor price of altering one line that looks like "public class > Foo implements SomeInterface" in the DSpaceObject Classes, we get a > explicitly defined API / Contract for the DSpace Domain model. The work > to add the Domain Model will actually be an easy merge for those who > have customized core dspace classes, as it is literally only changing > one line of code per class in dspace-api. > Mark, again, I'm not disagreeing with any of the code you've already done around either the Core Domain Model or Services -- I think both look very good so far. I'm really just trying to nail down a future release process that makes sense. I'm concerned more about whether we, as a trusted group of developers, will be able to maintain proper quality control over *everything* once every module has its own separate "trunk" and its own separate "release cycle" (which is the extreme version of 'asynchronous releases', I realize). A part of me is also playing devil's advocate here, to better tease out the proper balance between synchronous and asynchronous (as I think there *is* a balance). In my mind, we don't have a large enough development team to handle many different asynchronous releases, and still ensure proper quality control, and that someone didn't accidentally break our 'contract' (thus affecting some or all UIs) in a given release of a Core module. So, I'm trying to determine if there's a way to *both* perform some asynchronous fixes/releases, while also achieving a balance where we fully test & perform quality control on the more important pieces (the core APIs & their 'contracts') during our normal release processes. This is a great discussion so far, and I hope it continues! - Tim ------------------------------------------------------------------------------ Create and publish websites with WebMatrix Use the most popular FREE web apps or write code yourself; WebMatrix provides all the features you need to develop and publish your website. http://p.sf.net/sfu/ms-webmatrix-sf _______________________________________________ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
