Hi all: I have a few remarks inline, but my bottom line: I think this whole area needs more analysis before leaping into any solution, esp. right before a release. The Domain Model work is interesting and creative, but I'm not convinced we understand our use-cases well enough yet to judge it's applicability.
Richard On Jul 29, 2011, at 5:53 PM, Tim Donohue wrote: > Hi All, > > I have a few more background details to add to Mark's description, > followed by a few comments of my own. > > == A little more about the problems we are encountering == > > As Mark mentions, for 1.8.0, Richard R & I would like to release > 'dspace-replicate'. More info at: > https://jira.duraspace.org/browse/DS-876 > > Unfortunately, the reality here is that 'dspace-replicate' sits in SVN > Modules (svn/repo/modules) and also depends on 'dspace-api'. So, to > release 'dspace-replicate', we would need to *first* release > 'dspace-api'. But, at the same time, 'dspace-api' can only be released > if we perform the full packaged DSpace release from Trunk. > > What results is a chicken-or-egg issue, where 'dspace-replicate' cannot > be included in the full packaged DSpace release unless it's released > *first*, but it cannot be released first as it depends on dspace-api. I guess I don't really understand this point - I see no reason why the dependency here is of special concern, or why there is a problematic chicken-and-egg situation. You say "include replicate in full packaged release", but I see no requirement for this at all. Replicate is a good example of of a candidate for 'asynchronous' release, which means precisely *not* at the same time as the packaged release. Why is it a good idea to require that all add-ons be synchronized? I'm imagining this is the sort of thing (cloud replication) that one would not immediately decide on, but could elect to add to one's system later. > > So, the options for moving forward currently include: So I see a third option: do nothing (for the moment). I had no problem compiling the replicate code from the pom (including it's dependency on dspace-api, and 3rd party libraries), and deploying the jar artifacts, and configuration files to a DSpace installation. So I think we should be clear that not doing option 1 or 2 below will not *prevent* the use of the replication module. Of course, we could add scripting or other tooling to make it easier, but it is not a complicated process as is. > > Option 1: Continue to move *all modules* into SVN Trunk > (/svn/repo/dspace/trunk/) and release everything together from there. > This continues to make for a very crowded Trunk, unless we can find a > way to better organize it. > > OR > > Option 2: Move more towards refactoring the Domain Model (like Mark > mentions below), which allows modules like 'dspace-replicate' to be > changed to just depend on 'dspace-core-api' (which is just a series of > Java Interfaces). This allows some out-of-the-box DSpace modules to sit > outside of Trunk (like in /svn/repo/modules). It also does enable the > ability to perform asynchronous releases for those external modules > (though, to be clear, we don't *have* to release all external modules > asynchronously -- it's just available as an option). One of the reasons this is appealing is that it abstracts the DSpace API to interfaces. But that is a quite distinct concern, and I'm worried that that aspect colors our judgement about the module/release issues. Consider the strategy here, which I think is more general than dspace-api. Suppose I want to publish a new add-on that depends on our new 'business tier' library (or even, since that doesn't exist yet, on dspace-xmlui-api). If I understand the issues here, I would have to *first* refactor that latter into 'dspace-xmlui-api-api' and 'dspace-xmlui-api-impl', and have my add-on depend on the former. This model does not seem particularly developer-friendly or scalable. > > == My own comments on this issue == > > The more I look at this issue, the more I lean towards this Domain Model > refactoring (preferably even in 1.8.0, assuming we all agree it is > needed). The process of moving everything into Trunk is beginning making > Trunk a bit crowded. Even if modules are NOT in Trunk, we should still > be able to include the module source code in our packaged > 'dspace-src-release.zip' on SourceForge. > > I only have a few concerns that I want to make sure we keep in mind: > > 1. First, I think we need to be careful that this doesn't complicate our > release procedures too much. I just want to be sure the release process > doesn't become a painful full day affair (where the RC has to release > things in a very particular order, one-by-one-by-one). I believe there > should be ways to still automate things a bit better via Maven. But, we > just need to make sure we are constantly looking towards keeping > releases relatively easy. (As easier Maven release processes will make > for happy Release Coordinators!) > > 2. My only other (very minor) concern here is the names > "dspace-core-api" and "dspace-core-impl". A small part of me worries > about name confusion with the existing 'org.dspace.core.*' classes, > which obviously are completely different things. But, I'm not sure if > there is a better name, unless we went with something like > 'dspace-model-api' and 'dspace-model-impl'. All that being said, if no > one else sees this as a concern, then I'm fine with leaving it how it is. :) > > All in all, I think this refactoring of the Domain Model idea is very > important to decide on soon. That way we can ensure modules like > 'dspace-replicate' can be properly released as part of 1.8.0 (and moved > to Trunk, if we decide that's how it needs to be for 1.8) > > So, thoughts / questions are welcome & encouraged! Even letting us know > "I have no strong opinion one way or the other" is worthwhile, so that > we can gauge how controversial/non-controversial this idea may be. > Overall, I think I'm 90-95% on board here. I just need to make sure we > find ways to keep the Release Procedure straightforward, and not too > time-consuming overall. > > Thanks all! Enjoy your weekend. > > - Tim > > On 7/29/2011 11:30 AM, Mark Diggory wrote: >> Community, >> >> I just want to alert everyone that I've updated the patch for the DSpace >> Domain Model to reflect the latest status of the work >> >> https://jira.duraspace.org/browse/DS-845 >> >> <https://jira.duraspace.org/browse/DS-845>To assist others in >> understanding what this Domain Model Does or is for, please review the >> following wiki page(s). >> >> https://wiki.duraspace.org/display/DSPACE/Refactoring+the+DSpace+Domain+Model >> >> <https://wiki.duraspace.org/display/DSPACE/Refactoring+the+DSpace+Domain+Model>To >> give an overview of what this work intends to accomplish: >> >> Problem: Addon Modules that depend on DSpace artifacts such as >> dspace-api cannot be released asyncronously to DSpace releases because >> of dependency cycles created by all the projects residing under trunk >> and being released at the exact same time. This was observed with the >> dspace-stats, dspace-discovery and now the recent dspace-replicate work >> being done by Tim Donohue, Richard Rodgers and the dspace-sync work >> @mire completed for the Duracloud AV pilot. >> >> Solution: To enable Addons to stop relying directly on dependencies on >> dspace-api, a DSpace domain model project is proposed as >> dspace-core-api. This "core api" is a standalone DSpace project that >> enforces a specific API contract for interacting with DSpace Objects >> without being "bound" to a specific version of dspace-api. Addon >> modules can then write application code that interacts with this api >> (Icollection, IITem, IBitstream, IEPerson, etc) rather than depending on >> Concrete dspace-api implementation. >> >> Benefits: >> >> 1.) Measuring Compatability across releases: Separate releases of the >> dspace-core-api would happen on a different schedule than the regular >> DSpace release. Each DSpace release can then be "graded" based on use >> of this API to determine backwards/forwards compatability of the core >> DSpace implementation your custom code is using. >> >> http://scm.dspace.org/svn/repo/modules/dspace-core/trunk/api/src/main/java/org/dspace/content/ >> >> 2.) Contract lead to "boundaries" that cleanly separate application code >> in dspace from the core Domain Model. dspace-api "app" packages will be >> able to be broken out into separate projects and managed as addons, >> improving the ability of application developers to replace these areas >> of functionality in DSpace without the use of classpath overriding. >> >> 3.) The ability to write MockObject test suites more easily for DSpace >> to provide Unit testing for your addon. >> >> 4.) Finally, the ability for third party addons to release there addons >> prior to the official DSpace release, allowing the DSpace release >> process to assemble these third party addon modules as optional addons >> for DSpace without having to move them all under dspace/trunk as source. >> This will aid in defining a formal "space" for addons to DSpace and >> best practices for how they are authored and wired into DSpace without >> impacting the behavior and contract of the core libraries. >> An initial "bridge" implementation of this API is being authored that >> will allow users of the DSpace Spring Service Manager to be able >> complete "Repository" calls against DSpaceObject "Services". >> >> http://scm.dspace.org/svn/repo/modules/dspace-core/trunk/impl/src/main/java/org/dspace/content/ >> >> This Implemenation provides simple CRUD Repositories that are >> exemplified currently in the static methods provided in the >> DSpaceObjects themselves. Thus the following style methods: >> >> https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/content/Item.java#L140 >> >> public static Item find(Context context, int id) throws SQLException >> >> are provided via a LegacyItemDAO interface and implemented as: >> >> http://scm.dspace.org/svn/repo/modules/dspace-core/trunk/impl/src/main/java/org/dspace/content/LegacyItemDAO.java >> >> Thus Item.find(....) is replaced by the following when a reference is >> needed outside the Spring Application Context: >> >> Item item = dspace.getSingletonService(LegacyItemDAO.class).find(....) >> >> Whereas, when used within the Spring Application Context, Dependency >> Injection eliminates any lookup (this is the ideal usage). >> >> private ItemDAO itemDAO; >> >> public void setItemDAO(ItemDAO idao).... >> >> public void yourMethod() { >> >> Item item = itemDAO.find(...) >> >> } >> >> This allows for much of the implementation of the method to be >> extensible and to be abstracted away from the client application code >> calling it. >> >> This implementation not the final solution for porting DSpaceObject >> static methods to DAO but it is provided as a means for DSpace Addons to >> "Bridge" the gap as we work to move the implementations of these static >> methods to more formal DAO classes. Thus, Addons will jumpstart a >> migration of the DSpace "Core" codebase out of the dspace-api library. >> I propose that such an implementation, once stable, would be moved >> from dspace-core-impl into dspace-impl and represent a legacy >> implementation of the DSpace Domain Model against the relational >> postgres and oracle databases. Finally, I hope other can see how >> achiving such a solution will assist us in producing what will be a >> "swappable" implementation of the core DSpace API. Such a "swappable" >> implementation is an ideal architecture upon which to base a DSpace >> "With Fedora Inside" implementation, where such DAO would be replaced >> with interations with Fedora rather than a relational datastore. >> >> I hope to have what we consider to be very modest changes added to the >> dspace-api for release with DSpace 1.8.0 and invite other developers to >> begin working with and testing the proposed solution. I consider this a >> serious step towards a true separation of concerns (Application, Domain >> Model, Persistence) for DSpace. There will be examples of how to use >> this API in an addon by porting the DSpace Discovery and Statistics >> Addons to utilize it. >> >> To attain this a decision needs to be made within the community to adopt >> the signature changes in dspace-api, release and depend on >> dspace-core-api (just like dspace-services is today). This decision >> needs to be made rather quickly to get into the 1.8.0 release as a feature. >> >> Your thoughts and comments are welcome. >> >> Cheers, >> Mark >> >> p.s. I also want to clarify one point that came up in the last DSpace >> Commiter IRC Meeting, the patch to provide the Interfaces in the method >> signatures of dspace-api does not require the removal of any code from >> dspace-api to be placed into dspace-core-api, but does include some >> additional enhancements to the DCValue and MetadataVlaue objects to >> consolidate their implementations onto one "Interface" with Getters and >> Setters for DCValue properties. Thus the body of work continues to >> attempt to improve the DSpace Metadata API. >> >> -- >> Mark R. Diggory >> @mire - www.atmire.com <http://www.atmire.com/> >> 2888 Loker Avenue East - Suite 305 - Carlsbad - CA - 92010 >> Esperantolaan 4 - Heverlee 3001 - Belgium >> >> >> ------------------------------------------------------------------------------ >> Got Input? Slashdot Needs You. >> Take our quick survey online. Come on, we don't ask for help often. >> Plus, you'll get a chance to win $100 to spend on ThinkGeek. >> http://p.sf.net/sfu/slashdot-survey >> >> >> >> _______________________________________________ >> Dspace-devel mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/dspace-devel > > ------------------------------------------------------------------------------ > Got Input? Slashdot Needs You. > Take our quick survey online. Come on, we don't ask for help often. > Plus, you'll get a chance to win $100 to spend on ThinkGeek. > http://p.sf.net/sfu/slashdot-survey > _______________________________________________ > Dspace-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/dspace-devel ------------------------------------------------------------------------------ Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey _______________________________________________ Dspace-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dspace-devel
