[
https://jira.duraspace.org/browse/DS-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21531#action_21531
]
Mark Diggory commented on DS-976:
---------------------------------
> Our add-on will supply the duracloud replication functionality. This requires
> roughly 3 things to be added to a standard release:
> (1) the dspace code that provides this functionality (a set of curation
> tasks, and some shared other code)
To be "endorsed", these should be released as binary artifacts into the the
maven central repository (or other repo) with a formal release version number,
not copied as source into a local directory. One could copy the source
locally, but that should be a source build process (good for developers), not
the default "installing an addon" process (good for admins). In the original
Asynchronous release process, all addon modules would have binary releases.
Using binaries provides for a clean upgrade path for addons once added to a
codebase that does not require merging old and/or new versions of source, an
important component of the asynchronous release proces.
> (2) third party libraries that the code in (1) needs - viz duracloud java
> libraries
Yes, sure, thats just maven.
> (3) configuration properties files - e.g. duracloud.cfg (these normally live
> in config/modules)
If folks would use the ConfigurationService, defaults can live in
[addon-src]/src/main/resources/dspace/dspace/config-[addon].cfg and the service
will load them off the classpath prior to those being overriden in the
directory, so there is only a need to copy configs there if they are wanting to
customize them (likewise, we hope that in an "addon installer", by having a
default in the addon, copying it to config/modules can be automated to setup an
example config the user may alter, but I know this is a tangent).
-----
> The whole premise of add-ons is that none of this is present in the release
> build: specifically, that means that the
> dspace project does not know of or declare any dependencies on these
> libraries, and that what we can call the
> 'resource' files (like duracloud.cfg) are also *not* in the source tree.
> (which *does* contain 'dspace.cfg' etc)
Yes, that is clear, See my comment about default configs and
ConfigurationService above.
-----
Outstanding Comments:
1.) What are the changes it makes to the poms? dependencies, modules? Does it
add any dependencies that are required to the various dspace/modules/*/pom.xml?
2.) Where do you run your maven addon:insertany? its not clear in your
description
3.) per the following...
> The admin would download an add-on package, say: 'dspace-replicate-1.0.zip'
> from a trusted location (DSpace SF, e.g.)
> She would next unzip this package into dspace/addons (not
> dspace/modules/addons) in what we call in the doc the 'source' directory.
> Let us suppose that the unzipped package resulted in a 'dspace-replicate-1.0'
> directory under addons. She navigates to the config/modules
> sub-directory, and finds the duracloud.cfg file and edits it with her
> username and password for her DuraCloud account (this is an example of what I
> mean by
'Configuration changes').
> Then, she enters the command 'mvn addon:insertany' which will detect that
> there is a new add-on in the add-ons directory, programmatically examine that
add-on to make sure it obeys the 'rules' of good add-ons, and if it is OK,
*adds* the addon project to the pom project as a new maven module. Using this
command obviates the need to manually edit or touch any pom file in the project
hierarchy, including the dspace-pom.
> Finally, mvn package will operate as it usually does, but since there is a
> new addon in the list of modules, dependency management will take place, and
> all the new required pieces will be copied to the dspace/target/*build.dir
> area:
(the new config/modules/duracloud.cfg, the dspace-replicate.jar, duracloud.jar,
etc - it requires a bit of tweaking of assembly descriptor). From here they are
deployed using the usual ant targets..
> This process can be repeated with new add-ons and the list in the addons-pom
> just keeps growing. The whole site-specifc set of add-ons lives only in that
> *local* addons pom and its subdirectories - not in any files we distribute.
> Is this more comprehensible?
Yes, but we will still want to "distribute" these addons independently of the
release as separate addon releases.
If your addon:insertany goal is run outside the [addon-src], I would prefer
that we invert this, the plugin should be added to the "[addon-src]/pom.xml".
the call to install it into "dspace" should be run from the [addon-src] when
installing "source". But run from dspace/pom.xml when installing from binary.
This allows us to parameterize the plugin to either install from a source
project (copying it or linking it into the addon poms modules sections or from
a released artifact (either binary or source) in a maven repository.
Actually, for the "source" case, it does away with the need to have an "addons"
subproject outside "modules", the addons could just be installed into the
modules/pom.xml modules section and the source placed there with the rest of
the addons/webapps.
I would like to see the goal support providing optional "addon artifact maven
coordinates" that could be used rather than providing everything in a source
directory. They would be identified as command-line options or configuration
parameters for the plugin definition and the goals would be called as:
Something like the 4 following goals...
Install Released Binary Addon (binary released to maven repo):
Location: [dspace-src]/dspace
Command: mvn dspace:install-addon -DgroupId=org.dspace -DartifactID=my-addon
-Dversion=2.0.0 -Dtargets=cli,xmlui,common
Install Released Source Addon (source artifact deployed to maven repo):
Location: [dspace-src]/dspace
Command: mvn dspace:install-addon -DgroupId=org.dspace -DartifactID=my-addon
-Dversion=2.0.0 -Dpackaging=source -Dtargets=cli,xmlui,common
Install a Unreleased Binary Addon (binary "installed" into local repo):
Location: [addon-src]
Command: mvn dspace:install-addon -Dtargets=cli,xmlui,common
-DdspaceSourceDirectory=[dspace-src]
Install a Unreleased Source Addon (source not deployed to maven repo):
Location: [addon-src]
Command: mvn dspace:install-addon -Dtargets=cli,xmlui,common
-DdspaceSourceDirectory=[dspace-src]
Continue on to build your release:
Location: [dspace-src]/dspace
Command: mvn clean package
Thus, working with Maven should be a two part process to assure the reactor has
an opportunity to evaluate the full dependency tree once again after the addon
is installed.
mvn dspace:install-addon ....
mvn package ...
> Simple Asynchronous Add-on Facility for DSpace
> -----------------------------------------------
>
> Key: DS-976
> URL: https://jira.duraspace.org/browse/DS-976
> Project: DSpace
> Issue Type: New Feature
> Reporter: Richard Rodgers
> Assignee: Richard Rodgers
> Fix For: 1.8.0
>
> Attachments: addon-pom-template.xml, addon-tool.tar
>
>
> Placeholder for design ideas, proposals and discussions around supporting an
> asynchronous release process for add-ons, which are functional extensions to
> DSpace.
> Motivation: in addition to long-standing wishes to add greater flexibility,
> modularity and extensibility to DSpace, there is an immediate need to provide
> a low-risk, lightweight way to distribute the AIP backup & restore add-on
> (including DuraCloud-backed storage) which has been developed to be available
> for, and compatible with, the 1.8 release.
> I believe given the short time-frame and other resource constraints, it makes
> good sense to look at very simple designs that address these initial sets of
> add-on use-cases, but which hold the potential to be elaborated to
> accommodate more complex use-cases in the future (or at the very least, do
> not preclude different approaches later if needed). Therefore (to lean on a
> very tired metaphor), we should look for 'low-hanging fruit' by leveraging
> our existing build and deployment infrastructure as much as possible. In
> fact, I would lower the bar even further, and characterize the first system
> as 'fallen fruit' - essentially seeing what we can scavenge using current
> tools and practices. With that preamble, here are some initial definitions,
> scope considerations, design ideas, etc for a FF asynch add-on mechanism:
> (1) Definition/scope of a FF add-on:
> An add-on is a collection of code and discrete resources that extends DSpace
> functionality when added to a runtime (deployed) installation.
> Add-on source code can reside in any legal package that does not conflict
> with base DSpace code, or known published 3rd party code. The usual best
> practices/conventions should prevent collisions.
> An add-on must possess a maven pom compatible with current DSpace maven
> requirements in order to integrate with current build and deployment
> processes.
> An add-on must be available in a standard archive (zip, tar.gz, etc)
> containing any code and resources, together with the maven pom. That is, it
> must resemble an ordinary maven project.
> An add-on's code may be available in binary form (by pom reference) only if
> it published in a designated maven repository. Source code distributions of
> add-ons are optional, but it is desirable to have both source and binary
> available, as current DSpace practice is for the packaged releases.
> Add-on resources will be limited to a subset of those currently found in
> /dspace: specifically only files that reside in /bin and /config
> A resource is discrete if it does not require a 'merge' into an existing
> resource. Thus, an add-on will not contain information to perform edits,
> inserts, etc into other resources - these edits may in fact be required, but
> are regarded as out of scope for automated add-on operation. New
> configuration files, e.g. under config/modules, are good examples of discrete
> resources.
> An add-on will not require any database schema changes. Of course, this need
> is legitimate, but will not be supported initially.
> (2) FF Add-on life-cycle considerations
> An add-on will be installed into an existing (source) DSpace installation,
> rather than to a specific runtime deployment of same. As such, it can be
> deployed to many locations.
> Although possible, *uninstall* of an add-on is out-of scope.
> It will be possible to determine what add-ons are present in a system by
> examining the DSpace source tree - visibility in the runtime/Admin UI, etc is
> currently out-of scope.
> (3) A straw-man for add-on process:
> Current DSpace (scratch) installation has essentially 4 stages:
> (1) Download & stage (unzip)
> (2) Configure (manual process)
> (3) Prepare (which typically means maven compile and/or package)
> (4) Deploy* (usually via ant fresh_install or update, and copies to Tomcat,
> etc) - * indicates multiple targets
> I would propose that the installation of an add-on have this exact sequence:
> (1) Add-ons (as noted above) look like installations
> (2) Many add-ons require separate configuration, so we have to provide for
> this step
> (3) Prepare would look similar, but would also have to manage transfer of
> resource files
> (4) Deploy might always just be an 'ant update'
> This is just a rough initial set of thoughts - comments welcome (I see
> mdiggory already has some ;))
>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.duraspace.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
------------------------------------------------------------------------------
uberSVN's rich system and user administration capabilities and model
configuration take the hassle out of deploying and managing Subversion and
the tools developers use with it. Learn more about uberSVN and get a free
download at: http://p.sf.net/sfu/wandisco-dev2dev
_______________________________________________
Dspace-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dspace-devel
