Re: [Dspace-devel] DSpace deployment methods
Hi all,
Apologies for keeping this one running, but I'm still thinking about this
subject. My gut feeling is that I don't want to run a Maven build on a live
server, for two reasons...
1. Java has always claimed to be a 'Build once, run anywhere' language. So I
want to build and test elsewhere, then deploy the tested app.
2. I instinctively don't like having development tools on a live server, it
feels like a dodgy thing to do, although I would struggle to justify that
statement.
At the moment we only copy the target build directory to our live server, and
then run 'ant update...' on the live server. But even that feels a bit hacky.
At this point I should stop talking and read up on Tim's Commons Configuration
work.
Cheers.
Robin Taylor
Main Library
University of Edinburgh
From: Tim Donohue tdono...@duraspace.org
Sent: 17 August 2015 15:10
To: dspace-devel@lists.sourceforge.net
Subject: Re: [Dspace-devel] DSpace deployment methods
Hi Robin,
I am working to disentangle configuration from Maven Ant builds in the
Commons Configuration work (aimed for 6.0). Once we switch to Commons
Configuration, we should be able to simply remove all our custom config
variable interpolation magic that Maven Ant does. Commons Configuration
does interpolation on-the-fly, which means Maven and Ant should no longer need
to touch any of our configs (thought Ant will still need a dspace.dir to
install to obviously).
See https://github.com/DSpace/DSpace/pull/991 and
https://jira.duraspace.org/browse/DS-2654 for a bit more info.
- Tim
On 8/17/2015 5:07 AM, TAYLOR Robin wrote:
Hi Terry,
That's really useful, thanks. The main problem for me is that configuration is
so entwined with both the Maven build and the Ant installation, it makes it
difficult to find a process that avoids having to do a Maven build on a live
server, and hack various bits of configuration specific to that server. Sigh
[??]
Cheers, Robin.
Robin Taylor
Main Library
University of Edinburgh
From: Terry Brady
terry.br...@georgetown.edumailto:terry.br...@georgetown.edu
Sent: 14 August 2015 23:40
To: TAYLOR Robin
Cc: DSpace Developers
Subject: Re: [Dspace-devel] DSpace deployment methods
Robin,
I have been happy with the process that has evolved for us over time.
Since our repository code is private, we have cloned our github repository on
our server using an access token.
We have a build folder on each of our servers. That folder contains a script
that runs the following steps.
git fetch origin
git checkout ${BRANCH} -f
git pull
mvn clean package
cd dspace/target/dspace-build
ant update
#insert domain specific passwords/overrides into config files using sed
#restart webserver
When build.properties was first introduced, it did not allow us to override
what we needed to override. I have not revisited that file to see if it could
further simplify our build process.
On Thu, Aug 13, 2015 at 7:55 AM, TAYLOR Robin
robin.tay...@ed.ac.ukmailto:robin.tay...@ed.ac.uk wrote:
Hi all,
I notice there was a brief discussion about DSpace deployment methods yesterday
in the developer meeting. I would be very interested to hear what others do,
what sort of tools they use, what sort of infrastructure they have etc. If
anyone is willing to share please do reply. For our part we are still stuck in
a world of 'scp' and shell scripts, so could benefit greatly from a bit of
modernisation :)
Many thanks, Robin.
Robin Taylor
Main Library
University of Edinburgh
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
--
___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.netmailto:Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel
--
Terry Brady
Applications Programmer Analyst
Georgetown University Library Information Technology
https://www.library.georgetown.edu/lit/code
425-298-5498 (Seattle, WA)
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
--
___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.netmailto:Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel
--
___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel
Re: [Dspace-devel] DSpace deployment methods
Hi Robin, I'm hoping for the build/deploy/config process to be easier. If we eventually had configurations living within the DB (within the instance) that would be great. An instance could set config from the UI, and not-quite-reload, but just the config is updated, that would be great. That is however a different subject, a helpful one though. Build Once, Run Anywhere. Yep. We build on a server that doesn't run DSpace. Then scp the dspace-installer over to the app server. For each instance we have their config directory in its own git repository. So, locally I can upgrade configs for a site, push it to git. Then for that site, pull its configs back down. So on each app server, it might be ant -Dconfig=/home/some-university/config/dspace.cfg update, Since ant is a bit egregious with touching your configs, its followed with cd /home/some-university/config; git reset HEAD --hard; then restarting tomcat. There's more to do. SOLR 5 is going to change the game for solr. Such that DSpace won't be able to ship solr, it will no longer be a tomcat webapp, but it will instead be a standalone service / daemon. So, we'll need to clean up (if we have any custom solr code), and figure that out. And/or, an implementation for some other flexible no-sql data store, that can handle search/browse/statistics/faceting, ... The concept of a SITE within DSpace is interesting. Instead of having so many instances of DSpace, what.. really is distinct between instance A and instance B? The theme, the content, a few configurations. Expand the SITE DSO of DSpace to handle multiple DSpace sites. So then one DSpace instance can run dozens of SITES. Upgrading your instance of DSpace... upgrades all of your sites. boom. I'm dreaming here. I'm really trying to shed data OFF of the dspace instance, but rather into safe harbors. A dedicated postgres server, a log server, bitstreams into s3, configuration into the DB, sessions.. I don't know where. Thus, an instance of DSpace is really nothing, just the running application, your data is elsewhere. So instead of updating your instance. The eventual plan could be a stateless server. When your running DSpace build 100, and your ready to deploy DSpace build 105, just build new instances of 105, and then have your load balancer stop and shut down the build 100 instances. Do you update/upgrade, or just in with the new, out with the old. Peter Dietz Longsight www.longsight.com pe...@longsight.com p: 740-599-5005 x809 On Fri, Aug 21, 2015 at 10:19 AM, TAYLOR Robin robin.tay...@ed.ac.uk wrote: Hi all, Apologies for keeping this one running, but I'm still thinking about this subject. My gut feeling is that I don't want to run a Maven build on a live server, for two reasons... 1. Java has always claimed to be a 'Build once, run anywhere' language. So I want to build and test elsewhere, then deploy the tested app. 2. I instinctively don't like having development tools on a live server, it feels like a dodgy thing to do, although I would struggle to justify that statement. At the moment we only copy the target build directory to our live server, and then run 'ant update...' on the live server. But even that feels a bit hacky. At this point I should stop talking and read up on Tim's Commons Configuration work. Cheers. Robin Taylor Main Library University of Edinburgh -- *From:* Tim Donohue tdono...@duraspace.org *Sent:* 17 August 2015 15:10 *To:* dspace-devel@lists.sourceforge.net *Subject:* Re: [Dspace-devel] DSpace deployment methods Hi Robin, I am working to disentangle configuration from Maven Ant builds in the Commons Configuration work (aimed for 6.0). Once we switch to Commons Configuration, we should be able to simply remove all our custom config variable interpolation magic that Maven Ant does. Commons Configuration does interpolation on-the-fly, which means Maven and Ant should no longer need to touch any of our configs (thought Ant will still need a dspace.dir to install to obviously). See https://github.com/DSpace/DSpace/pull/991 and https://jira.duraspace.org/browse/DS-2654 for a bit more info. - Tim On 8/17/2015 5:07 AM, TAYLOR Robin wrote: Hi Terry, That's really useful, thanks. The main problem for me is that configuration is so entwined with both the Maven build and the Ant installation, it makes it difficult to find a process that avoids having to do a Maven build on a live server, and hack various bits of configuration specific to that server. Sigh [image: ] Cheers, Robin. Robin Taylor Main Library University of Edinburgh -- *From:* Terry Brady terry.br...@georgetown.edu terry.br...@georgetown.edu *Sent:* 14 August 2015 23:40 *To:* TAYLOR Robin *Cc:* DSpace Developers *Subject:* Re: [Dspace-devel] DSpace deployment methods Robin, I have been happy with the process that has evolved for us over time. Since our
Re: [Dspace-devel] DSpace deployment methods
On Mon, Aug 17, 2015 at 10:07:46AM +, TAYLOR Robin wrote: Hi Terry, That's really useful, thanks. The main problem for me is that configuration is so entwined with both the Maven build and the Ant installation, it makes it difficult to find a process that avoids having to do a Maven build on a live server, and hack various bits of configuration specific to that server. Sigh [] The build doesn't actually need any of the configuration. It's just that you have to fill out build.properties anyway or it complains, because it wants to customize various config. files for you. Installation needs dspace.dir. Before I had to do build.properties to make the build succeed, what I would do is just tell ant where to find the dspace.cfg it was to use. (-Dconfig=path/to/dspace.cfg) (I do build on the target host, because I've found no strong reason not to, but every site's needs are different.) If you've found other configuration settings that are used by Ant, maybe we can fix that. I keep a number of different configurations on file for use with various development and test deployments. At runtime, DSpace starts from a customized web.xml, but I'm not sure why we bother: the Context parameters (the customized bits) can be overridden in the Context descriptor if you're using Tomcat as the container, and I believe that the other popular containers have similar facilities. It seems to me that the ideal, in Servlet applications, is that the webapp. should be unaware of anything outside of its own existence until the container injects the specifics of this instance's deployment. We could make better use of that. -- Mark H. Wood Lead Technology Analyst University Library Indiana University - Purdue University Indianapolis 755 W. Michigan Street Indianapolis, IN 46202 317-274-0749 www.ulib.iupui.edu signature.asc Description: Digital signature -- ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
Re: [Dspace-devel] DSpace deployment methods
Hi Robin,
I am working to disentangle configuration from Maven Ant builds in the
Commons Configuration work (aimed for 6.0). Once we switch to Commons
Configuration, we should be able to simply remove all our custom config
variable interpolation magic that Maven Ant does. Commons
Configuration does interpolation on-the-fly, which means Maven and Ant
should no longer need to touch any of our configs (thought Ant will
still need a dspace.dir to install to obviously).
See https://github.com/DSpace/DSpace/pull/991 and
https://jira.duraspace.org/browse/DS-2654 for a bit more info.
- Tim
On 8/17/2015 5:07 AM, TAYLOR Robin wrote:
Hi Terry,
That's really useful, thanks. The main problem for me is that
configuration is so entwined with both the Maven build and the Ant
installation, it makes it difficult to find a process that avoids
having to do a Maven build on a live server, and hack various bits of
configuration specific to that server. Sigh
Cheers, Robin.
Robin Taylor
Main Library
University of Edinburgh
*From:* Terry Brady terry.br...@georgetown.edu
*Sent:* 14 August 2015 23:40
*To:* TAYLOR Robin
*Cc:* DSpace Developers
*Subject:* Re: [Dspace-devel] DSpace deployment methods
Robin,
I have been happy with the process that has evolved for us over time.
Since our repository code is private, we have cloned our github
repository on our server using an access token.
We have a build folder on each of our servers. That folder contains a
script that runs the following steps.
git fetch origin
git checkout ${BRANCH} -f
git pull
mvn clean package
cd dspace/target/dspace-build
ant update
#insert domain specific passwords/overrides into config files using sed
#restart webserver
When build.properties was first introduced, it did not allow us to
override what we needed to override. I have not revisited that file
to see if it could further simplify our build process.
On Thu, Aug 13, 2015 at 7:55 AM, TAYLOR Robin robin.tay...@ed.ac.uk
mailto:robin.tay...@ed.ac.uk wrote:
Hi all,
I notice there was a brief discussion about DSpace deployment
methods yesterday in the developer meeting. I would be very
interested to hear what others do, what sort of tools they use,
what sort of infrastructure they have etc. If anyone is willing to
share please do reply. For our part we are still stuck in a world
of 'scp' and shell scripts, so could benefit greatly from a bit of
modernisation :)
Many thanks, Robin.
Robin Taylor
Main Library
University of Edinburgh
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
--
___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
mailto:Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel
--
Terry Brady
Applications Programmer Analyst
Georgetown University Library Information Technology
https://www.library.georgetown.edu/lit/code
425-298-5498 (Seattle, WA)
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
--
___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel
--
___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel
