Re: [Dspace-devel] DSpace deployment methods

[TAYLOR Robin]

Hi all,


Apologies for keeping this one running, but I'm still thinking about this 
subject. My gut feeling is that I don't want to run a Maven build on a live 
server, for two reasons...


1. Java has always claimed to be a 'Build once, run anywhere' language. So I 
want to build and test elsewhere, then deploy the tested app.

2. I instinctively don't like having development tools on a live server, it 
feels like a dodgy thing to do, although I would struggle to justify that 
statement.


At the moment we only copy the target build directory to our live server, and 
then run 'ant update...' on the live server. But even that feels a bit hacky.


At this point I should stop talking and read up on Tim's Commons Configuration 
work.


Cheers.


Robin Taylor
Main Library
University of Edinburgh



From: Tim Donohue 
Sent: 17 August 2015 15:10
To: dspace-devel@lists.sourceforge.net
Subject: Re: [Dspace-devel] DSpace deployment methods

Hi Robin,

I am working to disentangle configuration from Maven & Ant builds in the 
Commons Configuration work (aimed for 6.0).  Once we switch to Commons 
Configuration, we should be able to simply remove all our custom config 
variable interpolation "magic" that Maven & Ant does.  Commons Configuration 
does interpolation on-the-fly, which means Maven and Ant should no longer need 
to touch any of our configs (thought Ant will still need a dspace.dir to 
install to obviously).

See https://github.com/DSpace/DSpace/pull/991 and 
https://jira.duraspace.org/browse/DS-2654 for a bit more info.

- Tim

On 8/17/2015 5:07 AM, TAYLOR Robin wrote:

Hi Terry,


That's really useful, thanks. The main problem for me is that configuration is 
so entwined with both the Maven build and the Ant installation, it makes it 
difficult to find a process that avoids having to do a Maven build on a live 
server, and hack various bits of configuration specific to that server. Sigh 
[??]


Cheers, Robin.


Robin Taylor
Main Library
University of Edinburgh



From: Terry Brady 

Sent: 14 August 2015 23:40
To: TAYLOR Robin
Cc: DSpace Developers
Subject: Re: [Dspace-devel] DSpace deployment methods

Robin,

I have been happy with the process that has evolved for us over time.

Since our repository code is private, we have cloned our github repository on 
our server using an access token.

We have a build folder on each of our servers.  That folder contains a script 
that runs the following steps.

git fetch origin
git checkout ${BRANCH} -f
git pull
mvn clean package
cd dspace/target/dspace-build
ant update
#insert domain specific passwords/overrides into config files using sed
#restart webserver

When "build.properties"  was first introduced, it did not allow us to override 
what we needed to override.  I have not revisited that file to see if it could 
further simplify our build process.

On Thu, Aug 13, 2015 at 7:55 AM, TAYLOR Robin 
mailto:robin.tay...@ed.ac.uk>> wrote:
Hi all,

I notice there was a brief discussion about DSpace deployment methods yesterday 
in the developer meeting. I would be very interested to hear what others do, 
what sort of tools they use, what sort of infrastructure they have etc. If 
anyone is willing to share please do reply. For our part we are still stuck in 
a world of 'scp' and shell scripts, so could benefit greatly from a bit of 
modernisation :)

Many thanks, Robin.

Robin Taylor
Main Library
University of Edinburgh
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.


--
___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel



--
Terry Brady
Applications Programmer Analyst
Georgetown University Library Information Technology
https://www.library.georgetown.edu/lit/code
425-298-5498 (Seattle, WA)



The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.




--




___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel


--
___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel

Re: [Dspace-devel] DSpace deployment methods

[Peter Dietz]

Hi Robin,

I'm hoping for the build/deploy/config process to be easier. If we
eventually had configurations living within the DB (within the instance)
that would be great. An instance could set config from the UI, and
not-quite-reload, but just the config is updated, that would be great. That
is however a different subject, a helpful one though.


Build Once, Run Anywhere. Yep.
We build on a server that doesn't run DSpace. Then scp the dspace-installer
over to the app server.
For each instance we have their config directory in its own git repository.
So, locally I can upgrade configs for a site, push it to git. Then for that
site, pull its configs back down. So on each app server, it might be ant
-Dconfig=/home/some-university/config/dspace.cfg update, Since ant is a bit
egregious with touching your configs, its followed with cd
/home/some-university/config; git reset HEAD --hard; then restarting tomcat.

There's more to do. SOLR 5 is going to change the game for solr. Such that
DSpace won't be able to ship solr, it will no longer be a tomcat webapp,
but it will instead be a standalone service / daemon. So, we'll need to
clean up (if we have any custom solr code), and figure that out. And/or, an
implementation for some other flexible no-sql data store, that can handle
search/browse/statistics/faceting, ...

The concept of a SITE within DSpace is interesting. Instead of having so
many instances of DSpace, what.. really is distinct between instance A and
instance B? The theme, the content, a few configurations. Expand the SITE
DSO of DSpace to handle multiple DSpace sites. So then one DSpace instance
can run dozens of SITES. Upgrading your instance of DSpace... upgrades all
of your sites. boom.

I'm dreaming here.
I'm really trying to shed data OFF of the dspace instance, but rather into
safe harbors. A dedicated postgres server, a log server, bitstreams into
s3, configuration into the DB, sessions.. I don't know where. Thus, an
instance of DSpace is really nothing, just the running application, your
data is elsewhere. So instead of updating your instance. The eventual plan
could be a "stateless" server. When your running DSpace build 100, and your
ready to deploy DSpace build 105, just build new instances of 105, and then
have your load balancer stop and shut down the build 100 instances. Do you
update/upgrade, or just in with the new, out with the old.



Peter Dietz
Longsight
www.longsight.com
pe...@longsight.com
p: 740-599-5005 x809

On Fri, Aug 21, 2015 at 10:19 AM, TAYLOR Robin 
wrote:

> Hi all,
>
>
> Apologies for keeping this one running, but I'm still thinking about this
> subject. My gut feeling is that I don't want to run a Maven build on a live
> server, for two reasons...
>
>
> 1. Java has always claimed to be a 'Build once, run anywhere' language. So
> I want to build and test elsewhere, then deploy the tested app.
>
> 2. I instinctively don't like having development tools on a live server,
> it feels like a dodgy thing to do, although I would struggle to justify
> that statement.
>
>
> At the moment we only copy the target build directory to our live server,
> and then run 'ant update...' on the live server. But even that feels a bit
> hacky.
>
>
> At this point I should stop talking and read up on Tim's Commons
> Configuration work.
>
>
> Cheers.
>
>
> Robin Taylor
> Main Library
> University of Edinburgh
>
>
> --
> *From:* Tim Donohue 
> *Sent:* 17 August 2015 15:10
> *To:* dspace-devel@lists.sourceforge.net
>
> *Subject:* Re: [Dspace-devel] DSpace deployment methods
>
> Hi Robin,
>
> I am working to disentangle configuration from Maven & Ant builds in the
> Commons Configuration work (aimed for 6.0).  Once we switch to Commons
> Configuration, we should be able to simply remove all our custom config
> variable interpolation "magic" that Maven & Ant does.  Commons
> Configuration does interpolation on-the-fly, which means Maven and Ant
> should no longer need to touch any of our configs (thought Ant will still
> need a dspace.dir to install to obviously).
>
> See https://github.com/DSpace/DSpace/pull/991 and
> https://jira.duraspace.org/browse/DS-2654 for a bit more info.
>
> - Tim
>
> On 8/17/2015 5:07 AM, TAYLOR Robin wrote:
>
> Hi Terry,
>
>
> That's really useful, thanks. The main problem for me is that
> configuration is so entwined with both the Maven build and the Ant
> installation, it makes it difficult to find a process that avoids having to
> do a Maven build on a live server, and hack various bits of configuration
> specific to that server. Sigh [image: 😊]
>
>
> Cheers, Robin.
>
>
> Robin Taylor
> Main Library
> University of Edinburgh
>
>
> --
> *From:* Terry Brady 
> 
> *Sent:* 14 August 2015 23:40
> *To:* TAYLOR Robin
> *Cc:* DSpace Developers
> *Subject:* Re: [Dspace-devel] DSpace deployment methods
>
> Robin,
>
> I have been happy with the process that has evolved for us over time.
>
> Since our reposit

[Dspace-devel] NOTICE: DSpace Developer list migrating to Google Groups TODAY

[Tim Donohue]

Hello DSpace-Devel subscribers,

This dspace-devel SourceForge mailing list will be migrated to Google 
Groups TODAY  (Aug 21).

No action on your part should be required.

* You will be subscribed to the new dspace-devel Google Groups mailing 
list (and receive a notice of that new subscription)
* You will be unsubscribed from the old Sourceforge list (So, please 
disregard any unsubscription email you may receive today.)
* Your existing subscription options (no-mail or digest mode setting) 
should be carried over to the new Google Group.

Once the migration is complete, a final notice will be sent to all 
subscribers providing additional information.

This migration from SourceForge to Google Groups is taking place for the 
following reasons:
1) SourceForge hosting has been unstable recently. Our SF mailing lists 
have gone down twice in the last six months.
2) Google Groups provides better list search and management capabilities.
3) Other open source repository projects use Google Groups and have had 
good experiences with it

Thanks,

Tim Donohue

-- 
Tim Donohue
Technical Lead for DSpace & DSpaceDirect
DuraSpace.org | DSpace.org | DSpaceDirect.org


--
___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel

[Dspace-devel] NOTICE: DSpace Developers list moved to Google Groups. Please RESUBSCRIBE yourself!

[Tim Donohue]

Hello DSpace-Devel subscribers,

The migration of the dspace-devel list archives from SourceForge to 
Google Groups has succeeded.

 From this point forward, please use dspace-de...@googlegroup.com
https://groups.google.com/d/forum/dspace-devel

Unfortunately, it seems Google has implemented a new policy which 
doesn't allow us to automatically re-subscribe more than 100 members per 
day. This policy is per Google Group, so even logging in as a separate 
admin account doesn't allow us to add more than 100 members per day. 
(The dspace-devel mailing list currently has ~500 subscribers, so only 
1/5 of you were resubscribed automatically.)

If you did NOT receive an email that you were subscribed to the Google 
Group, then you MUST subscribe yourself to continue to receive 
dspace-devel messages.

TO RESUBSCRIBE to dspace-devel, please choose one of the following options:
* Send an email dspace-devel+subscr...@googlegroups.com
* OR, visit https://groups.google.com/d/forum/dspace-devel and click 
"Join Group" at the top. You can then also modify your subscription 
options (digest mode, etc)

This will be the FINAL email sent across the dspace-devel SourceForge 
mailing list.  To continue to receive dspace-devel emails, you must 
subscribe yourself to the new dspace-devel Google Group.

Sorry for the inconvenience,

Tim

-- 
Tim Donohue
Technical Lead for DSpace & DSpaceDirect
DuraSpace.org | DSpace.org | DSpaceDirect.org


--
___
Dspace-devel mailing list
Dspace-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-devel