Thanks Mark and Tim! Mark, could you speak more to "use the Servlet container's usual clustering features to share UI <-> REST session state"? I'm not familiar with Tomcat's clustering features or how DSpace manages sessions, but I'm a little familiar with Keycloak (which uses Wildfly) using session caching in Inifinispan, and JGroups for communicating between cluster nodes (about sessions) and node discovery (when multicast isn't available), while also using sticky sessions in the load balancer, so that a user-initiated session/request hits the same node. Out of the box, Keycloak doesn't do full session replication, but sessions are looked up from other cluster nodes on demand. A little bit of reading suggests Tomcat's node discovery is only done via multicast or static addresses (that's unusable with a third party cloud provider like AWS or Azure). Tomcat seems to use full session replication out of the box, although that has pros and cons. Looks like more reading to do there. I'm more worried about availability than capacity. I like to have at least 2 nodes of everything, so that if 1 goes down or needs downtime, then 1 node can keep serving users.
We tend to use object storage instead of filesystem storage these days for the filestore, so there's no problem there. That's interesting to know DSpace 7 will expect a separate Solr! That helps. David Cook Software Engineer Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Online: 02 8005 0595 -----Original Message----- From: Mark H. Wood,UL 0115A,+1 317 274 0749, <[email protected]> On Behalf Of Mark H. Wood Sent: Saturday, 12 September 2020 12:44 AM To: [email protected] Subject: Re: [dspace-tech] Insight into DSpace 7 On Fri, Sep 11, 2020 at 10:37:41AM +1000, [email protected] wrote: [...] > I have scale on my mind at lot these days. Would it be possible to run > multiple REST API nodes and multiple UI nodes? How much of the existing > DSpace system is going to be retained? I imagine a lot of the OAI-PMH code, > handle server, and such will remain untouched? I haven't tried, but I think it may be possible. The UI only communicates with the REST service. Multiple back-end instances should use the Servlet container's usual clustering features to share UI <-> REST session state. Maybe someone else can comment on session management across multiple front-end instances. But I would guess that this is not the first place you'll see capacity issues. The back end uses several data stores: o the database. The DBMS is responsible for coordinating concurrent access and should do it well. The DSpace business logic layer is careful to handle concurrent database access, since it may be driven by concurrent request threads. o the assetstore(s). Multiple back-ends would need a shared filesystem. o Solr. Unless you are using administrator tools (e.g. during upgrade), I think that all four cores simply accumulate records and answer search queries. o If you are offering RDF, the triplestore. I don't know whether there are concurrency issues here. o Each instance would have its own separate logs. The DBMS and Solr each have their own scale-out features, if needed. The Handle resolver communicates with DSpace solely through the database, so it should have no issues related to scaling out the DSpace back-end. DSpace 7 will expect you to install Solr separately, which means that you can put it on a different host. If you are concerned about scaling, you probably should do that. Here we already run (distinct, not scaled-out) DSpace 5 and 6 instances with a single DBMS on a separate host. This works well. -- Mark H. Wood Lead Technology Analyst University Library Indiana University - Purdue University Indianapolis 755 W. Michigan Street Indianapolis, IN 46202 317-274-0749 www.ulib.iupui.edu -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/20200911144421.GB17073%40IUPUI.Edu. -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/008a01d68a33%2448072000%24d8156000%24%40prosentient.com.au.
