On Tue, 2020-08-11 at 11:07 -0400, Mark H. Wood wrote: > On Tue, Aug 11, 2020 at 01:34:48PM +0000, Michael White wrote: > > Our DSpace instance has been steadily growing over the years and > > now has over 18,000 records, all with one or more full text files > > attached (taking up around 106GB of disk space). We are on DSpace > > 6.2 and currently only have one assetstore configured. > > > > Whilst we have no issues with dspace operation/performance, my > > System and Network colleagues are reporting problems when their > > scheduled backup jobs of the DSpace server are trying to run as > > they are timing out, which they suspect is due to the large number > > of files on the server. > > > > To resolve this, they have asked me about the possibility of > > splitting the current dspace filesystem in to a number of smaller > > filesystems (which they can then back up in parallel, reducing the > > overall time to back up the dspace server). > > > > To that end they have asked about the possibility of splitting the > > assetstore in to a number of filesystems and creating softlinks to > > these from the assetstore - e.g. adding 9 new filesystems for > > clusters of 10 assetstore subdirectories - i.e.: > > > > .../dspace/assetstore/10 -> .../assetstores10-19/10 > > .../dspace/assetstore/11 -> .../assetstores10-19/11 > > .../dspace/assetstore/12 -> .../assetstores10-19/12 > > .... ... > > .../dspace/assetstore/20 -> ../assetstores20-29/20 > > > > Has anyone ever done anything like this? Any reasons why that > > wouldn't work?
Not really. However one should consider using mountpoints in the assetstore itself instead. That would make it easier to administrate. > > I see no obvious reason why it wouldn't work. > > "They" must think that the backup storage device is the bottleneck, > if > they believe that parallel backups will improve throughput. Ehm, not following you here, Mark. What I think they want to achieve is a higher client parallelism. This means that instead of one save stream running on one file system they can have several savestreams running, one on each filesystem. If they had backup storage performance issues that would not make sense. So, going for higher parallelism by having several smaller file systems makes sense. > I would > do some exploring to see if this is true, before doing a lot of work > to redistribute all those files. Is there another lurking > bottleneck, > nearly as narrow, currently covered by the one they are working on? Excellent point, can the DSpace server handle the extra network traffic. Can the network? > > Each of those ten volumes will continue to grow, but at 1/10 the > current rate, so this puts the problem off for some time but doesn't > solve it in the long run. That will give you some time to ponder > whether there is a smarter way to back up your repository. > > > I can, of course, add another assetstore alongside the existing > > one, but my understanding is that this would only be used going > > forwards, and so wouldn't address the large number of files in the > > current assetstore - unless there is a way to then distribute the > > existing assetstore across a number of smaller assetstores? > > Your understanding is correct: all new submissions go into the > "current assetstore". > > There is a "BitStore migration tool" which moves *all* content from > one > assetstore to another: > > https://wiki.lyrasis.org/display/DSDOC6x/Storage+Layer#StorageLayer-MigrateBitStores > > That isn't quite what you want, but it might serve as a model for a > tool to split up an assetstore. > > If you do go this route, you might look at changing the way you do > backup. The materials in the "old" assetstores are probably subject > to very infrequent deletions but are otherwise static storage -- all > new and replacement content goes into the "current" assetstore. You > might devise a very different backup schedule for the old assetstores > vs. current. I totally agree. For mostly static content you could do a full backup once or twice a year combined with incremental and cumulative incremental backups. I'd suggest something like this: .../dspace/assetstore/10 --mountpoint, filesystem10 .../dspace/assetstore/11 --mountpoint, filesystem11 .../dspace/assetstore/12 --mountpoint, filesystem12 .... ... .../dspace/assetstore/20 --mountpoint, filesystem20 That would do for the few houndreds of gigabytes you have. If you had > 10x that, I'd probably go for something like ZFS with replication and snapshot backup Good luck, /tony > > -- > Mark H. Wood > Lead Technology Analyst > > University Library > Indiana University - Purdue University Indianapolis > 755 W. Michigan Street > Indianapolis, IN 46202 > 317-274-0749 > www.ulib.iupui.edu > -- Tony Albers - Systems Architect - IT Development Royal Danish Library, Victor Albecks Vej 1, 8000 Aarhus C, Denmark Tel: +45 2566 2383 - CVR/SE: 2898 8842 - EAN: 5798000792142 -- All messages to this mailing list should adhere to the DuraSpace Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/f798ffb3b357e6f0d7b4855c5c8d55849ddf0ce5.camel%40kb.dk.
