> the nodes don’t work while we copy the TDB (it’s a limitation that comes from > Jena/TDB)
Most people doing this would copy the new db to a separate location on the remote machine (while Fuseki continues to run), then quickly take Fuseki down, swap the dbs (possible using a symlink) and then bring Fuseki back up-- it should take a few seconds at most. If your node storage cannot store more than one copy on a node, that is not a limitation of Jena. Otherwise, as Lorenz pointed out, wholesale copying the db is not the most network-efficient way to do this. Have you considered some delta-distribution system like RDF Delta? https://afs.github.io/rdf-delta/ Adam > On Dec 22, 2017, at 7:55 PM, Dimov, Stefan <[email protected]> wrote: > > Yes, this is high av., deployment and backup. > > We have cloud system with multiple nodes and TDB needs to be copied to/from > these nodes. Copied from – when we need to back it up and copied to – when we > deploy it. So, the problems come when we copy from/to the nodes – it takes > time, operations sometimes fail, downtime - the nodes don’t work while we > copy the TDB (it’s a limitation that comes from Jena/TDB) and also sometimes > the hardware OS is throwing errors just because there are big files on it > (without even copying them). > > Before you say “you need a better hardware”, I agree, but that’s what we have > right now, so I was just wondering if somebody had similar problems and how > they resolved it (apart from replacing the hardware with faster, bigger and > high av. one ( ) > > Regards, > Stefan > > > On 12/22/17, 4:24 PM, "ajs6f" <[email protected]> wrote: > > Can you tell us about the workflow that is managing them? You mention > copying and maintaining TDB files-- is that for high availability? Are you > writing updates to a master and replicating the resulting store, or some > other scenario? > > ajs6f > >> On Dec 22, 2017, at 7:19 PM, Dimov, Stefan <[email protected]> wrote: >> >> Our TDB now is about 32G and I see that some of its files are almost 5G >> (single file size), but let’s assume it can/will grow 3, 4, 5 times … >> >> On 12/22/17, 2:16 PM, "Dick Murray" <[email protected]> wrote: >> >> How big? How many? >> >> On 22 Dec 2017 8:37 pm, "Dimov, Stefan" <[email protected]> wrote: >> >>> Hi all, >>> >>> We have a project, which we’re trying to productize and we’re facing >>> certain operational issues with big size files. Especially with copying and >>> maintaining them on the productive cloud hardware (application nodes). >>> >>> Did anybody have similar issues? How did you resolve them? >>> >>> I will appreciate if someone shares their experience/problems/solutions. >>> >>> Regards, >>> Stefan >>> >> >> > > >
