ERRATA: > I don't know why then. Maybe SSD is making all the difference. Try to load it > (or "latest-all") on a comparable machine using a single SATA disk instead of > SSD.
s/SATA/HDD ---------------------------- > I loaded 2.2B on a 16G machine which wasn't even server class (i.e. it's > I/O path to SSD isn't very quick). I don't know why then. Maybe SSD is making all the difference. Try to load it (or "latest-all") on a comparable machine using a single SATA disk instead of SSD. Around 100-150M my computer slows dows significantly, and then always down from here. All I know is that it's either because of too little RAM, or because the disk can't keep up. > If RAM really is at 1G , even on your small 8G server, suggests your > setup is configured in the OS to restrict the RAM for mapping. RAM per > process should be > real RAM (remember memory mapped files are used) or > the VM is setup in some odd way. Or 32bit java. Yeah sorry I was looking at shared memory. Right now resident memory is ~3.5GB and virtual ~5.5GB. Process started with 150K triples per second, now after 250M triples processed is at 50K triples/second and slowing down (processing batches of 25K). I don't know what to say, I think the conclusion is simply that tdbloader (any version) just doesn't work with large graphs on HDDs. So the only solution has to be to use an SSD, or find a way to split the graph into smaller stores, or simply give up. $ java -version openjdk version "1.8.0_151" OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12) OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode) $ ulimit -a -t: cpu time (seconds) unlimited -f: file size (blocks) unlimited -d: data seg size (kbytes) unlimited -s: stack size (kbytes) 8192 -c: core file size (blocks) 0 -m: resident set size (kbytes) unlimited -u: processes 31370 -n: file descriptors 1024 -l: locked-in-memory size (kbytes) unlimited -v: address space (kbytes) unlimited -x: file locks unlimited -i: pending signals 31370 -q: bytes in POSIX msg queues 819200 -e: max nice 0 -r: max rt priority 95 -N 15: unlimited
