Re: TDB2 parallel load on cloud SSD and other observations/questions

2020-06-22 Thread Isroel Kogan
thank you Rob - I have confused the terminology. Indeed each run processes 0.5m quads. what is UOM of the batch loading? M/s? Looking at the output of iotop - the 3 main threads - which comprise the lionshare of the activity - have pretty steady reads - of about 900-950 M/s w little variation

Re: TDB2 parallel load on cloud SSD and other observations/questions

2020-06-22 Thread Rob Vesse
Isabel I think there might be a fundamental misunderstanding happening about batch sizes here. The batch sizes are fixed for a run and never changes, the "batch size" you refer to is a speed calculation e.g 19:03:24 INFO loader :: Add: 248,000,000 github_1_fixed.nq (Batch: 3,562 /