On 18/03/14 00:32, Martino Buffolino wrote:
Hi,
I ran tdbloader2 overnight on an amazon box and it seemed to run out of
disk space and ultimately crashed. Is there a way I can start the process
up, beginning at the sort step?
Here is the log:
INFO Add: 2,647,900,000 Data (Batch: 54,704 / Avg: 24,949)
INFO Add: 2,647,950,000 Data (Batch: 54,347 / Avg: 24,949)
INFO Add: 2,648,000,000 Data (Batch: 83,752 / Avg: 24,950)
INFO Elapsed: 106,130.99 seconds [2014/03/17 22:44:16 UTC]
INFO Total: 2,648,011,531 tuples : 106,438.23 seconds : 24,878.39
tuples/sec [2014/03/17 22:49:23 UTC]
22:49:25 Index phase
22:49:25 Index SPO
sort: write failed: /tmp/sortxRql3B: No space left on device
Thanks for any help,
Martino
Maybe - if the work files are still there.
You can edit the script tdbloader2worker:
1/ Set DATA_TRIPLES and DATA_QUADS to the relevant work files.
2/ Remove the data phase (comment out the first call to java for
CmdNodeTableBuilder)
3/ set KEEPWORKFILES=1
By the way, loading is much faster on Amazon if you use one of the
instances with a large SSD.
In fact, running plan "tdbloader" and using an SSD machine maybe the
best way. tdbloader does not have large intermediate files.
Andy
