GitHub user maxx-ukoo added a comment to the discussion: How to load big dataset to new database
@afs Thanks for the update. After a few extra tests, I think I found the issue — I need a lot of memory for the upload. I ran everything from scratch, starting with a 16 GB VM and a 4 GB Java heap. After the upload speed decreased, I increased the memory to 32 GB, then to 64 GB. Each memory upgrade improved the upload speed. However, when I reduced the memory from 64 GB to 32 GB, the upload speed decreased again. I believe memory is responsible for the upload speed. I didn’t use tdb2.xloader because it can’t upload data to the named graph. Also, I don’t know how to provide 400–600 files as command-line arguments for the upload. GitHub link: https://github.com/apache/jena/discussions/3701#discussioncomment-15671362 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
