Hi Hugh,
Thank you for your comments.
When the loader fails, I continue from the point it failed.
I want to clarify, that virtuoso server, was not crashing, but the loader.
Our custom loader uses only one thread. We submit one file at time,
that contains 10 000 000 triples
When I increased waiting time in our virtuoso loader, then it loads the
batch file, but for every batch it takes more and more time, which we know
is expected.
But it still doesn't explain why virtuoso is working so slow.
I tested read speed of disks (using "hdparm -Tt /dev/sdb" command )
that I have attached and where database files are located and for each of
them read speed of 46.19 MB/sec.
When I run loader and observe virtuoso disk utilization, it doesn't go
above 4 MB/sec.
And in Goolge developers console, I can see that disk IO and CPU
utilization is close to zero.
Using SSD disks is not an option, as Googel cloud doesn't provide such
service.
In virtuoso logs I see these messages:
...
14:33:50 * Monitor: CPU% is low while there are large numbers of runnable
threads
14:34:06 * Monitor: High disk read (2)
14:35:50 * Monitor: CPU% is low while there are large numbers of runnable
threads
14:36:06 * Monitor: High disk read (2)
14:38:53 * Monitor: High disk read (2)
14:40:31 * Monitor: CPU% is low while there are large numbers of runnable
threads
14:40:53 * Monitor: High disk read (2)
14:42:43 * Monitor: CPU% is low while there are large numbers of runnable
threads
14:42:53 * Monitor: High disk read (2)
14:44:53 * Monitor: High disk read (2)
14:45:31 * Monitor: CPU% is low while there are large numbers of runnable
threads
14:46:59 * Monitor: High disk read (2)
14:47:37 * Monitor: CPU% is low while there are large numbers of runnable
threads
14:49:11 * Monitor: High disk read (2)
14:51:08 * Monitor: CPU% is low while there are large numbers of runnable
threads
14:51:12 * Monitor: High disk read (2)
...
Is it because it is cloud, that virtuoso has problems to fully utilize the
disks?
Are there any parameters I could configure?
As length of mail is limited, below is link to output of LDMeter.
http://pastebin.com/ASNSamtf
Regards,
Andrejs
------------------------------------------------------------------------------
Put Bad Developers to Shame
Dominate Development with Jenkins Continuous Integration
Continuously Automate Build, Test & Deployment
Start a new project now. Try Jenkins in the cloud.
http://p.sf.net/sfu/13600_Cloudbees
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
