On Tue, Nov 27, 2012 at 7:30 PM, Gunaratna, Dalkandura Arachchige Kalpa Shashika Silva <[email protected]> wrote: > Hi Hugh, > There was a bug in the script (ini file when uncommenting and had to > remove spaces). I think it is working fine now and loaded within few hours > and I can see its memory usage using top command and isql status(''); > message for number of buffers. > > I have a another issue though, I downloaded all the files for DBpedia > 3.8 dump but when I query an instance, it doesn't give me interconnections > (sameAs link) for other datasets like Freebase but only sameAs links within > dbpedia.
Also load the files in http://downloads.dbpedia.org/3.8/links/ Regards, Christopher > Following is the list of loaded files (which was downloaded from > dump and loaded). There if the folder, I have created a file called > global.graph as mentioned in the openLink website documentations but it > didn't get loaded. I can also see that external interlining file is also > loaded but I do not get sameAs links to Freebase. I checked this by running > the same query in dbpedia.org sparql endpoint and my local endpoint. Do you > have any idea what would have happened if anything went wrong in my > installation? Thank you very much for the help. > > > > > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/article_categories_en.nt.gz > http://dbpedia.org > 2 2012.11.21 21:47.14 0 2012.11.21 21:57.23 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/category_labels_en.nt.gz > http://dbpedia.org > 2 2012.11.21 21:57.23 0 2012.11.21 21:58.39 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/disambiguations_en.nt.gz > http://dbpedia.org > 2 2012.11.21 21:58.39 0 2012.11.21 21:59.43 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/disambiguations_unredirected_en.nt.gz > http://dbpedia.org > 2 2012.11.21 21:59.43 0 2012.11.21 22:0.5 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/external_links_en.nt.gz > http://dbpedia.org > 2 2012.11.21 22:0.5 0 2012.11.21 22:5.30 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/geo_coordinates_en.nt.gz > http://dbpedia.org > 2 2012.11.21 22:5.30 0 2012.11.21 22:7.6 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/global.graph > http://dbpedia.org > 2 2012.11.21 22:7.6 0 2012.11.21 22:7.6 0 0 NULL > 37000 SP029: TURTLE RDF loader, line 1: Undefined namespace prefix at > http://dbpedia.org > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/homepages_en.nt.gz > http://dbpedia.org > 2 2012.11.21 22:7.6 0 2012.11.21 22:7.33 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/images_en.nt.gz > http://dbpedia.org > 2 2012.11.21 22:7.33 0 2012.11.21 22:12.56 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/infobox_properties_en.nt.gz > http://dbpedia.org > 2 2012.11.21 22:12.56 0 2012.11.21 23:10.20 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/infobox_properties_unredirected_en.nt.gz > http://dbpedia.org > 2 2012.11.21 23:10.20 0 2012.11.21 23:37.1 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/infobox_property_definitions_en.nt.gz > http://dbpedia.org > 2 2012.11.21 23:37.1 0 2012.11.21 23:37.6 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/infobox_test_en.nt.gz > http://dbpedia.org > 2 2012.11.21 23:37.1 0 2012.11.22 0:28.20 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/instance_types_en.nt.gz > http://dbpedia.org > 2 2012.11.22 0:28.20 0 2012.11.22 0:36.7 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/interlanguage_links_en.nt.gz > http://dbpedia.org > 2 2012.11.22 0:36.7 0 2012.11.22 0:48.43 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/interlanguage_links_same_as_chapters_en.nt.gz > http://dbpedia.org > 2 2012.11.22 0:48.43 0 2012.11.22 0:55.18 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/interlanguage_links_same_as_en.nt.gz > http://dbpedia.org > 2 2012.11.22 0:55.19 0 2012.11.22 1:8.38 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/interlanguage_links_see_also_chapters_en.nt.gz > http://dbpedia.org > 2 2012.11.22 1:8.39 0 2012.11.22 1:9.6 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/interlanguage_links_see_also_en.nt.gz > http://dbpedia.org > 2 2012.11.22 1:9.6 0 2012.11.22 1:9.37 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/iri_same_as_uri_en.nt.gz > http://dbpedia.org > 2 2012.11.22 1:9.37 0 2012.11.22 1:10.41 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/labels_en.nt.gz > http://dbpedia.org > 2 2012.11.22 1:10.41 0 2012.11.22 1:23.53 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/long_abstracts_en.nt.gz > http://dbpedia.org > 2 2012.11.22 1:23.53 0 2012.11.22 1:36.56 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/mappingbased_properties_en.nt.gz > http://dbpedia.org > 2 2012.11.22 1:36.56 0 2012.11.22 1:59.15 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/mappingbased_properties_unredirected_en.nt.gz > http://dbpedia.org > 2 2012.11.22 1:59.15 0 2012.11.22 2:7.58 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/page_ids_en.nt.gz > http://dbpedia.org > 2 2012.11.22 2:7.58 0 2012.11.22 2:23.26 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/page_links_en.nt.gz > http://dbpedia.org > 2 2012.11.22 2:23.26 0 2012.11.22 4:59.20 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/page_links_unredirected_en.nt.gz > http://dbpedia.org > 2 2012.11.22 4:59.20 0 2012.11.22 6:15.17 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/persondata_en.nt.gz > http://dbpedia.org > 2 2012.11.22 6:15.17 0 2012.11.22 6:28.46 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/persondata_unredirected_en.nt.gz > http://dbpedia.org > 2 2012.11.22 6:28.46 0 2012.11.22 6:32.14 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/pnd_en.nt.gz > http://dbpedia.org > 2 2012.11.22 6:32.14 0 2012.11.22 6:32.14 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/redirects_en.nt.gz > http://dbpedia.org > 2 2012.11.22 6:32.14 0 2012.11.22 6:40.57 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/redirects_transitive_en.nt.gz > http://dbpedia.org > 2 2012.11.22 6:40.58 0 2012.11.22 6:45.28 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/revision_ids_en.nt.gz > http://dbpedia.org > 2 2012.11.22 6:45.28 0 2012.11.22 7:10.23 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/revision_uris_en.nt.gz > http://dbpedia.org > 2 2012.11.22 7:10.23 0 2012.11.22 7:46.7 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/short_abstracts_en.nt.gz > http://dbpedia.org > 2 2012.11.22 7:46.7 0 2012.11.22 8:1.54 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/skos_categories_en.nt.gz > http://dbpedia.org > 2 2012.11.22 8:1.54 0 2012.11.22 8:8.2 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/specific_mappingbased_properties_en.nt.gz > http://dbpedia.org > 2 2012.11.22 8:8.2 0 2012.11.22 8:9.19 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/topical_concepts_en.nt.gz > http://dbpedia.org > 2 2012.11.22 8:9.19 0 2012.11.22 8:9.28 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/topical_concepts_unredirected_en.nt.gz > http://dbpedia.org > 2 2012.11.22 8:9.19 0 2012.11.22 8:9.30 0 0 NULL > NULL > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/wikipedia_links_en.nt.gz > http://dbpedia.org > 2 2012.11.22 8:9.30 0 2012.11.22 8:50.6 0 0 NULL > NULL > > > > > ________________________________ > From: Hugh Williams [[email protected]] > Sent: Thursday, November 22, 2012 6:10 AM > > To: Gunaratna, Dalkandura Arachchige Kalpa Shashika Silva > Cc: [email protected] > Subject: Re: [Dbpedia-discussion] local dbpedia dump load to virtuoso server > > Hi Kalpa, > > Did you restart your Virtuoso instance having made these changes to enable > the new settings to take effect ? Also please provide a copy of the INI file > changes you have made so we can see what should be reported in the status > output ? > > Best Regards > Hugh Williams > Professional Services > OpenLink Software, Inc. // http://www.openlinksw.com/ > Weblog -- http://www.openlinksw.com/blogs/ > LinkedIn -- http://www.linkedin.com/company/openlink-software/ > Twitter -- http://twitter.com/OpenLink > Google+ -- http://plus.google.com/100570109519069333827/ > Facebook -- http://www.facebook.com/OpenLinkSoftware > Universal Data Access, Integration, and Management Technology Providers > > On 20 Nov 2012, at 17:50, Gunaratna, Dalkandura Arachchige Kalpa Shashika > Silva wrote: > > Hi Hugh, > Yes, I uncommented the lines and commented out the default number of > buffers before I started the server. I made the changes in virtuoso.ini in > var/lib/db folder. But following is the status I see when I type status(''); > in isql. I think it is still using 2000 buffers and I do not know why it is > using the default setting after I have changed the parameters. > > > > OpenLink Virtuoso Server > Version 06.01.3127-pthreads for Linux as of Nov 15 2012 > Started on: 2012/11/18 16:10 GMT-300 > > Database Status: > File size 29255270400, 3571200 pages, 760183 free. > 2000 buffers, 2000 used, 1175 dirty 6 wired down, repl age 429 441 w. io 3 > w/crsr. > Disk Usage: 352003762 reads avg 0 msec, 0% r 0% w last 0 s, 198597618 > writes, > 1311 read ahead, batch = 1. Autocompact 4406099 in 3148871 out, 28% > saved. > Gate: 3381 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap. > Log = /home/kalpa/Virtuoso/var/lib/virtuoso/db/virtuoso.trx, 2394315279 > bytes > 1545766 pages have been changed since last backup (in checkpoint state) > Current backup timestamp: 0x0000-0x00-0x00 > Last backup date: unknown > Clients: 3 connects, max 2 concurrent > RPC: 37 calls, 1 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0 > second brk=159555584 > Checkpoint Remap 339966 pages, 0 mapped back. 1997 s atomic time. > DB master 3571200 total 760183 free 339966 remap 44104 mapped back > temp 256 total 250 free > > Lock Status: 0 deadlocks of which 0 2r1w, 105 waits, > Currently 3 threads running 0 threads waiting 0 threads in vdb. > Pending: > > 23 Rows. -- 2234 msec. > > > > ________________________________ > From: Hugh Williams [[email protected]] > Sent: Tuesday, November 20, 2012 4:21 AM > To: Gunaratna, Dalkandura Arachchige Kalpa Shashika Silva > Cc: [email protected] > Subject: Re: [Dbpedia-discussion] local dbpedia dump load to virtuoso server > > Hi Kalpa, > > Have you tuning your Virtuoso instance to take advantage of the 32GB of > available RAM as detailed at: > > http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtRDFPerformanceTuning > > I suspect you are running out of allocated memory for the Virtuoso server as > the NumberOfBuffers param is low by default and need to be increase to match > the available memory as indicated in the above document. Run the > “status(‘’);” command from isql to see the status of the server in > particular the number of buffers available to ensure the setting has taken > effect. > > Best Regards > Hugh Williams > Professional Services > OpenLink Software, Inc. // http://www.openlinksw.com/ > Weblog -- http://www.openlinksw.com/blogs/ > LinkedIn -- http://www.linkedin.com/company/openlink-software/ > Twitter -- http://twitter.com/OpenLink > Google+ -- http://plus.google.com/100570109519069333827/ > Facebook -- http://www.facebook.com/OpenLinkSoftware > Universal Data Access, Integration, and Management Technology Providers > > On 20 Nov 2012, at 04:21, Gunaratna, Dalkandura Arachchige Kalpa Shashika > Silva wrote: > > Hi, > I am loading dbpedia version 3.8 into Virtuoso 6.1. I unzipped the files > into .gz format and ran ld_dir method in isql mode to load all files in a > folder (whole dump). Then I am running rdf_loader_run() to process the > files. I have two questions, > > 1. the server is having 32 GB of ram and 8 cores. But still it has taken 2 > days to load half the files. Why is it taking this much time? is it normal? > > > 2. When I check the status using SELECT * FROM DB.DBA.LOAD_LIST; one file > has a start timestamp but no end timestamp. Following is the relevant row > for that file in the output, > > /home/kalpa/Virtuoso/data/datasets/dbpedia/3.8/3.8/en/infobox_properties_en.nt.gz > http://dbpedia.org > 1 2012.11.17 2:12.5 0 NULL 0 NULL > NULL > > If I re-load that file only after the other files are processed, will it > add duplicate tuples or I have to remove that file and then load that file > only? If so, what is the process to do that? > If I commit after processing all files are finished (I assume I have to > commit using command commit WORK; to make changes permanent) and only load > that infobox_properties_en.nt.gz file only again, is it fine? Please let me > know some details about these two problems and make the local dbpedia server > working without any errors. Thank you. > > regards, > kalpa > > > ------------------------------------------------------------------------------ > Monitor your physical, virtual and cloud infrastructure from a single > web console. Get in-depth insight into apps, servers, databases, vmware, > SAP, cloud infrastructure, etc. Download 30-day Free Trial. > Pricing starts from $795 for 25 servers or applications! > http://p.sf.net/sfu/zoho_dev2dev_nov_______________________________________________ > Dbpedia-discussion mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion > > > > > > ------------------------------------------------------------------------------ > Monitor your physical, virtual and cloud infrastructure from a single > web console. Get in-depth insight into apps, servers, databases, vmware, > SAP, cloud infrastructure, etc. Download 30-day Free Trial. > Pricing starts from $795 for 25 servers or applications! > http://p.sf.net/sfu/zoho_dev2dev_nov > _______________________________________________ > Dbpedia-discussion mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion > ------------------------------------------------------------------------------ Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov _______________________________________________ Dbpedia-discussion mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
