Hi Hugh,
To give a detailed look into the load and the insert statements, i have
provided the statements below:
DB.DBA.RDF_LOAD_RDFXML(file_to_string_output('/data/graph1.rdf'),'http://test/','http://test/graph1');
sparql
BASE <http://test/>
PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>
PREFIX test:<http://test/#>
INSERT INTO GRAPH <graph1> {
?class ?rel_id ?class.
}
WHERE {
GRAPH <graph1> {
?class a ?type.
}
GRAPH <graph3> {
?rel_id ssb:type_instance_of ssb:METAREL_0000023. #reflexive relation type
}
}
;
The first statement RDF_LOAD_RDFXML creates the graph and loads the graph1.rdf
file.
The next insert query, matches the existing records from graph1 and compares it
to the records in graph3 and then inserts into graph1.
during this insert, the output was exactly like what i have shared earlier.
Count of Triples:
The number of triples that are loaded into the graph1 cannot be determined as
the triples are generated on the fly during insert.
This has let me to suspect the output i got.
cheers,
Nirmala
-----Original Message-----
From: Hugh Williams [mailto:[email protected]]
Sent: Saturday, November 21, 2009 5:14 PM
To: Nirmala Seehappan
Cc: [email protected]
Subject: Re: [Virtuoso-users] Upper Limit of triples inserted
Hi Nirmala,
I wanted to see the complete isql output on screen showing which function you
are using to perform these dataset uploads. As you stated
DB.DBA.RDF_LOAD_RDFXML was being used and I suggested you use the
multi-threaded DB.DBA.RDF_LOAD_RDFXML_MT variant for large datasets, but you
have not confirmed if this is being used.
Also, as stated previously I note you are running the Virtuoso 5.0.8 build, and
would thus recommend you also try with the latest 5.0.12 archive available for
download as their have been some fixes for loading RDF data in this build:
http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSDownload
You indicate their should be 300,127 triples, how has this been determined, and
if you run the query "sparql seclect count(*) from <http://test/graph1> where
{?s ?p ?o}" what triple count does this return in comparison to the number of
triples you are expecting ?
Best Regards
Hugh Williams
Professional Services
OpenLink Software
Web: http://www.openlinksw.com
Support: http://support.openlinksw.com
Forums: http://boards.openlinksw.com/support
Twitter: http://twitter.com/OpenLink
On 19 Nov 2009, at 13:24, Nirmala Seehappan wrote:
> Hi Hugh,
>
> yes. you are right. I hit the "TransactionAfterImageLimit" error earlier. But
> now, i donot get the same error.
> Indeed, i do not get any errors, also the virtuoso logs also do not report
> errors.
> But on looking into the output of the insert graph statement, i suspect that
> there might be something wrong with the output or the way the records are
> inserted into the graph.
>
> I am uploading an input rdf file of size 124 MB, the number of triples that
> is uploaded is 300,127 triples.
>
>
> Here is the output of the insert statement:
>
> Clear <http://test/graph1> -- done
>
> 1 Rows. -- 470466 msec.
>
> Done. -- 71766 msec.
> callret-0
> VARCHAR
> ______________________________________________________________________
> _________
>
> Insert into <http://test/graph1>, 100001 triples -- done
>
> 1 Rows. -- 33144 msec.
> callret-0
> VARCHAR
> ______________________________________________________________________
> _________
>
> Insert into <http://test/graph1>, 124 triples -- done
>
> 1 Rows. -- 1798 msec.
> callret-0
> VARCHAR
> ______________________________________________________________________
> _________
>
> Insert into <http://test/graph1>, 100001 triples -- done
>
> 1 Rows. -- 10846 msec.
> callret-0
> VARCHAR
> ______________________________________________________________________
> _________
>
> Insert into <http://test/graph1>, 100001 triples -- done
>
> 1 Rows. -- 46552 msec.
>
> Done. -- 293 msec.
> callret-0
> VARCHAR
>
> I actually doubt if i have the correct number of triples uploaded into the
> store.
>
> cheers,
> Nirmala
>
>
>
> -----Original Message-----
> From: Hugh Williams [mailto:[email protected]]
> Sent: Wednesday, November 18, 2009 6:06 PM
> To: Nirmala Seehappan
> Cc: [email protected]
> Subject: Re: [Virtuoso-users] Upper Limit of triples inserted
>
> Hi Nirmala,
>
> I am slightly confused, back in July you where getting "Transaction aborted
> because it's log after image size went above the limit" errors performing
> SPARUL insert queries and advised to the the RDF_LOAD_RDFXML_MT function for
> loaded large datasets or increase the "TransactionAfterImageLimit" ini file
> param which controls the log size limit, are you not hitting this issue again
> here ?
>
> How many triples are you attempting to insert and what if any errors are
> occurring when attempting this ? Do check the virtuoso.log file to see if any
> errors are being logged their also ?
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software
> Web: http://www.openlinksw.com
> Support: http://support.openlinksw.com
> Forums: http://boards.openlinksw.com/support
> Twitter: http://twitter.com/OpenLink
>
> On 18 Nov 2009, at 12:32, Nirmala Seehappan wrote:
>
>> Hi Hugh,
>>
>> i don't face any problem with the load of the data,but my problem is with
>> the insert statements.
>>
>> INSERT INTO GRAPH <http://test/graph1> {....};
>>
>> The result of this is broken into groups of 100001 triples(instead of
>> a single insert result statement)
>>
>> cheers,
>> Nirmala
>>
>>
>> -----Original Message-----
>> From: Hugh Williams [mailto:[email protected]]
>> Sent: Tuesday, November 17, 2009 3:23 PM
>> To: Nirmala Seethappan
>> Cc: [email protected]
>> Subject: Re: [Virtuoso-users] Upper Limit of triples inserted
>>
>> Hi Nirmala,
>>
>> Their should be no limits to the number of triples that can be loaded. For
>> large datasets however we do recommend the use of the rdf_load_rdfxml_mt
>> function which is a multi-threaded variant of the function you are using as
>> detailed at:
>>
>> http://docs.openlinksw.com/virtuoso/fn_rdf_load_rdfxml_mt.html
>>
>> What is the actual number of triples in the dataset being loaded ? Assuming
>> their is one per line you can use the following to determine this:
>>
>> wc -l <dataset_file_name>
>>
>> I note you are running the Virtuoso 5.0.8 build, and would thus recommend
>> you also try with the latest 5.0.12 archive available for download as their
>> have been some fixes for loading RDF data in this build:
>>
>> http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSDownload
>>
>>
>> Best Regards
>> Hugh Williams
>> Professional Services
>> OpenLink Software
>> Web: http://www.openlinksw.com
>> Support: http://support.openlinksw.com
>> Forums: http://boards.openlinksw.com/support
>> Twitter: http://twitter.com/OpenLink
>>
>> On 17 Nov 2009, at 13:21, Nirmala Seehappan wrote:
>>
>>> Hi,
>>>
>>> I'm working in virtuoso 5.0.8 and trying to upload data into the graphs
>>> using DB.DBA.RDF_LOAD_RDFXML.
>>> The output of the insert statement is as follows:
>>> _______________________________________________
>>> Insert into <http://test/graph1>, 100001 triples -- done
>>>
>>> 1 Rows. -- 10846 msec.
>>> callret-0
>>> VARCHAR
>>> _______________________________________________
>>>
>>> The datafile has more triples(>100001), but i doubt if the insert statement
>>> is limited to 100001 triples per insert statement. This same line gets
>>> repeated more often when the input file size is bigger.
>>>
>>> Is there any limit for the insert statements ?
>>> If so, are there any parameters in the virtuoso.ini file that controls this
>>> limit?
>>>
>>> Please do provide me more understanding on this issue.
>>>
>>> cheers,
>>> Nirmala
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> -
>>> -------- Let Crystal Reports handle the reporting - Free Crystal
>>> Reports 2008 30-Day trial. Simplify your report design, integration
>>> and deployment - and focus on what you do best, core application
>>> coding. Discover what's new with Crystal Reports now.
>>> http://p.sf.net/sfu/bobj-july
>>> _______________________________________________
>>> Virtuoso-users mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> -
>> -------- Let Crystal Reports handle the reporting - Free Crystal
>> Reports 2008 30-Day trial. Simplify your report design, integration
>> and deployment - and focus on what you do best, core application
>> coding. Discover what's new with Crystal Reports now.
>> http://p.sf.net/sfu/bobj-july
>> _______________________________________________
>> Virtuoso-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
>
>
>
