Hi Hugh,
Please find attached the virtuoso.log file and the exception received by the
java client when we try to execute a big insert query.
Regards,
Jonathan Dray
2009/7/28 Hugh Williams <[email protected]>
> Hi Jonathan,
> Can you please use the Virtuoso trace_on() function to enable additional
> server side tracing to the virtuoso.log file as detailed at:
>
> http://docs.openlinksw.com/virtuoso/fn_trace_on.html
>
>
> Then attempt to run the 2 test cases to produce the error and provide the
> resultant virtuoso.log where the trace information will be written to.
>
>
> Note you can use the complementary trace_off() function to immediately turn
> off tracing when done ...
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software
> Web: http://www.openlinksw.com
> Support: http://support.openlinksw.com
> Forums: http://boards.openlinksw.com/support
>
>
>
> On 28 Jul 2009, at 11:05, Jonathan Dray wrote:
>
> Hi,
>
> We have a problem with the construction of valid insert queries using Jena
> and virtuoso.
> We often manipulate a big volume of data which are converted to rdf and
> inserted into virtuoso.
>
> A virtuoso update request is constructed and the exec method is called.
>
> Now when the insert request is big, there are 2 cases of failure :
>
> * The number of lines is too important
> * A line size in the insert query is too big
>
> Here is the excerpt from the stack trace containg the returned error
> message :
>
> Caused by: com.hp.hpl.jena.update.UpdateException: Convert results are
> FAILED.
> at virtuoso.jena.driver.VirtuosoUpdateRequest.exec(Unknown Source)
> at
> fr.bnf.spar.rdf.jena.query.InsertQuery.execInsert(InsertQuery.java:44)
> ... 54 common frames omitted
> Caused by: virtuoso.jdbc3.VirtuosoException: SQ199: Maximum size (32767) of
> a code vector exceeded by 131745 bytes. Please split the code in smaller
> units.
> at virtuoso.jdbc3.VirtuosoResultSet.process_result(Unknown Source)
> at virtuoso.jdbc3.VirtuosoResultSet.<init>(Unknown Source)
> at virtuoso.jdbc3.VirtuosoStatement.sendQuery(Unknown Source)
> at virtuoso.jdbc3.VirtuosoStatement.execute(Unknown Source)
> ... 56 common frames omitted
>
> Here is the code that creates a VirtuosoUpdateRequest from a Jena Model
> Object with named graph support :
>
> StringBuilder request = new StringBuilder();
> request.append(PrefixContants.PREFIXES)
> .append("INSERT INTO GRAPH <")
> .append("namedgraph")
> .append("> {\n");
> StmtIterator it = model.listStatements();
> while(it.hasNext()) {
> Statement st = it.nextStatement();
> request.append(FmtUtils.stringForTriple(st.asTriple()))
> .append(" . ");
> }
> request.append("}");
> VirtuosoUpdateRequest vur =
> VirtuosoUpdateFactory.create(request.toString(), virtGraph);
> vur.exec()
>
>
> First, is our construction of insert queries correct / efficient ?
> Is this behaviour normal ? and if it is the case, do you provide any tool /
> librairy to split a query according to these limitations ?
> How do we preserve transactional behaviour if we still have to split an
> insert query into multiple queries ?
>
> Thanks.
> Regards,
>
> Jonathan Dray
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
>
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.
> http://p.sf.net/sfu/bobj-july_______________________________________________
> Virtuoso-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel
>
>
>
com.hp.hpl.jena.update.UpdateException: Convert results are FAILED.:
at virtuoso.jena.driver.VirtuosoUpdateRequest.exec(Unknown Source)
at
fr.bnf.spar.rdf.jena.query.InsertQuery.execInsert(InsertQuery.java:50)
... 28 more
Caused by: virtuoso.jdbc3.VirtuosoException: SQ199: Maximum size (32767) of a
code vector exceeded by 709361 bytes. Please split the code in smaller units.
at virtuoso.jdbc3.VirtuosoResultSet.process_result(Unknown Source)
at virtuoso.jdbc3.VirtuosoResultSet.<init>(Unknown Source)
at virtuoso.jdbc3.VirtuosoStatement.sendQuery(Unknown Source)
at virtuoso.jdbc3.VirtuosoStatement.execute(Unknown Source)
11:54:19 LTRS_0 <DBA> 127.0.0.1 1111:10 Begin transact 0xe8547d0
11:54:19 LTRS_1 <DBA> 127.0.0.1 1111:10 Commit transact 0xe8547d0 0
11:54:19 LTRS_2 <DBA> 127.0.0.1 1111:10 Restart transact 0xe8547d0
11:54:19 LTRS_0 dba 127.0.0.1 1111:10 Begin transact 0xe8547d0
11:54:19 USER_1 dba 127.0.0.1 1111:10 login
11:54:19 CSLQ_0 dba 127.0.0.1 1111:10 s62 sparql
define output:format '_JAVA_'
PREFIX dc:<http://purl.org/dc/terms/>
PREFIX sparcontext:<info:bnf/spar/context#>
PREFIX sparstructure:<info:bnf/spar/structure#>
PREFIX sparfixity:<info:bnf/spar/fixity#>
PREFIX oai-ore:<http://www.openarchives.org/ore/terms/>
PREFIX spartextmd:<info:bnf/spar/textmd#>
PREFIX dcterms:<http://purl.org/dc/terms/>
PREFIX sparreference:<info:bnf/spar/reference#>
PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>
PREFIX sparmix:<info:bnf/spar/mix#>
PREFIX spar
11:54:19 COMP_2 dba 127.0.0.1 1111:10 Compile text: sparql
define output:format '_JAVA_'
PREFIX dc:<http://purl.org/dc/terms/>
PREFIX sparcontext:<info:bnf/spar/context#>
PREFIX sparstructure:<info:bnf/spar/structure#>
PREFIX sparfixity:<info:bnf/spar/fixity#>
PREFIX oai-ore:<http://www.openarchives.org/ore/terms/>
PREFIX spartextmd:<info:bnf/spar/textmd#>
PREFIX dcterms:<http://purl.org/dc/terms/>
PREFIX sparreference:<info:bnf/spar/reference#>
PREFIX rdfs:<http://www.w3.org/2000/01/rdf-schema#>
PREFIX sparmix:<info:bnf/spar/mix#>
PREFIX sparprovenance:<info:bnf/spar/provenance#>
PREFIX owl:<http://www.w3.org/2002/07/owl#>
PREFIX rdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX sparagent:<info:bnf/spar/agent#>
PREFIX sparrepresentation:<info:bnf/spar/representation#>
PREFIX xsi:<http://www.w3.org/2001/XMLSchema-instance>
INSERT INTO GRAPH <ark:/12148/bc6p07n65.namedGraph.version0> {
<ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/provenance#hasEvent> <info:bnf/spar/provenance/b329848b-d465-45ab-9e9c-896e5917559b> . <ark:/12148/bc6p07n65/f445/ocr.> <http://purl.org/dc/terms/format> <ark:/12148/br2d2gz> . <ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/representation#isUsedAs> "ocr" . <ark:/12148/bc6p07n65/f445/ocr.> rdf:type <info:bnf/spar/structure#file> . <ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/textmd#character_size> "1" . <ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/textmd#byte_size> "8" . <ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/textmd#charset> "ISO_8859-1:1987" . <ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/textmd#markup_language> "http://bibnum.bnf.fr/ns/alto_prod"; . <ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/textmd#byte_order> "big" . <ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/textmd#linebreak> "CR/LF" . <ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/fixity#size> "49958" . <ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/representation#location> "./ocr/X0000445.XML" . <ark:/12148/bc6p07n65/f445/ocr.> <info:bnf/spar/provenance#hasEvent> <info:bnf/spar/provenance/2944cb90-7dc4-11de-bd0b-00212814c89c> . <info:bnf/spar/provenance/ce820484-7f2c-48a8-8a06-45cc888e76c3> <info:bnf/spar/provenance#hasPerformer> <ark:/12148/br2d2dj> . <info:bnf/spar/provenance/ce820484-7f2c-48a8-8a06-45cc888e76c3> <info:bnf/spar/provenance#hasPerformer> <ark:/12148/br2d2cb> . <info:bnf/spar/provenance/ce820484-7f2c-48a8-8a06-45cc888e76c3> <info:bnf/spar/provenance#eventDetail> "fileProcessing is valid" . <info:bnf/spar/provenance/ce820484-7f2c-48a8-8a06-45cc888e76c3> <http://purl.org/dc/terms/date> "2009-07-31T13:32:41Z" . <info:bnf/spar/provenance/ce820484-7f2c-48a8-8a06-45cc888e76c3> rdf:type <info:bnf/spar/provenance#fileProcessing> . _:b0 <info:bnf/spar/mix#scanningSoftwareName> "Conversion multipage et tatouage BnF 2003" . _:b0 rdf:type <info:bnf/spar/mix#scannerCapture> . <info:bnf/spar/provenance/35feb966-81c6-4ce7-b90c-5c1c8f24aa8d> <info:bnf/spar/provenance#hasPerformer> <ark:/12148/br2d2dj> . <info:bnf/spar/provenance/35feb966-81c6-4ce7-b90c-5c1c8f24aa8d> <info:bnf/spar/provenance#hasPerformer> <ark:/12148/br2d2cb> . <info:bnf/spar/provenance/35feb966-81c6-4ce7-b90c-5c1c8f24aa8d> <info:bnf/spar/provenance#eventDetail> "fileProcessing is valid" . <info:bnf/spar/provenance/35feb966-81c6-4ce7-b90c-5c1c8f24aa8d> <http://purl.org/dc/terms/date> "2009-07-31T13:23:29Z" . <info:bnf/spar/provenance/35feb966-81c6-4ce7-b90c-5c1c8f24aa8d> rdf:type <info:bnf/spar/provenance#fileProcessing> . _:b1 <info:bnf/spar/mix#orientation> "1" . _:b1 <info:bnf/spar/mix#usesCaptureDevice> _:b2 . _:b1 <info:bnf/spar/mix#imageProducer> "Biblioth?que nationale de France - cote : N0208210" . _:b1 rdf:type <info:bnf/spar/mix#Capture> . _:b3 <info:bnf/spar/mix#orientation> "1" . _:b3 <info:bnf/spar/mix#usesCaptureDevice> _:b4 . _:b3 <info:bnf/spar/mix#imageProducer> "Biblioth?que nationale de France - cote : N0208210" . _:b3 rdf:type <info:bnf/spar/mix#Capture> . _:b5 <info:bnf/spar/mix#orientation> "1" . _:b5 <info:bnf/spar/mix#usesCaptureDevice> _:b6 . _:b5 <info:bnf/spar/mix#imageProducer> "Biblioth?que nationale de France - cote : N0208210" . _:b5 rdf:type <info:bnf/spar/mix#Capture> . <ark:/12148/bc6p07n65/f476/master.> <http://purl.org/dc/terms/format> <ark:/12148/br2d2wf> . <ark:/12148/bc6p07n65/f476/master.> <info:bnf/spar/mix#compressionScheme> "4" . <ark:/12148/bc6p07n65/f476/master.> rdf:type <info:bnf/spar/structure#file> . <ark:/12148/bc6p07n65/f476/master.> <http://purl.org/dc/terms/source> _:b7 . <ark:/12148/bc6p07n65/f476/master.> <info:bnf/spar/provenance#hasEvent> <info:bnf/spar/provenance/29333f60-7dc4-11de-bd0b-00212814c89c> . <ark:/12148/bc6p07n65/f476/master.> <info:bnf/spar/mix#byteOrder> "little_endian" . <ark:/12148/bc6p07n65/f476/master.> <info:bnf/spar/provenance#hasEvent> <info:bnf/spar/provenance/e0ab0afe-e8d1-4a87-8f91-6c75e728dce8> . <ark:/12148/bc6p07n65/f476/master.> <info:bnf/spar/representation#location> "./master/T0000476.tif" . <ark:/12148/bc6p07n65/f476/master.> <info:bnf/spar/mix#represents> _:b8 . <ark:/12148/bc6p07n65/f476/master.> <info:bnf/spar/fixity#size> "12737" . <ark:/12148/bc6p07n65/f476/master.> <info:bnf/spar/representation#isUsedAs> "master" . _:b9 <info:bnf/spar/mix#orientation> "1" . _:b9 <info:bnf/spar/mix#usesCaptureDevice> _:b10 . _:b9 <info:bnf/spar/mix#imageProducer> "Biblioth?que nationale de France - cote : N0208210" . _:b9 rdf:type <info:bnf/spar/mix#Capture> . <info:bnf/spar/provenance/5b4934da-39f7-41a7-b473-f02fea1ef995> <info:bnf/spar/provenance#hasPerformer> <ark:/12148/br2d2dj> . <info:bnf/spar/provenance/5b4934da-39f7-41a7-b473-f02fea1ef995> <info:bnf/spar/provenance#hasPerformer> <ark:/12148/br2d2cb> . <info:bnf/spar/provenance/5b4934da-39f7-41a7-b473-f02fea1ef995> <info:bnf/spar/provenance#eventDetail> "fileProcessing is valid" . <info:bnf/spar/provenance/5b4934da-39f7-41a7-b473-f02fea1ef995> <http://purl.org/dc/terms/date> "2009-07-31T13:37:41Z" . <info:bnf/spar/provenance/5b4934da-39f7-41a7-b473-f02fea1ef995> rdf:type <info:bnf/spar/provenance#fileProcessing> . _:b11 <info:bnf/spar/mix#orientation> "1" . _:b11 <info:bnf/spar/mix#usesCaptureDevice> _:b12 . _:b11 <info:bnf/spar/mix#imageProducer> "Biblioth?que nationale de France - cote : N0208210" . _:b11 rdf:type <info:bnf/spar/mix#Capture> . _:b13 <info:bnf/spar/mix#samplingFrequencyUnit> "2" . _:b13 <info:bnf/spar/mix#ySamplingFrequencyNumerator> "300" . _:b13 <info:bnf/spar/mix#samplesPerPixel> "1" . _:b13 <info:bnf/spar/mix#colorSpace> "0" . _:b13 rdf:type <info:bnf/spar/mix#image> . _:b13 <info:bnf/spar/mix#imageheight> "2481" . _:b13 <info:bnf/spar/mix#isCapturedBy> _:b14 . _:b13 <info:bnf/spar/mix#bitsPerSampleValue> "1" . _:b13 <info:bnf/spar/mix#xSamplingFrequencyNumerator> "300" . _:b13 <info:bnf/spar/mix#imagewidth> "1749" . _:b15 <info:bnf/spar/mix#bitsPerSampleValue> "1" . _:b15 <info:bnf/spar/mix#isCapturedBy> _:b16 . _:b15 <info:bnf/spar/mix#xSamplingFrequencyNumerator> "300" . _:b15 <info:bnf/spar/mix#imagewidth> "1749" . _:b15 rdf:type <info:bnf/spar/mix#image> . _:b15 <info:bnf/spar/mix#samplesPerPixel> "1" . _:b15 <info:bnf/spar/mix#samplingFrequencyUnit> "2" . _:b15 <info:bnf/spar/mix#ySamplingFrequencyNumerator> "300" . _:b15 <info:bnf/spar/mix#colorSpace> "0" . _:b15 <info:bnf/spar/mix#imageheight> "2481" . <ark:/12148/bc6p07n65/f275/master.> <info:bnf/spar/mix#compressionScheme> "4" . <ark:/12148/bc6p07n65/f275/master.> <http://purl.org/dc/terms/format> <ark:/12148/br2d2wf> . <ark:/12148/bc6p07n65/f275/master.> <info:bnf/spar/fixity#size> "22073" . <ark:/12148/bc6p07n65/f275/master.> <http://purl.org/dc/terms/source> _:b17 . <ark:/12148/bc6p07n65/f275/master.> <info:bnf/spar/provenance#hasEvent> <info:bnf/spar/provenance/29333f60-7dc4-11de-bd0b-00212814c89c> . <ark:/12148/bc6p07n65/f275/master.> rdf:type <info:bnf/spar/structure#file> . <ark:/12148/bc6p07n65/f275/master.> <info:bnf/spar/representation#location> "./master/T0000275.tif" . <ark:/12148/bc6p07n65/f275/master.> <info:bnf/spar/provenance#hasEvent> <info:bnf/spar/provenance/bde65351-0124-44f8-8a62-a179778103b7> . <ark:/12148/bc6p07n65/f275/master.> <info:bnf/spar/mix#byteOrder> "little_endian" . <ark:/12148/bc6p07n65/f275/master.> <11:54:27 LTRS_0 <DBA> Internal Internal Begin transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:55:29 LTRS_1 <DBA> Internal Internal Commit transact 0xaa4a5a60 0
11:55:29 LTRS_2 <DBA> Internal Internal Restart transact 0xaa4a5a60
11:57:20 ERRS_0 42000 SQ199 Maximum size (32767) of a code vector exceeded by 709361 bytes. Please split the code in smaller units.
11:57:20 LTRS_1 dba 127.0.0.1 1111:8 Rollback transact 0xf2f53d0 0
11:57:20 LTRS_2 dba 127.0.0.1 1111:8 Restart transact 0xf2f53d0
11:57:20 USER_0 dba 127.0.0.1 1111:8 logout
11:57:20 LTRS_1 dba 127.0.0.1 1111:9 Rollback transact 0xc01f488 0
11:57:20 LTRS_2 dba 127.0.0.1 1111:9 Restart transact 0xc01f488
11:57:20 USER_0 dba 127.0.0.1 1111:9 logout
11:57:20 LTRS_1 dba 127.0.0.1 1111:10 Rollback transact 0xe8547d0 0
11:57:20 LTRS_2 dba 127.0.0.1 1111:10 Restart transact 0xe8547d0
11:57:20 USER_0 dba 127.0.0.1 1111:10 logout
