Hi Harald, Can you please try load the data in batch mode setting the "uselazyAdd" parameter in your connect string to true ie:
- VirtuosoRepository virtRepo = new
VirtuosoRepository("jdbc:virtuoso://localhost:1111,dba,dba", "dba", "dba",
"http://test.com/");
+ VirtuosoRepository virtRepo = new
VirtuosoRepository("jdbc:virtuoso://localhost:1111,dba,dba", "dba", "dba",
"http://test.com/", true);
This parameter enabling the batch optimized loading of data as detailed in the
Javadocs for the VirtuosoRepository method at:
http://docs.openlinksw.com/sesame/
Development are continuing to look into the cause of the "too many open
statements" error when performing a single load, as that should work even if
slower than the batch mode ...
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
On 6 Mar 2013, at 14:12, Harald Beck <[email protected]> wrote:
> Hi Hugh,
>
> What I'm using is (essentially) something like this:
>
> import org.openrdf.model.Statement;
> import org.openrdf.repository.Repository;
> import org.openrdf.repository.RepositoryConnection;
> import org.openrdf.repository.RepositoryException;
> import org.openrdf.rio.RDFHandler;
> import org.openrdf.rio.RDFHandlerException;
> import org.openrdf.rio.helpers.RDFHandlerBase;
> import virtuoso.sesame2.driver.VirtuosoRepository;
>
>
> public class Test {
>
> public void export(Repository projectRepo) throws RepositoryException,
> RDFHandlerException {
>
> RepositoryConnection projectConn = projectRepo.getConnection();
>
> try {
> VirtuosoRepository virtRepo = new
> VirtuosoRepository("jdbc:virtuoso://localhost:1111,dba,dba", "dba", "dba",
> "http://test.com/");
> final RepositoryConnection virtConn = virtRepo.getConnection();
>
>
> RDFHandler virtHandler = new RDFHandlerBase() {
> @Override
> public void handleStatement(Statement st) throws
> RDFHandlerException {
> try {
> virtConn.add(st);
> } catch (RepositoryException ex) {
> throw new RDFHandlerException(ex);
> }
> }
> };
>
> projectConn.export(virtHandler);
> virtConn.commit();
> virtConn.close();
>
> } finally {
> projectConn.close();
> }
> }
>
> }
>
> In the meantime, I could narrow down the problem: I can also trigger the
> error by using the following method:
>
> void export(Collection<Statement> statements) throws RepositoryException {
> VirtuosoRepository virtRepo = new
> VirtuosoRepository("jdbc:virtuoso://localhost:1111,dba,dba", "dba", "dba",
> "http://test.com/");
> RepositoryConnection virtConn = virtRepo.getConnection();
> for (Statement s : statements) {
> virtConn.add(s);
> }
> virtConn.commit();
> virtConn.close();
> }
>
> However, if I use
>
> virtConn.add(statements)
>
> instead of the above for-loop, the problem does NOT occur. So, there seems to
> be a problem with too many .add calls.
>
> Most of the time, it happens already before 10.000 triples. Committing
> manually during the loop, or setting auto commit to true does not help.
>
> By the way, which performance should I expect to have? With the above method,
> I can only insert between 250 and 650 triples/sec (inserting seems to slow
> down on larger files), while DB.DBA.TTLP_MT achieves around 8.000-13.000
> triples/sec on my machine (ubuntu 12.04.1, 64bit, 8 GB RAM, Intel Core
> i7-2670QM CPU @ 2.20GHz × 8).
>
> Best,
> Harald
>
>
>
>
>
>
> ----- Original Message -----
> From: "Hugh Williams" <[email protected]>
> To: "Harald Beck" <[email protected]>
> Cc: [email protected]
> Sent: Tuesday, March 5, 2013 1:55:25 PM
> Subject: Re: [Virtuoso-users] Too many open statements
>
> Hi Harald,
>
> This issue should be resolved unless you are hitting another use case which
> forces the condition to occur, thus are you able to provide a simple
> reproducible test case we can setup in-house to see this issue ?
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc. // http://www.openlinksw.com/
> Weblog -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter -- http://twitter.com/OpenLink
> Google+ -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
>
> On 4 Mar 2013, at 13:14, Harald Beck <[email protected]> wrote:
>
>> Hi,
>>
>> similarly as Andrea in
>> http://www.mail-archive.com/[email protected]/msg05185.html
>>
>> I get the
>>
>> virtuoso.jdbc3.VirtuosoException: SR491: Too many open statements
>>
>> In the last attempt, it happened after inserting 17838 triples (into
>> Virtuoso Opensource 6.1.6).
>>
>> I'm using the current virtjdbc3 driver (Build 3.62).
>>
>> Any suggestions what to inspect?
>>
>> Best,
>> Harald
>>
>> --
>> Harald Beck
>> Software Developer
>>
>> Semantic Web Company GmbH
>> Mariahilfer Straße 70 / 8
>> A - 1070 Vienna, Austria
>> Tel +43 1 402 12 35
>> Fax +43 1 402 12 35 - 22
>>
>> http://www.semantic-web.at
>> http://blog.semantic-web.at
>> http://poolparty.biz
>>
>> LOD2 - Creating Knowledge out of Interlinked Data - http://lod2.eu/
>>
>> ------------------------------------------------------------------------------
>> Everyone hates slow websites. So do we.
>> Make your web apps faster with AppDynamics
>> Download AppDynamics Lite for free today:
>> http://p.sf.net/sfu/appdyn_d2d_feb
>> _______________________________________________
>> Virtuoso-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
>
>
> --
> Harald Beck
> Software Developer
>
> Semantic Web Company GmbH
> Mariahilfer Straße 70 / 8
> A - 1070 Vienna, Austria
> Tel +43 1 402 12 35
> Fax +43 1 402 12 35 - 22
>
> http://www.semantic-web.at
> http://blog.semantic-web.at
> http://poolparty.biz
>
> LOD2 - Creating Knowledge out of Interlinked Data - http://lod2.eu/
>
> ------------------------------------------------------------------------------
> Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester
> Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the
> endpoint security space. For insight on selecting the right partner to
> tackle endpoint security challenges, access the full report.
> http://p.sf.net/sfu/symantec-dev2dev
> _______________________________________________
> Virtuoso-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
smime.p7s
Description: S/MIME cryptographic signature
------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________ Virtuoso-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/virtuoso-users
