[jira] [Commented] (JENA-1758) Exception when preparing InfModel after TDB loading operation
[
https://issues.apache.org/jira/browse/JENA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17044491#comment-17044491
]
ASF subversion and git services commented on JENA-1758:
---
Commit d2e23164a08082f3e2c52c43b99b0bf0e66dcc91 in jena's branch
refs/heads/master from Andy Seaborne
[ https://gitbox.apache.org/repos/asf?p=jena.git;h=d2e2316 ]
JENA-1758: Use a semaphore to notify thread has finished
> Exception when preparing InfModel after TDB loading operation
> -
>
> Key: JENA-1758
> URL: https://issues.apache.org/jira/browse/JENA-1758
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
>Affects Versions: Jena 3.12.0
>Reporter: Damien Obrist
>Assignee: Andy Seaborne
>Priority: Critical
> Fix For: Jena 3.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h2. Exception
> I'm loading a few million triples into a TDB2 dataset. I'm using a custom
> loader (extending {{LoaderMain}}), since the triples being loaded are
> generated by a separate application and streamed in over HTTP.
> After the loading is done, I try to reset an {{InfModel}} to recompute the
> inference taking into account the new triples, but I encounter the following
> exception:
> {noformat}
> org.apache.jena.atlas.RuntimeIOException: Out of bounds: (limit
> 32834204)32834205
> at org.apache.jena.atlas.io.IO.exception(IO.java:254)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.checkRead(TransBinaryDataFile.java:190)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.read(TransBinaryDataFile.java:184)
> at
> org.apache.jena.tdb2.store.nodetable.TReadAppendFileTransport.read(TReadAppendFileTransport.java:71)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:637)
> at
> org.apache.thrift.protocol.TCompactProtocol.readFieldBegin(TCompactProtocol.java:543)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:318)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:311)
> at org.apache.jena.riot.thrift.wire.RDF_IRI.read(RDF_IRI.java:258)
> at
> org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:319)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
> at org.apache.thrift.TUnion.read(TUnion.java:138)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:80)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:112)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:108)
> at
> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
> at
> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
> at
> java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
> at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:903)
> at org.apache.jena.atlas.iterator.Iter$1.hasNext(Iter.java:192)
> at
> org.apache.jena.util.iterator.WrappedIterator.hasNext(Wra
[jira] [Commented] (JENA-1758) Exception when preparing InfModel after TDB loading operation
[
https://issues.apache.org/jira/browse/JENA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17044492#comment-17044492
]
ASF subversion and git services commented on JENA-1758:
---
Commit 9a398d6a24541da107c16aa5dda347aad91caaad in jena's branch
refs/heads/master from Andy Seaborne
[ https://gitbox.apache.org/repos/asf?p=jena.git;h=9a398d6 ]
Merge pull request #695 from afs/jena1758-semaphore
JENA-1758: Use a semaphore to notify thread has finished
> Exception when preparing InfModel after TDB loading operation
> -
>
> Key: JENA-1758
> URL: https://issues.apache.org/jira/browse/JENA-1758
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
>Affects Versions: Jena 3.12.0
>Reporter: Damien Obrist
>Assignee: Andy Seaborne
>Priority: Critical
> Fix For: Jena 3.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h2. Exception
> I'm loading a few million triples into a TDB2 dataset. I'm using a custom
> loader (extending {{LoaderMain}}), since the triples being loaded are
> generated by a separate application and streamed in over HTTP.
> After the loading is done, I try to reset an {{InfModel}} to recompute the
> inference taking into account the new triples, but I encounter the following
> exception:
> {noformat}
> org.apache.jena.atlas.RuntimeIOException: Out of bounds: (limit
> 32834204)32834205
> at org.apache.jena.atlas.io.IO.exception(IO.java:254)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.checkRead(TransBinaryDataFile.java:190)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.read(TransBinaryDataFile.java:184)
> at
> org.apache.jena.tdb2.store.nodetable.TReadAppendFileTransport.read(TReadAppendFileTransport.java:71)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:637)
> at
> org.apache.thrift.protocol.TCompactProtocol.readFieldBegin(TCompactProtocol.java:543)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:318)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:311)
> at org.apache.jena.riot.thrift.wire.RDF_IRI.read(RDF_IRI.java:258)
> at
> org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:319)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
> at org.apache.thrift.TUnion.read(TUnion.java:138)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:80)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:112)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:108)
> at
> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
> at
> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
> at
> java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
> at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:903)
> at org.apache.jena.atlas.iterator.Iter$1.hasNext(Iter.java:192)
> at
> org.a
[jira] [Commented] (JENA-1758) Exception when preparing InfModel after TDB loading operation
[
https://issues.apache.org/jira/browse/JENA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930338#comment-16930338
]
Andy Seaborne commented on JENA-1758:
-
{quote}I see the fix version is Jena 3.13.0. Do you have a rough estimate on
when it is going to be released?
{quote}
"Soon" :) We hope to release this month but everyone is fitting in the open
source work around everything else.
There are development builds if you want to try it out now or build locally.
The fix is in master now.
> Exception when preparing InfModel after TDB loading operation
> -
>
> Key: JENA-1758
> URL: https://issues.apache.org/jira/browse/JENA-1758
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
>Affects Versions: Jena 3.12.0
>Reporter: Damien Obrist
>Assignee: Andy Seaborne
>Priority: Critical
> Fix For: Jena 3.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h2. Exception
> I'm loading a few million triples into a TDB2 dataset. I'm using a custom
> loader (extending {{LoaderMain}}), since the triples being loaded are
> generated by a separate application and streamed in over HTTP.
> After the loading is done, I try to reset an {{InfModel}} to recompute the
> inference taking into account the new triples, but I encounter the following
> exception:
> {noformat}
> org.apache.jena.atlas.RuntimeIOException: Out of bounds: (limit
> 32834204)32834205
> at org.apache.jena.atlas.io.IO.exception(IO.java:254)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.checkRead(TransBinaryDataFile.java:190)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.read(TransBinaryDataFile.java:184)
> at
> org.apache.jena.tdb2.store.nodetable.TReadAppendFileTransport.read(TReadAppendFileTransport.java:71)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:637)
> at
> org.apache.thrift.protocol.TCompactProtocol.readFieldBegin(TCompactProtocol.java:543)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:318)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:311)
> at org.apache.jena.riot.thrift.wire.RDF_IRI.read(RDF_IRI.java:258)
> at
> org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:319)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
> at org.apache.thrift.TUnion.read(TUnion.java:138)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:80)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:112)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:108)
> at
> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
> at
> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
> at
> java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
> at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:903)
> at org.apache.jena.atlas.iterator.Iter$1.hasNext(Iter.jav
[jira] [Commented] (JENA-1758) Exception when preparing InfModel after TDB loading operation
[
https://issues.apache.org/jira/browse/JENA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930286#comment-16930286
]
Damien Obrist commented on JENA-1758:
-
[~andy.seaborne] you're welcome, thank you for reviewing and merging this issue
so quickly.
{quote}I hope you haven't had to make a custom loader simply for that.
{quote}
No, thankfully not. All I did was extend {{LoaderMain}} to, in addition to
loading from files, be able to load from a stream. I just realized that my code
snippet is a bit misleading and actually looks more like this:
{code:java}
CustomLoader loader = new CustomLoader(LoaderPlans.loaderPlanParallel,
dataset.asDatasetGraph);
loader.load(inputStream);
{code}
And the loading code as described is actually inside the custom loader.
{quote}FYI: The variant of the loader that uses as much parallelism as possible
{quote}
Yes, I have noticed. However, in my case that's perfect as I want the loading
to be as fast as possible and use all available resources. I haven't had any
problems with TCP connections so far, but thanks for the heads up!
I see the fix version is Jena 3.13.0. Do you have a rough estimate on when it
is going to be released?
> Exception when preparing InfModel after TDB loading operation
> -
>
> Key: JENA-1758
> URL: https://issues.apache.org/jira/browse/JENA-1758
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
>Affects Versions: Jena 3.12.0
>Reporter: Damien Obrist
>Assignee: Andy Seaborne
>Priority: Critical
> Fix For: Jena 3.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h2. Exception
> I'm loading a few million triples into a TDB2 dataset. I'm using a custom
> loader (extending {{LoaderMain}}), since the triples being loaded are
> generated by a separate application and streamed in over HTTP.
> After the loading is done, I try to reset an {{InfModel}} to recompute the
> inference taking into account the new triples, but I encounter the following
> exception:
> {noformat}
> org.apache.jena.atlas.RuntimeIOException: Out of bounds: (limit
> 32834204)32834205
> at org.apache.jena.atlas.io.IO.exception(IO.java:254)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.checkRead(TransBinaryDataFile.java:190)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.read(TransBinaryDataFile.java:184)
> at
> org.apache.jena.tdb2.store.nodetable.TReadAppendFileTransport.read(TReadAppendFileTransport.java:71)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:637)
> at
> org.apache.thrift.protocol.TCompactProtocol.readFieldBegin(TCompactProtocol.java:543)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:318)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:311)
> at org.apache.jena.riot.thrift.wire.RDF_IRI.read(RDF_IRI.java:258)
> at
> org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:319)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
> at org.apache.thrift.TUnion.read(TUnion.java:138)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:80)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:112)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:108)
> at
> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
> at
> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
> at
> java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.lamb
[jira] [Commented] (JENA-1758) Exception when preparing InfModel after TDB loading operation
[
https://issues.apache.org/jira/browse/JENA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929236#comment-16929236
]
Andy Seaborne commented on JENA-1758:
-
[~dobrist]-- thank you for the excellent report and investigation.
The
[PR|https://github.com/apache/jena/pull/606|https://github.com/apache/jena/pull/606]
looks spot on.
The way to feed the loader with data that isn't from local file is
{{loader.stream()}} as your code has it. I hope you haven't had to make a
custom loader simply for that.
FYI: The variant of the loader that uses as much parallelism as possible can be
too aggressive and use all of the CPU cores and I/O bandwidth of the machine
that general OS scheduling is sluggish (minutes). This might impact a TCP
connection feeding a large amount of data, locking it out for long enough to
cause the TCP connection to drop out. I haven't seen this happen with the data
loader but from experience with other cases of machine saturation.
> Exception when preparing InfModel after TDB loading operation
> -
>
> Key: JENA-1758
> URL: https://issues.apache.org/jira/browse/JENA-1758
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
>Affects Versions: Jena 3.12.0
>Reporter: Damien Obrist
>Assignee: Andy Seaborne
>Priority: Critical
> Fix For: Jena 3.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h2. Exception
> I'm loading a few million triples into a TDB2 dataset. I'm using a custom
> loader (extending {{LoaderMain}}), since the triples being loaded are
> generated by a separate application and streamed in over HTTP.
> After the loading is done, I try to reset an {{InfModel}} to recompute the
> inference taking into account the new triples, but I encounter the following
> exception:
> {noformat}
> org.apache.jena.atlas.RuntimeIOException: Out of bounds: (limit
> 32834204)32834205
> at org.apache.jena.atlas.io.IO.exception(IO.java:254)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.checkRead(TransBinaryDataFile.java:190)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.read(TransBinaryDataFile.java:184)
> at
> org.apache.jena.tdb2.store.nodetable.TReadAppendFileTransport.read(TReadAppendFileTransport.java:71)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:637)
> at
> org.apache.thrift.protocol.TCompactProtocol.readFieldBegin(TCompactProtocol.java:543)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:318)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:311)
> at org.apache.jena.riot.thrift.wire.RDF_IRI.read(RDF_IRI.java:258)
> at
> org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:319)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
> at org.apache.thrift.TUnion.read(TUnion.java:138)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:80)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:112)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:108)
> at
> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
> at
> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
> at
> java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
> at
> java.util.stream.Stre
[jira] [Commented] (JENA-1758) Exception when preparing InfModel after TDB loading operation
[
https://issues.apache.org/jira/browse/JENA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929233#comment-16929233
]
ASF subversion and git services commented on JENA-1758:
---
Commit cc3c75c800c68cb298c283e2ddeef047b205ae8b in jena's branch
refs/heads/master from Andy Seaborne
[ https://gitbox.apache.org/repos/asf?p=jena.git;h=cc3c75c ]
Merge pull request #606 from dobrist/JENA-1758
JENA-1758: Wait for the thread to complete when finishing bulk loading
> Exception when preparing InfModel after TDB loading operation
> -
>
> Key: JENA-1758
> URL: https://issues.apache.org/jira/browse/JENA-1758
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
>Affects Versions: Jena 3.12.0
>Reporter: Damien Obrist
>Assignee: Andy Seaborne
>Priority: Critical
> Fix For: Jena 3.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h2. Exception
> I'm loading a few million triples into a TDB2 dataset. I'm using a custom
> loader (extending {{LoaderMain}}), since the triples being loaded are
> generated by a separate application and streamed in over HTTP.
> After the loading is done, I try to reset an {{InfModel}} to recompute the
> inference taking into account the new triples, but I encounter the following
> exception:
> {noformat}
> org.apache.jena.atlas.RuntimeIOException: Out of bounds: (limit
> 32834204)32834205
> at org.apache.jena.atlas.io.IO.exception(IO.java:254)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.checkRead(TransBinaryDataFile.java:190)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.read(TransBinaryDataFile.java:184)
> at
> org.apache.jena.tdb2.store.nodetable.TReadAppendFileTransport.read(TReadAppendFileTransport.java:71)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:637)
> at
> org.apache.thrift.protocol.TCompactProtocol.readFieldBegin(TCompactProtocol.java:543)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:318)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:311)
> at org.apache.jena.riot.thrift.wire.RDF_IRI.read(RDF_IRI.java:258)
> at
> org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:319)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
> at org.apache.thrift.TUnion.read(TUnion.java:138)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:80)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:112)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:108)
> at
> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
> at
> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
> at
> java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
> at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:903)
> at org.apache.jena.atlas.iterator.Iter$1.hasNext(Iter.java:192)
> at
[jira] [Commented] (JENA-1758) Exception when preparing InfModel after TDB loading operation
[
https://issues.apache.org/jira/browse/JENA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929231#comment-16929231
]
ASF subversion and git services commented on JENA-1758:
---
Commit 6f7db99a0c9d0b20fe9848c4abc4cd2444e4b557 in jena's branch
refs/heads/master from damienobrist
[ https://gitbox.apache.org/repos/asf?p=jena.git;h=6f7db99 ]
JENA-1758: Wait for the thread to complete when finishing bulk loading
This prevents the thread from running until after a loading operation has
returned, which can cause problems when the dataset is used immediately
thereafter.
> Exception when preparing InfModel after TDB loading operation
> -
>
> Key: JENA-1758
> URL: https://issues.apache.org/jira/browse/JENA-1758
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
>Affects Versions: Jena 3.12.0
>Reporter: Damien Obrist
>Assignee: Andy Seaborne
>Priority: Critical
> Fix For: Jena 3.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h2. Exception
> I'm loading a few million triples into a TDB2 dataset. I'm using a custom
> loader (extending {{LoaderMain}}), since the triples being loaded are
> generated by a separate application and streamed in over HTTP.
> After the loading is done, I try to reset an {{InfModel}} to recompute the
> inference taking into account the new triples, but I encounter the following
> exception:
> {noformat}
> org.apache.jena.atlas.RuntimeIOException: Out of bounds: (limit
> 32834204)32834205
> at org.apache.jena.atlas.io.IO.exception(IO.java:254)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.checkRead(TransBinaryDataFile.java:190)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.read(TransBinaryDataFile.java:184)
> at
> org.apache.jena.tdb2.store.nodetable.TReadAppendFileTransport.read(TReadAppendFileTransport.java:71)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:637)
> at
> org.apache.thrift.protocol.TCompactProtocol.readFieldBegin(TCompactProtocol.java:543)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:318)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:311)
> at org.apache.jena.riot.thrift.wire.RDF_IRI.read(RDF_IRI.java:258)
> at
> org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:319)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
> at org.apache.thrift.TUnion.read(TUnion.java:138)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:80)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:112)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:108)
> at
> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
> at
> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
> at
> java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
> at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
> at org.apache.jena.atlas.iterat
[jira] [Commented] (JENA-1758) Exception when preparing InfModel after TDB loading operation
[
https://issues.apache.org/jira/browse/JENA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929232#comment-16929232
]
ASF subversion and git services commented on JENA-1758:
---
Commit cc3c75c800c68cb298c283e2ddeef047b205ae8b in jena's branch
refs/heads/master from Andy Seaborne
[ https://gitbox.apache.org/repos/asf?p=jena.git;h=cc3c75c ]
Merge pull request #606 from dobrist/JENA-1758
JENA-1758: Wait for the thread to complete when finishing bulk loading
> Exception when preparing InfModel after TDB loading operation
> -
>
> Key: JENA-1758
> URL: https://issues.apache.org/jira/browse/JENA-1758
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
>Affects Versions: Jena 3.12.0
>Reporter: Damien Obrist
>Assignee: Andy Seaborne
>Priority: Critical
> Fix For: Jena 3.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h2. Exception
> I'm loading a few million triples into a TDB2 dataset. I'm using a custom
> loader (extending {{LoaderMain}}), since the triples being loaded are
> generated by a separate application and streamed in over HTTP.
> After the loading is done, I try to reset an {{InfModel}} to recompute the
> inference taking into account the new triples, but I encounter the following
> exception:
> {noformat}
> org.apache.jena.atlas.RuntimeIOException: Out of bounds: (limit
> 32834204)32834205
> at org.apache.jena.atlas.io.IO.exception(IO.java:254)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.checkRead(TransBinaryDataFile.java:190)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.read(TransBinaryDataFile.java:184)
> at
> org.apache.jena.tdb2.store.nodetable.TReadAppendFileTransport.read(TReadAppendFileTransport.java:71)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:637)
> at
> org.apache.thrift.protocol.TCompactProtocol.readFieldBegin(TCompactProtocol.java:543)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:318)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:311)
> at org.apache.jena.riot.thrift.wire.RDF_IRI.read(RDF_IRI.java:258)
> at
> org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:319)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
> at org.apache.thrift.TUnion.read(TUnion.java:138)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:80)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:112)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:108)
> at
> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
> at
> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
> at
> java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
> at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:903)
> at org.apache.jena.atlas.iterator.Iter$1.hasNext(Iter.java:192)
> at
[jira] [Commented] (JENA-1758) Exception when preparing InfModel after TDB loading operation
[
https://issues.apache.org/jira/browse/JENA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928625#comment-16928625
]
Damien Obrist commented on JENA-1758:
-
h2. Solution
The thread started in {{DataToTuples#startBulk}} needs to be waited for before
finishing the loading operation. Else it might still be running after the
loading has returned, which causes the observed problems.
Pull request:
https://github.com/apache/jena/pull/606.
I was able to validate in my environment that with this fix the exception no
longer occurs.
> Exception when preparing InfModel after TDB loading operation
> -
>
> Key: JENA-1758
> URL: https://issues.apache.org/jira/browse/JENA-1758
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
>Affects Versions: Jena 3.12.0
>Reporter: Damien Obrist
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h2. Exception
> I'm loading a few million triples into a TDB2 dataset. I'm using a custom
> loader (extending {{LoaderMain}}), since the triples being loaded are
> generated by a separate application and streamed in over HTTP.
> After the loading is done, I try to reset an {{InfModel}} to recompute the
> inference taking into account the new triples, but I encounter the following
> exception:
> {noformat}
> org.apache.jena.atlas.RuntimeIOException: Out of bounds: (limit
> 32834204)32834205
> at org.apache.jena.atlas.io.IO.exception(IO.java:254)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.checkRead(TransBinaryDataFile.java:190)
> at
> org.apache.jena.dboe.trans.data.TransBinaryDataFile.read(TransBinaryDataFile.java:184)
> at
> org.apache.jena.tdb2.store.nodetable.TReadAppendFileTransport.read(TReadAppendFileTransport.java:71)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:637)
> at
> org.apache.thrift.protocol.TCompactProtocol.readFieldBegin(TCompactProtocol.java:543)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:318)
> at
> org.apache.jena.riot.thrift.wire.RDF_IRI$RDF_IRIStandardScheme.read(RDF_IRI.java:311)
> at org.apache.jena.riot.thrift.wire.RDF_IRI.read(RDF_IRI.java:258)
> at
> org.apache.jena.riot.thrift.wire.RDF_Term.standardSchemeReadValue(RDF_Term.java:319)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:224)
> at org.apache.thrift.TUnion$TUnionStandardScheme.read(TUnion.java:213)
> at org.apache.thrift.TUnion.read(TUnion.java:138)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableTRDF.readNodeFromTable(NodeTableTRDF.java:80)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:197)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:108)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:52)
> at
> org.apache.jena.tdb2.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:66)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:112)
> at org.apache.jena.tdb2.lib.TupleLib.quad(TupleLib.java:108)
> at
> org.apache.jena.tdb2.lib.TupleLib.lambda$convertToQuads$3(TupleLib.java:53)
> at org.apache.jena.atlas.iterator.Iter$2.next(Iter.java:270)
> at
> org.apache.jena.atlas.iterator.IteratorWrapper.next(IteratorWrapper.java:36)
> at
> org.apache.jena.tdb2.store.IteratorTxnTracker.next(IteratorTxnTracker.java:43)
> at
> java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
> at
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
> at
> java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
> at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:903)
> at org.apache.jena.atlas.iterator.Iter$1.hasNext(Iter.java:192)
> at
> org.apache.jena.
[jira] [Commented] (JENA-1758) Exception when preparing InfModel after TDB loading operation
[
https://issues.apache.org/jira/browse/JENA-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928617#comment-16928617
]
Damien Obrist commented on JENA-1758:
-
h2. Investigation
Playing around I have observed the following:
* when catching the exception and trying to prepare the model a second time,
the call succeeded and no exception was thrown
* when sleep for a few seconds after the loading and before preparing the
model, the call succeeded and no exception was thrown at all
So I suspected that, even though the loading is complete, something must still
be going on in the background.
Next I tried to close and reopen the dataset after the loading, in order to
make sure that potential pending writes are terminated and have been flushed to
disk:
{code:java}
TDBInternal.expel(dataset.asDatasetGraph());
dataset = TDB2Factory.connectDataset(path);
infModel = ModelFactory.createInfModel(reasoner, unionModel);
Txn.executeRead(dataset, infModel::prepare);
{code}
This lead to the following exception:
{noformat}
org.apache.jena.atlas.lib.InternalErrorException: BlockMgrFileAccess : already
closed
at
org.apache.jena.dboe.base.block.BlockMgrFileAccess.checkNotClosed(BlockMgrFileAccess.java:78)
at
org.apache.jena.dboe.base.block.BlockMgrFileAccess.allocLimit(BlockMgrFileAccess.java:150)
at
org.apache.jena.dboe.base.block.BlockMgrWrapper.allocLimit(BlockMgrWrapper.java:87)
at
org.apache.jena.dboe.base.page.PageBlockMgr.allocLimit(PageBlockMgr.java:45)
at
org.apache.jena.dboe.trans.bplustree.BPlusTree._commitPrepare(BPlusTree.java:544)
at
org.apache.jena.dboe.trans.bplustree.BPlusTree._commitPrepare(BPlusTree.java:73)
at
org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.commitPrepare(TransactionalComponentLifecycle.java:98)
at
org.apache.jena.dboe.transaction.txn.TransactionCoordinator.lambda$executePrepare$12(TransactionCoordinator.java:687)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at
org.apache.jena.dboe.transaction.txn.TransactionCoordinator.executePrepare(TransactionCoordinator.java:685)
at
org.apache.jena.dboe.transaction.txn.Transaction.prepare(Transaction.java:151)
at
org.apache.jena.dboe.transaction.txn.Transaction.commit(Transaction.java:160)
at
org.apache.jena.tdb2.loader.main.DataToTuples.action(DataToTuples.java:139)
at
org.apache.jena.tdb2.loader.main.DataToTuples.lambda$startBulk$0(DataToTuples.java:101)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "Thread-67" java.lang.NullPointerException
at
org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.getDataState(TransactionalComponentLifecycle.java:251)
at
org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.abort(TransactionalComponentLifecycle.java:126)
at org.apache.jena.dboe.transaction.txn.SysTrans.abort(SysTrans.java:45)
at
org.apache.jena.dboe.transaction.txn.Transaction.lambda$null$10(Transaction.java:189)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at
org.apache.jena.dboe.transaction.txn.Transaction.lambda$abort$$11(Transaction.java:189)
at
org.apache.jena.dboe.transaction.txn.TransactionCoordinator.executeAbort(TransactionCoordinator.java:726)
at
org.apache.jena.dboe.transaction.txn.Transaction.abort$(Transaction.java:189)
at
org.apache.jena.dboe.transaction.txn.Transaction.abort(Transaction.java:181)
at
org.apache.jena.tdb2.loader.main.DataToTuples.action(DataToTuples.java:142)
at
org.apache.jena.tdb2.loader.main.DataToTuples.lambda$startBulk$0(DataToTuples.java:101)
at java.lang.Thread.run(Thread.java:748)
{noformat}
This shows that {{DataToTuples}} is the culprit still running after the loading
operation has returned.
> Exception when preparing InfModel after TDB loading operation
> -
>
> Key: JENA-1758
> URL: https://issues.apache.org/jira/browse/JENA-1758
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB2
>Affects Versions: Jena 3.12.0
>Reporter: Damien Obrist
>Priority: Major
>
> h2. Exception
> I'm loading a few million triples into a TDB2 dataset. I'm using a custom
> loader (extending {{LoaderMain}}), since the triples being loaded are
> generated by a separate application and streamed in over HTTP.
> After the loading is done, I try to reset an {{InfModel}} to recompute the
> inference taking into account the new triples, but I encounter the following
> exception:
> {noformat}
> org.apache.jena.atlas.RuntimeIOException: Out of bounds: (limit
> 32834204)32834205
> at org.apache.jena.atlas.io.IO.exception(IO.java:254)
> at
> org.apache.jena.dboe.
