date:20180305

Possible fuseki/memory store bug

2018-03-05 Thread Dave Reynolds

I've been trying to debug some weird behaviour in my test cases and 
think they are due to a bug in memory-backed fuseki stores.


However, the behaviour is so odd and hard to reproduce I'd like some 
confirmation that someone else sees the same effect before opening a JIRA.


# Steps to reproduce

[Sorry this is convoluted but all my attempts to simplify fail to show 
the suspected bug.]


1. Fresh download of fuseki 3.6.0.

2. Start an in memory server in one shell:

fuseki-server --mem --update /ds

3. Create test data with two statements about one resource:

echo 'prefix rdfs:  
prefix eg:
eg:r4 a eg:Resource; rdfs:label "r 4" .' > update.ttl

4. Use the graph REST API to put the data into a named graph:

s-put http://localhost:3030/ds/data http://localhost/graph5 update.ttl

5. Run a sparql update which will delete the original statements from 
the graph and add some replacement statements to a new graph:


rupdate --service=http://localhost:3030/ds/update '
DELETE {GRAPH ?G { ?p ?o}} WHERE {GRAPH 
?G { ?p ?o}};

INSERT DATA { GRAPH  {
 
 "r 4 modified" .
 
 
 .

} }'

6. Check the contents of the store:

rsparql --service http://localhost:3030/ds/query "SELECT * WHERE 
{Graph ?G {?s ?p ?o}} ORDER BY ?G"


At this point the store *should* contain two statements in graph 
http://localhost/test/r4. With a TDB-backed fuseki that's what I see. 
With the memory backed fuseki I see an apparently empty store.


If other named graphs are populated with other unrelated data they will 
seem to have disappeared as well.


7. Now reinsert the original data, running step 4 again and check by 
running step 6 again. At this point both the "missing" statements from 
graph http://localhost/test/r4 reappear, as does the reinserted original 
statements in http://localhost/graph5.


# Simplifying the test case

So far ...

  - I've failed to reproduce this outside of fuseki.

  - I've failed to reproduce this without mixing graph operations and 
update operations.


  - If I reduce the inserted/updated data to a single statement instead 
of a pair of statements it passes.


  - If I try with at empty TDB store it passes.

  - I get the same behaviour from 3.4.0 as from 3.6.0.

Any of these failures may be user error on my part, it has been so hard 
turning an apparently non-deterministic error into something 
reproducible that I'm no longer sure of anything :(


Am I going mad or does anyone else see the same behaviour?

Dave

[jira] [Commented] (JENA-1495) Covariant return type for Model's PrefixMapping methods.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386949#comment-16386949
 ] 

ASF GitHub Bot commented on JENA-1495:
--

Github user jaco0646 commented on a diff in the pull request:

https://github.com/apache/jena/pull/368#discussion_r172364101
  
--- Diff: jena-core/src/main/java/org/apache/jena/rdf/model/Model.java ---
@@ -1051,4 +1051,12 @@ Remove all the statements matching (s, p, o) from 
this model.
 Answer true iff .close() has been called on this Model.
 */
 public boolean isClosed();
+
+// Override return type for methods inherited from PrefixMapping
--- End diff --

Done.


> Covariant return type for Model's PrefixMapping methods.
> 
>
> Key: JENA-1495
> URL: https://issues.apache.org/jira/browse/JENA-1495
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: Jena 3.6.0
>Reporter: Adam Jacobs
>Priority: Trivial
>  Labels: easyfix
>
> The {{Model}} interface inherits several methods from {{PrefixMapping}} which 
> return {{this}}.
>  The return type of these methods can be refined to allow call chaining with 
> the {{Model}} object.
> There appear to be two concrete implementations of {{Model}} which override 
> the {{PrefixMapping}} methods: {{ModelCom}} and {{SecuredModelImpl}}. Steps 
> to add covariant return types for prefix mapping are as follows.
> Add these six methods to the Model interface.
>  # Model setNsPrefix( String prefix, String uri );
>  # Model removeNsPrefix( String prefix );
>  # Model clearNsPrefixMap();
>  # Model setNsPrefixes( PrefixMapping other );
>  # Model setNsPrefixes( Map map );
>  # Model withDefaultMappings( PrefixMapping map );
> ...and update ModelCom accordingly.
> Add these six methods to the SecuredModel interface.
>  # SecuredModel setNsPrefix( String prefix, String uri );
>  # SecuredModel removeNsPrefix( String prefix ); // This method is already 
> defined.
>  # SecuredModel clearNsPrefixMap();
>  # SecuredModel setNsPrefixes( PrefixMapping other );
>  # SecuredModel setNsPrefixes( Map map );
>  # SecuredModel withDefaultMappings( PrefixMapping map );
> ...and update SecuredModelImpl accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jena pull request #368: JENA-1495 Return Model from PrefixMapping methods.

2018-03-05 Thread jaco0646

Github user jaco0646 commented on a diff in the pull request:

https://github.com/apache/jena/pull/368#discussion_r172364101
  
--- Diff: jena-core/src/main/java/org/apache/jena/rdf/model/Model.java ---
@@ -1051,4 +1051,12 @@ Remove all the statements matching (s, p, o) from 
this model.
 Answer true iff .close() has been called on this Model.
 */
 public boolean isClosed();
+
+// Override return type for methods inherited from PrefixMapping
--- End diff --

Done.


---

[GitHub] jena pull request #367: JENA-1391: Add methods for ModelCollectors to API in...

2018-03-05 Thread ajs6f

GitHub user ajs6f reopened a pull request:

https://github.com/apache/jena/pull/367

JENA-1391: Add methods for ModelCollectors to API in ModelUtils



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajs6f/jena JENA-1391b

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/367.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #367


commit 81cd7a1c1bf362413986dced749d750abdc035c5
Author: ajs6f 
Date:   2018-03-01T14:13:35Z

JENA-1391: Add methods for ModelCollectors to API in ModelUtils




---

[jira] [Commented] (JENA-1391) Add Convenience Methods to Dataset

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386658#comment-16386658
 ] 

ASF GitHub Bot commented on JENA-1391:
--

GitHub user ajs6f reopened a pull request:

https://github.com/apache/jena/pull/367

JENA-1391: Add methods for ModelCollectors to API in ModelUtils



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajs6f/jena JENA-1391b

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/367.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #367


commit 81cd7a1c1bf362413986dced749d750abdc035c5
Author: ajs6f 
Date:   2018-03-01T14:13:35Z

JENA-1391: Add methods for ModelCollectors to API in ModelUtils




> Add Convenience Methods to Dataset
> --
>
> Key: JENA-1391
> URL: https://issues.apache.org/jira/browse/JENA-1391
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: ARQ
>Affects Versions: Jena 3.4.0
>Reporter: Adam Jacobs
>Assignee: A. Soroka
>Priority: Trivial
> Fix For: Jena 3.7.0
>
>
> The Dataset interface could provide several convenience methods similar to 
> the Model interface, allowing usability of RDF quads on par with RDF triples. 
> Specific examples include,
> # add(Dataset)
> # remove(Dataset)
> # union(Dataset)
> # intersection(Dataset)
> # difference(Dataset)
> # isEmpty()
> Following is a possible implementation of these methods.
> {code:java}
> default Dataset add(Dataset d) {
> this.getDefaultModel().add(d.getDefaultModel());
> d.listNames().forEachRemaining(name -> 
> this.getNamedModel(name).add(d.getNamedModel(name)));
> return this;
> }
> default Dataset remove(Dataset d) {
> this.getDefaultModel().remove(d.getDefaultModel());
> d.listNames().forEachRemaining(name -> 
> this.getNamedModel(name).remove(d.getNamedModel(name)));
> return this;
> }
> default Dataset union(Dataset d) {
> return DatasetFactory.create().add(this).add(d);
> }
> default Dataset difference(Dataset d) {
> Dataset output = DatasetFactory.create();
> 
> output.setDefaultModel(this.getDefaultModel().difference(d.getDefaultModel()));
> this.listNames().forEachRemaining(name -> {
> Model difference = 
> this.getNamedModel(name).difference(d.getNamedModel(name));
> if (!difference.isEmpty()) output.addNamedModel(name, difference);
> });
> return output;
> }
> default Dataset intersection(Dataset d) {
> Dataset output = DatasetFactory.create();
> 
> output.setDefaultModel(this.getDefaultModel().intersection(d.getDefaultModel()));
> Set names = this.names();
> names.retainAll(d.names());
> names.forEach(name -> {
> Model intersection = 
> this.getNamedModel(name).intersection(d.getNamedModel(name));
> if (!intersection.isEmpty()) output.addNamedModel(name, 
> intersection);
> });
> return output;
> }
> default Set names() {
> Set names = new HashSet<>();
> this.listNames().forEachRemaining(names::add);
> return names;
> }
> default boolean isEmpty() {
> return this.asDatasetGraph().isEmpty();
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1391) Add Convenience Methods to Dataset

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386654#comment-16386654
 ] 

ASF GitHub Bot commented on JENA-1391:
--

Github user ajs6f closed the pull request at:

https://github.com/apache/jena/pull/367


> Add Convenience Methods to Dataset
> --
>
> Key: JENA-1391
> URL: https://issues.apache.org/jira/browse/JENA-1391
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: ARQ
>Affects Versions: Jena 3.4.0
>Reporter: Adam Jacobs
>Assignee: A. Soroka
>Priority: Trivial
> Fix For: Jena 3.7.0
>
>
> The Dataset interface could provide several convenience methods similar to 
> the Model interface, allowing usability of RDF quads on par with RDF triples. 
> Specific examples include,
> # add(Dataset)
> # remove(Dataset)
> # union(Dataset)
> # intersection(Dataset)
> # difference(Dataset)
> # isEmpty()
> Following is a possible implementation of these methods.
> {code:java}
> default Dataset add(Dataset d) {
> this.getDefaultModel().add(d.getDefaultModel());
> d.listNames().forEachRemaining(name -> 
> this.getNamedModel(name).add(d.getNamedModel(name)));
> return this;
> }
> default Dataset remove(Dataset d) {
> this.getDefaultModel().remove(d.getDefaultModel());
> d.listNames().forEachRemaining(name -> 
> this.getNamedModel(name).remove(d.getNamedModel(name)));
> return this;
> }
> default Dataset union(Dataset d) {
> return DatasetFactory.create().add(this).add(d);
> }
> default Dataset difference(Dataset d) {
> Dataset output = DatasetFactory.create();
> 
> output.setDefaultModel(this.getDefaultModel().difference(d.getDefaultModel()));
> this.listNames().forEachRemaining(name -> {
> Model difference = 
> this.getNamedModel(name).difference(d.getNamedModel(name));
> if (!difference.isEmpty()) output.addNamedModel(name, difference);
> });
> return output;
> }
> default Dataset intersection(Dataset d) {
> Dataset output = DatasetFactory.create();
> 
> output.setDefaultModel(this.getDefaultModel().intersection(d.getDefaultModel()));
> Set names = this.names();
> names.retainAll(d.names());
> names.forEach(name -> {
> Model intersection = 
> this.getNamedModel(name).intersection(d.getNamedModel(name));
> if (!intersection.isEmpty()) output.addNamedModel(name, 
> intersection);
> });
> return output;
> }
> default Set names() {
> Set names = new HashSet<>();
> this.listNames().forEachRemaining(names::add);
> return names;
> }
> default boolean isEmpty() {
> return this.asDatasetGraph().isEmpty();
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jena pull request #367: JENA-1391: Add methods for ModelCollectors to API in...

2018-03-05 Thread ajs6f

Github user ajs6f closed the pull request at:

https://github.com/apache/jena/pull/367


---

[GitHub] jena pull request #371: JENA-1498: Fuseki backup script

2018-03-05 Thread afs

GitHub user afs opened a pull request:

https://github.com/apache/jena/pull/371

JENA-1498: Fuseki backup script



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/afs/jena fuseki-backup-script

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/371.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #371


commit 1cb843be5435a65e99538faa1f4d7eaaeee1d92d
Author: Andy Seaborne 
Date:   2018-03-05T18:54:22Z

JENA-1498: Fuseki backup script




---

[jira] [Commented] (JENA-1498) Provide an example Fuseki backup script.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386545#comment-16386545
 ] 

ASF GitHub Bot commented on JENA-1498:
--

Github user afs commented on the issue:

https://github.com/apache/jena/pull/371
  
Git has decided this is a rename when it is actually a delete/new file.


> Provide an example Fuseki backup script.
> 
>
> Key: JENA-1498
> URL: https://issues.apache.org/jira/browse/JENA-1498
> Project: Apache Jena
>  Issue Type: Improvement
>Reporter: Andy Seaborne
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1498) Provide an example Fuseki backup script.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386544#comment-16386544
 ] 

ASF GitHub Bot commented on JENA-1498:
--

GitHub user afs opened a pull request:

https://github.com/apache/jena/pull/371

JENA-1498: Fuseki backup script



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/afs/jena fuseki-backup-script

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/371.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #371


commit 1cb843be5435a65e99538faa1f4d7eaaeee1d92d
Author: Andy Seaborne 
Date:   2018-03-05T18:54:22Z

JENA-1498: Fuseki backup script




> Provide an example Fuseki backup script.
> 
>
> Key: JENA-1498
> URL: https://issues.apache.org/jira/browse/JENA-1498
> Project: Apache Jena
>  Issue Type: Improvement
>Reporter: Andy Seaborne
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jena issue #371: JENA-1498: Fuseki backup script

2018-03-05 Thread afs

Github user afs commented on the issue:

https://github.com/apache/jena/pull/371
  
Git has decided this is a rename when it is actually a delete/new file.


---

[jira] [Created] (JENA-1498) Provide an example Fuseki backup script.

2018-03-05 Thread Andy Seaborne (JIRA)

Andy Seaborne created JENA-1498:
---

 Summary: Provide an example Fuseki backup script.
 Key: JENA-1498
 URL: https://issues.apache.org/jira/browse/JENA-1498
 Project: Apache Jena
  Issue Type: Improvement
Reporter: Andy Seaborne






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386445#comment-16386445
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172276168
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( !

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

2018-03-05 Thread ajs6f

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172276168
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported() )
+handlers.put(key, th) ;
+}
+
+// Determine the key - an object that is the unit

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386444#comment-16386444
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user afs commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172275879
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported()

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

2018-03-05 Thread afs

Github user afs commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172275879
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported() )
+handlers.put(key, th) ;
+}
+
+// Determine the key - an object that is the unit of

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386431#comment-16386431
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172272325
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( !

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

2018-03-05 Thread ajs6f

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172272325
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported() )
+handlers.put(key, th) ;
+}
+
+// Determine the key - an object that is the unit

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386428#comment-16386428
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user afs commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172271582
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported()

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

2018-03-05 Thread afs

Github user afs commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172271582
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported() )
+handlers.put(key, th) ;
+}
+
+// Determine the key - an object that is the unit of

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386356#comment-16386356
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172252112
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( !

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

2018-03-05 Thread ajs6f

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172252346
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported() )
+handlers.put(key, th) ;
+}
+
+// Determine the key - an object that is the unit

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

2018-03-05 Thread ajs6f

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172254605
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
--- End diff --

typo missing `)`


---

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386359#comment-16386359
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172251788
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
--- End diff --

`onld` => `old`


> Transactions not passed down for nested models.
> ---
>
> Key: JENA-1492
> URL: https://issues.apache.org/jira/browse/JENA-1492
> Project: Apache Jena
>  Issue Type: Bug
>  Components: Fuseki, TDB, TDB2
>Affects Versions: Jena 3.7.0
>Reporter: Andy Seaborne
>Priority: Major
> Attachments: config-tdb2-model.ttl
>
>
> From [users@ 
> email|https://lists.apache.org/thread.html/eacd92488360298a4916b2b1b0cf2d299797140be1960f84410f9b5d@%3Cusers.jena.apache.org%3E],
>  after the configuration sorted out.
> Hierarchies of models, such as inference models, on top of a TBD2 
> backed-graph do not pass down the transaction leading to
> {noformat}
> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a 
> transaction
>     at 
> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
>     at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
>     at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
>     at 
> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
>     at 
> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
>     at 
>

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386358#comment-16386358
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172254605
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
--- End diff --

typo missing `)`


> Transactions not passed down for nested models.
> ---
>
> Key: JENA-1492
> URL: https://issues.apache.org/jira/browse/JENA-1492
> Project: Apache Jena
>  Issue Type: Bug
>  Components: Fuseki, TDB, TDB2
>Affects Versions: Jena 3.7.0
>Reporter: Andy Seaborne
>Priority: Major
> Attachments: config-tdb2-model.ttl
>
>
> From [users@ 
> email|https://lists.apache.org/thread.html/eacd92488360298a4916b2b1b0cf2d299797140be1960f84410f9b5d@%3Cusers.jena.apache.org%3E],
>  after the configuration sorted out.
> Hierarchies of models, such as inference models, on top of a TBD2 
> backed-graph do not pass down the transaction leading to
> {noformat}
> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a 
> transaction
>     at 
> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
>     at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
>     at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
>     at 
> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
>     at 
> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
>     at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
>     at 
> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:110)
>     at 
>

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386354#comment-16386354
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/369
  
Okay. `::wrap` vs. `::create` is not what I am referring to as 
reformatting. It's stuff like 
[this](https://github.com/apache/jena/pull/369/files#diff-27cb8b74e951d2872a91a48066930e2d)
 or 
[this](https://github.com/apache/jena/pull/369/files#diff-9d27f4f9095b5d190b6fe8977b6c393b).

From what I understand, this all makes sense. Datasets that are "collected" 
from multiple graphs can, at best, pass the transaction through to their 
graphs, those that are in control of their own data can retain the 
responsibility for transactions themselves. That does seem to be the least 
surprising behavior and the one that veers towards "dataset as the fundamental 
unit" which we have been working towards.



> Transactions not passed down for nested models.
> ---
>
> Key: JENA-1492
> URL: https://issues.apache.org/jira/browse/JENA-1492
> Project: Apache Jena
>  Issue Type: Bug
>  Components: Fuseki, TDB, TDB2
>Affects Versions: Jena 3.7.0
>Reporter: Andy Seaborne
>Priority: Major
> Attachments: config-tdb2-model.ttl
>
>
> From [users@ 
> email|https://lists.apache.org/thread.html/eacd92488360298a4916b2b1b0cf2d299797140be1960f84410f9b5d@%3Cusers.jena.apache.org%3E],
>  after the configuration sorted out.
> Hierarchies of models, such as inference models, on top of a TBD2 
> backed-graph do not pass down the transaction leading to
> {noformat}
> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a 
> transaction
>     at 
> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
>     at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
>     at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
>     at 
> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
>     at 
> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
>     at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
>     at 
> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:110)
>     at 
> org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:47)
>     at 
> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
>     at org.apache.jena.sparql.core.GraphView.graphBaseFind(GraphView.java:125)
>     at org.apache.jena.graph.impl.GraphBase.find(GraphBase.java:255)
> {noformat}
> Example configuration (full version attached):
> {noformat}
> :service  a   fuseki:Service ;
>     fuseki:dataset    :dataset ;
> ...
> ## Wrapper: Datset containing one TDB2-backed graph
> :dataset a ja:RDFDataset ;
>     ja:defaultGraph   :graph .
> ##  TDB2-backed graph
> :graph rdf:type tdb2:GraphTDB ;
>     tdb2:dataset :datasetTDB2 .
> ## TDB2 database
> :datasetTDB2 rdf:type tdb2:DatasetTDB2 ;
>     tdb2:location "DB2".
> {noformat}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386357#comment-16386357
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172254303
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( !

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

2018-03-05 Thread ajs6f

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172251788
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
--- End diff --

`onld` => `old`


---

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

2018-03-05 Thread ajs6f

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172254303
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported() )
+handlers.put(key, th) ;
+}
+
+// Determine the key - an object that is the unit

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386360#comment-16386360
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172252346
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( !

[GitHub] jena pull request #369: JENA-1492: Pass down transactions from general datas...

2018-03-05 Thread ajs6f

Github user ajs6f commented on a diff in the pull request:

https://github.com/apache/jena/pull/369#discussion_r172252112
  
--- Diff: 
jena-arq/src/main/java/org/apache/jena/sparql/core/TxnDataset2Graph.java ---
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.*;
+import java.util.function.Consumer;
+
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.TransactionHandler;
+import org.apache.jena.query.ReadWrite;
+import org.apache.jena.query.TxnType;
+import org.apache.jena.reasoner.InfGraph;
+import org.apache.jena.shared.LockMRSW;
+import org.apache.jena.sparql.JenaTransactionException;
+
+/**
+ * A {@link Transactional} that passes the transaction operations down to 
transactions on
+ * independent graphs.
+ * 
+ * There are limitations:
+ * 
+ * we can't atomically do all the commits together in the crash 
situation.
+ * This {@code Transactional} maintains a MRSW policy because that is 
all that is
+ * required of graphs in general.
+ * 
+ * It does cover the important case of one graph ({@link DatasetGraphOne}) 
where the one
+ * graph is an InfGraph and should work when the graphs in the dataset is 
not changing or
+ * when a new memory graph is added mid-transaction.
+ * 
+ * This is not "nested transactions" - theer is no overall "commit" or 
"abort". If
+ * failure/restart occurs, some graphs may have commited and others not. 
It is the best
+ * that can be done given for an arbitrary collection of graphs, backed by 
different
+ * storage and having different capabilities.
+ * 
+ * Best practice is to change the graph membership outside of any 
transaction,
+ * ideally at setup time of the object using this class. (Caution: SPARQL 
Update
+ * can create graphs.   
+ * @See {@link DatasetGraphMapLink}
+ * @See {@link DatasetGraphOne}
+ */
+public class TxnDataset2Graph extends TransactionalLock {
+/**
+ * Control whether to pass down transactions from the dataset to the 
graph in the
+ * dataset. This should be set to "true"; setting it "false" causes 
the onld,
+ * no-transaction passing behaviour.
+ * 
+ * This is temporary flag during the transition because the change at 
Jena 3.7.0 needs
+ * to be proven in real deployments as well as testing. "false" 
restores the Jena
+ * 3.6.0 and before behaviour (transactions not passed down). See 
JENA-1492.
+ * 
+ * @deprecated This flag will be removed.
+ */
+@Deprecated
+public static boolean TXN_DSG_GRAPH = true;
+
+private Graph primary;
+// Object key may be a graph or a DSG is the graph is a GraphView.
+// This avoids starting a tranasction on the same storage unit twice. 
+private Map handlers = new HashMap<>();
+
+private Object lock = new Object();
+
+public TxnDataset2Graph(Graph primaryGraph, Graph ... otherGraphs) {
+super(new LockMRSW());
+primary = primaryGraph;
+handlers = buildHandlerSet(primary, Arrays.asList(otherGraphs));
+}
+
+private static Map buildHandlerSet(Graph 
primary, Collection graphs) {
+Map handlers = new HashMap<>();
+addHandler(handlers, primary);
+graphs.forEach(g->addHandler(handlers,g));
+return handlers;
+}
+
+private static void addHandler(Map 
handlers, Graph graph) {
+TransactionHandler th = graph.getTransactionHandler();
+if ( ! th.transactionsSupported() )
+return;
+Object key = calcKey(graph);
+if ( th.transactionsSupported() )
+handlers.put(key, th) ;
+}
+
+// Determine the key - an object that is the unit

[GitHub] jena issue #369: JENA-1492: Pass down transactions from general datasets

2018-03-05 Thread ajs6f

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/369
  
Okay. `::wrap` vs. `::create` is not what I am referring to as 
reformatting. It's stuff like 
[this](https://github.com/apache/jena/pull/369/files#diff-27cb8b74e951d2872a91a48066930e2d)
 or 
[this](https://github.com/apache/jena/pull/369/files#diff-9d27f4f9095b5d190b6fe8977b6c393b).

From what I understand, this all makes sense. Datasets that are "collected" 
from multiple graphs can, at best, pass the transaction through to their 
graphs, those that are in control of their own data can retain the 
responsibility for transactions themselves. That does seem to be the least 
surprising behavior and the one that veers towards "dataset as the fundamental 
unit" which we have been working towards.



---

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386317#comment-16386317
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user afs commented on the issue:

https://github.com/apache/jena/pull/369
  
Yes, this is models passing down transactions.  It only became possible 
with dataset-transaction promotion (new to 3.7.0).  

This all has a lot of knock on consequences. The wrap/create distinction is 
quite important and that touches a lot of files; it is not reformatting. "wrap" 
a model and the dataset can not have named graphs added, "create" with a model 
and it starts with that model and can have named graphs added. We only used to 
have "create" which has overhead and complexity as well as occasionally leading 
to unexpected behaviour with SPARQL Updates (updates work but the new NG are 
in-memory and disappear on restart - better to have a closed dataset and catch 
the issue at update time).

While testing, I found lack of test coverage and integration testing, which 
is the root of the users@ email that triggered this, was the place to put this. 
There as a lack of assemblers for certain other special cases so I included 
them as well as they arose naturally from checking test coverage.




> Transactions not passed down for nested models.
> ---
>
> Key: JENA-1492
> URL: https://issues.apache.org/jira/browse/JENA-1492
> Project: Apache Jena
>  Issue Type: Bug
>  Components: Fuseki, TDB, TDB2
>Affects Versions: Jena 3.7.0
>Reporter: Andy Seaborne
>Priority: Major
> Attachments: config-tdb2-model.ttl
>
>
> From [users@ 
> email|https://lists.apache.org/thread.html/eacd92488360298a4916b2b1b0cf2d299797140be1960f84410f9b5d@%3Cusers.jena.apache.org%3E],
>  after the configuration sorted out.
> Hierarchies of models, such as inference models, on top of a TBD2 
> backed-graph do not pass down the transaction leading to
> {noformat}
> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a 
> transaction
>     at 
> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
>     at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
>     at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
>     at 
> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
>     at 
> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
>     at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
>     at 
> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:110)
>     at 
> org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:47)
>     at 
> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
>     at org.apache.jena.sparql.core.GraphView.graphBaseFind(GraphView.java:125)
>     at org.apache.jena.graph.impl.GraphBase.find(GraphBase.java:255)
> {noformat}
> Example configuration (full version attached):
> {noformat}
> :service  a   fuseki:Service ;
>     fuseki:dataset    :dataset ;
> ...
> ## Wrapper: Datset containing one TDB2-backed graph
> :dataset a ja:RDFDataset ;
>     ja:defaultGraph   :graph .
> ##  TDB2-backed graph
> :graph rdf:type tdb2:GraphTDB ;
>     tdb2:dataset :datasetTDB2 .
> ## TDB2 database
> :datasetTDB2 rdf:type tdb2:DatasetTDB2 ;
>     tdb2:location "DB2".
> {noformat}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jena issue #369: JENA-1492: Pass down transactions from general datasets

2018-03-05 Thread afs

Github user afs commented on the issue:

https://github.com/apache/jena/pull/369
  
Yes, this is models passing down transactions.  It only became possible 
with dataset-transaction promotion (new to 3.7.0).  

This all has a lot of knock on consequences. The wrap/create distinction is 
quite important and that touches a lot of files; it is not reformatting. "wrap" 
a model and the dataset can not have named graphs added, "create" with a model 
and it starts with that model and can have named graphs added. We only used to 
have "create" which has overhead and complexity as well as occasionally leading 
to unexpected behaviour with SPARQL Updates (updates work but the new NG are 
in-memory and disappear on restart - better to have a closed dataset and catch 
the issue at update time).

While testing, I found lack of test coverage and integration testing, which 
is the root of the users@ email that triggered this, was the place to put this. 
There as a lack of assemblers for certain other special cases so I included 
them as well as they arose naturally from checking test coverage.




---

[jira] [Commented] (JENA-1495) Covariant return type for Model's PrefixMapping methods.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386290#comment-16386290
 ] 

ASF GitHub Bot commented on JENA-1495:
--

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/368
  
This looks basically good to me: with an additional commit addressing 
https://github.com/apache/jena/pull/368#pullrequestreview-101006841, I'll be 
happy to merge this.


> Covariant return type for Model's PrefixMapping methods.
> 
>
> Key: JENA-1495
> URL: https://issues.apache.org/jira/browse/JENA-1495
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: Core
>Affects Versions: Jena 3.6.0
>Reporter: Adam Jacobs
>Priority: Trivial
>  Labels: easyfix
>
> The {{Model}} interface inherits several methods from {{PrefixMapping}} which 
> return {{this}}.
>  The return type of these methods can be refined to allow call chaining with 
> the {{Model}} object.
> There appear to be two concrete implementations of {{Model}} which override 
> the {{PrefixMapping}} methods: {{ModelCom}} and {{SecuredModelImpl}}. Steps 
> to add covariant return types for prefix mapping are as follows.
> Add these six methods to the Model interface.
>  # Model setNsPrefix( String prefix, String uri );
>  # Model removeNsPrefix( String prefix );
>  # Model clearNsPrefixMap();
>  # Model setNsPrefixes( PrefixMapping other );
>  # Model setNsPrefixes( Map map );
>  # Model withDefaultMappings( PrefixMapping map );
> ...and update ModelCom accordingly.
> Add these six methods to the SecuredModel interface.
>  # SecuredModel setNsPrefix( String prefix, String uri );
>  # SecuredModel removeNsPrefix( String prefix ); // This method is already 
> defined.
>  # SecuredModel clearNsPrefixMap();
>  # SecuredModel setNsPrefixes( PrefixMapping other );
>  # SecuredModel setNsPrefixes( Map map );
>  # SecuredModel withDefaultMappings( PrefixMapping map );
> ...and update SecuredModelImpl accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jena issue #368: JENA-1495 Return Model from PrefixMapping methods.

2018-03-05 Thread ajs6f

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/368
  
This looks basically good to me: with an additional commit addressing 
https://github.com/apache/jena/pull/368#pullrequestreview-101006841, I'll be 
happy to merge this.


---

[jira] [Commented] (JENA-1492) Transactions not passed down for nested models.

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386286#comment-16386286
 ] 

ASF GitHub Bot commented on JENA-1492:
--

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/369
  
There is some new `Assembler` stuff and reformatting, so I'm not totally 
clear what the thrust of this PR is: is it to ensure that `Model`s wrapping 
other `Model`s (e.g. for inferencing) correctly pass transaction control calls 
into their wrapped models (which should pass them into their underlying 
`Dataset`)?


> Transactions not passed down for nested models.
> ---
>
> Key: JENA-1492
> URL: https://issues.apache.org/jira/browse/JENA-1492
> Project: Apache Jena
>  Issue Type: Bug
>  Components: Fuseki, TDB, TDB2
>Affects Versions: Jena 3.7.0
>Reporter: Andy Seaborne
>Priority: Major
> Attachments: config-tdb2-model.ttl
>
>
> From [users@ 
> email|https://lists.apache.org/thread.html/eacd92488360298a4916b2b1b0cf2d299797140be1960f84410f9b5d@%3Cusers.jena.apache.org%3E],
>  after the configuration sorted out.
> Hierarchies of models, such as inference models, on top of a TBD2 
> backed-graph do not pass down the transaction leading to
> {noformat}
> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a 
> transaction
>     at 
> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
>     at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
>     at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
>     at 
> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
>     at 
> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
>     at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
>     at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
>     at 
> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:110)
>     at 
> org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:47)
>     at 
> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
>     at org.apache.jena.sparql.core.GraphView.graphBaseFind(GraphView.java:125)
>     at org.apache.jena.graph.impl.GraphBase.find(GraphBase.java:255)
> {noformat}
> Example configuration (full version attached):
> {noformat}
> :service  a   fuseki:Service ;
>     fuseki:dataset    :dataset ;
> ...
> ## Wrapper: Datset containing one TDB2-backed graph
> :dataset a ja:RDFDataset ;
>     ja:defaultGraph   :graph .
> ##  TDB2-backed graph
> :graph rdf:type tdb2:GraphTDB ;
>     tdb2:dataset :datasetTDB2 .
> ## TDB2 database
> :datasetTDB2 rdf:type tdb2:DatasetTDB2 ;
>     tdb2:location "DB2".
> {noformat}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jena issue #369: JENA-1492: Pass down transactions from general datasets

2018-03-05 Thread ajs6f

Github user ajs6f commented on the issue:

https://github.com/apache/jena/pull/369
  
There is some new `Assembler` stuff and reformatting, so I'm not totally 
clear what the thrust of this PR is: is it to ensure that `Model`s wrapping 
other `Model`s (e.g. for inferencing) correctly pass transaction control calls 
into their wrapped models (which should pass them into their underlying 
`Dataset`)?


---

[jira] [Resolved] (JENA-1497) ParameterizedSparqlString detects delimiters incorrectly

2018-03-05 Thread Rob Vesse (JIRA)


 [ 
https://issues.apache.org/jira/browse/JENA-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rob Vesse resolved JENA-1497.
-
   Resolution: Fixed
Fix Version/s: Jena 3.7.0

Fixed for the 3.7.0 release

> ParameterizedSparqlString detects delimiters incorrectly
> 
>
> Key: JENA-1497
> URL: https://issues.apache.org/jira/browse/JENA-1497
> Project: Apache Jena
>  Issue Type: Bug
>  Components: ARQ
>Affects Versions: Jena 3.6.0
>Reporter: Rob Vesse
>Assignee: Rob Vesse
>Priority: Major
> Fix For: Jena 3.7.0
>
>
> As reported on the mailing list - 
> [https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E|https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E]
> Investigation shows that the delimiter parsing logic has some flaws that 
> causes it to do the wrong thing resulting in the possibility of incorrect 
> detection of injection attacks leading to some valid SPARQL strings being 
> rejected when attempting to inject parameters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1497) ParameterizedSparqlString detects delimiters incorrectly

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386160#comment-16386160
 ] 

ASF GitHub Bot commented on JENA-1497:
--

Github user asfgit closed the pull request at:

https://github.com/apache/jena/pull/370


> ParameterizedSparqlString detects delimiters incorrectly
> 
>
> Key: JENA-1497
> URL: https://issues.apache.org/jira/browse/JENA-1497
> Project: Apache Jena
>  Issue Type: Bug
>  Components: ARQ
>Affects Versions: Jena 3.6.0
>Reporter: Rob Vesse
>Assignee: Rob Vesse
>Priority: Major
>
> As reported on the mailing list - 
> [https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E|https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E]
> Investigation shows that the delimiter parsing logic has some flaws that 
> causes it to do the wrong thing resulting in the possibility of incorrect 
> detection of injection attacks leading to some valid SPARQL strings being 
> rejected when attempting to inject parameters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jena pull request #370: Fix delimiter parsing logic (JENA-1497)

2018-03-05 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/jena/pull/370


---

[jira] [Commented] (JENA-1497) ParameterizedSparqlString detects delimiters incorrectly

2018-03-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386159#comment-16386159
 ] 

ASF subversion and git services commented on JENA-1497:
---

Commit 2e732b48500f5856d9b4ba639960b8bd8baa8a1a in jena's branch 
refs/heads/master from [~rvesse]
[ https://git-wip-us.apache.org/repos/asf?p=jena.git;h=2e732b4 ]

Merge branch 'JENA-1497'

This closes #370


> ParameterizedSparqlString detects delimiters incorrectly
> 
>
> Key: JENA-1497
> URL: https://issues.apache.org/jira/browse/JENA-1497
> Project: Apache Jena
>  Issue Type: Bug
>  Components: ARQ
>Affects Versions: Jena 3.6.0
>Reporter: Rob Vesse
>Assignee: Rob Vesse
>Priority: Major
>
> As reported on the mailing list - 
> [https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E|https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E]
> Investigation shows that the delimiter parsing logic has some flaws that 
> causes it to do the wrong thing resulting in the possibility of incorrect 
> detection of injection attacks leading to some valid SPARQL strings being 
> rejected when attempting to inject parameters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1497) ParameterizedSparqlString detects delimiters incorrectly

2018-03-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386157#comment-16386157
 ] 

ASF subversion and git services commented on JENA-1497:
---

Commit 411c1031a0f51885f6966914c58202654614be13 in jena's branch 
refs/heads/master from [~rvesse]
[ https://git-wip-us.apache.org/repos/asf?p=jena.git;h=411c103 ]

Fix delimiter parsing logic (JENA-1497)

Logical flaws in using continue vs break inside inner loops where
causing the wrong delimiter positions to be detected and leading to
false positives being reported for potential injection attacks.  Fixing
the logic allows the user test case to pass.


> ParameterizedSparqlString detects delimiters incorrectly
> 
>
> Key: JENA-1497
> URL: https://issues.apache.org/jira/browse/JENA-1497
> Project: Apache Jena
>  Issue Type: Bug
>  Components: ARQ
>Affects Versions: Jena 3.6.0
>Reporter: Rob Vesse
>Assignee: Rob Vesse
>Priority: Major
>
> As reported on the mailing list - 
> [https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E|https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E]
> Investigation shows that the delimiter parsing logic has some flaws that 
> causes it to do the wrong thing resulting in the possibility of incorrect 
> detection of injection attacks leading to some valid SPARQL strings being 
> rejected when attempting to inject parameters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1497) ParameterizedSparqlString detects delimiters incorrectly

2018-03-05 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386158#comment-16386158
 ] 

ASF subversion and git services commented on JENA-1497:
---

Commit 44683c424da9bea3b261151f7243524813917055 in jena's branch 
refs/heads/master from [~rvesse]
[ https://git-wip-us.apache.org/repos/asf?p=jena.git;h=44683c4 ]

Rename test cases and fix warnings (JENA-1497)


> ParameterizedSparqlString detects delimiters incorrectly
> 
>
> Key: JENA-1497
> URL: https://issues.apache.org/jira/browse/JENA-1497
> Project: Apache Jena
>  Issue Type: Bug
>  Components: ARQ
>Affects Versions: Jena 3.6.0
>Reporter: Rob Vesse
>Assignee: Rob Vesse
>Priority: Major
>
> As reported on the mailing list - 
> [https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E|https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E]
> Investigation shows that the delimiter parsing logic has some flaws that 
> causes it to do the wrong thing resulting in the possibility of incorrect 
> detection of injection attacks leading to some valid SPARQL strings being 
> rejected when attempting to inject parameters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1497) ParameterizedSparqlString detects delimiters incorrectly

2018-03-05 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385908#comment-16385908
 ] 

ASF GitHub Bot commented on JENA-1497:
--

GitHub user rvesse opened a pull request:

https://github.com/apache/jena/pull/370

Fix delimiter parsing logic (JENA-1497)

Logical flaws in using continue vs break inside inner loops where
causing the wrong delimiter positions to be detected and leading to
false positives being reported for potential injection attacks.  Fixing
the logic allows the user test case to pass.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rvesse/jena JENA-1497

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/370.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #370


commit 411c1031a0f51885f6966914c58202654614be13
Author: Rob Vesse 
Date:   2018-03-05T10:25:24Z

Fix delimiter parsing logic (JENA-1497)

Logical flaws in using continue vs break inside inner loops where
causing the wrong delimiter positions to be detected and leading to
false positives being reported for potential injection attacks.  Fixing
the logic allows the user test case to pass.




> ParameterizedSparqlString detects delimiters incorrectly
> 
>
> Key: JENA-1497
> URL: https://issues.apache.org/jira/browse/JENA-1497
> Project: Apache Jena
>  Issue Type: Bug
>  Components: ARQ
>Affects Versions: Jena 3.6.0
>Reporter: Rob Vesse
>Assignee: Rob Vesse
>Priority: Major
>
> As reported on the mailing list - 
> [https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E|https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E]
> Investigation shows that the delimiter parsing logic has some flaws that 
> causes it to do the wrong thing resulting in the possibility of incorrect 
> detection of injection attacks leading to some valid SPARQL strings being 
> rejected when attempting to inject parameters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jena pull request #370: Fix delimiter parsing logic (JENA-1497)

2018-03-05 Thread rvesse

GitHub user rvesse opened a pull request:

https://github.com/apache/jena/pull/370

Fix delimiter parsing logic (JENA-1497)

Logical flaws in using continue vs break inside inner loops where
causing the wrong delimiter positions to be detected and leading to
false positives being reported for potential injection attacks.  Fixing
the logic allows the user test case to pass.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rvesse/jena JENA-1497

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/370.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #370


commit 411c1031a0f51885f6966914c58202654614be13
Author: Rob Vesse 
Date:   2018-03-05T10:25:24Z

Fix delimiter parsing logic (JENA-1497)

Logical flaws in using continue vs break inside inner loops where
causing the wrong delimiter positions to be detected and leading to
false positives being reported for potential injection attacks.  Fixing
the logic allows the user test case to pass.




---

[jira] [Commented] (JENA-1497) ParameterizedSparqlString detects delimiters incorrectly

2018-03-05 Thread Rob Vesse (JIRA)


[ 
https://issues.apache.org/jira/browse/JENA-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385897#comment-16385897
 ] 

Rob Vesse commented on JENA-1497:
-

Basic problem is use of {{continue}} vs {{break}} in the parsing logic, 
{{continue}} is used inside inner loops which of course doesn't do what is 
desired resulting in invalid delimiter detection which for complex SPARQL 
strings results in potential false positives of potential injection attacks.  
Changing to {{break}} in the appropriate places resolves the issue

> ParameterizedSparqlString detects delimiters incorrectly
> 
>
> Key: JENA-1497
> URL: https://issues.apache.org/jira/browse/JENA-1497
> Project: Apache Jena
>  Issue Type: Bug
>  Components: ARQ
>Affects Versions: Jena 3.6.0
>Reporter: Rob Vesse
>Assignee: Rob Vesse
>Priority: Major
>
> As reported on the mailing list - 
> [https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E|https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E]
> Investigation shows that the delimiter parsing logic has some flaws that 
> causes it to do the wrong thing resulting in the possibility of incorrect 
> detection of injection attacks leading to some valid SPARQL strings being 
> rejected when attempting to inject parameters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (JENA-1497) ParameterizedSparqlString detects delimiters incorrectly

2018-03-05 Thread Rob Vesse (JIRA)

Rob Vesse created JENA-1497:
---

 Summary: ParameterizedSparqlString detects delimiters incorrectly
 Key: JENA-1497
 URL: https://issues.apache.org/jira/browse/JENA-1497
 Project: Apache Jena
  Issue Type: Bug
  Components: ARQ
Affects Versions: Jena 3.6.0
Reporter: Rob Vesse
Assignee: Rob Vesse


As reported on the mailing list - 
[https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E|https://lists.apache.org/thread.html/3855aa8046cfea61433042655144f071c56baa7c5d61a78544730455@%3Cusers.jena.apache.org%3E]

Investigation shows that the delimiter parsing logic has some flaws that causes 
it to do the wrong thing resulting in the possibility of incorrect detection of 
injection attacks leading to some valid SPARQL strings being rejected when 
attempting to inject parameters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

48 matches

Mail list logo