[GitHub] jena issue #503: JENA-1645: Use uri predicate in concrete subject query.
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/503 cc @osma @xristy for review as main devs in this area ---
[GitHub] jena pull request #501: JENA-1643: Use transactional dataset in clear().
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/501#discussion_r237831448
--- Diff: jena-tdb/src/main/java/org/apache/jena/tdb/store/GraphTDB.java ---
@@ -171,7 +171,10 @@ protected final int graphBaseSize() {
@Override
public void clear() {
-getDatasetGraphTDB().deleteAny(getGraphName(), Node.ANY, Node.ANY,
Node.ANY) ;
+// Logically, this is "super.clear()" except the default
implementation
+// is a loop that materializes nodes. We want to call the dataset
directly
+// so that nodes don't get materialized, just deleted from indexes.
--- End diff --
Loaded from the node table into nodes in the cache I think?
---
[GitHub] jena pull request #492: JENA-1623: Endpoint access control lists
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/492#discussion_r235398663
--- Diff:
jena-fuseki2/jena-fuseki-core/src/main/java/org/apache/jena/fuseki/build/FusekiBuilder.java
---
@@ -70,16 +69,42 @@ public static void populateStdServices(DataService
dataService, boolean allowUpd
/** Add an operation to a {@link DataService} with a given endpoint
name */
public static void addServiceEP(DataService dataService, Operation
operation, String endpointName) {
-dataService.addEndpoint(operation, endpointName) ;
+dataService.addEndpoint(operation, endpointName) ;
+}
+
+/** Add an operation to a {@link DataService} with a given endpoint
name */
+public static void addServiceEP(DataService dataService, Operation
operation, String endpointName, AuthPolicy requestAuth) {
+dataService.addEndpoint(operation, endpointName, requestAuth) ;
}
public static void addServiceEP(DataService dataService, Operation
operation, Resource svc, Property property) {
String p = "<"+property.getURI()+">" ;
ResultSet rs = FusekiBuildLib.query("SELECT * { ?svc " + p + "
?ep}", svc.getModel(), "svc", svc) ;
for ( ; rs.hasNext() ; ) {
QuerySolution soln = rs.next() ;
-String epName = soln.getLiteral("ep").getLexicalForm() ;
-addServiceEP(dataService, operation, epName);
+AuthPolicy requestAuth = null;
--- End diff --
Looks like `FusekiBuilder` populates this at a later stage so maybe a
comment to that effect here?
---
[GitHub] jena pull request #492: JENA-1623: Endpoint access control lists
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/492#discussion_r235395467
--- Diff:
jena-fuseki2/jena-fuseki-core/src/main/java/org/apache/jena/fuseki/build/FusekiBuilder.java
---
@@ -70,16 +69,42 @@ public static void populateStdServices(DataService
dataService, boolean allowUpd
/** Add an operation to a {@link DataService} with a given endpoint
name */
public static void addServiceEP(DataService dataService, Operation
operation, String endpointName) {
-dataService.addEndpoint(operation, endpointName) ;
+dataService.addEndpoint(operation, endpointName) ;
+}
+
+/** Add an operation to a {@link DataService} with a given endpoint
name */
+public static void addServiceEP(DataService dataService, Operation
operation, String endpointName, AuthPolicy requestAuth) {
+dataService.addEndpoint(operation, endpointName, requestAuth) ;
}
public static void addServiceEP(DataService dataService, Operation
operation, Resource svc, Property property) {
String p = "<"+property.getURI()+">" ;
ResultSet rs = FusekiBuildLib.query("SELECT * { ?svc " + p + "
?ep}", svc.getModel(), "svc", svc) ;
for ( ; rs.hasNext() ; ) {
QuerySolution soln = rs.next() ;
-String epName = soln.getLiteral("ep").getLexicalForm() ;
-addServiceEP(dataService, operation, epName);
+AuthPolicy requestAuth = null;
--- End diff --
Should this be getting populated with a non-null value here? Or is the
intention to set to `null` by default and then potentially load in the auth
policy later elsewhere?
---
[GitHub] jena pull request #492: JENA-1623: Endpoint access control lists
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/492#discussion_r235394477
--- Diff:
jena-fuseki2/jena-fuseki-core/src/main/java/org/apache/jena/fuseki/auth/AuthPolicy.java
---
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.fuseki.auth;
+
+/**
+ * Policy for authorization to a resource.
+ * Assumes the user has already been authenticated.
+ */
+public interface AuthPolicy {
+/**
+ * Is the use authorized for this resource?
+ */
+public boolean isAllowed(String user);
+
+/**
+ * Is the use denied for this resource? Both {@linkplain #isAllowed}
and
+ * {@linkplain #isDenied} could be false if the policy does not knwo
one way of the
--- End diff --
Typo - `knwo` -> `know`
---
[GitHub] jena issue #489: Merged store literals once for JENA-1630
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/489 So basically this is just avoiding redundantly storing the literal value multiple times. The actual literal value is always stored at least once but may be indexed multiple times in many non-stored fields with different analysers for different languages? ---
[GitHub] jena issue #474: JENA-1609 Jena master does not build under JDK10
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/474 @afs Agreed, many folks are going to migrate Java 8 -> Java 11 because of the LTS release status are are skipping interim versions ---
[GitHub] jena pull request #476: Fix JDK10 Build (JENA-1609)
GitHub user rvesse opened a pull request: https://github.com/apache/jena/pull/476 Fix JDK10 Build (JENA-1609) Simplified version of PR #474 avoiding the dependency weirdness by adding the missing dependency only for the execution of the Javadoc plugin on the offending module cc @lewismc You can merge this pull request into a Git repository by running: $ git pull https://github.com/rvesse/jena jdk10-build Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/476.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #476 commit ae06ecf498c3cd7e909b76d81ae72a05709c6941 Author: Rob Vesse Date: 2018-10-03T17:33:25Z Get Jena JDBC building on JDK 10 (JENA-1609) Pass needed log4j dependency directly to Javadoc plugin when bundling the Jena JDBC javadocs. Also upgrade Javadoc plugin to 3.0.1 to avoid bugs commit 4ee971b42a018dc8887db274283a1606b2f2b32a Author: Rob Vesse Date: 2018-10-03T18:02:30Z Remove Javadoc link that JDK 10 dislikes (JENA-1609) ---
[GitHub] jena issue #474: JENA-1609 Jena master does not build under JDK10
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/474 I think there is a simpler solution to this, I have Jena JDBC building locally under JDK10 and will push a new PR shortly ---
[GitHub] jena pull request #474: JENA-1609 Jena master does not build under JDK10
Github user rvesse commented on a diff in the pull request: https://github.com/apache/jena/pull/474#discussion_r222388084 --- Diff: jena-jdbc/jena-jdbc-core/pom.xml --- @@ -39,17 +39,28 @@ 3.10.0-SNAPSHOT - - - org.slf4j - slf4j-log4j12 - + + + org.slf4j + slf4j-log4j12 + + + + log4j + log4j + + + + org.apache.logging.log4j + log4j-api + + + + + org.apache.logging.log4j --- End diff -- This code predates Log4j 2.x so shouldn't have any dependency upon it ---
[GitHub] jena issue #474: JENA-1609 Jena master does not build under JDK10
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/474 @afs https://github.com/rvesse/airline/blob/master/pom.xml#L495-L497 which I discovered via JIRA https://issues.apache.org/jira/browse/MJAVADOC-475 ---
[GitHub] jena issue #474: JENA-1609 Jena master does not build under JDK10
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/474 @afs For some projects where I have added JDK 9/10 support I have had to create separate profiles activated by JDK version for the Javadoc plugin because there are breaking changes between the plugin versions and supported flags for the underlying `javadoc` tool Also if you have updated the Javadoc plugin version to a 3.x release then you need to add an `` sub-element within `` rather than just specifying the options directly ---
[GitHub] jena pull request #474: JENA-1609 Jena master does not build under JDK10
Github user rvesse commented on a diff in the pull request: https://github.com/apache/jena/pull/474#discussion_r221884001 --- Diff: jena-jdbc/jena-jdbc-core/pom.xml --- @@ -39,17 +39,28 @@ 3.10.0-SNAPSHOT - - - org.slf4j - slf4j-log4j12 - + + + org.slf4j + slf4j-log4j12 + + + + log4j + log4j + + + --- End diff -- `JenaDriver` can optionally configure Log4j, this is because JDBC drivers can be used in contexts where the end user can't directly configure logging e.g. when dropped into other applications. In those contexts users can add `logging=` to have the driver configure logging for them. However the default behaviour is to not configure anything and all the code internally uses SLF4J so in environments where users can control logging configuration they are free to do that. So in environments where the user fully controls things they can configure logging as appropriate. ---
[GitHub] jena pull request #474: JENA-1609 Jena master does not build under JDK10
Github user rvesse commented on a diff in the pull request: https://github.com/apache/jena/pull/474#discussion_r221885106 --- Diff: jena-jdbc/jena-jdbc-core/pom.xml --- @@ -39,17 +39,28 @@ 3.10.0-SNAPSHOT - - - org.slf4j - slf4j-log4j12 - + + + org.slf4j + slf4j-log4j12 + + + + log4j + log4j + + + + org.apache.logging.log4j + log4j-api + + + + + org.apache.logging.log4j --- End diff -- I think this is unnecessary, the only Log4j direct usage is to configure Log4j, the code itself always uses SLF4J for its logging ---
[GitHub] jena pull request #465: JENA-1594: Per-graph access control for Fuseki
Github user rvesse commented on a diff in the pull request: https://github.com/apache/jena/pull/465#discussion_r213250204 --- Diff: jena-fuseki2/jena-fuseki-access/src/main/java/org/apache/jena/fuseki/access/DataAccessCtl.java --- @@ -0,0 +1,163 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.jena.fuseki.access; + +import java.util.function.Function; + +import javax.servlet.ServletContext; +import javax.servlet.http.HttpServletRequest; + +import org.apache.jena.fuseki.embedded.FusekiServer; +import org.apache.jena.fuseki.server.Operation; +import org.apache.jena.fuseki.servlets.ActionService; +import org.apache.jena.fuseki.servlets.HttpAction; +import org.apache.jena.fuseki.servlets.ServiceDispatchRegistry; +import org.apache.jena.query.Dataset; +import org.apache.jena.query.DatasetFactory; +import org.apache.jena.riot.WebContent; +import org.apache.jena.sparql.core.DatasetGraph; +import org.apache.jena.sparql.core.DatasetGraphFilteredView; +import org.apache.jena.sparql.util.Context; +import org.apache.jena.sparql.util.Symbol; +import org.apache.jena.sys.JenaSystem; +import org.eclipse.jetty.security.SecurityHandler; + +/** A library of operations related to data acess sexurity for Fuseki */ --- End diff -- Typo -> `sexurity` ---
[GitHub] jena pull request #465: JENA-1594: Per-graph access control for Fuseki
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/465#discussion_r213249005
--- Diff:
jena-arq/src/main/java/org/apache/jena/sparql/core/DatasetGraphFilteredView.java
---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.core;
+
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.function.Predicate;
+
+import org.apache.jena.atlas.iterator.Iter;
+import org.apache.jena.atlas.logging.Log;
+import org.apache.jena.graph.Graph;
+import org.apache.jena.graph.Node;
+import org.apache.jena.sparql.graph.GraphReadOnly;
+import org.apache.jena.sparql.graph.GraphUnionRead;
+
+/**
+ * A read-only {@link DatasetGraph} that applies a filter testing all
triples and quads
+ * returned by accessing the data. Only quads where the filter tests for
"true" are exposed.
+ */
+public class DatasetGraphFilteredView extends DatasetGraphReadOnly
implements DatasetGraphWrapperView {
+ /*
+ Write operations
+add(Quad)
+delete(Quad)
+add(Node, Node, Node, Node)
+delete(Node, Node, Node, Node)
+deleteAny(Node, Node, Node, Node)
+clear()
+
+ Read operations
+listGraphNodes()
+isEmpty()
+find()
+find(Quad)
+find(Node, Node, Node, Node)
+findNG(Node, Node, Node, Node)
+contains(Quad)
+contains(Node, Node, Node, Node)
+size()
+toString()
+
+ Graph operations
+listGraphNodes()
+getGraph(Node)
+getDefaultGraph()
+containsGraph(Node)
+ */
+
+private final Predicate quadFilter;
+private final Collection visibleGraphs;
+
+public DatasetGraphFilteredView(DatasetGraph dsg, Predicate
filter, Collection visibleGraphs) {
+super(dsg);
+this.quadFilter = filter;
+if ( visibleGraphs.contains(Quad.defaultGraphIRI) ||
visibleGraphs.contains(Quad.defaultGraphNodeGenerated) ) {
+Log.warn(DatasetGraphFilteredView.class, "default graph Node
in visibleGraphs colelction - fix up applied");
--- End diff --
Typo -> `colelction`
---
[GitHub] jena pull request #463: Cleanup
Github user rvesse commented on a diff in the pull request: https://github.com/apache/jena/pull/463#discussion_r211191248 --- Diff: jena-arq/src/main/java/org/apache/jena/sparql/core/DynamicDatasets.java --- @@ -56,8 +55,8 @@ public static DatasetGraph dynamicDataset(DatasetDescription description, Datase if (description.isEmpty() ) return dsg; -Set defaultGraphs = NodeUtils.convertToNodes(description.getDefaultGraphURIs()) ; -Set namedGraphs = NodeUtils.convertToNodes(description.getNamedGraphURIs()) ; +Collection defaultGraphs = NodeUtils.convertToNodes(description.getDefaultGraphURIs()) ; --- End diff -- Isn't the point of using sets here to remove duplicates or is that now done in the `NodeUtils.convertToNodes()` method? ---
[GitHub] jena pull request #459: Fuseki code maintenance.
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/459#discussion_r209539897
--- Diff:
jena-fuseki2/jena-fuseki-core/src/main/java/org/apache/jena/fuseki/mgt/ActionDatasets.java
---
@@ -288,28 +294,77 @@ protected void execDeleteItem(HttpAction action) {
if ( ! action.getDataAccessPointRegistry().isRegistered(name) )
ServletOps.errorNotFound("No such dataset registered: "+name);
+// This acts as a lock.
systemDSG.begin(ReadWrite.WRITE) ;
boolean committed = false ;
+
try {
// Here, go offline.
// Need to reference count operations when they drop to zero
// or a timer goes off, we delete the dataset.
DataAccessPoint ref =
action.getDataAccessPointRegistry().get(name) ;
+
// Redo check inside transaction.
if ( ref == null )
ServletOps.errorNotFound("No such dataset registered:
"+name);
+
+String filename = name.startsWith("/") ? name.substring(1) :
name;
+List configurationFiles =
FusekiSystem.existingConfigurationFile(filename);
+if ( configurationFiles.size() != 1 ) {
+// This should not happen.
+action.log.warn(format("[%d] There are %d configuration
files, not one.", action.id, configurationFiles.size()));
+ServletOps.errorOccurred(
+format(
+"There are %d configuration files, not one. Delete
not performed; clearup of the filesystem needed.",
+action.id, configurationFiles.size()));
+}
+
+String cfgPathname = configurationFiles.get(0);
+
+// Delete configuration file.
+// Once deleted, server restart will not have the database.
+FileOps.deleteSilent(cfgPathname);
-// Make it invisible to the outside.
+// Get before removing.
+DataService dataService = ref.getDataService();
+
+// Make it invisible in this running server.
action.getDataAccessPointRegistry().remove(name);
-// Delete configuration file.
-// Should be only one, undo damage if multiple.
-String filename = name.startsWith("/") ? name.substring(1) :
name;
-
FusekiSystem.existingConfigurationFile(filename).stream().forEach(FileOps::deleteSilent);
-// Leave the database in place. if it is in /databases/,
recreating the
-// configuration will make the database reappear. This is
intentional.
-// Deleting a large database by accident is a major mistake.
+// Delete the database for real only when it is in the server
"run/databases"
+// area. Don't delete databases that reside elsewhere. We do
delete the
+// configuration file, so the databases will not be associated
with the server
+// anymore.
--- End diff --
Where is the code that actually enforces the above comment? I don't see
any check against the location of the database in the subsequent code
---
[GitHub] jena pull request #449: JENA-1578
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/449#discussion_r207151396
--- Diff:
jena-arq/src/main/java/org/apache/jena/query/ParameterizedSparqlString.java ---
@@ -1734,4 +1739,250 @@ public String toString() {
}
}
+
+/**
+ * Assign a VALUES varName with a multiple items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL
query.
+ * See setGroupedValues to assign multiple values to multiple
variables.
+ * Using "var" with list(prop_A, obj_A) on query "VALUES (?p ?o)
{?var}"
+ * would produce "VALUES (?p ?o) {(prop_A obj_A)}".
+ *
+ *
+ * @param varName
+ * @param items
+ */
+public void setValues(String varName, Collection
items) {
+items.forEach(item -> validateParameterValue(item.asNode()));
+this.valuesReplacements.put(varName, new ValueReplacement(varName,
items));
+}
+
+/**
+ * Assign a VALUES varName with a single item.
+ * Using "var" with Literal obj_A on query "VALUES ?o {?var}" would
produce
+ * "VALUES ?o {obj_A}".
+ *
+ * @param varName
+ * @param item
+ */
+public void setValues(String varName, RDFNode item) {
+setValues(varName, Arrays.asList(item));
+}
+
+/**
+ * **
+ * Sets a map of VALUES varNames and their items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL
query.
+ * See setGroupedValues to assign multiple values to multiple
variables.
+ *
+ * @param itemsMap
+ */
+public void setValues(Map>
itemsMap) {
+itemsMap.forEach(this::setValues);
+}
+
+/**
+ * Allocate multiple lists of variables to a single VALUES varName.
+ * Using "vars" with list(list(prop_A, obj_A), list(prop_B, obj_B)) on
query
+ * "VALUES (?p ?o) {?vars}" would produce "VALUES (?p ?o) {(prop_A
obj_A)
+ * (prop_B obj_B)}".
+ *
+ * @param varName
+ * @param groupedItems
+ */
+public void setGroupedValues(String varName, Collection> groupedItems) {
+groupedItems.forEach(collection -> collection.forEach(item ->
validateParameterValue(item.asNode(;
+this.valuesReplacements.put(varName, new ValueReplacement(varName,
groupedItems, true));
+}
+
+private String applyValues(String command) {
+
+for (ValueReplacement valueReplacement :
valuesReplacements.values()) {
+command = valueReplacement.apply(command);
+}
+return command;
+}
+
+private static final String VALUES_KEYWORD = "values";
+
+protected static String[] extractTargetVars(String command, String
varName) {
+String[] targetVars;
+
+int varIndex = command.indexOf(varName);
+if (varIndex > -1) {
+String subCmd = command.substring(0, varIndex).toLowerCase();
//Truncate the command at the varName. Lowercase to search both types of values.
+int valuesIndex = subCmd.lastIndexOf(VALUES_KEYWORD);
+int bracesIndex = subCmd.lastIndexOf("{");
+String vars = command.substring(valuesIndex +
VALUES_KEYWORD.length(), bracesIndex);
+targetVars = vars.replaceAll("[(?)]", "").trim().split(" ");
+} else {
+targetVars = new String[]{};
+}
+return targetVars;
+}
+
+protected static boolean checkParenthesis(String command, String
varName) {
+boolean isNeeded;
+
+int varIndex = command.indexOf(varName);
+if (varIndex > -1) {
+String subCmd = command.substring(0, varIndex).toLowerCase();
//Truncate the command at the varName. Lowercase to search both types of values.
+int valuesIndex = subCmd.lastIndexOf(VALUES_KEYWORD);
+int parenthesisIndex = subCmd.indexOf("(", valuesIndex +
VALUES_KEYWORD.length());
+isNeeded = parenthesisIndex > -1;
+} else {
+isNeeded = false;
+}
+return isNeeded;
+}
+
+/**
+ * Performs replacement of VALUES in query string.
+ *
+ */
+private class ValueReplacement {
+
+private final String varName;
+private final Collection items;
[GitHub] jena pull request #449: JENA-1578
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/449#discussion_r207152306
--- Diff:
jena-arq/src/main/java/org/apache/jena/query/ParameterizedSparqlString.java ---
@@ -1734,4 +1739,250 @@ public String toString() {
}
}
+
+/**
+ * Assign a VALUES varName with a multiple items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL
query.
+ * See setGroupedValues to assign multiple values to multiple
variables.
+ * Using "var" with list(prop_A, obj_A) on query "VALUES (?p ?o)
{?var}"
+ * would produce "VALUES (?p ?o) {(prop_A obj_A)}".
+ *
+ *
+ * @param varName
+ * @param items
+ */
+public void setValues(String varName, Collection
items) {
+items.forEach(item -> validateParameterValue(item.asNode()));
--- End diff --
Later code requires that `items` be a `List` so this code should either
enforce it here or otherwise sanitise `items` into a `List` at this point
---
[GitHub] jena pull request #449: JENA-1578
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/449#discussion_r207148170
--- Diff:
jena-arq/src/test/java/org/apache/jena/query/TestParameterizedSparqlString.java
---
@@ -1925,4 +1926,190 @@ public void test_param_string_bug_04() {
pss.toString();
}
+
+@Test
+public void test_set_values_item() {
+// Tests a single value being added.
+String str = "SELECT * WHERE { VALUES ?o {?objs} ?s ?p ?o }";
+ParameterizedSparqlString pss = new ParameterizedSparqlString(str);
+pss.setValues("objs", ResourceFactory.createPlainLiteral("test"));
+
+String exp = "SELECT * WHERE { VALUES ?o {\"test\"} ?s ?p ?o }";
+String res = pss.toString();
+//System.out.println("Exp: " + exp);
+//System.out.println("Res: " + res);
+Assert.assertEquals(exp, res);
+}
+
+@Test
+public void test_set_values_items() {
+// Tests two values for same variable.
+String str = "SELECT * WHERE { VALUES ?o {?objs} ?s ?p ?o }";
+ParameterizedSparqlString pss = new ParameterizedSparqlString(str);
+List objs = new ArrayList<>();
+objs.add(ResourceFactory.createPlainLiteral("obj_A"));
+objs.add(ResourceFactory.createPlainLiteral("obj_B"));
+pss.setValues("objs", objs);
+
+String exp = "SELECT * WHERE { VALUES ?o {\"obj_A\" \"obj_B\"} ?s
?p ?o }";
+String res = pss.toString();
+//System.out.println("Exp: " + exp);
+//System.out.println("Res: " + res);
+Assert.assertEquals(exp, res);
+}
+
+@Test
+public void test_set_values_items_parenthesis() {
+// Tests two values for same variable.
+String str = "SELECT * WHERE { VALUES (?o) {?objs} ?s ?p ?o }";
+ParameterizedSparqlString pss = new ParameterizedSparqlString(str);
+List objs = new ArrayList<>();
+objs.add(ResourceFactory.createPlainLiteral("obj_A"));
+objs.add(ResourceFactory.createPlainLiteral("obj_B"));
+pss.setValues("objs", objs);
+
+String exp = "SELECT * WHERE { VALUES (?o) {(\"obj_A\")
(\"obj_B\")} ?s ?p ?o }";
+String res = pss.toString();
+//System.out.println("Exp: " + exp);
+//System.out.println("Res: " + res);
+Assert.assertEquals(exp, res);
+}
+
+@Test
+public void test_set_values_multiple_variables_parenthesis() {
+// Tests two values for same variable.
+String str = "SELECT * WHERE { VALUES (?p ?o) {?vars} ?s ?p ?o }";
+ParameterizedSparqlString pss = new ParameterizedSparqlString(str);
+List vars = new ArrayList<>();
+
vars.add(ResourceFactory.createProperty("http://example.org/prop_A;));
+vars.add(ResourceFactory.createPlainLiteral("obj_A"));
+pss.setValues("vars", vars);
+
+String exp = "SELECT * WHERE { VALUES (?p ?o)
{(<http://example.org/prop_A> \"obj_A\")} ?s ?p ?o }";
+String res = pss.toString();
+System.out.println("Exp: " + exp);
+System.out.println("Res: " + res);
+Assert.assertEquals(exp, res);
+}
+
+@Test
+public void test_set_values_multi_var() {
+// Tests two variables.
+String str = "SELECT * WHERE { VALUES ?p {?props} VALUES ?o
{?objs} ?s ?p ?o }";
+ParameterizedSparqlString pss = new ParameterizedSparqlString(str);
+List objs = new ArrayList<>();
+objs.add(ResourceFactory.createPlainLiteral("obj_A"));
+objs.add(ResourceFactory.createPlainLiteral("obj_B"));
+pss.setValues("objs", objs);
+
+List props = new ArrayList<>();
+
props.add(ResourceFactory.createProperty("http://example.org/prop_A;));
+
props.add(ResourceFactory.createProperty("http://example.org/prop_B;));
+pss.setValues("props", props);
+
+String exp = "SELECT * WHERE { VALUES ?p
{<http://example.org/prop_A> <http://example.org/prop_B>} VALUES ?o {\"obj_A\"
\"obj_B\"} ?s ?p ?o }";
+String res = pss.toString();
+//System.out.println("Exp: " + exp);
+//System
[GitHub] jena pull request #449: JENA-1578
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/449#discussion_r207150857
--- Diff:
jena-arq/src/main/java/org/apache/jena/query/ParameterizedSparqlString.java ---
@@ -1734,4 +1739,250 @@ public String toString() {
}
}
+
+/**
+ * Assign a VALUES varName with a multiple items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL
query.
+ * See setGroupedValues to assign multiple values to multiple
variables.
+ * Using "var" with list(prop_A, obj_A) on query "VALUES (?p ?o)
{?var}"
+ * would produce "VALUES (?p ?o) {(prop_A obj_A)}".
+ *
+ *
+ * @param varName
+ * @param items
+ */
+public void setValues(String varName, Collection
items) {
+items.forEach(item -> validateParameterValue(item.asNode()));
+this.valuesReplacements.put(varName, new ValueReplacement(varName,
items));
+}
+
+/**
+ * Assign a VALUES varName with a single item.
+ * Using "var" with Literal obj_A on query "VALUES ?o {?var}" would
produce
+ * "VALUES ?o {obj_A}".
+ *
+ * @param varName
+ * @param item
+ */
+public void setValues(String varName, RDFNode item) {
+setValues(varName, Arrays.asList(item));
+}
+
+/**
+ * **
+ * Sets a map of VALUES varNames and their items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL
query.
+ * See setGroupedValues to assign multiple values to multiple
variables.
+ *
+ * @param itemsMap
+ */
+public void setValues(Map>
itemsMap) {
+itemsMap.forEach(this::setValues);
+}
+
+/**
+ * Allocate multiple lists of variables to a single VALUES varName.
+ * Using "vars" with list(list(prop_A, obj_A), list(prop_B, obj_B)) on
query
+ * "VALUES (?p ?o) {?vars}" would produce "VALUES (?p ?o) {(prop_A
obj_A)
+ * (prop_B obj_B)}".
+ *
+ * @param varName
+ * @param groupedItems
+ */
+public void setGroupedValues(String varName, Collection> groupedItems) {
+groupedItems.forEach(collection -> collection.forEach(item ->
validateParameterValue(item.asNode(;
+this.valuesReplacements.put(varName, new ValueReplacement(varName,
groupedItems, true));
+}
+
+private String applyValues(String command) {
+
+for (ValueReplacement valueReplacement :
valuesReplacements.values()) {
+command = valueReplacement.apply(command);
+}
+return command;
+}
+
+private static final String VALUES_KEYWORD = "values";
+
+protected static String[] extractTargetVars(String command, String
varName) {
+String[] targetVars;
+
+int varIndex = command.indexOf(varName);
+if (varIndex > -1) {
+String subCmd = command.substring(0, varIndex).toLowerCase();
//Truncate the command at the varName. Lowercase to search both types of values.
+int valuesIndex = subCmd.lastIndexOf(VALUES_KEYWORD);
+int bracesIndex = subCmd.lastIndexOf("{");
+String vars = command.substring(valuesIndex +
VALUES_KEYWORD.length(), bracesIndex);
+targetVars = vars.replaceAll("[(?)]", "").trim().split(" ");
+} else {
+targetVars = new String[]{};
+}
+return targetVars;
+}
+
+protected static boolean checkParenthesis(String command, String
varName) {
--- End diff --
Per @afs's comments it might be simpler to just always include the
parenthesis since the un-paranthesised form is simply a shortcut for the single
variable case
---
[GitHub] jena pull request #449: JENA-1578
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/449#discussion_r207147377
--- Diff:
jena-arq/src/main/java/org/apache/jena/query/ParameterizedSparqlString.java ---
@@ -1734,4 +1739,250 @@ public String toString() {
}
}
+
+/**
+ * Assign a VALUES varName with a multiple items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL
query.
+ * See setGroupedValues to assign multiple values to multiple
variables.
+ * Using "var" with list(prop_A, obj_A) on query "VALUES (?p ?o)
{?var}"
+ * would produce "VALUES (?p ?o) {(prop_A obj_A)}".
+ *
+ *
+ * @param varName
+ * @param items
+ */
+public void setValues(String varName, Collection
items) {
+items.forEach(item -> validateParameterValue(item.asNode()));
+this.valuesReplacements.put(varName, new ValueReplacement(varName,
items));
+}
+
+/**
+ * Assign a VALUES varName with a single item.
+ * Using "var" with Literal obj_A on query "VALUES ?o {?var}" would
produce
+ * "VALUES ?o {obj_A}".
+ *
+ * @param varName
+ * @param item
+ */
+public void setValues(String varName, RDFNode item) {
+setValues(varName, Arrays.asList(item));
+}
+
+/**
+ * **
+ * Sets a map of VALUES varNames and their items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL
query.
+ * See setGroupedValues to assign multiple values to multiple
variables.
+ *
+ * @param itemsMap
+ */
+public void setValues(Map>
itemsMap) {
+itemsMap.forEach(this::setValues);
+}
+
+/**
+ * Allocate multiple lists of variables to a single VALUES varName.
+ * Using "vars" with list(list(prop_A, obj_A), list(prop_B, obj_B)) on
query
+ * "VALUES (?p ?o) {?vars}" would produce "VALUES (?p ?o) {(prop_A
obj_A)
+ * (prop_B obj_B)}".
+ *
+ * @param varName
+ * @param groupedItems
+ */
+public void setGroupedValues(String varName, Collection> groupedItems) {
--- End diff --
I wonder if these should be named `setRowValues()` since you are
effectively substituting several rows of data and that terminology more closely
matches how the SPARQL spec and Jena algebra treats VALUES
---
[GitHub] jena pull request #449: JENA-1578
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/449#discussion_r207150287
--- Diff:
jena-arq/src/main/java/org/apache/jena/query/ParameterizedSparqlString.java ---
@@ -1734,4 +1739,250 @@ public String toString() {
}
}
+
+/**
+ * Assign a VALUES varName with a multiple items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL
query.
+ * See setGroupedValues to assign multiple values to multiple
variables.
+ * Using "var" with list(prop_A, obj_A) on query "VALUES (?p ?o)
{?var}"
+ * would produce "VALUES (?p ?o) {(prop_A obj_A)}".
+ *
+ *
+ * @param varName
+ * @param items
+ */
+public void setValues(String varName, Collection
items) {
+items.forEach(item -> validateParameterValue(item.asNode()));
+this.valuesReplacements.put(varName, new ValueReplacement(varName,
items));
+}
+
+/**
+ * Assign a VALUES varName with a single item.
+ * Using "var" with Literal obj_A on query "VALUES ?o {?var}" would
produce
+ * "VALUES ?o {obj_A}".
+ *
+ * @param varName
+ * @param item
+ */
+public void setValues(String varName, RDFNode item) {
+setValues(varName, Arrays.asList(item));
+}
+
+/**
+ * **
+ * Sets a map of VALUES varNames and their items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL
query.
+ * See setGroupedValues to assign multiple values to multiple
variables.
+ *
+ * @param itemsMap
+ */
+public void setValues(Map>
itemsMap) {
+itemsMap.forEach(this::setValues);
+}
+
+/**
+ * Allocate multiple lists of variables to a single VALUES varName.
+ * Using "vars" with list(list(prop_A, obj_A), list(prop_B, obj_B)) on
query
+ * "VALUES (?p ?o) {?vars}" would produce "VALUES (?p ?o) {(prop_A
obj_A)
+ * (prop_B obj_B)}".
+ *
+ * @param varName
+ * @param groupedItems
+ */
+public void setGroupedValues(String varName, Collection> groupedItems) {
+groupedItems.forEach(collection -> collection.forEach(item ->
validateParameterValue(item.asNode(;
+this.valuesReplacements.put(varName, new ValueReplacement(varName,
groupedItems, true));
+}
+
+private String applyValues(String command) {
+
+for (ValueReplacement valueReplacement :
valuesReplacements.values()) {
+command = valueReplacement.apply(command);
+}
+return command;
+}
+
+private static final String VALUES_KEYWORD = "values";
+
+protected static String[] extractTargetVars(String command, String
varName) {
+String[] targetVars;
+
+int varIndex = command.indexOf(varName);
+if (varIndex > -1) {
+String subCmd = command.substring(0, varIndex).toLowerCase();
//Truncate the command at the varName. Lowercase to search both types of values.
+int valuesIndex = subCmd.lastIndexOf(VALUES_KEYWORD);
+int bracesIndex = subCmd.lastIndexOf("{");
+String vars = command.substring(valuesIndex +
VALUES_KEYWORD.length(), bracesIndex);
--- End diff --
Per my comments on test cases, does this potentially find too much if the
query string never contains `VALUES` or the query is otherwise syntactically
malformed e.g. missing braces?
---
[GitHub] jena pull request #449: JENA-1578
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/449#discussion_r206083636
--- Diff:
jena-arq/src/main/java/org/apache/jena/query/ParameterizedSparqlString.java ---
@@ -1734,4 +1739,237 @@ public String toString() {
}
}
+
+/**
+ * Assign a varName with a multiple items and whether to include
+ * parenthesis.
+ *
+ * @param varName
+ * @param items
+ * @param isParenthesisNeeded
+ */
+public void setValues(String varName, Collection
items, boolean isParenthesisNeeded) {
+this.valuesReplacements.put(varName, new ValueReplacement(varName,
items, isParenthesisNeeded));
+}
+
+/**
+ * Assign a varName with a multiple items.
+ * Can be used to assign multiple values to a single variable or single
+ * value to multiple variables (if using a List) in the SPARQL
query.
+ * See setGroupedValues to assign multiple values to multiple
variables.
+ *
+ * @param varName
+ * @param items
+ */
+public void setValues(String varName, Collection
items) {
+setValues(varName, items, false);
+}
+
+/**
+ * Assign a varName with a single item and whether to include
parenthesis.
+ *
+ * @param varName
+ * @param item
+ * @param isParenthesisNeeded
+ */
+public void setValues(String varName, RDFNode item, boolean
isParenthesisNeeded) {
--- End diff --
I will take a proper look at this tomorrow.
My first reaction though is that I am a little worried that we would expose
to the user (even if they are a developer in this scenario) the decision as to
whether parenthesis are needed both from a security (SPARQL injection) and a
validity perspective. The code should be able to determine this based upon how
many variables are being inserted and do the right thing.
---
[GitHub] jena pull request #441: Improved Travis Configuration
GitHub user rvesse opened a pull request: https://github.com/apache/jena/pull/441 Improved Travis Configuration - Skips Travis install phase - Test on both Oracle and Open JDK - Build full profile - Fix environment specification You can merge this pull request into a Git repository by running: $ git pull https://github.com/rvesse/jena travis Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/441.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #441 commit ac70cb5276f358839ef42bd61d3858d8d974fade Author: Rob Vesse Date: 2018-06-27T17:23:24Z Modified Travis config - Skip dependency:go-offline - Add -B to avoid download progress junk in log - Build full project - Also build with Open JDK 8 commit 9ac9c0c54432e9ae3ca589db259bbadadc57b48f Author: Rob Vesse Date: 2018-06-27T17:25:58Z Fix Travis environment setup ---
[GitHub] jena pull request #440: Fix SSE parsing on table with boolean values (JENA-1...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/440#discussion_r198545901
--- Diff:
jena-arq/src/test/java/org/apache/jena/sparql/syntax/TestSSE_Builder.java ---
@@ -233,4 +249,42 @@ public void testBuildExpr_14() {
"(!= ?x ?y)") ;
}
+@Test
+public void testBuildTable_01() {
+Op expected = OpTable.unit();
+Op actual = SSE.parseOp("(table unit)");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_02() {
+Op expected = OpTable.empty();
+Op actual = SSE.parseOp("(table empty)");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_03() {
+Op expected = OpTable.create(TableFactory.create(Var.alloc("x"),
NodeConst.nodeTrue));
+Op actual = SSE.parseOp("(table (vars ?x) (row (?x true)))");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_04() {
+// Can't test for equality because can't create a BNode in a way
that equality will
+// succeed because OpTable does strict equality and ignores
NodeIsomorphismMap
+SSE.parseOp("(table (vars ?x) (row (?x _:test)))");
+}
+
+@Test(expected = ExprBuildException.class)
--- End diff --
Apparently it's because we extend `TestCase` so JUnit assumes a JUnit 3.8
test which doesn't support expected, adding `@RunWith(JUnit4.class)` to the
test class resolves this
See https://stackoverflow.com/a/50867198/107591
---
[GitHub] jena pull request #440: Fix SSE parsing on table with boolean values (JENA-1...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/440#discussion_r198490652
--- Diff:
jena-arq/src/test/java/org/apache/jena/sparql/syntax/TestSSE_Builder.java ---
@@ -233,4 +249,42 @@ public void testBuildExpr_14() {
"(!= ?x ?y)") ;
}
+@Test
+public void testBuildTable_01() {
+Op expected = OpTable.unit();
+Op actual = SSE.parseOp("(table unit)");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_02() {
+Op expected = OpTable.empty();
+Op actual = SSE.parseOp("(table empty)");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_03() {
+Op expected = OpTable.create(TableFactory.create(Var.alloc("x"),
NodeConst.nodeTrue));
+Op actual = SSE.parseOp("(table (vars ?x) (row (?x true)))");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_04() {
+// Can't test for equality because can't create a BNode in a way
that equality will
+// succeed because OpTable does strict equality and ignores
NodeIsomorphismMap
+SSE.parseOp("(table (vars ?x) (row (?x _:test)))");
+}
+
+@Test(expected = ExprBuildException.class)
--- End diff --
For some reason `expected` is not working as expected ;)
---
[GitHub] jena pull request #440: Fix SSE parsing on table with boolean values (JENA-1...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/440#discussion_r198489213
--- Diff:
jena-arq/src/test/java/org/apache/jena/sparql/syntax/TestSSE_Builder.java ---
@@ -233,4 +247,54 @@ public void testBuildExpr_14() {
"(!= ?x ?y)") ;
}
+@Test
+public void testBuildTable_01() {
+Op expected = OpTable.unit();
+Op actual = SSE.parseOp("(table unit)");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_02() {
+Op expected = OpTable.empty();
+Op actual = SSE.parseOp("(table empty)");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_03() {
+Op expected = OpTable.create(TableFactory.create(Var.alloc("x"),
NodeConst.nodeTrue));
+Op actual = SSE.parseOp("(table (vars ?x) (row (?x true)))");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTableBad_01() {
+try {
+SSE.parseOp("(table (vars ?x) (row (?x (table unit");
+Assert.fail("Parsing should fail");
+} catch (ExprBuildException e) {
+// OK
--- End diff --
I get the following when using the preferred form:
```
[-1,-1]: Not a node: (table ...)
[1,27]: Only concrete nodes as table values: ANY: _
[ERROR] Tests run: 139, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
0.036 s <<< FAILURE! - in org.apache.jena.sparql.syntax.TS_SSE
[ERROR] testBuildTableBad_01(org.apache.jena.sparql.syntax.TestSSE_Builder)
Time elapsed: 0.006 s <<< ERROR!
org.apache.jena.sparql.sse.builders.ExprBuildException: [-1,-1]: Not a node
at
org.apache.jena.sparql.syntax.TestSSE_Builder.testBuildTableBad_01(TestSSE_Builder.java:282)
[ERROR] testBuildTableBad_02(org.apache.jena.sparql.syntax.TestSSE_Builder)
Time elapsed: 0 s <<< ERROR!
org.apache.jena.sparql.sse.builders.ExprBuildException: [1,27]: Only
concrete nodes as table values: ANY
at
org.apache.jena.sparql.syntax.TestSSE_Builder.testBuildTableBad_02(TestSSE_Builder.java:288)
```
Any idea why?
---
[GitHub] jena pull request #440: Fix SSE parsing on table with boolean values (JENA-1...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/440#discussion_r198484085
--- Diff:
jena-arq/src/test/java/org/apache/jena/sparql/syntax/TestSSE_Builder.java ---
@@ -233,4 +247,54 @@ public void testBuildExpr_14() {
"(!= ?x ?y)") ;
}
+@Test
+public void testBuildTable_01() {
+Op expected = OpTable.unit();
+Op actual = SSE.parseOp("(table unit)");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_02() {
+Op expected = OpTable.empty();
+Op actual = SSE.parseOp("(table empty)");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_03() {
+Op expected = OpTable.create(TableFactory.create(Var.alloc("x"),
NodeConst.nodeTrue));
+Op actual = SSE.parseOp("(table (vars ?x) (row (?x true)))");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTableBad_01() {
+try {
+SSE.parseOp("(table (vars ?x) (row (?x (table unit");
+Assert.fail("Parsing should fail");
+} catch (ExprBuildException e) {
+// OK
+}
+}
+
+@Test
+public void testBuildTableBad_02() {
+try {
+SSE.parseOp("(table (vars ?x) (row (?x _:anon)))");
+Assert.fail("Parsing should fail");
--- End diff --
Yeah I was unsure on this, I added a new check for this but I am happy to
allow this to parse if desired
---
[GitHub] jena pull request #440: Fix SSE parsing on table with boolean values (JENA-1...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/440#discussion_r198484184
--- Diff:
jena-arq/src/test/java/org/apache/jena/sparql/syntax/TestSSE_Builder.java ---
@@ -233,4 +247,54 @@ public void testBuildExpr_14() {
"(!= ?x ?y)") ;
}
+@Test
+public void testBuildTable_01() {
+Op expected = OpTable.unit();
+Op actual = SSE.parseOp("(table unit)");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_02() {
+Op expected = OpTable.empty();
+Op actual = SSE.parseOp("(table empty)");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTable_03() {
+Op expected = OpTable.create(TableFactory.create(Var.alloc("x"),
NodeConst.nodeTrue));
+Op actual = SSE.parseOp("(table (vars ?x) (row (?x true)))");
+assertEquals(expected, actual);
+}
+
+@Test
+public void testBuildTableBad_01() {
+try {
+SSE.parseOp("(table (vars ?x) (row (?x (table unit");
+Assert.fail("Parsing should fail");
+} catch (ExprBuildException e) {
+// OK
--- End diff --
Yeah I know, wasn't working for me for some reason
---
[GitHub] jena pull request #440: Fix SSE parsing on table with boolean values (JENA-1...
GitHub user rvesse opened a pull request: https://github.com/apache/jena/pull/440 Fix SSE parsing on table with boolean values (JENA-1566) JENA-1566 reports an issue with round tripping through SSE of tables that contain boolean literals i.e. `true` and `false` This was due to overly strict checking in `BuilderBinding` which has been removed although a missing test for non-blank-ness was added. Various unit tests are added around this issue. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rvesse/jena JENA-1566 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/440.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #440 commit 1a3f662f31533700685eda774458eba9a6766229 Author: Rob Vesse Date: 2018-06-27T10:38:19Z Remove unecessarily strict check in BuilderTable (JENA-1566) Check prevents valid node values being parsed for row entries and there is already checking of invalid values that happens after the attempt to parse the node value. Adds various unit tests around BuilderTable as well. commit e6d8c52ad0416cbb81ab393b8b9bdaccab30a3e5 Author: Rob Vesse Date: 2018-06-27T10:45:50Z Add missing node test in BuilderBinding (JENA-1566) ---
[GitHub] jena issue #439: JENA-1565: Print the throwable and stacktrace
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/439 @afs If you want to merge this now I was about to cut the RC for 3.8.0 ---
[GitHub] jena issue #439: JENA-1565: Print the throwable and stacktrace
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/439 Do you want this in 3.8.0? ---
[GitHub] jena issue #436: JENA-1556 implementation
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/436 @xristy I think you can just merge as you would normally though you probably need to be explicit about the remote and branch you are using for remote operations: ``` > git checkout master > git pull apache-origin master > git merge Jena-1556-MutilingualEnhancements-3.8.0 > git push apache-origin master ``` ---
[GitHub] jena pull request #433: JENA-1561: Enable TDB2 in Fuseki UI and on the comma...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/433#discussion_r194342614
--- Diff:
jena-fuseki2/jena-fuseki-core/src/main/java/org/apache/jena/fuseki/mgt/ActionDatasets.java
---
@@ -351,15 +356,13 @@ private static void assemblerFromForm(HttpAction
action, StreamRDF dest) {
FusekiSystem.addGlobals(params);
//action.log.info(format("[%d] Create database : name = %s, type =
%s", action.id, dbName, dbType )) ;
-if ( ! dbType.equals(tDatabasetTDB) && !
dbType.equals(tDatabasetMem) )
-ServletOps.errorBadRequest(format("dbType can be only '%s' or
'%s'", tDatabasetTDB, tDatabasetMem)) ;
-String template = null ;
-if ( dbType.equalsIgnoreCase(tDatabasetTDB))
-template =
TemplateFunctions.templateFile(Template.templateTDBFN, params, Lang.TTL) ;
-if ( dbType.equalsIgnoreCase(tDatabasetMem))
-template =
TemplateFunctions.templateFile(Template.templateMemFN, params, Lang.TTL) ;
-RDFParser.create().source(new
StringReader(template)).base("http://base/;).lang(Lang.TTL).parse(dest);
+String template =
dbTypeToTemplate.get(dbType.toLowerCase(Locale.ROOT));
+if ( template == null )
+ServletOps.errorBadRequest(format("dbType can be only '%s'
or '%s'", tDatabaseTDB, tDatabaseMem)) ;
--- End diff --
This PR now allows three possibilities so the error message is now
inaccurate
---
[GitHub] jena pull request #431: JENA-1559: Alternative PrefixMapping implementation;...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/431#discussion_r194339397
--- Diff:
jena-arq/src/main/java/org/apache/jena/sparql/graph/PrefixMappingOver.java ---
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.graph;
+
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.function.BiConsumer;
+
+import org.apache.jena.iri.IRI;
+import org.apache.jena.riot.system.PrefixMap;
+
+public class PrefixMappingOver extends PrefixMappingBase {
--- End diff --
Would `PrefixMappingDecorator` be a more descriptive name?
---
[GitHub] jena pull request #431: JENA-1559: Alternative PrefixMapping implementation;...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/431#discussion_r194338419
--- Diff:
jena-arq/src/main/java/org/apache/jena/sparql/graph/PrefixMappingBase.java ---
@@ -0,0 +1,277 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.graph;
+
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.StringJoiner;
+import java.util.function.BiConsumer;
+
+import org.apache.jena.ext.xerces.util.XMLChar;
+import org.apache.jena.shared.PrefixMapping;
+import org.apache.jena.util.SplitIRI;
+
+/**
+ * Framework for implementing {@link PrefixMapping}. It is stateless
(unlike
+ * {@code PrefixMappingImpl}) and implements the contract of {@link
PrefixMapping},
+ * providing the key algorithm and delegating storage to the subclasses.
+ *
+ * Reverse mapping, looking up a URI to find a prefix is complex. There
may be several
+ * possibilities. Applications should not reliy on every implementat being
consistent,
+ * espeically when writing data when there has to be a choice on on prefix
to use to
+ * shorten a URI.
+ *
+ */
+public abstract class PrefixMappingBase implements PrefixMapping {
+/* Reverse mappings.
+ * The strict contract of PrefixMapping requires a separate
reversemapping to be storoed and manipuated
--- End diff --
Typos:
- Missing space in `reverse mapping`
- `storoed`
- `manipuated`
---
[GitHub] jena pull request #431: JENA-1559: Alternative PrefixMapping implementation;...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/431#discussion_r194338879
--- Diff:
jena-arq/src/main/java/org/apache/jena/sparql/graph/PrefixMappingBase.java ---
@@ -0,0 +1,277 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.graph;
+
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.StringJoiner;
+import java.util.function.BiConsumer;
+
+import org.apache.jena.ext.xerces.util.XMLChar;
+import org.apache.jena.shared.PrefixMapping;
+import org.apache.jena.util.SplitIRI;
+
+/**
+ * Framework for implementing {@link PrefixMapping}. It is stateless
(unlike
+ * {@code PrefixMappingImpl}) and implements the contract of {@link
PrefixMapping},
+ * providing the key algorithm and delegating storage to the subclasses.
+ *
+ * Reverse mapping, looking up a URI to find a prefix is complex. There
may be several
+ * possibilities. Applications should not reliy on every implementat being
consistent,
+ * espeically when writing data when there has to be a choice on on prefix
to use to
+ * shorten a URI.
+ *
+ */
+public abstract class PrefixMappingBase implements PrefixMapping {
+/* Reverse mappings.
+ * The strict contract of PrefixMapping requires a separate
reversemapping to be storoed and manipuated
+ * which in turn adds complexity to the storage implementations.
+ * However, applications removing prefixes is unusual so we end up
with a lot of complexity with little value.
+ *
+ * Beware of the details of removing a mapping when there is another
to the same URI.
+ * If we had:
+ *
+ * Add (pref1, U)
+ * Add (pref2, U)
+ *
+ * so that {@code U} reverse maps ({@link #getNsURIPrefix}) to {@code
pref2} (it was
+ * done second) then
+ *
+ * Remove (pref2)
+ *
+ * it causes {@code U} to reverse map to {@code pref1}.
+ *
+ * This feature is quite a burden on implementations and should be
regarded as "legacy" -
+ * an implementation may not support this complex effect.
+ *
+ * PrefixMappingMem does.
+ * Database backed ones typically don't.
+ */
+
+protected PrefixMappingBase() {}
+
+// The storage operations of an implementation.
+
+/** Add prefix */
+abstract protected void add(String prefix, String uri);
+
+/** Remove prefix. */
+abstract protected void remove(String prefix);
+
+/** Clear all mappings */
+abstract protected void clear();
+
+abstract protected boolean isEmpty();
+
+abstract protected int size();
+
+/** Return the URI that the prefix maps to. */
+abstract protected String prefixToUri(String prefix);
+
+/** Return a prefix that maps to the URI.
+ * There may be several; the answer is any one of them.
+ */
+abstract protected String uriToPrefix(String uri);
+
+/** Return as a map. This map is only used within this class.
+ * It can be as efficient as possible.
+ * It will not be modifed.
+ * It wil not be retuned to a caller of {@code PrefixMappingBase}.
+ */
+abstract protected Map asMap();
+
+/**
+ * Return as a map. The map return is not connected to the prefix
mapping implementation,
+ * does not reflect subsequent prefix mapping changes.
+ */
+abstract protected Map asMapCopy();
+
+/** Apply the {@link BiConsumer} to each (prefix, uri) pair. */
+abstract protected void apply(BiConsumer action);
+
+/**
+ * This part of the subclass API and may be overriden if an
implementation can do
+ * better This general implementation is based on asMap
[GitHub] jena pull request #431: JENA-1559: Alternative PrefixMapping implementation;...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/431#discussion_r194338271
--- Diff:
jena-arq/src/main/java/org/apache/jena/sparql/graph/PrefixMappingBase.java ---
@@ -0,0 +1,277 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.sparql.graph;
+
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.StringJoiner;
+import java.util.function.BiConsumer;
+
+import org.apache.jena.ext.xerces.util.XMLChar;
+import org.apache.jena.shared.PrefixMapping;
+import org.apache.jena.util.SplitIRI;
+
+/**
+ * Framework for implementing {@link PrefixMapping}. It is stateless
(unlike
+ * {@code PrefixMappingImpl}) and implements the contract of {@link
PrefixMapping},
+ * providing the key algorithm and delegating storage to the subclasses.
+ *
+ * Reverse mapping, looking up a URI to find a prefix is complex. There
may be several
+ * possibilities. Applications should not reliy on every implementat being
consistent,
--- End diff --
Typos
- `implementat` this line
- `espeically` on the following line
---
[GitHub] jena pull request #426: JENA-1552: Phased loader
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/426#discussion_r192681609
--- Diff:
jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/loader/parallel/LoaderPlan.java
---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.tdb2.loader.parallel;
+
+import org.apache.jena.tdb2.store.NodeId;
+
+/**
+ * A {@code LoaderPlan}
+ *
+ * For triples and for quads there is a first phase to parse the input,
+ * convert to tuples of {@link NodeId NodeIds}, including allocating the
ids,
+ * and do at least one tuple index for each of triples quads to capture
the input.
+ *
+ * After that, a number of phases builds the other indexes.
+ *
+ * The {@code mulithreadedInput} f;ag indicates whether the first phase is
+ * done in parallel (threads for parer, node table building and primary
indexes)
+ * or as a single threaded process.
+ */
+public class LoaderPlan {
--- End diff --
I really like this design, since the class has a public constructor
end-users could potentially test out alternative strategies if they so desired?
---
[GitHub] jena pull request #426: JENA-1552: Phased loader
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/426#discussion_r192680143
--- Diff:
jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/loader/parallel/LoaderPlan.java
---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.tdb2.loader.parallel;
+
+import org.apache.jena.tdb2.store.NodeId;
+
+/**
+ * A {@code LoaderPlan}
+ *
+ * For triples and for quads there is a first phase to parse the input,
+ * convert to tuples of {@link NodeId NodeIds}, including allocating the
ids,
+ * and do at least one tuple index for each of triples quads to capture
the input.
+ *
+ * After that, a number of phases builds the other indexes.
+ *
+ * The {@code mulithreadedInput} f;ag indicates whether the first phase is
--- End diff --
Typo `f;ag` -> `flag`
---
[GitHub] jena pull request #426: JENA-1552: Phased loader
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/426#discussion_r192680996
--- Diff:
jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/loader/parallel/LoaderPlan.java
---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.tdb2.loader.parallel;
+
+import org.apache.jena.tdb2.store.NodeId;
+
+/**
+ * A {@code LoaderPlan}
+ *
+ * For triples and for quads there is a first phase to parse the input,
+ * convert to tuples of {@link NodeId NodeIds}, including allocating the
ids,
+ * and do at least one tuple index for each of triples quads to capture
the input.
+ *
+ * After that, a number of phases builds the other indexes.
+ *
+ * The {@code mulithreadedInput} f;ag indicates whether the first phase is
+ * done in parallel (threads for parer, node table building and primary
indexes)
+ * or as a single threaded process.
+ */
+public class LoaderPlan {
+private final InputStage dataInput;
+private final String[] loadGroup3;
+private final String[] loadGroup4;
+private final String[][] secondaryGroups3;
+private final String[][] secondaryGroups4;
+
+public LoaderPlan(InputStage dataInput,
+String[] loadGroup3, String[] loadGroup4,
+String[][] secondaryGroups3, String[][] secondaryGroups4) {
+this.dataInput = dataInput;
+this.loadGroup3 = loadGroup3;
+this.loadGroup4 = loadGroup4;
+this.secondaryGroups3 = secondaryGroups3;
+this.secondaryGroups4 = secondaryGroups4;
+}
+public InputStage dataInputType() { return dataInput; }
+public String[] primaryLoad3() { return loadGroup3; }
+public String[] primaryLoad4() { return loadGroup4; }
+public String[][] secondaryIndex3() { return secondaryGroups3; }
+public String[][] secondaryIndex4() { return secondaryGroups4; }
--- End diff --
The naming of these seems somewhat esoteric, I assume that the 3 and 4
refers to Triples and Quads respectively?
---
[GitHub] jena pull request #414: JENA-1542: Integrate Lucene index in transaction lif...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/414#discussion_r190532549
--- Diff:
jena-db/jena-dboe-transaction/src/main/java/org/apache/jena/dboe/transaction/txn/TransactionCoordinator.java
---
@@ -142,41 +141,56 @@ public TransactionCoordinator
add(TransactionalComponent elt) {
* @see #add
*/
public TransactionCoordinator remove(TransactionalComponent elt) {
-checkSetup() ;
-synchronized(coordinatorLock) {
-components.remove(elt.getComponentId()) ;
-}
+checklAllowModification() ;
+components.remove(elt.getComponentId()) ;
return this ;
}
+
+/**
+ * Perform modification of this {@code TransaxctionCoordiator} after
it has been started.
--- End diff --
Typo in the `{@code }` here?
---
[GitHub] jena pull request #414: JENA-1542: Integrate Lucene index in transaction lif...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/414#discussion_r190533210
--- Diff:
jena-text/src/test/java/org/apache/jena/query/text/TestTextTxn.java ---
@@ -0,0 +1,341 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.query.text;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assume.assumeTrue;
+
+import java.io.Reader;
+import java.io.StringReader;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List ;
+
+import org.apache.jena.atlas.iterator.Iter ;
+import org.apache.jena.atlas.lib.Creator;
+import org.apache.jena.atlas.lib.StrUtils ;
+import org.apache.jena.query.* ;
+import org.apache.jena.rdf.model.Model ;
+import org.apache.jena.sparql.core.Quad ;
+import org.apache.jena.sparql.sse.SSE ;
+import org.apache.jena.system.Txn;
+import org.apache.jena.tdb.TDBFactory ;
+import org.apache.jena.tdb2.TDB2Factory;
+import org.apache.jena.vocabulary.RDFS ;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexReader;
+import org.apache.lucene.index.IndexWriter;
+import org.apache.lucene.store.Directory ;
+import org.apache.lucene.store.RAMDirectory ;
+import org.junit.Test ;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
+
+/** Text dataset tests using datasets transactionally, named
unionDefaultGraph.
+ *
+ * Note that in Lucene, writes are not visible to a reader of the {@code
Directory}
+ * until committed. A special {@link IndexReader} is needed. See
+ * {@link DirectoryReader#open(IndexWriter)}. jena-text does not currentyl
do this.
--- End diff --
Typo - `currentyl`
---
[GitHub] jena pull request #424: JENA-1550: TDB2 loader
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/424#discussion_r189542354
--- Diff:
jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/loader/parallel/LoaderConst.java
---
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.tdb2.loader.parallel;
+
+import java.util.Collections;
+import java.util.List;
+
+import org.apache.jena.atlas.lib.tuple.Tuple;
+import org.apache.jena.tdb2.store.NodeId;
+
+public class LoaderConst {
+
+/** Chunk size for the triple->tuples output pipe */
+public final static int ChunkSize = 100_000 ;
--- End diff --
Didn't even know this little syntax trick for making numeric constants more
readable existed in Java!
---
[GitHub] jena pull request #424: JENA-1550: TDB2 loader
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/424#discussion_r189542444
--- Diff:
jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/loader/parallel/LoaderParallel.java
---
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.tdb2.loader.parallel;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.function.Consumer;
+
+import org.apache.jena.atlas.lib.tuple.Tuple;
+import org.apache.jena.graph.Node;
+import org.apache.jena.riot.system.StreamRDF;
+import org.apache.jena.sparql.core.DatasetGraph;
+import org.apache.jena.tdb2.loader.DataLoader;
+import org.apache.jena.tdb2.loader.base.LoaderBase;
+import org.apache.jena.tdb2.loader.base.LoaderOps;
+import org.apache.jena.tdb2.loader.base.MonitorOutput;
+import org.apache.jena.tdb2.store.DatasetGraphTDB;
+import org.apache.jena.tdb2.store.DatasetPrefixesTDB;
+import org.apache.jena.tdb2.store.NodeId;
+import org.apache.jena.tdb2.sys.TDBInternal;
+
+/**
+ * The parallel Loader.
+ *
+ * The process is:
+ *
+ * {@code DataBatcher -> DataToTuples -> Indexer}
+ *
+ * {@link DataBatcher} produces {@link DataBlock}s - grouping of triples
and quads. It uses
--- End diff --
Unterminated sentence here?
---
[GitHub] jena pull request #424: JENA-1550: TDB2 loader
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/424#discussion_r189542491
--- Diff:
jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/loader/parallel/LoaderParallel.java
---
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.tdb2.loader.parallel;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.function.Consumer;
+
+import org.apache.jena.atlas.lib.tuple.Tuple;
+import org.apache.jena.graph.Node;
+import org.apache.jena.riot.system.StreamRDF;
+import org.apache.jena.sparql.core.DatasetGraph;
+import org.apache.jena.tdb2.loader.DataLoader;
+import org.apache.jena.tdb2.loader.base.LoaderBase;
+import org.apache.jena.tdb2.loader.base.LoaderOps;
+import org.apache.jena.tdb2.loader.base.MonitorOutput;
+import org.apache.jena.tdb2.store.DatasetGraphTDB;
+import org.apache.jena.tdb2.store.DatasetPrefixesTDB;
+import org.apache.jena.tdb2.store.NodeId;
+import org.apache.jena.tdb2.sys.TDBInternal;
+
+/**
+ * The parallel Loader.
+ *
+ * The process is:
+ *
+ * {@code DataBatcher -> DataToTuples -> Indexer}
+ *
+ * {@link DataBatcher} produces {@link DataBlock}s - grouping of triples
and quads. It uses
+ *
+ * {@link DataToTuples} processes {@link DataBlock} to create 2 outputs
blocks of {@code Tuple}, one output for triples, oue for quads.
--- End diff --
Typo `oue` -> `one`
---
[GitHub] jena pull request #424: JENA-1550: TDB2 loader
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/424#discussion_r189540060
--- Diff: jena-base/src/main/java/org/apache/jena/atlas/lib/Timer.java ---
@@ -69,4 +69,13 @@ static public String timeStr(long timeInterval) {
protected String timeStr(long timePoint, long startTimePoint) {
return timeStr(timePoint - startTimePoint) ;
}
+
+/** Time an operation. Return the elapsed time in milliseconds. */
+public static long time(Runnable action) {
--- End diff --
There's an implied assumption here that any exceptions thrown by the
runnable are handled by the caller
---
[GitHub] jena pull request #407: Cleanup
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/407#discussion_r183809337
--- Diff:
jena-db/jena-tdb2/src/main/java/org/apache/jena/tdb2/store/DatasetGraphSwitchable.java
---
@@ -111,7 +112,8 @@ public Graph getDefaultGraph() {
@Override
public Graph getGraph(Node gn) {
-return ngCache.getOrFill(gn,
()->GraphViewSwitchable.createNamedGraph(this, gn));
+Node key = ( gn != null ) ? gn : Quad.defaultGraphNodeGenerated;
+return ngCache.getOrFill(gn,
()->GraphViewSwitchable.createNamedGraph(this, key));
--- End diff --
Shouldn't the first `gn` reference on this line also be changed to `key`
otherwise you are adding a potentially `null` key to the cache
---
[GitHub] jena pull request #395: JENA-1488: add a selective folding analyzer
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/395#discussion_r181083359
--- Diff:
jena-text/src/test/java/org/apache/jena/query/text/filter/TestSelectiveFoldingFilter.java
---
@@ -0,0 +1,135 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.query.text.filter;
+
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.io.StringReader;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+
+import org.apache.lucene.analysis.CharArraySet;
+import org.apache.lucene.analysis.standard.StandardTokenizer;
+import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
+import org.junit.Before;
+import org.junit.Test;
+
+/**
+ * Test {@link SelectiveFoldingFilter}.
+ */
+
+public class TestSelectiveFoldingFilter {
+
+private StringReader inputText;
+private CharArraySet whitelisted;
+
+@Before
+public void setUp() {
+inputText = new StringReader("Señora Siobhán, look at that
façade");
+}
+
+/**
+ * An empty white list means that the default behaviour of the
Lucene's ASCIIFoldingFilter applies.
+ * @throws IOException from Lucene API
+ */
+@Test
+public void testEmptyWhiteListIsOkay() throws IOException {
+whitelisted = new CharArraySet(Collections.emptyList(), false);
+List tokens = collectTokens(inputText, whitelisted);
+List expected = Arrays.asList("Senora", "Siobhan", "look",
"at", "that", "facade");
+assertTrue(tokens.equals(expected));
+}
+
+@Test
+public void testSingleCharacterWhiteListed() throws IOException {
+whitelisted = new CharArraySet(Arrays.asList("ç"), false);
+List tokens = collectTokens(inputText, whitelisted);
+List expected = Arrays.asList("Senora", "Siobhan", "look",
"at", "that", "façade");
+assertTrue(tokens.equals(expected));
+}
+
+@Test
+public void testCompleteWhiteListed() throws IOException {
+whitelisted = new CharArraySet(Arrays.asList("ñ", "á", "ç"),
false);
+List tokens = collectTokens(inputText, whitelisted);
+// here we should have the complete input
+List expected = Arrays.asList("Señora", "Siobhán",
"look", "at", "that", "façade");
+assertTrue(tokens.equals(expected));
+}
+
+@Test
+public void testCaseMatters() throws IOException {
+// note the first capital letter
+whitelisted = new CharArraySet(Arrays.asList("Ñ", "á", "ç"),
false);
+List tokens = collectTokens(inputText, whitelisted);
+List expected = Arrays.asList("Senora", "Siobhán",
"look", "at", "that", "façade");
+assertTrue(tokens.equals(expected));
+}
+
+@Test
+public void testMismatchWhiteList() throws IOException {
+whitelisted = new CharArraySet(Arrays.asList("ú", "ć", "ž"),
false);
+List tokens = collectTokens(inputText, whitelisted);
+List expected = Arrays.asList("Senora", "Siobhan", "look",
"at", "that", "facade");
+assertTrue(tokens.equals(expected));
+}
+
+@Test(expected = NullPointerException.class)
+public void testNullWhiteListThrowsError() throws IOException {
+collectTokens(inputText, null);
+}
+
+@Test
+
[GitHub] jena issue #393: JENA-1517: Implement missing override in OpVarsPattern visi...
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/393 +1 ---
[GitHub] jena pull request #391: JENA-1516: ObjectFileStorage fix
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/391#discussion_r179407258
--- Diff: jena-tdb/src/main/java/org/apache/jena/tdb/lib/NodeLib.java ---
@@ -53,26 +52,32 @@
// Characters in IRIs that are illegal and cause SSE problems, but we
wish to keep.
final private static char MarkerChar = '_' ;
final private static char[] invalidIRIChars = { MarkerChar , ' ' } ;
+final private static int SIZE = 1024;
+// Marshalling space.
+final private static ByteBuffer workspace = ByteBuffer.allocate(SIZE);
--- End diff --
This buffer is a `static` and there is no thread safety or synchronisation
around its usage. Is this safe?
---
[GitHub] jena issue #385: defined filters and tokenizers for ConfigurableAnalyzer
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/385 @xristy Yes I don't think there is any harm in supporting both the explicit form as well as supporting literal datatypes directly ---
[GitHub] jena issue #385: defined filters and tokenizers for ConfigurableAnalyzer
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/385 Btw as you are now a committer you can follow the instructions at https://reference.apache.org/committer/github to get added to the Apache GitHub org so you'll show up as `Member` and not just as `Contributor` on here ---
[GitHub] jena issue #385: defined filters and tokenizers for ConfigurableAnalyzer
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/385 Looks like a really nice extension to the current machinery! I notice there is a unit test for the assembler support but no unit tests for using some of these new features, would it be possible to include some of those to verify that this actually allows addressing the use cases that led to this PR? One other thought is whether the explicit param type definitions are necessary for the simpler cases (integer, boolean) or whether we could just rely on literal datatypes to reduce the verbosity of the configuration for those simpler cases? For the complex cases like sets, token streams, analysers etc then explicit types makes total sense. ---
[GitHub] jena pull request #370: Fix delimiter parsing logic (JENA-1497)
GitHub user rvesse opened a pull request: https://github.com/apache/jena/pull/370 Fix delimiter parsing logic (JENA-1497) Logical flaws in using continue vs break inside inner loops where causing the wrong delimiter positions to be detected and leading to false positives being reported for potential injection attacks. Fixing the logic allows the user test case to pass. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rvesse/jena JENA-1497 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/370.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #370 commit 411c1031a0f51885f6966914c58202654614be13 Author: Rob Vesse <rvesse@...> Date: 2018-03-05T10:25:24Z Fix delimiter parsing logic (JENA-1497) Logical flaws in using continue vs break inside inner loops where causing the wrong delimiter positions to be detected and leading to false positives being reported for potential injection attacks. Fixing the logic allows the user test case to pass. ---
[GitHub] jena pull request #365: JENA-1490: Handle blank nodes client-server when usi...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/365#discussion_r170621100
--- Diff:
jena-rdfconnection/src/main/java/org/apache/jena/rdfconnection/RDFConnectionRemoteBuilder.java
---
@@ -0,0 +1,324 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.rdfconnection;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.Objects;
+import java.util.function.Function;
+
+import org.apache.http.client.HttpClient;
+import org.apache.http.protocol.HttpContext;
+import org.apache.jena.riot.*;
+import org.apache.jena.sparql.core.Transactional;
+import org.apache.jena.sparql.core.TransactionalLock;
+import org.apache.jena.sparql.engine.http.QueryEngineHTTP;
+
+public class RDFConnectionRemoteBuilder {
+/*package*/ static String SameAsDestination = "";
+
+protected Transactional txnLifecycle =
TransactionalLock.createMRPlusSW();
+protected HttpClienthttpClient= null;
+protected HttpContext httpContext = null;
+protected Stringdestination = null;
+
+protected StringsQuery= SameAsDestination;
+protected StringsUpdate = SameAsDestination;
+protected StringsGSP = SameAsDestination;
+
+protected StringqueryURL = null;
+protected StringupdateURL = null;
+protected StringgspURL= null;
+
+// On-the-wire settings.
+protected RDFFormat outputQuads= RDFFormat.NQUADS;
+protected RDFFormat outputTriples = RDFFormat.NTRIPLES;
+
+protected StringacceptGraph=
WebContent.defaultGraphAcceptHeader;
+protected StringacceptDataset =
WebContent.defaultDatasetAcceptHeader;
+
+protected StringacceptSelectResult =
QueryEngineHTTP.defaultSelectHeader();
+protected StringacceptAskResult=
QueryEngineHTTP.defaultAskHeader();
+// All-purpose head that works for any query type (but is quite long!)
+protected StringacceptSparqlResults =
acceptSelectResult+","+acceptGraph;
+// Whether to parse SPARQL Queries and Updates for checkign purposes.
+protected boolean parseCheckQueries = true;
+protected boolean parseCheckUpdates = true;
+
+RDFConnectionRemoteBuilder() {
+// Default settings are the meber declarations.
+}
+
+RDFConnectionRemoteBuilder(RDFConnectionRemote base) {
+Objects.requireNonNull(base);
+txnLifecycle = base.txnLifecycle;
+if ( txnLifecycle == null )
+txnLifecycle = TransactionalLock.createMRPlusSW();
+httpClient = base.httpClient;
+httpContext = base.httpContext;
+destination = base.destination;
+sQuery = base.svcQuery;
+sUpdate = base.svcUpdate;
+sGSP= base.svcGraphStore;
+outputQuads = base.outputQuads;
+outputTriples = base.outputTriples;
+
+acceptGraph = base.acceptGraph;
+acceptDataset = base.acceptDataset;
+
+acceptSelectResult = base.acceptSelectResult;
+acceptAskResult = base.acceptAskResult;
+parseCheckQueries = base.parseCheckQueries;
+parseCheckUpdates = base.parseCheckUpdates;
+}
+
+/** URL of the remote SPARQL endpoint.
+ * For Fuseki, this is the URL of the dataset e.g.
http:/localhost:3030/dataset
+ */
+public RDFConnectionRemoteBuilder destination(String destination) {
+Objects.requireNonNull(destination);
+this.destination = destination;
+return this;
+}
+
+
[GitHub] jena pull request #366: JENA-1494: Reader properties
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/366#discussion_r170616197
--- Diff: jena-arq/src/main/java/org/apache/jena/riot/SysRIOT.java ---
@@ -50,14 +51,20 @@
/**
* Context key for old style RDFWriter properties. The value of this
in a
* {@link Context} must be a {@code Map<String, Object>}. The entries
of the
- * map are used to set writer properties before the Jena legalacy
+ * map are used to set writer properties before the Jena legacy
* {@link RDFWriter} is called. Only has any effect on RDF/XML and
* RDF/XML-ABBREV.
*/
-
-/** Context key for old style RDFWriter properties */
public static final Symbol sysRdfWriterProperties =
Symbol.create(riotBase+"rdfWriter_properties") ;
-
+
+/**
+ * Context key for old style RDFReader properties. The value of this
in a
+ * {@link Context} must be a {@code Map<String, Object>}. The entries
of the
+ * map are used to set reader properties before the Jena legalacy
--- End diff --
Typo - `legalacy`
---
[GitHub] jena pull request #366: JENA-1494: Reader properties
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/366#discussion_r170616897
--- Diff:
jena-arq/src/main/java/org/apache/jena/riot/lang/ReaderRIOTRDFXML.java ---
@@ -116,17 +154,29 @@ public void parse() {
arp.getHandlers().setErrorHandler(rslt) ;
arp.getHandlers().setNamespaceHandler(rslt) ;
+// ARPOptions.
+ARPOptions arpOptions = arp.getOptions() ;
if ( RiotUniformCompatibility ) {
-ARPOptions options = arp.getOptions() ;
// Convert some warnings to errors for compatible behaviour
for all parsers.
for ( int code : additionalErrors )
-options.setErrorMode(code, ARPErrorNumbers.EM_ERROR) ;
-arp.setOptionsWith(options) ;
+arpOptions.setErrorMode(code, ARPErrorNumbers.EM_ERROR) ;
}
if ( JenaRuntime.isRDF11 )
arp.getOptions().setIRIFactory(IRIResolver.iriFactory());
+if ( context != null ) {
+try {
+@SuppressWarnings("unchecked")
+Map<String, Object> p = (Map<String,
Object>)(context.get(SysRIOT.sysRdfReaderProperties)) ;
+if ( p != null )
+p.forEach((k,v) -> oneProperty(arpOptions, k, v)) ;
+} catch (Throwable ex) {
+Log.warn(AdapterRDFWriter.class, "Problem setting
properties", ex);
--- End diff --
Should this message more clearly state that some/all properties may have
been ignored?
---
[GitHub] jena pull request #360: Make sure to use datasets transactionally (JENA-1486...
GitHub user rvesse opened a pull request: https://github.com/apache/jena/pull/360 Make sure to use datasets transactionally (JENA-1486) textindexer did not use datasets transactionally which could fail for dataset implementations that require transactional usage. And it is safer to always use transactions where provided anyway. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rvesse/jena JENA-1486 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/360.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #360 commit 0eeb40db8eb8b64617b232aaf8806769c00df964 Author: Rob Vesse <rvesse@...> Date: 2018-02-12T14:19:01Z Make sure to use datasets transactionally (JENA-1486) textindexer did not use datasets transactionally which could fail for dataset implementations that require transactional usage. And it is safer to always use transactions where provided anyway. ---
[GitHub] jena issue #345: JENA-1468: Retain variables in op (table)
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/345 @afs Thanks for the more detailed explanation +1 to merge ---
[GitHub] jena issue #345: JENA-1468: Retain variables in op (table)
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/345 Does https://github.com/apache/jena/blob/cc038809fb622779933831011909714e22ef494c/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformPromoteTableEmpty.java#L160 need to change to use the new test? ---
[GitHub] jena pull request #330: [WIP] Obfuscation Support
GitHub user rvesse opened a pull request: https://github.com/apache/jena/pull/330 [WIP] Obfuscation Support Early WIP on obfuscation support based on our internal implementation approach. Note various TODOs here and there, I am focusing on getting the basics working before extending it to be more general or adding user facing utilities like CLI tools etc You can merge this pull request into a Git repository by running: $ git pull https://github.com/rvesse/jena obfuscate Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/330.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #330 commit 9294d37a939356d088405535c0475213962fb6d0 Author: Rob Vesse <rve...@apache.org> Date: 2017-12-15T11:24:40Z Initial stub for obfuscation Interface and trivial implementations commit d3069453a718721248f8c14b6cf38e6de6e8844c Author: Rob Vesse <rve...@apache.org> Date: 2017-12-15T14:59:08Z Additional work on obfuscation support - Add a multi-pass implementation - Add basic unit tests ---
[GitHub] jena issue #315: Add --stop-warnings to riot (JENA-1434)
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/315 Reworked based on @afs's feedback in the JIRA ---
[GitHub] jena pull request #315: Add --stop-warnings to riot (JENA-1434)
GitHub user rvesse opened a pull request: https://github.com/apache/jena/pull/315 Add --stop-warnings to riot (JENA-1434) Add the option to allow riot to abort on warnings You can merge this pull request into a Git repository by running: $ git pull https://github.com/rvesse/jena JENA-1434 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/315.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #315 commit 1f7082cb88a4f796bf305d680ab49509049eec11 Author: Rob Vesse <rve...@apache.org> Date: 2017-11-23T17:56:23Z Add --stop-warnings to riot (JENA-1434) Add the option to allow riot to abort on warnings ---
[GitHub] jena pull request #308: JENA-1384: Canonical literals: lexical form and lang...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/308#discussion_r152523288
--- Diff:
jena-arq/src/main/java/org/apache/jena/riot/process/normalize/CanonicalizeLiteral.java
---
@@ -73,6 +76,36 @@ public Node apply(Node node) {
return n2 ;
}
+/** Convert the lexical form to a canonical form if one of the known
datatypes,
+ * otherwise return the node argument. (same object :: {@code ==})
+ */
+public static Node canonicalValue(Node node) {
+if ( ! node.isLiteral() )
+return node ;
+// Fast-track
+if ( NodeUtils.isLangString(node) )
+return node;
+if ( NodeUtils.isSimpleString(node) )
+return node;
+
+if ( !
node.getLiteralDatatype().isValid(node.getLiteralLexicalForm()) )
+// Invalid lexical form for the datatype - do nothing.
+return node;
+
+RDFDatatype dt = node.getLiteralDatatype() ;
+// Datatype, not rdf:langString (RDF 1.1).
+DatatypeHandler handler = dispatch.get(dt) ;
+if ( handler == null )
+return node ;
+Node n2 = handler.handle(node, node.getLiteralLexicalForm(), dt) ;
+if ( n2 == null )
+return node ;
+return n2 ;
+}
+
+/** Convert the language tag of a lexical form to a canonical form if
one of the known datatypes,
+ * otherwise return the node argument. (same object; compare by {@code
==})
+ */
private static Node canonicalLangtag(String lexicalForm, String
langTag) {
String langTag2 = LangTag.canonical(langTag);
if ( langTag2.equals(langTag) )
--- End diff --
Shouldn't we be returning `node` not `null` in the subsequent line?
---
[GitHub] jena pull request #297: Enable parallel testing for Elephas IO (JENA-1407)
Github user rvesse closed the pull request at: https://github.com/apache/jena/pull/297 ---
[GitHub] jena issue #297: Enable parallel testing for Elephas IO (JENA-1407)
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/297 I think leaving at 2 is good, I was playing with using `false` though obviously that lessens the benefit. I think we can close this out though if we've delivered noticeable improvements already, not really worth tweaking the configs too much as any further gain is likely limited ---
[GitHub] jena issue #297: Enable parallel testing for Elephas IO (JENA-1407)
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/297 I have now changed this to dynamically set test parallelism to use 50% of available cores ---
[GitHub] jena issue #297: Enable parallel testing for Elephas IO (JENA-1407)
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/297 Parallelism of 2 gives similar benefits. I have a quad core machine personally. One possible approach is to use the `build-helper:cpu-count` plugin - http://www.mojohaus.org/build-helper-maven-plugin/cpu-count-mojo.html - to dynamically set the level of parallelism so that we don't over subscribe the users system ---
[GitHub] jena pull request #297: Enable parallel testing for Elephas IO (JENA-1407)
GitHub user rvesse opened a pull request: https://github.com/apache/jena/pull/297 Enable parallel testing for Elephas IO (JENA-1407) Using parallelism of 4 reduces build time for Elephas IO to approximately 3-4 minutes in my testing. Also fixed one test that was not parallel safe to use the pre-existing temporary folder. Ran this a ton of times in a tight loop on my system to check that there weren't any transient errors or serial test assumptions encoded in the tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rvesse/jena JENA-1407 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/297.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #297 commit b26643b4cf19704cd4ad969d65f59095b235d667 Author: Rob Vesse <rve...@apache.org> Date: 2017-10-26T17:03:33Z Enable parallel testing for Elephas IO (JENA-1407) Using parallelism of 4 reduces build time for Elephas IO to approximately 3-4 minutes in my testing. Also fixed one test that was not parallel safe to use the pre-existing temporary folder. ---
[GitHub] jena pull request #294: Fix and tests for possible NPE (JENA-1405)
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/294#discussion_r146852236
--- Diff:
jena-arq/src/main/java/org/apache/jena/riot/adapters/AdapterFileManager.java ---
@@ -285,6 +286,12 @@ protected Model readModelWorker(Model model, String
filenameOrURI, String baseUR
if ( baseURI == null )
baseURI = SysRIOT.chooseBaseIRI(filenameOrURI) ;
try(TypedInputStream in =
streamManager.openNoMapOrNull(mappedURI)) {
+if ( in == null )
+{
+if ( log.isDebugEnabled() )
+log.debug("Failed to locate '"+mappedURI+"'") ;
--- End diff --
Yes, I just copied and pasted this from the parent class where the same
logic is present
---
[GitHub] jena pull request #294: Fix and tests for possible NPE (JENA-1405)
GitHub user rvesse opened a pull request: https://github.com/apache/jena/pull/294 Fix and tests for possible NPE (JENA-1405) Fixes a possible NPE in AdaptorFileManager by aligning its behaviour with FileManager to throw NotFoundException Adds specific tests for these cases You can merge this pull request into a Git repository by running: $ git pull https://github.com/rvesse/jena JENA-1405 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/294.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #294 commit 643bd7a944e23554d21d1f01c207fe7e1a3d3098 Author: Rob Vesse <rve...@apache.org> Date: 2017-10-25T10:26:37Z Fix and tests for possible NPE (JENA-1405) Fixes a possible NPE in AdaptorFileManager by aligning its behaviour with FileManager to throw NotFoundException Adds specific tests for these cases ---
[GitHub] jena issue #291: Bad regular expression patterns should throw ExprEvalExcept...
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/291 LGTM +1 ---
[GitHub] jena issue #285: JENA-1398: Marked archaic FOAF terms in the vocabulary as d...
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/285 Thanks for the updates, looks good to me We aim to release approximately every three months, the last release was July so we are probably due another release soon. However everything is done on volunteer time so it will really depend on peoples availability ---
[GitHub] jena pull request #285: JENA-1398: Marked archaic FOAF terms in the vocabula...
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/285#discussion_r144011336
--- Diff:
jena-arq/src/main/java/org/apache/jena/sparql/vocabulary/FOAF.java ---
@@ -19,242 +19,282 @@
package org.apache.jena.sparql.vocabulary ;
/* CVS $Id: $ */
-
+
import org.apache.jena.rdf.model.Model ;
import org.apache.jena.rdf.model.ModelFactory ;
import org.apache.jena.rdf.model.Property ;
import org.apache.jena.rdf.model.Resource ;
-
+
/**
- * Vocabulary definitions from FOAF.ttl
- * @author Auto-generated by schemagen on 09 Jul 2007 16:50
+ * Vocabulary definitions from FOAF.rdf
+ * @author Auto-generated by schemagen on 11 Oct 2017 20:53
*/
public class FOAF {
/** The RDF model that holds the vocabulary terms */
-private static Model m_model = ModelFactory.createDefaultModel();
-
+private static final Model M_MODEL = ModelFactory.createDefaultModel();
+
/** The namespace of the vocabulary as a string */
public static final String NS = "http://xmlns.com/foaf/0.1/;;
-
+
/** The namespace of the vocabulary as a string
- * @see #NS */
+ * @return namespace as String
+ * @see #NS */
public static String getURI() {return NS;}
-
+
/** The namespace of the vocabulary as a resource */
-public static final Resource NAMESPACE = m_model.createResource( NS );
-
-/** A jabber ID for something. */
-public static final Property jabberID = m_model.createProperty(
"http://xmlns.com/foaf/0.1/jabberID; );
-
-/** A short informal nickname characterising an agent (includes
login identifiers,
- * IRC and other chat nicknames).
- */
-public static final Property nick = m_model.createProperty(
"http://xmlns.com/foaf/0.1/nick; );
-
-/** A checksum for the DNA of some thing. Joke. */
-public static final Property dnaChecksum = m_model.createProperty(
"http://xmlns.com/foaf/0.1/dnaChecksum; );
-
-/** A topic of some page or document. */
-public static final Property topic = m_model.createProperty(
"http://xmlns.com/foaf/0.1/topic; );
-
-/** A theme. */
-public static final Property theme = m_model.createProperty(
"http://xmlns.com/foaf/0.1/theme; );
-
-/** An MSN chat ID */
-public static final Property msnChatID = m_model.createProperty(
"http://xmlns.com/foaf/0.1/msnChatID; );
-
-/** An OpenID for an Agent. */
-public static final Property openid = m_model.createProperty(
"http://xmlns.com/foaf/0.1/openid; );
-
-/** The family_name of some person. */
-public static final Property family_name = m_model.createProperty(
"http://xmlns.com/foaf/0.1/family_name; );
-
-/** A homepage of a school attended by the person. */
-public static final Property schoolHomepage = m_model.createProperty(
"http://xmlns.com/foaf/0.1/schoolHomepage; );
-
-/** A project this person has previously worked on. */
-public static final Property pastProject = m_model.createProperty(
"http://xmlns.com/foaf/0.1/pastProject; );
-
-/** A .plan comment, in the tradition of finger and '.plan'
files. */
-public static final Property plan = m_model.createProperty(
"http://xmlns.com/foaf/0.1/plan; );
-
-/** A Myers Briggs (MBTI) personality classification. */
-public static final Property myersBriggs = m_model.createProperty(
"http://xmlns.com/foaf/0.1/myersBriggs; );
-
-/** A personal mailbox, ie. an Internet mailbox associated with
exactly one owner,
- * the first owner of this mailbox. This is a 'static inverse
functional property',
- * in that there is (across time and change) at most one individual
that ever
- * has any particular value for foaf:mbox.
- */
-public static final Property mbox = m_model.createProperty(
"http://xmlns.com/foaf/0.1/mbox; );
-
-/** The gender of this Agent (typically but not necessarily 'male'
or 'female'). */
-public static final Property gender = m_model.createProperty(
"http://xmlns.com/foaf/0.1/gender; );
-
-/** A sha1sum hash, in hex. */
-public static final Property sha1 = m_model.createProperty(
"http://xmlns.com/foaf/0.1/sha1; );
-
-/** A link to the publications of this person. */
-public static final Property publications = m_model.createProperty(
"http://xmlns.com/foaf/0.1/publications; );
-
+public static final Resource NAMESPACE = M_MODEL.createResource( NS );
+
/** Indicates
[GitHub] jena issue #281: Slice out old Codehaus JXR Maven plugin invocation
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/281 +1 ---
[GitHub] jena pull request #237: JENA-1313: compare using a Collator when both litera...
Github user rvesse commented on a diff in the pull request: https://github.com/apache/jena/pull/237#discussion_r111201417 --- Diff: jena-arq/src/main/java/org/apache/jena/sparql/expr/NodeValue.java --- @@ -784,10 +786,15 @@ private static int compare(NodeValue nv1, NodeValue nv2, boolean sortOrderingCom return x ; } -// same lang tag (case insensitive) -x = StrUtils.strCompare(node1.getLiteralLexicalForm(), node2.getLiteralLexicalForm()) ; +// same lang tag, handle collation +// TBD: cache locales? cache collators? pre define both/any? a simple in-memory lru-map-cache? +Locale desiredLocale = Locale.forLanguageTag(node1.getLiteralLanguage()); +Collator collator = Collator.getInstance(desiredLocale); --- End diff -- This needs to be much more defensively coded. Language tags are not constrained to be valid languages so they could be no locale or collation associated with a given tag to make sure that we fall back to the existing behaviour for unknown languages --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #234: Changes to jena-elephas to update it for RIOT/RDFParser cha...
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/234 Thanks for starting on this, one of the first things I thought of when you started on the API improvements was that it would make life much easier for components like Elephas that need to manipulate the parser set up Looks great so far --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #230: JENA-733 add documentation for narrower data type case
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/230 +1 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #219: Upgrade to Lucene 6.4.1 in jena-text and jena-spatial (JENA...
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/219 Thanks for adding the more informative error message, the wording is extremely clear and should avoid lots of user confusion --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #219: Upgrade to Lucene 6.4.1 in jena-text and jena-spatial (JENA...
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/219 Is there a logical place where we could catch that specific exception and provide a more obvious error e.g. Index format is no longer supported, please rebuild the text index with Jena X.Y.Z or higher If we do that proactively we are likely to get less support questions that if we leave the default error --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #222: JENA-1306: RDFParser
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/222#discussion_r104627436
--- Diff: jena-arq/src/main/java/org/apache/jena/riot/RDFParserBuilder.java
---
@@ -0,0 +1,431 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.riot;
+
+import java.io.InputStream;
+import java.io.Reader;
+import java.io.StringReader;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.http.Header;
+import org.apache.http.client.HttpClient;
+import org.apache.http.impl.client.cache.CachingHttpClientBuilder;
+import org.apache.http.message.BasicHeader;
+import org.apache.jena.atlas.lib.IRILib;
+import org.apache.jena.graph.BlankNodeId;
+import org.apache.jena.graph.Graph;
+import org.apache.jena.riot.lang.LabelToNode;
+import org.apache.jena.riot.system.*;
+import org.apache.jena.riot.web.HttpNames;
+import org.apache.jena.sparql.core.DatasetGraph;
+import org.apache.jena.sparql.util.Context;
+
+/**
+ * An {@link RDFParser} is a process that will generate triples;
+ * {@link RDFParserBuilder} provides the means to setup the parser.
+ *
+ * An {@link RDFParser} has a predefined source; the target for output is
given when the "parse" method is called.
+ * It can be used multiple times in which case the same source is reread.
The destination can vary.
+ * The application is responsible for concurrency of the destination of
the parse operation.
+ *
+ * The process is
+ *
+ * StreamRDF destination = ...
+ * RDFParser parser = RDFParser.create()
+ * .source("filename.ttl")
+ * .build();
+ * parser.parse(destination);
+ *
+ * or using a short cut:
+ *
+ * RDFParser parser = RDFParser.create()
+ * .source("filename.ttl")
+ * .parse(destination);
+ *
+ */
+public class RDFParserBuilder {
+// Source
+private String uri = null;
+private Path path = null;
+private InputStream inputStream;
+// StringReader - charset problems with any other kind.
+private Reader javaReader = null;
+
+// HTTP
+private Map<String, String> httpHeaders = new HashMap<>();
+private HttpClient httpClient = null;
+
+// Syntax
+private Lang hintLang = null;
+private Lang forceLang = null;
+
+private String baseUri = null;
+
+// Unused but left in case required in the future.
+private boolean strict = SysRIOT.isStrictMode();
+private boolean resolveURIs = true;
+private IRIResolver resolver = null;
+//
+
+// Construction for the StreamRDF
+private FactoryRDF factory = null;
+private LabelToNode labelToNode = null;
+
+// Bad news.
+private ErrorHandler errorHandler = null;
+
+// Parsing process
+private Context context = null;
+
+public static RDFParserBuilder create() { return new
RDFParserBuilder() ; }
+private RDFParserBuilder() {}
+
+/**
+ * Set the source to {@link Path}.
+ * This clears any other source setting.
+ * @param path
+ * @return this
+ */
+public RDFParserBuilder source(Path path) {
+clearSource();
+this.path = path;
+return this;
+}
+
+/**
+ * Set the source to a URI; this includes OS file names.
+ * File URL shoudl be of the form {@code file:///...}.
+ * This clears any other source setting.
+ * @param uri
+ * @return this
+ */
+public RDFParserBuilder source(String uri) {
+clearSource();
+this.uri = uri;
+return th
[GitHub] jena pull request #222: JENA-1306: RDFParser
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/222#discussion_r104627292
--- Diff: jena-arq/src/main/java/org/apache/jena/riot/RDFParserBuilder.java
---
@@ -0,0 +1,431 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.riot;
+
+import java.io.InputStream;
+import java.io.Reader;
+import java.io.StringReader;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.http.Header;
+import org.apache.http.client.HttpClient;
+import org.apache.http.impl.client.cache.CachingHttpClientBuilder;
+import org.apache.http.message.BasicHeader;
+import org.apache.jena.atlas.lib.IRILib;
+import org.apache.jena.graph.BlankNodeId;
+import org.apache.jena.graph.Graph;
+import org.apache.jena.riot.lang.LabelToNode;
+import org.apache.jena.riot.system.*;
+import org.apache.jena.riot.web.HttpNames;
+import org.apache.jena.sparql.core.DatasetGraph;
+import org.apache.jena.sparql.util.Context;
+
+/**
+ * An {@link RDFParser} is a process that will generate triples;
+ * {@link RDFParserBuilder} provides the means to setup the parser.
+ *
+ * An {@link RDFParser} has a predefined source; the target for output is
given when the "parse" method is called.
+ * It can be used multiple times in which case the same source is reread.
The destination can vary.
+ * The application is responsible for concurrency of the destination of
the parse operation.
+ *
+ * The process is
+ *
+ * StreamRDF destination = ...
+ * RDFParser parser = RDFParser.create()
+ * .source("filename.ttl")
+ * .build();
+ * parser.parse(destination);
+ *
+ * or using a short cut:
+ *
+ * RDFParser parser = RDFParser.create()
+ * .source("filename.ttl")
+ * .parse(destination);
+ *
+ */
+public class RDFParserBuilder {
+// Source
+private String uri = null;
+private Path path = null;
+private InputStream inputStream;
+// StringReader - charset problems with any other kind.
+private Reader javaReader = null;
+
+// HTTP
+private Map<String, String> httpHeaders = new HashMap<>();
+private HttpClient httpClient = null;
+
+// Syntax
+private Lang hintLang = null;
+private Lang forceLang = null;
+
+private String baseUri = null;
+
+// Unused but left in case required in the future.
+private boolean strict = SysRIOT.isStrictMode();
+private boolean resolveURIs = true;
+private IRIResolver resolver = null;
+//
+
+// Construction for the StreamRDF
+private FactoryRDF factory = null;
+private LabelToNode labelToNode = null;
+
+// Bad news.
+private ErrorHandler errorHandler = null;
+
+// Parsing process
+private Context context = null;
+
+public static RDFParserBuilder create() { return new
RDFParserBuilder() ; }
+private RDFParserBuilder() {}
+
+/**
+ * Set the source to {@link Path}.
+ * This clears any other source setting.
+ * @param path
+ * @return this
+ */
+public RDFParserBuilder source(Path path) {
+clearSource();
+this.path = path;
+return this;
+}
+
+/**
+ * Set the source to a URI; this includes OS file names.
+ * File URL shoudl be of the form {@code file:///...}.
+ * This clears any other source setting.
+ * @param uri
+ * @return this
+ */
+public RDFParserBuilder source(String uri) {
+clearSource();
+this.uri = uri;
+return th
[GitHub] jena issue #212: JENA-1284: Improvements for bulk graph operations in GraphU...
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/212 My concern is that depending on the underlying storage of the data in the graphs a call to `contains()` might be quite expensive and it would be more efficient to simply call `delete()` directly. You appear to have provided a flag to force the old behaviour so I think go ahead and merge this. Since if any user does encounter performance regressions we can advise them of the workaround and use a concrete report as the basis for any further optimisation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #205: JENA-1277: don't use sorting in spatial queries, for much b...
Github user rvesse commented on the issue:
https://github.com/apache/jena/pull/205
We should already have support for most of the trigonometric functions
built into ARQ as extension functions -
https://github.com/apache/jena/tree/59e5e06c85c98c9c65a9c1addbc4252626e9d061/jena-arq/src/main/java/org/apache/jena/sparql/function/library/leviathan
For example:
```
sparql --data ~/Documents/Work/Datasets/minimal.nt "SELECT
(<http://www.dotnetrdf.org/leviathan#sin>(90) AS ?x) { }"
| x|
| 0.8939966636005579e0 |
```
Notes:
- Inverse functions are named as `sin-1` etc.
- function arguments are expected to be radians
- convenience functions `degreesToRadians` and `radiansToDegrees` are
provided
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #212: JENA-1284: Improvements for bulk graph operations in GraphU...
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/212 I presume this was motivated by a real-world performance concern? It would be interesting to know how much difference it makes. My only concern is what happens when there is only a slight difference between the size of the affected graphs? It's looks like when there is a slight difference then you potentially do twice the work because on one code path you do both a `find()` and a `delete()`/`add()` for every triple. Would it be worth making the behaviour based a configurable percentage difference e.g. If the graphs are within 10% of each others size don't use the new path. Also is materialising the list of triples a potential memory issue when the iterator is over a large amount of data? Click --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #197: JENA-1267: RDFConnection
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/197#discussion_r92839505
--- Diff:
jena-integration-tests/src/test/java/org/apache/jena/test/rdfconnection/AbstractTestRDFConnection.java
---
@@ -0,0 +1,382 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.test.rdfconnection;
+
+import java.util.concurrent.atomic.AtomicInteger ;
+
+import org.apache.jena.atlas.iterator.Iter ;
+import org.apache.jena.atlas.junit.BaseTest ;
+import org.apache.jena.atlas.lib.StrUtils ;
+import org.apache.jena.query.Dataset ;
+import org.apache.jena.query.DatasetFactory ;
+import org.apache.jena.query.ReadWrite ;
+import org.apache.jena.rdf.model.Model ;
+import org.apache.jena.rdf.model.ModelFactory ;
+import org.apache.jena.rdfconnection.RDFConnection;
+import org.apache.jena.riot.RDFDataMgr ;
+import org.apache.jena.sparql.core.DatasetGraph ;
+import org.apache.jena.sparql.sse.SSE ;
+import org.apache.jena.sparql.util.IsoMatcher ;
+import org.apache.jena.system.Txn ;
+import org.junit.Assume ;
+import org.junit.Test ;
+
+public abstract class AbstractTestRDFConnection extends BaseTest {
+// Testing data.
+static String DIR = "testing/RDFConnection/" ;
+
+protected abstract RDFConnection connection() ;
+// Not all RDFConenction types support.
--- End diff --
Typo: conenction -> connection
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #197: JENA-1267: RDFConnection
Github user rvesse commented on a diff in the pull request: https://github.com/apache/jena/pull/197#discussion_r92839346 --- Diff: jena-integration-tests/README.md --- @@ -0,0 +1,11 @@ +Jena Intrgration Tests --- End diff -- Typo r -> e --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #197: JENA-1267: RDFConnection
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/197#discussion_r92839569
--- Diff:
jena-integration-tests/src/test/java/org/apache/jena/test/rdfconnection/AbstractTestRDFConnection.java
---
@@ -0,0 +1,382 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.test.rdfconnection;
+
+import java.util.concurrent.atomic.AtomicInteger ;
+
+import org.apache.jena.atlas.iterator.Iter ;
+import org.apache.jena.atlas.junit.BaseTest ;
+import org.apache.jena.atlas.lib.StrUtils ;
+import org.apache.jena.query.Dataset ;
+import org.apache.jena.query.DatasetFactory ;
+import org.apache.jena.query.ReadWrite ;
+import org.apache.jena.rdf.model.Model ;
+import org.apache.jena.rdf.model.ModelFactory ;
+import org.apache.jena.rdfconnection.RDFConnection;
+import org.apache.jena.riot.RDFDataMgr ;
+import org.apache.jena.sparql.core.DatasetGraph ;
+import org.apache.jena.sparql.sse.SSE ;
+import org.apache.jena.sparql.util.IsoMatcher ;
+import org.apache.jena.system.Txn ;
+import org.junit.Assume ;
+import org.junit.Test ;
+
+public abstract class AbstractTestRDFConnection extends BaseTest {
+// Testing data.
+static String DIR = "testing/RDFConnection/" ;
+
+protected abstract RDFConnection connection() ;
+// Not all RDFConenction types support.
+// (may acquite RDFConnection.supportTransactionalAbort but ATM that
isn't included)
--- End diff --
Typo: acquite -> acquire
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #197: JENA-1267: RDFConnection
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/197#discussion_r92840442
--- Diff: jena-rdfconnection/Documentation.md ---
@@ -0,0 +1,205 @@
+# RDF Connection : SPARQL operations API
+
+`RDFConnection` provides a unified set of operations for working on RDF
+with SPARQL operations. It provides http://www.w3.org/TR/sparql11-query/;>SPARQL Query, http://www.w3.org/TR/sparql11-update/;>SPARQL Update and the http://www.w3.org/TR/sparql11-http-rdf-update/;>SPARQL Graph
+Store operations. The interface is uniform - the same interface
+applies to local data and to remote data using HTTP and the SPARQL
+protocols ( http://www.w3.org/TR/sparql11-protocol/;>SPARQL
+protocol and http://www.w3.org/TR/sparql11-http-rdf-update/;>SPARQL Graph Store
+Protocol).
+
+## Outline
+
+`RDFConnection` provides a number of different styles for working with RDF
+data in Java. It provides support for try-resource and functional code
+passing styles, as well the more basic sequence of methods calls.
+
+`try-resources` to manage the connection, and two operations, one to load
+some data, and one to make a query:
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+conn.load("data.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+}
+```
+This could have been written as (approximately -- the error handling is
better
+in the example above):
+
+```
+RDFConnection conn = RDFConnectionFactory.connect(...)
+conn.load("data.ttl") ;
+QueryExecution qExec = conn.query("SELECT DISTINCT ?s { ?s ?p ?o }") ;
+ResultSet rs = qExec.execSelect() ;
+while(rs.hasNext()) {
+QuerySolution qs = rs.next() ;
+Resource subject = qs.getResource("s") ;
+System.out.println("Subject: "+subject) ;
+}
+qExec.close() ;
+conn.close() ;
+```
+
+Jena also provides a separate
+[SPARQL over JDBC
driver](http://jena.staging.apache.org/documentation/jdbc/index.html)
+library.
+
+## Transactions
+
+Transactions are the preferred way to work with RDF data.
+Operations on an `RDFConnection` outside of an application-controlled
+transaction will cause the system to add one for the duration of the
+operation. This "autocommit" feature may lead to inefficient operations due
+to excessive overhead.
+
+The `Txn` class provides a Java8-style transaction API. Transactions are
+code passed in the `Txn` library that handles the transaction lifecycle.
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+Txn.execWrite(conn, ()-> {
+conn.load("data1.ttl") ;
+conn.load("data2.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+}) ;
+}
+```
+
+The traditional style of explicit `begin`, `commit`, `abort` is also
available.
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+conn.begin(ReadWrite.WRITE) ;
+try {
+conn.load("data1.ttl") ;
+conn.load("data2.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+conn.commit() ;
+} finally { conn.end() ; }
+}
+```
+
+The use of `try-finally` ensures that transactions are properly finished.
+The `conn.end()` provides an abort in case an exception occurs in the
+transaction and a commit has not been issued. The use of `try-finally`
+ensures that transactions are properly finished.
+
+`Txn` is wrapping these steps up and calling the application supplied code
+for the transaction body.
+
+### Remote Transactions
+
+SPARQL does not define a remote transaction standard protocol. Each remote
+operation shuld be atomic (all happens or nothing happens) - this is the
+responsibility of the remote server.
+
+An `RDFConenction` will at least provide the client-side locking features.
+This means that overlapping operations that chnage data are naturally
+handled by the transaction pattern within a single JVM.
+
+## Graph Store Protocol
+
+The http://www.w3.org/TR/sparql11-http-rdf-update/
[GitHub] jena pull request #197: JENA-1267: RDFConnection
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/197#discussion_r92840570
--- Diff: jena-rdfconnection/Documentation.md ---
@@ -0,0 +1,205 @@
+# RDF Connection : SPARQL operations API
+
+`RDFConnection` provides a unified set of operations for working on RDF
+with SPARQL operations. It provides http://www.w3.org/TR/sparql11-query/;>SPARQL Query, http://www.w3.org/TR/sparql11-update/;>SPARQL Update and the http://www.w3.org/TR/sparql11-http-rdf-update/;>SPARQL Graph
+Store operations. The interface is uniform - the same interface
+applies to local data and to remote data using HTTP and the SPARQL
+protocols ( http://www.w3.org/TR/sparql11-protocol/;>SPARQL
+protocol and http://www.w3.org/TR/sparql11-http-rdf-update/;>SPARQL Graph Store
+Protocol).
+
+## Outline
+
+`RDFConnection` provides a number of different styles for working with RDF
+data in Java. It provides support for try-resource and functional code
+passing styles, as well the more basic sequence of methods calls.
+
+`try-resources` to manage the connection, and two operations, one to load
+some data, and one to make a query:
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+conn.load("data.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+}
+```
+This could have been written as (approximately -- the error handling is
better
+in the example above):
+
+```
+RDFConnection conn = RDFConnectionFactory.connect(...)
+conn.load("data.ttl") ;
+QueryExecution qExec = conn.query("SELECT DISTINCT ?s { ?s ?p ?o }") ;
+ResultSet rs = qExec.execSelect() ;
+while(rs.hasNext()) {
+QuerySolution qs = rs.next() ;
+Resource subject = qs.getResource("s") ;
+System.out.println("Subject: "+subject) ;
+}
+qExec.close() ;
+conn.close() ;
+```
+
+Jena also provides a separate
+[SPARQL over JDBC
driver](http://jena.staging.apache.org/documentation/jdbc/index.html)
+library.
+
+## Transactions
+
+Transactions are the preferred way to work with RDF data.
+Operations on an `RDFConnection` outside of an application-controlled
+transaction will cause the system to add one for the duration of the
+operation. This "autocommit" feature may lead to inefficient operations due
+to excessive overhead.
+
+The `Txn` class provides a Java8-style transaction API. Transactions are
+code passed in the `Txn` library that handles the transaction lifecycle.
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+Txn.execWrite(conn, ()-> {
+conn.load("data1.ttl") ;
+conn.load("data2.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+}) ;
+}
+```
+
+The traditional style of explicit `begin`, `commit`, `abort` is also
available.
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+conn.begin(ReadWrite.WRITE) ;
+try {
+conn.load("data1.ttl") ;
+conn.load("data2.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+conn.commit() ;
+} finally { conn.end() ; }
+}
+```
+
+The use of `try-finally` ensures that transactions are properly finished.
+The `conn.end()` provides an abort in case an exception occurs in the
+transaction and a commit has not been issued. The use of `try-finally`
+ensures that transactions are properly finished.
+
+`Txn` is wrapping these steps up and calling the application supplied code
+for the transaction body.
+
+### Remote Transactions
+
+SPARQL does not define a remote transaction standard protocol. Each remote
+operation shuld be atomic (all happens or nothing happens) - this is the
+responsibility of the remote server.
+
+An `RDFConenction` will at least provide the client-side locking features.
+This means that overlapping operations that chnage data are naturally
+handled by the transaction pattern within a single JVM.
+
+## Graph Store Protocol
+
+The http://www.w3.org/TR/sparql11-http-rdf-update/
[GitHub] jena pull request #197: JENA-1267: RDFConnection
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/197#discussion_r92840274
--- Diff: jena-rdfconnection/Documentation.md ---
@@ -0,0 +1,205 @@
+# RDF Connection : SPARQL operations API
+
+`RDFConnection` provides a unified set of operations for working on RDF
+with SPARQL operations. It provides http://www.w3.org/TR/sparql11-query/;>SPARQL Query, http://www.w3.org/TR/sparql11-update/;>SPARQL Update and the http://www.w3.org/TR/sparql11-http-rdf-update/;>SPARQL Graph
+Store operations. The interface is uniform - the same interface
+applies to local data and to remote data using HTTP and the SPARQL
+protocols ( http://www.w3.org/TR/sparql11-protocol/;>SPARQL
+protocol and http://www.w3.org/TR/sparql11-http-rdf-update/;>SPARQL Graph Store
+Protocol).
+
+## Outline
+
+`RDFConnection` provides a number of different styles for working with RDF
+data in Java. It provides support for try-resource and functional code
+passing styles, as well the more basic sequence of methods calls.
+
+`try-resources` to manage the connection, and two operations, one to load
+some data, and one to make a query:
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+conn.load("data.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+}
+```
+This could have been written as (approximately -- the error handling is
better
+in the example above):
+
+```
+RDFConnection conn = RDFConnectionFactory.connect(...)
+conn.load("data.ttl") ;
+QueryExecution qExec = conn.query("SELECT DISTINCT ?s { ?s ?p ?o }") ;
+ResultSet rs = qExec.execSelect() ;
+while(rs.hasNext()) {
+QuerySolution qs = rs.next() ;
+Resource subject = qs.getResource("s") ;
+System.out.println("Subject: "+subject) ;
+}
+qExec.close() ;
+conn.close() ;
+```
+
+Jena also provides a separate
+[SPARQL over JDBC
driver](http://jena.staging.apache.org/documentation/jdbc/index.html)
+library.
+
+## Transactions
+
+Transactions are the preferred way to work with RDF data.
+Operations on an `RDFConnection` outside of an application-controlled
+transaction will cause the system to add one for the duration of the
+operation. This "autocommit" feature may lead to inefficient operations due
+to excessive overhead.
+
+The `Txn` class provides a Java8-style transaction API. Transactions are
+code passed in the `Txn` library that handles the transaction lifecycle.
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+Txn.execWrite(conn, ()-> {
+conn.load("data1.ttl") ;
+conn.load("data2.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+}) ;
+}
+```
+
+The traditional style of explicit `begin`, `commit`, `abort` is also
available.
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+conn.begin(ReadWrite.WRITE) ;
+try {
+conn.load("data1.ttl") ;
+conn.load("data2.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+conn.commit() ;
+} finally { conn.end() ; }
+}
+```
+
+The use of `try-finally` ensures that transactions are properly finished.
+The `conn.end()` provides an abort in case an exception occurs in the
+transaction and a commit has not been issued. The use of `try-finally`
+ensures that transactions are properly finished.
+
+`Txn` is wrapping these steps up and calling the application supplied code
+for the transaction body.
+
+### Remote Transactions
+
+SPARQL does not define a remote transaction standard protocol. Each remote
+operation shuld be atomic (all happens or nothing happens) - this is the
+responsibility of the remote server.
+
+An `RDFConenction` will at least provide the client-side locking features.
+This means that overlapping operations that chnage data are naturally
--- End diff --
Typo: chnage -> change
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #197: JENA-1267: RDFConnection
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/197#discussion_r92841987
--- Diff:
jena-rdfconnection/src/main/java/org/apache/jena/rdfconnection/RDFConnectionRemote.java
---
@@ -0,0 +1,478 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.rdfconnection;
+
+import static java.util.Objects.requireNonNull ;
+
+import java.io.File ;
+import java.io.InputStream ;
+import java.util.concurrent.locks.ReentrantLock ;
+import java.util.function.Supplier ;
+
+import org.apache.http.HttpEntity ;
+import org.apache.http.client.HttpClient ;
+import org.apache.http.entity.EntityTemplate ;
+import org.apache.http.protocol.HttpContext ;
+import org.apache.jena.atlas.io.IO ;
+import org.apache.jena.atlas.web.HttpException ;
+import org.apache.jena.atlas.web.TypedInputStream ;
+import org.apache.jena.graph.Graph ;
+import org.apache.jena.query.* ;
+import org.apache.jena.rdf.model.Model ;
+import org.apache.jena.rdf.model.ModelFactory ;
+import org.apache.jena.riot.* ;
+import org.apache.jena.riot.web.HttpCaptureResponse ;
+import org.apache.jena.riot.web.HttpOp ;
+import org.apache.jena.riot.web.HttpResponseLib ;
+import org.apache.jena.sparql.ARQException ;
+import org.apache.jena.sparql.core.DatasetGraph ;
+import org.apache.jena.sparql.core.Transactional ;
+import org.apache.jena.system.Txn ;
+import org.apache.jena.update.UpdateExecutionFactory ;
+import org.apache.jena.update.UpdateProcessor ;
+import org.apache.jena.update.UpdateRequest ;
+import org.apache.jena.web.HttpSC ;
+
+/**
+ *
+ */
+public class RDFConnectionRemote implements RDFConnection {
+private static final String fusekiDftSrvQuery = "sparql" ;
+private static final String fusekiDftSrvUpdate = "update" ;
+private static final String fusekiDftSrvGSP = "data" ;
+
+private boolean isOpen = true ;
+private final String destination ;
+private final String svcQuery ;
+private final String svcUpdate ;
+private final String svcGraphStore ;
+private HttpClient httpClient ;
+private HttpContext httpContext = null ;
+
+// Builder?
+// HttpContext, HttpClient.
+// Statics for "query", "query+update" : SparqlQueryConnectionRemote >
SparqlUpdateConnectionRemote > RDFConnectionRemote
+// XXX Very long "HttpOp.execHttpPost"
+
+/** Create connection that wil use the {@link HttpClient} using URL of
the dataset and default service names */
+public RDFConnectionRemote(HttpClient httpClient, String destination) {
+this(httpClient,
+ requireNonNull(destination),
+ fusekiDftSrvQuery,
+ fusekiDftSrvUpdate,
+ fusekiDftSrvGSP) ;
+}
+
+
+/** Create connection, using URL of the dataset and default service
names */
+public RDFConnectionRemote(String destination) {
+this(requireNonNull(destination),
+ fusekiDftSrvQuery,
+ fusekiDftSrvUpdate,
+ fusekiDftSrvGSP) ;
+}
+
+// ??
+/** Create connection, using full URLs for services. Pass a null for
"no service endpoint". */
+public RDFConnectionRemote(String sQuery, String sUpdate, String sGSP)
{
+this(null, sQuery, sUpdate, sGSP) ;
+}
+
+/** Create connection, using URL of the dataset and short names for
the services */
+public RDFConnectionRemote(String destination, String sQuery, String
sUpdate, String sGSP) {
+this(null, destination, sQuery, sUpdate, sGSP) ;
+}
+
+/** Create connection, using URL of the dataset and short names for
the services */
+public RDFConnectionRemote(HttpClient httpClient, String destina
[GitHub] jena pull request #197: JENA-1267: RDFConnection
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/197#discussion_r92841380
--- Diff:
jena-rdfconnection/src/main/java/org/apache/jena/rdfconnection/RDFConnectionFactory.java
---
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.jena.rdfconnection;
+
+import org.apache.jena.query.Dataset ;
+import org.apache.jena.system.JenaSystem ;
+
+// Pool stuff
+public class RDFConnectionFactory {
+static { JenaSystem.init(); }
+
+/** Create a connection to a remote location by URL.
+ * This is the URL for the dataset.
+ *
+ * This call assumes the names of services as:
+ *
+ * SPARQL Query endpoint : "sparql"
+ * SPARQL Update endpoint : "update"
+ * SPARQL Graph Store Protocol : "data"
+ *
+ * These are the default names in http://jena.apache.org/documentation/fuseki2;>Fuseki
+ * Other names can be specificied using {@link #connect(String,
String, String, String)}
+ *
+ * @param destination
+ * @return RDFConnection
+ * @see #connect(String, String, String, String)
+ */
+public static RDFConnection connect(String destination) {
+return new RDFConnectionRemote(destination) ;
+}
+
+/** Create a connection to a remote location by URL.
+ * This is the URL for the dataset.
+ *
+ * This call requires specifying the names of the service.
+ */
+public static RDFConnection connect(String queryServiceEndpoint,
+String updateServiceEndpoint,
+String graphStoreProtocolEndpoint)
{
+return new RDFConnectionRemote(queryServiceEndpoint,
updateServiceEndpoint, graphStoreProtocolEndpoint) ;
+ }
+
+
+/** Create a connection to a remote location by URL.
+ * This is the URL for the dataset.
+ *
+ * This call requires specifying the names of the servicerelative to
the dataset URL.
--- End diff --
Missing space
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #197: JENA-1267: RDFConnection
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/jena/pull/197#discussion_r92840841
--- Diff: jena-rdfconnection/Documentation.md ---
@@ -0,0 +1,205 @@
+# RDF Connection : SPARQL operations API
+
+`RDFConnection` provides a unified set of operations for working on RDF
+with SPARQL operations. It provides http://www.w3.org/TR/sparql11-query/;>SPARQL Query, http://www.w3.org/TR/sparql11-update/;>SPARQL Update and the http://www.w3.org/TR/sparql11-http-rdf-update/;>SPARQL Graph
+Store operations. The interface is uniform - the same interface
+applies to local data and to remote data using HTTP and the SPARQL
+protocols ( http://www.w3.org/TR/sparql11-protocol/;>SPARQL
+protocol and http://www.w3.org/TR/sparql11-http-rdf-update/;>SPARQL Graph Store
+Protocol).
+
+## Outline
+
+`RDFConnection` provides a number of different styles for working with RDF
+data in Java. It provides support for try-resource and functional code
+passing styles, as well the more basic sequence of methods calls.
+
+`try-resources` to manage the connection, and two operations, one to load
+some data, and one to make a query:
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+conn.load("data.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+}
+```
+This could have been written as (approximately -- the error handling is
better
+in the example above):
+
+```
+RDFConnection conn = RDFConnectionFactory.connect(...)
+conn.load("data.ttl") ;
+QueryExecution qExec = conn.query("SELECT DISTINCT ?s { ?s ?p ?o }") ;
+ResultSet rs = qExec.execSelect() ;
+while(rs.hasNext()) {
+QuerySolution qs = rs.next() ;
+Resource subject = qs.getResource("s") ;
+System.out.println("Subject: "+subject) ;
+}
+qExec.close() ;
+conn.close() ;
+```
+
+Jena also provides a separate
+[SPARQL over JDBC
driver](http://jena.staging.apache.org/documentation/jdbc/index.html)
+library.
+
+## Transactions
+
+Transactions are the preferred way to work with RDF data.
+Operations on an `RDFConnection` outside of an application-controlled
+transaction will cause the system to add one for the duration of the
+operation. This "autocommit" feature may lead to inefficient operations due
+to excessive overhead.
+
+The `Txn` class provides a Java8-style transaction API. Transactions are
+code passed in the `Txn` library that handles the transaction lifecycle.
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+Txn.execWrite(conn, ()-> {
+conn.load("data1.ttl") ;
+conn.load("data2.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+}) ;
+}
+```
+
+The traditional style of explicit `begin`, `commit`, `abort` is also
available.
+
+```
+try ( RDFConnection conn = RDFConnectionFactory.connect(...) ) {
+conn.begin(ReadWrite.WRITE) ;
+try {
+conn.load("data1.ttl") ;
+conn.load("data2.ttl") ;
+conn.querySelect("SELECT DISTINCT ?s { ?s ?p ?o }", (qs)->
+ Resource subject = qs.getResource("s") ;
+ System.out.println("Subject: "+subject) ;
+}) ;
+conn.commit() ;
+} finally { conn.end() ; }
+}
+```
+
+The use of `try-finally` ensures that transactions are properly finished.
+The `conn.end()` provides an abort in case an exception occurs in the
+transaction and a commit has not been issued. The use of `try-finally`
+ensures that transactions are properly finished.
+
+`Txn` is wrapping these steps up and calling the application supplied code
+for the transaction body.
+
+### Remote Transactions
+
+SPARQL does not define a remote transaction standard protocol. Each remote
+operation shuld be atomic (all happens or nothing happens) - this is the
+responsibility of the remote server.
+
+An `RDFConenction` will at least provide the client-side locking features.
+This means that overlapping operations that chnage data are naturally
+handled by the transaction pattern within a single JVM.
+
+## Graph Store Protocol
+
+The http://www.w3.org/TR/sparql11-http-rdf-update/
[GitHub] jena issue #180: JENA-1248: Fuseki registry update
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/180 Looks good to me My only concern is how Shiro behaves in a environment where each invaded server may have a different security configuration. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/151 The incorrect/ overridden plug-ins can certainly be removed. They are probably just cruft from the early days of development. Historically AspectJ was used in order to inject trace level logging into every single method in the JDBC code base. The logic behind this was that the intent was for the driver to be used with generic JDBC tools where attaching a debugger would have been difficult/impossible and so you would've had to rely on logging to do debugging. This probably doesn't apply any more so you may as well remove it. There may be associated entries in the license/notice that should also be removed. But this was three years ago so my memory maybe incorrect. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/151 @ajs6f provided that those examples are available at some point then this is fine with me. I'm not sure how we could incorporate test cases to this aspect because you would need an endpoint protected by forms authentication for the purposes of the tests. In theory you could configure shiro with Fuseki to do that but as Fuseki lacks a login page that wouldn't work in practice. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
