[GitHub] jena issue #148: JENA-508: implemented fn:adjust-datetime-to-timezone, fn:ad...
Github user afs commented on the issue: https://github.com/apache/jena/pull/148 Same function that checks for xsd:dateTime, xsd:date or xsd:time is the right thing to do. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #148: JENA-508: implemented fn:adjust-datetime-to-timezone, fn:ad...
Github user afs commented on the issue:
https://github.com/apache/jena/pull/148
It's a good thing it's summer where I am (British Summer Time)!
My reading of https://www.w3.org/TR/xpath-functions-3/#timezone.functions
is that conversions where there is no provided timezone use local time,
including daylight savings time.
In the tests `TimeZone.getDefault().getRawOffset()` returns the offset
without regard for the local daylight savings time.
Also allowing for timezones which are negative offsets, I came up with:
```
private String getDynamicDurationString(){
int tzOffset = TimeZone.getDefault().getOffset(new
Date().getTime())/(1000*60) ;
String x = "PT"+Math.abs(tzOffset)+"M";
if ( tzOffset < 0 )
x = "-"+x ;
return x ;
}
```
`XSDFuncOp.adjustDatetimeToTimezone` also needs this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #148: JENA-508: implemented fn:adjust-datetime-to-timezone...
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/148#discussion_r66244619
--- Diff:
jena-arq/src/main/java/org/apache/jena/sparql/expr/nodevalue/XSDFuncOp.java ---
@@ -1554,4 +1551,55 @@ private static Duration
valueCanonicalDuration(NodeValue nv) {
//dur = ...
return dur ;
}
+
+public static NodeValue adjustDatetimeToTimezone(NodeValue
nv1,NodeValue nv2){
+if(nv1 == null)
+return null;
+
+if(!nv1.isDateTime() && !nv1.isDate() && !nv1.isTime()){
+throw new ExprEvalException("Not a valid date, datetime or
time:"+nv1);
+}
+
+XMLGregorianCalendar calValue = nv1.getDateTime();
+Boolean hasTz = calValue.getTimezone() !=
DatatypeConstants.FIELD_UNDEFINED;
+int inputOffset = 0;
+if(hasTz){
+inputOffset = calValue.getTimezone();
+}
+
+int tzOffset = TimeZone.getDefault().getRawOffset() / (1000*60);
+if(nv2 != null){
--- End diff --
`int tzOffset = TimeZone.getDefault().getOffset(new
Date().getTime())/(1000*60) ;`
Ideally, don't set this with a call to `new Date()` until later -- see
below.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #148: JENA-508: implemented fn:adjust-datetime-to-timezone...
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/148#discussion_r66244682
--- Diff:
jena-arq/src/main/java/org/apache/jena/sparql/expr/nodevalue/XSDFuncOp.java ---
@@ -1554,4 +1551,55 @@ private static Duration
valueCanonicalDuration(NodeValue nv) {
//dur = ...
return dur ;
}
+
+public static NodeValue adjustDatetimeToTimezone(NodeValue
nv1,NodeValue nv2){
+if(nv1 == null)
+return null;
+
+if(!nv1.isDateTime() && !nv1.isDate() && !nv1.isTime()){
+throw new ExprEvalException("Not a valid date, datetime or
time:"+nv1);
+}
+
+XMLGregorianCalendar calValue = nv1.getDateTime();
+Boolean hasTz = calValue.getTimezone() !=
DatatypeConstants.FIELD_UNDEFINED;
+int inputOffset = 0;
+if(hasTz){
+inputOffset = calValue.getTimezone();
+}
+
+int tzOffset = TimeZone.getDefault().getRawOffset() / (1000*60);
+if(nv2 != null){
+if(!nv2.isDuration()) {
+String nv2StrValue = nv2.getString();
+if(nv2StrValue.equals("")){
+
calValue.setTimezone(DatatypeConstants.FIELD_UNDEFINED);
+if(nv1.isDateTime())
+return NodeValue.makeDateTime(calValue);
+else if(nv1.isTime())
+return
NodeValue.makeNode(calValue.toXMLFormat(),XSDDatatype.XSDtime);
+else
+return NodeValue.makeDate(calValue);
+}
+throw new ExprEvalException("Not a valid duration:" + nv2);
+}
+Duration tzDuration = nv2.getDuration();
--- End diff --
```} else {
tzOffset = TimeZone.getDefault().getOffset(new
Date().getTime())/(1000*60) ;
}
```
so `new Date` is only called if needed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #148: JENA-508: implemented fn:adjust-datetime-to-timezone...
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/148#discussion_r66245039
--- Diff:
jena-arq/src/main/java/org/apache/jena/sparql/expr/nodevalue/XSDFuncOp.java ---
@@ -1554,4 +1551,55 @@ private static Duration
valueCanonicalDuration(NodeValue nv) {
//dur = ...
return dur ;
}
+
+public static NodeValue adjustDatetimeToTimezone(NodeValue
nv1,NodeValue nv2){
+if(nv1 == null)
+return null;
+
+if(!nv1.isDateTime() && !nv1.isDate() && !nv1.isTime()){
+throw new ExprEvalException("Not a valid date, datetime or
time:"+nv1);
+}
+
+XMLGregorianCalendar calValue = nv1.getDateTime();
+Boolean hasTz = calValue.getTimezone() !=
DatatypeConstants.FIELD_UNDEFINED;
+int inputOffset = 0;
+if(hasTz){
+inputOffset = calValue.getTimezone();
+}
+
+int tzOffset = TimeZone.getDefault().getRawOffset() / (1000*60);
+if(nv2 != null){
+if(!nv2.isDuration()) {
+String nv2StrValue = nv2.getString();
+if(nv2StrValue.equals("")){
+
calValue.setTimezone(DatatypeConstants.FIELD_UNDEFINED);
+if(nv1.isDateTime())
+return NodeValue.makeDateTime(calValue);
+else if(nv1.isTime())
+return
NodeValue.makeNode(calValue.toXMLFormat(),XSDDatatype.XSDtime);
+else
+return NodeValue.makeDate(calValue);
+}
+throw new ExprEvalException("Not a valid duration:" + nv2);
+}
+Duration tzDuration = nv2.getDuration();
+tzOffset = tzDuration.getSign()*(tzDuration.getMinutes() +
60*tzDuration.getHours());
+if(tzDuration.getSeconds() > 0)
+throw new ExprEvalException("The timezone duration should
be an integral number of minutes");
+int absTzOffset = java.lang.Math.abs(tzOffset);
+if(absTzOffset > 14*60)
+throw new ExprEvalException("The timezone should be a
duration between -PT14H and PT14H.");
+}
+String tzSign = (tzOffset-inputOffset) > 0 ? "" : "-";
+Duration durToAdd =
NodeValue.makeDuration(tzSign+"PT"+java.lang.Math.abs(tzOffset-inputOffset)+"M").getDuration();
--- End diff --
Could this be done with something like
`NodeValue.xmlDatatypeFactory.createDurationDayTime(tzOffset-inputOffset)`?
It avoids string parsing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #148: JENA-508: implemented fn:adjust-datetime-to-timezone, fn:ad...
Github user afs commented on the issue: https://github.com/apache/jena/pull/148 Other than my comments above - this looks good to include. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #47: JENA-901 LPDRuleEngine cache Guava from jena-shadowed-ext
Github user afs commented on the issue: https://github.com/apache/jena/pull/47 +1 : Looks good to me. I'll merge the java changes if there are no other comments. The POM change isn't needed. jena-shaded-guava comes via jena-base. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #152: JENA-1158: Union default graph
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/152 JENA-1158: Union default graph This competes general handling of `urn:x-arq:UnionGraph`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena union-graph Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/152.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #152 commit 1b5d0fc7700e77d0bc62add733cf52dea208f68f Author: Andy Seaborne Date: 2016-07-07T20:46:32Z Reformat commit 193b1441500803d83cc37cd95f53f53e3534ff09 Author: Andy Seaborne Date: 2016-07-07T20:47:24Z Remove incorrect comment. commit ba00301f644d15783dd7fa39a6751c58e9603502 Author: Andy Seaborne Date: 2016-07-07T20:48:07Z Reformat. commit cfa17c12e0af352ee6e3a2ba5a614defc167e25f Author: Andy Seaborne Date: 2016-07-07T20:49:21Z JENA-1158: Provide the case of the union being all named graphs. commit 65e1228bb3847e109f55454874178120f308b3c9 Author: Andy Seaborne Date: 2016-07-07T20:50:08Z Fix formatting (minor). commit 145b7139ac1024fb6007c130ca12494bedad297d Author: Andy Seaborne Date: 2016-07-07T20:50:26Z Add comments. commit 9ddfccac29b4d7dbc17c920035045846b38523cc Author: Andy Seaborne Date: 2016-07-08T17:05:38Z JENA-1158: union default graph tests and clearup. commit e7e13f8cd085f7d98e87c94df5a54b2bfd130775 Author: Andy Seaborne Date: 2016-07-08T17:13:21Z Merge branch 'master' into union-graph --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #152: JENA-1158: Union default graph
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/152#discussion_r70122439
--- Diff: jena-arq/src/main/java/org/apache/jena/sparql/graph/GraphOps.java
---
@@ -29,60 +29,49 @@
import org.apache.jena.sparql.core.Quad ;
// Combine with Jena GraphUtils.
-public class GraphOps
-{
-
-public static boolean containsGraph(DatasetGraph dsg, Node gn)
-{
-// [[DynDS]]
-if ( Quad.isDefaultGraph(gn))
+public class GraphOps {
+public static boolean containsGraph(DatasetGraph dsg, Node gn) {
+if ( Quad.isDefaultGraph(gn) )
return true ;
-if ( Quad.isUnionGraph(gn))
+if ( Quad.isUnionGraph(gn) )
return true ;
return dsg.containsGraph(gn) ;
}
-
-public static Graph getGraph(DatasetGraph dsg, Node gn)
-{
-// [[DynDS]]
+
+public static Graph getGraph(DatasetGraph dsg, Node gn) {
if ( gn == null )
return dsg.getDefaultGraph() ;
if ( Quad.isDefaultGraph(gn) )
// Explicit or generated.
return dsg.getDefaultGraph() ;
-if ( Quad.isUnionGraph(gn))
+if ( Quad.isUnionGraph(gn) )
return unionGraph(dsg) ;
return dsg.getGraph(gn) ;
}
-
-public static Graph unionGraph(DatasetGraph dsg)
-{
+
+public static Graph unionGraph(DatasetGraph dsg) {
+// Snapshot it now.
--- End diff --
Yes, and it's just what was there before, though it does seem unlikely that
CCME happens and the alternative of a dynamic union makes sense as well.
I'll update the PR so there are two operations.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user afs commented on the issue: https://github.com/apache/jena/pull/151 I haven't got my head around the new API yet - this is just a preliminary comment. It is a bit of a change around authentication but I think it is the right thing to do. It seems to me that the only other choice is not to migrate to the new API which over time will become a nuisance. Documenting the migration (for the release notes) will be helpful - maybe an example for user/password in the HttpOp class javadoc. If someone can grok that, they can see how to migrate their own code. One thought - it looks like the new style is to treat `CloseableHttpClient` as a one time use object. That means a try-resource in `HttpOp.exec`. I'm not sure how widespread this is. `getDefaultHttpClient` `setDefaultHttpClient` should be changed to be a `HttpClientBuilder`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user afs commented on the issue: https://github.com/apache/jena/pull/151 Another thought: Does `HttpContext` with every `HttpClient` usage make an sense now? We now rely on `HttpClient` a lot more and `HttpContext` seems mainly for cookies. Maybe, versions of calls with `HttpClient` but without `HttpContext` might make sense or might not. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #149: Jena 1206
Github user afs commented on the issue: https://github.com/apache/jena/pull/149 +1 : Looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #153: JENA-1090: Txn - a java8-centric transaction API
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/153 JENA-1090: Txn - a java8-centric transaction API You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena txn Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/153.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #153 commit 952aebd0ac8f2d5d590811f0ac97d3c17c9bea5a Author: Andy Seaborne Date: 2016-07-09T21:40:03Z JENA-1090 : Txn - a java8-centric transaction API commit 52ff200c2756c9844b4346ffa1bff2a09081b649 Author: Andy Seaborne Date: 2016-07-09T21:41:01Z Merge branch 'master' of https://github.com/apache/jena into txn --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #153: JENA-1090: Txn - a java8-centric transaction API
Github user afs commented on the issue: https://github.com/apache/jena/pull/153 There is markdown documentation to go with this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #139: JSON-LD output
Github user afs commented on the issue:
https://github.com/apache/jena/pull/139
Apologies that this has taken so long.
> TODO some documentation. Where should it be added? The best form would be
small sample code.
Examples can go in `jena-arq/src-examples/arq/examples/riot/`. A new
section about JSON-LD output for
https://jena.apache.org/documentation/io/rdf-output.html, and use code links to
apache/jena github, master branch.
Symbols are fine but URI's please! (changing `RIOT.riotIRI` to
"http://jena.apache.org/riot";):
e.g. in JsonLDWriter
```
public static final Symbol JSONLD_CONTEXT =
Symbol.create("JSONLD_CONTEXT");
```
**RDFDataMgr**
The way to set things specially for a writing is to do:
```
WriterDatasetRIOT w =
RDFDataMgr.createDatasetWriter(RDFFormat.JSONLD_COMPACT_FLAT) ;
w.write(System.out, dataset.asDatasetGraph(), null, "http://base";,
RIOT.getContext()) ;
```
rather than add `RDFDataMgr.write(..., Context cxt)`
See
[examples/riot/ExRIOT_2](https://github.com/apache/jena/blob/master/jena-arq/src-examples/arq/examples/riot/ExRIOT_2.java)
for an example for a reader.
If you want, add a new API class specific to JSON output `JsonLdMgr` (to go
in the same package as `RDFDataMgr`) with JSON-LD-specific functions to wrap
common ways to call JSON-LD output.
**Constants**
This could be tidied out with a subclass of `RDFFormatVariant` that carries
JSON-LD information:
```
// From JSON writer
public static enum JSONLD_FORMAT {
COMPACT, FLATTEN, EXPAND, FRAME
}
static class JSONLDFormatVariant extends RDFFormatVariant {
private JSONLD_FORMAT option ;
private boolean prettyJson ;
public JSONLDFormatVariant(String name, boolean prettyJson,
JSONLD_FORMAT option) {
super(name) ;
this.options = option ;
this.prettyJson = prettyJson ;
}
public boolean prettyJson() { return prettyJson ; }
public boolean option(JSONLD_FORMAT fmt) {
for ( JSONLD_FORMAT f : options ) {
if ( fmt == f )
return true ;
}
return false ;
}
}
private static final RDFFormatVariant JSONLD_EXPAND_PRETTY = new
JSONLDFormatVariant("expand pretty", true, JSONLD_FORMAT.EXPAND) ;
private static final RDFFormatVariant JSONLD_EXPAND_FLAT= new
JSONLDFormatVariant("expand flat", false, JSONLD_FORMAT.COMPACT) ;
...
```
then use in JsonLDWriter by casting to and JSONLDFormatVariant
```
JSONLDFormatVariant format = (JSONLDFormatVariant)format ;
```
```
if ( format.option(COMPACT) ) {
...
}
```
**Tests**
I ran the tests for JSONLD but got:
```
10:13:04 WARN JsonLDWriter :: JSONLD_CONTEXT value is not a
String. Assuming a context object expected by JSON-LD JsonLdProcessor.compact
or flatten
10:13:04 INFO TestJsonLDWriter :: Sorry to get this exception
org.apache.jena.riot.RiotException: com.github.jsonldjava.core.JsonLdError:
loading remote context failed: http://schema.org/
at
org.apache.jena.riot.out.JsonLDWriter.serialize(JsonLDWriter.java:218)
at org.apache.jena.riot.out.JsonLDWriter.write(JsonLDWriter.java:123)
at org.apache.jena.riot.out.JsonLDWriter.write(JsonLDWriter.java:129)
at
org.apache.jena.riot.system.RiotLib$WriterAdapter.write(RiotLib.java:376)
at org.apache.jena.riot.RDFDataMgr.write$(RDFDataMgr.java:1232)
...
```
We need to be able to run the tests without assuing that external network
resources are availanle and functioning.
The tests and build should work if there is no extenral connectivity.
**Code organisation**
If you feel there classes, fell free to create a package
org.apache.jena.riot.out.jsonld to put them together.
**Other**
Indentation: spaces, not tabs please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user afs commented on the issue: https://github.com/apache/jena/pull/151 What would a sidecar parameter look like in this situation? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #95: JENA-626 SPARQL Query Caching
Github user afs commented on the issue: https://github.com/apache/jena/pull/95 Another use case: In my day-job, we have a local (same JVM) database because calling to a remote database (even same datacenter) is too expensive. RDF is used to drive the UI for HTTP/HTML requests. This means we get a lot of small calls into the database for every single HTTP request. Going off machine for each call does not work. Getting larger parts of the RDF (i.e. a few large remote calls) only gets us into having a local graph and many calls to that. We can of course add our own caching because we control the whole stack but it suggests a design whereby caching is done by the `QueryExecution` and not as part of Fuseki HTTP request handling, either specific to Fuseki or as a general feature in ARQ. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #139: JSON-LD output
Github user afs commented on the issue: https://github.com/apache/jena/pull/139 JIRA created to track this: https://issues.apache.org/jira/browse/JENA-1208 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #153: JENA-1090: Txn - a java8-centric transaction API
Github user afs commented on the issue: https://github.com/apache/jena/pull/153 Semaphore hand out permits so 0 means no permit. There is a `semaStart.release()` in `run()` and the corresponding `acquire` in the lambda in `create` which has comments. Renaming the local `semaStart` would help. The `AtomicReference` can be combined in `AtomicReference`. `ThreadTxn` came about for testing purposes - `Txn` is the main contribution. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #95: JENA-626 SPARQL Query Caching
Github user afs commented on the issue: https://github.com/apache/jena/pull/95 That's possible though they come into existence a bit late in the process of setting up the query. That are quite a few of possibilities up to an including making `QueryExecutionFactory` have a policy object where all the internal calls like `makePlan` become calls to a singleton object. For local: `QueryEngineFactory` is the interface - it can choose to handle a request or not via `accept` so a policy on what to cache is possible, (e.g. some datasets not others, some queries no others). It would accept whether it has it cached or will cache it and call back into setup. That might be special code in `QueryEngineRegistry.find`, a special policy for `QueryExecutionFactory` Or maybe as part of `QueryExecutionFactory.makePlan` looking aside to the cache or adding `QueryEngineFactory` to the usual dispatch chain. That seems cleaner in some ways but it'll need a way to to allow it to also call the `QueryEngineRegistry` to find the factory that will actually produce new results else it'll loop on the recursion. A note of caution: the dispatch chain does get modified to add and remove the reference query engine for example. For remote: To support Fuseki, we don't need remote caching. The remote case and the local case follow different paths but again a singleton policy at `QueryExecutionFactory.createServiceRequest/3` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #153: JENA-1090: Txn - a java8-centric transaction API
Github user afs commented on the issue: https://github.com/apache/jena/pull/153 If one of the transaction operations throws anything then the system is broken. It should not happen and would indicate an unrepairable error condition (e.g. NPE, broken journal). After that, all bets are off! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #149: Jena 1206
Github user afs commented on the issue: https://github.com/apache/jena/pull/149 I was assuming these vocabularies would be part of the main distribution and build. If so, the POM file is not needed (nor invoked). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #149: Jena 1206
Github user afs commented on the issue: https://github.com/apache/jena/pull/149 If we were starting from scratch, `vocabularies/w3c` makes sense. We already have `RDF`, `RDFS`, `OWL`, `SKOS` which are W3C vocabularies and they are in `vocabularies/`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #150: [JENA-1204] support for multiple BuiltinRegistry for differ...
Github user afs commented on the issue: https://github.com/apache/jena/pull/150 Ping @der --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #153: JENA-1090: Txn - a java8-centric transaction API
Github user afs commented on the issue: https://github.com/apache/jena/pull/153 Merged but this code always benefits from ongoing code review! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #149: Jena 1206
Github user afs commented on the issue: https://github.com/apache/jena/pull/149 I don't have any strong feelings - I'm just following the current pattern (for better or worse). I doubt we will have name clashes (independent vocabs - not versions). The minor downside of classifying by package is implicit domain. DCAT will be hopefully used much more widely than eGov. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #133: Support for try-with-resources Statements in ClosableIterat...
Github user afs commented on the issue: https://github.com/apache/jena/pull/133 This seems like a high-impact change (even though it will result in warnings not errors) for users and the code. I think we should not make this change. The option of new API over the same core is worthwhile pursuing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #133: Support for try-with-resources Statements in ClosableIterat...
Github user afs commented on the issue: https://github.com/apache/jena/pull/133 I have no idea! Maybe it's a revised current API, maybe it's influenced by different design patterns. Any API is more likely to work over Graph/Triple/Node than add another layer on Model. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user afs commented on the issue: https://github.com/apache/jena/pull/151 The need for closing responses isn't new - all typed input streams should be closed in the codebase currently because I had to go hunting them all down with the old `HttpOp` code. So, hopefully, theer are no issues here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user afs commented on the issue: https://github.com/apache/jena/pull/151 So to check my understanding: For each operation that takes a `HttpClient` there is one that takes a `HttpClient` and an `HttpContext`. Not so bad - we don't have the `HttpAutheticator` ones any more. Of course, if java had default parameters values like scala ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #154: JENA-1209: Exclusive mode for TDB
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/154 JENA-1209: Exclusive mode for TDB You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena tdb-exclusive Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/154.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #154 commit ee6a67864f784e6e2bbfc63eb8f96276a587fbe5 Author: Andy Seaborne Date: 2016-07-13T16:17:04Z JENA-1209: Exclusive mode for TDB --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user afs commented on the issue: https://github.com/apache/jena/pull/151 @rvesse so the proposal is operations that take a `HttpClient` and operations that take an `HttpClientBuilder` and `HttpAuthenticator`? (It seems odd that the can't get the setting out of an `HttpClient` to feed into a builder.) `HttpAuthenticator.apply` takes a `HttpClientBuilder`. That would work for me. We have to change the existing design because `HttpClient` is now immutable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user afs commented on the issue: https://github.com/apache/jena/pull/151 My understanding of the new 4.3+ API is that building per call should not be a burden. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #155: JENA-508: fn:format-number
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/155 JENA-508: fn:format-number Implementation and tests for `fn:format-number`. For the "decimal-format-name" third argument. F&O says it is something defined in the static context but does not provide any more guidance. You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena functions Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/155.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #155 commit 679129ce6fa4c87f477e69b28b7b59edbe41bd22 Author: Andy Seaborne Date: 2016-07-15T16:45:28Z fn:format-number --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #156: JENA-508: fn:replace
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/156 JENA-508: fn:replace fn:replace, which is REPLACE, as a custom function. You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena fn-replace Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/156.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #156 commit 0e36d5340373d48d72fcde65827f2b7e59857543 Author: Andy Seaborne Date: 2016-07-16T12:35:26Z JENA-508: fn:replace --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #157: Fixes for JENA-1212
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/157#discussion_r71170935
--- Diff:
jena-arq/src/main/java/org/apache/jena/atlas/data/SortedDataBag.java ---
@@ -200,9 +263,13 @@ public void flush()
if (!finishedAdding && memSize > 1)
{
// Again, some ugliness for speed
-Object[] array = memory.toArray();
-Arrays.sort(array, (Comparator)comparator);
-memory = Arrays.asList((E[])array);
+E[] array = (E[]) memory.toArray();
+if (comparator.abortableSort(array))
+{
+ // if we comment this back in, we lose the timeout
message!
+ // return Iter.nullIterator();
+}
--- End diff --
What's this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #157: Fixes for JENA-1212
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/157#discussion_r71314599
--- Diff:
jena-arq/src/main/java/org/apache/jena/atlas/data/SortedDataBag.java ---
@@ -69,19 +69,81 @@
protected final ThresholdPolicy policy;
protected final SerializationFactory serializationFactory;
-protected final Comparator comparator;
+protected final CanAbortComparator comparator;
protected boolean finishedAdding = false;
protected boolean spilled = false;
protected boolean closed = false;
+protected volatile boolean cancelled;
public SortedDataBag(ThresholdPolicy policy,
SerializationFactory serializerFactory, Comparator comparator)
{
this.policy = policy;
this.serializationFactory = serializerFactory;
-this.comparator = comparator;
+this.comparator = new CanAbortComparator(comparator);
}
+private final class CanAbortComparator implements Comparator
+ {
+ /**
+ The test for whether the sort has been cancelled is
+ performed every cancelTestFrequency
comparisons.
+ This reduces the (presumed) overhead of access to a
+ volatile boolean.
+ */
+ static final int cancelTestFrequency = 1;
+
+ /**
+ Count of the number of times this comparator has been
called.
+ */
+ int count = 0;
+
+ final Comparator baseComparator;
+
+ public CanAbortComparator(Comparator comparator)
+ {
+ this.baseComparator = comparator;
+ }
+
+ @Override public int compare(E o1, E o2)
+ {
+ count += 1;
+ if (count % cancelTestFrequency == 0)
+ {
+ if (cancelled) throw new AbandonSort();
+ }
+ return baseComparator.compare(o1, o2);
+ }
+
+ /**
+ Sort the array e using this comparator
+ with the additional ability to abort the sort.
+ */
+ public boolean abortableSort(E[] e) {
+ try { Arrays.sort(e, this); }
--- End diff --
Yes - with two notes: both are easily testable:
What happens if the comparator throws an exception on a separate thread (I
expect the std library handles this somehow)?
Is there a risk that one sort impacts the whole server?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #157: Fixes for JENA-1212
Github user afs commented on the issue: https://github.com/apache/jena/pull/157 or be fancy and use "AtomicBoolean" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #157: Fixes for JENA-1212
Github user afs commented on the issue: https://github.com/apache/jena/pull/157 I thin it is better to consider them separately - first, get cancellable sorts into the code base, then look at `Arrays.parallelSort`. `Arrays.parallelSort` is going to need some care to make sure it does not degrade concurrent requests and that is it properly behaves when cancelled. What I'd like to know is how the current PR behaves in a live system. As it builds on an existing facility, we can't take a "experimental feature" approach. 1. Does it completely address the motivating use case in production usage? (May be it is only partial - we want to make one set of changes on master, not a series changes.) 2. Does it lead to any advere/different effects? (including performance comparisons though I doubt it will have an impact, because it replaces, it would be good to know before it goes into becomes the master branch.) I hope @ehedgehog can run tests in the with a modified Fuseki in a staging setup for the system(s) that illustrated the problem in the first place. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #157: Fixes for JENA-1212
Github user afs commented on the issue: https://github.com/apache/jena/pull/157 It does not need logging, and definitely not at level `INFO` - cancellation is a normal feature (and should cause an exception) and Jena is a library. Libraries should not log at `INFO` in normal use. Consider Fuseki - it logs a cancelled request anyway, catching the query cancellation exception. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #158: JENA-1211 "trix"
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/158 JENA-1211 "trix" Accept "TriX" and "trix" elements Write "trix". "TriX" is the element name in the HPL techreport. "trix" is the element name in the W3C DTD and schema. You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena jena-1211-trix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/158.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #158 commit bcd7e400975b45c36d40636ee06f7930dfbac374 Author: Andy Seaborne Date: 2016-07-24T10:07:13Z JENA-1211: Read both 'TriX' and 'trix'; write 'trix' commit 3cc29462470f86c07e12581611f0e67ff1cc2a61 Author: Andy Seaborne Date: 2016-07-24T11:04:34Z Remove unnecessary suppressed warnings. commit ea83371ee2938fa7828802889067500d5a384818 Author: Andy Seaborne Date: 2016-07-24T11:23:51Z JENA-1211: Tests commit 0927884002ec8f42c198616eb9b1ee92cd8ba331 Author: Andy Seaborne Date: 2016-07-24T11:24:24Z Misplaced start/finish - fix for streaming oputput --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #159: JENA-1210: Call model.read
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/159 JENA-1210: Call model.read You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena filemanager-ontimports Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/159.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #159 commit 66f1126eb1e0ac62ab947bda29886f7779eaf81c Author: Andy Seaborne Date: 2016-07-15T16:07:56Z Combine FileManger-style model.read with RDFDataMgr style processing. Call mode.read tso that it can be overriden by OntModlImpl which does "loadImports". commit 9e976b64b88f237ca9d82fe69b8b3de6ef31d434 Author: Andy Seaborne Date: 2016-07-24T19:08:31Z JENA-1210: Call model.read in AdapterFileManager. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #157: Fixes for JENA-1212
Github user afs commented on the issue: https://github.com/apache/jena/pull/157 Looks OK. `TestSortedDataBagCancellation` has a warning of missing `@Override`. Some formatting/whitespace issue issues: 1. Tabs used - not spaces. 2. `AbortableComparator` - old style layout. 3. `QueryIterSort` is reformatted to current style but tabs used and assumed to be 8. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #159: JENA-1210: Call model.read
Github user afs commented on the issue: https://github.com/apache/jena/pull/159 That will go in circles! Plain `model.read` goes to `RDFReaderRIOT` (The old RDFReader interface adapter to RIOT) which makes all calls go to `RDFDataMgr.read(model,...)`. If that calls `model.read` we have a loop. `OntModel.read` overrides `Model.read` and does "get basic data" via a call to `super.read` (which is `ModelCom.read`) and then "load imports". So the layering is that RDFDataMgr does the basic read operations, with the indirection of `StreamManager`/`LocationMapping`. The key here is going via `FileManager.get().readModel` bypasses an overridable `model.read`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #159: JENA-1210: Call model.read
Github user afs commented on the issue: https://github.com/apache/jena/pull/159 There's a comment in `AdapterFileManager.readModelWorker` about override of `Model.read`. `RDFDataMgr` does say at the top "Read data from a location into a Model/Dataset etc" so it's not promising anything more. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #157: Fixes for JENA-1212
Github user afs commented on the issue: https://github.com/apache/jena/pull/157 Various dev@ email. It's the java code conventions with spaces not tabs (so code displays in github!), indent of 4, opening brace on same line (these are the eclipse defaults or very close to them IIRC). No trailing whitespace if possible. Use common sense to make it readable for the next person. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #159: JENA-1210: Call model.read
Github user afs commented on the issue: https://github.com/apache/jena/pull/159 No problem. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #157: Fixes for JENA-1212
Github user afs commented on the issue: https://github.com/apache/jena/pull/157 http://jena.apache.org/getting_involved/reviewing_contributions.html --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user afs commented on the issue: https://github.com/apache/jena/pull/151 I came across "preemptive authentication" for HttpClient which apparently does not a preconfigured `HttpClient`. [HttpComponents 4.5.x tutorial section, 4.6](https://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/authentication.html#d5e717) It works pre-populating the authentication data cache and calling: ``` CloseableHttpResponse response = httpclient.execute(target, http-request, HttpClientContext); ``` It's not quite the style of `org.apache.jena.atlas.web.auth.HttpAuthenticator` but it's close. Would this be better than `HttpClientBuilder`+`HttpAuthenticator`? Mu first impression is that it is much the same - just different. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #157: Fixes for JENA-1212
Github user afs commented on the issue: https://github.com/apache/jena/pull/157 +1 to merging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #157: Fixes for JENA-1212
Github user afs commented on the issue:
https://github.com/apache/jena/pull/157
It's sorting time inside of Fuseki that will really count. A few % is OK
for me - optimizing other things, e.g. the comparison process of
{{BindingComparator}} itself, is better to recover any loss.
I am a bit surprised the code makes any difference because the comparison
of two items is rather flexible ... which means not cheap.
I agree with @ajs6f - let's merge it and the see what's what such as
folding the cancellability into {{BindingComparator}} is one option.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #160: JENA-1189: Resolve problems with jena-jdbc-driver-tdb tests...
Github user afs commented on the issue: https://github.com/apache/jena/pull/160 I ran the PR on Linux and the build still builds. +1 to merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #133: Support for try-with-resources Statements in ClosableIterat...
Github user afs commented on the issue: https://github.com/apache/jena/pull/133 @rwm : The impact is not solely on Jena codebase but also on all the application code written to the Jena API. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #161: Plain begin
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/161 Plain begin This PR is for discussion and is not yet ready for merging. It adds to TDB the ability for a read transaction to promote to a write transaction. Currently (to avoid general API changes outside TDB) this happens automatically in `DatasetGraphTransaction.getW()`. It needs to be enabled with `DatasetGraphTransaction.promotion = true` If this is useful, we can add new modes to `ReadWrite` and/or add `begin()` (no args). There is a big design point: at the point at which a transaction becomes a writer their are two choices as to what to do if another write transaction happened between this one starting and this one promoting. 1. The transaction can now see changes made by the other writer. A limited form of "read committed" behaviour. Downside: results from actions during the read phase may be invalidated. 1. The transaction is blocked from promoting. This is purer (results are not invalidated; an error is returned) but the code must deal with it. Having it as a choice is possible - one should be the default for `begin()`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena plain-begin Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/161.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #161 commit 0bd93192b54c818852c033bcc644ccc6169e0abf Author: Andy Seaborne Date: 2016-07-16T19:51:57Z Split different categories of get() : R, W, G and T. commit 1d6c6b91e5453e36ec423f1f54aed584dfac8f18 Author: Andy Seaborne Date: 2016-07-16T19:53:16Z Typo commit 5c507cc5bcd28a87c79550657e1e45301e02579e Author: Andy Seaborne Date: 2016-07-16T19:53:50Z Remove pointless private constructor. commit 99d99e802c91d8e14afe4d66c189ae15c6b6fbdb Author: Andy Seaborne Date: 2016-07-16T19:55:19Z Typo commit 8afea3b2972837ce6c7abb68512b7842fd2e0fc3 Author: Andy Seaborne Date: 2016-07-16T19:55:45Z Make some methods package access. commit 66425363776e9071497d70bd50e07be94bf3bb4c Author: Andy Seaborne Date: 2016-07-16T19:58:00Z Promotable TDB transactions. commit d00f3b48b65c22dc5e010b7efa2ccd15801b81d1 Author: Andy Seaborne Date: 2016-07-17T09:38:24Z Remove unused operations. commit af12f3b7f042925f51fec7a0b5324af1d4a8ef09 Author: Andy Seaborne Date: 2016-07-17T13:34:25Z Graphs across transaction boundaries. commit 58002d91b999aa44e838d23e4e713f14e1006884 Author: Andy Seaborne Date: 2016-07-17T16:21:23Z Do not cache default model. Assumes too much about the DatasetGraph. commit 4c1847b02d7023bd39c76e35cbfaca9bdb191948 Author: Andy Seaborne Date: 2016-07-17T16:22:17Z Refactor GraphTDB. commit b08c60ce65ccfcf114b0cf21c3c96f0fee987b5f Author: Andy Seaborne Date: 2016-07-18T15:24:58Z Switchable promotion. Select txn/non-txn graph versions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin
Github user afs commented on the issue:
https://github.com/apache/jena/pull/161
Example:
```
public static void dsgTxn() {
DatasetGraphTransaction.promotion = true ;
DatasetGraph dsg = TDBFactory.createDatasetGraph() ;
Quad q1 = SSE.parseQuad("(_ :s :p1 1)") ;
dsg.begin(ReadWrite.READ);
// Start R->W here
dsg.add(q1) ;
dsg.commit();
dsg.end() ;
Txn.execRead(dsg, ()->{
RDFDataMgr.write(System.out, dsg.getDefaultGraph(), Lang.NT);
});
System.out.println("DONE") ;
}
```
NB Current `Txn` does not work for promotion - it needs to `commit()`, not
just `end()` a read transaction.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 `DatasetGraphTransaction.promotion = true` is only needed for this illustrative PR. It means for exploration, so you can swap modes. It will removed when the code is real. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 I'd like to get his code into master even if it is switched off. If we are comfortable with it the general way to go, it's easier to have it live. If not enabled, the code should make no difference to current behaviour. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 The proposal is not to adopt every detail of this PR; it's to include but not activate it. I think we should try to add the feature to all the transactional implementations (TDB, TIM, MRSW - not SDB) together. Having this in the code base, not activated by default, is a step on that road. Automatically allowing writes is normal behaviour for JDBC which has implicit ``begin()``. Exceptions don't work well. Some transaction-unaware library code might make the update causing the transaction to promote. An exception will crash out of the middle of the library code. The name `ReadWrite` may not be a good choice if start adding more modes. Also, we need to check how old code ignores a new value. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #162: JENA-1221: Embedded Fuseki server
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/162 JENA-1221: Embedded Fuseki server This add a new module under jena-fuseki2 for running Fuseki embedded. It is a new module so it can have it's own POM. The POM excludes all unecessary dependencies from jena-fuseki-core including logging provider, and modules jena-text and jena-spatial. You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena fuseki-embedded Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/162.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #162 commit c809629d096ec5279913638952c03716ea1117d1 Author: Andy Seaborne Date: 2016-08-06T13:49:36Z JENA-1221: Embedded Fuseki server --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #162: JENA-1221: Embedded Fuseki server
Github user afs commented on the issue:
https://github.com/apache/jena/pull/162
Examples:
```
/** Create a SPARQL endpoint for an application dataset and start the
server */
private static void example0() {
DatasetGraph dsg = dataset() ;
// Run a Fuseki server with "/ds" as the dataset.
// Default set up : query, update, graph store and quads
operations.
FusekiEmbeddedServer.make(3330, "/ds", dsg).start() ;
}
```
```
/** Create a SPARQL endpoint for an application dataset */
private static void example1() {
DatasetGraph dsg = dataset() ;
// Run a Fuseki server with "/ds" as the dataset.
// Default set up : query, update, graph store and quads
operations.
FusekiEmbeddedServer server = FusekiEmbeddedServer.create()
.setPort()
.add("/ds", dsg)
.build() ;
server.start() ;
}
```
```
/** Create a Fuseki server with a just a SPAQRL query endpoint.
* The SPARQL endpoint URLs look like {@code /rdf/sparql?query=}
*/
private static void example2() {
DatasetGraph dsg = dataset() ;
DataService queryService = new DataService(dsg) ;
queryService.addEndpoint(OperationName.Query, "sparql");
FusekiEmbeddedServer server = FusekiEmbeddedServer.create()
.setPort(3334)
.add("/rdf", queryService)
.build() ;
server.start() ;
// Sync with the server - this is blocking.
server.join() ;
}
```
```
/** Create a Fuseki server with two sets of services. One is the usual
set of read-only endpoints,
* the other is just being able to do quads operations
* GET, POST, PUT on "/ds2" in N-quads and TriG.
*/
private static void example3() {
DatasetGraph dsg = DatasetGraphFactory.createTxnMem() ;
// A service with just being able to do quads operations
// That is, GET, POST, PUT on "/ds2" in N-quads and TriG.
DataService dataService = new DataService(dsg) ;
dataService.addEndpoint(OperationName.Quads_RW, "");
FusekiEmbeddedServer server = FusekiEmbeddedServer.create()
.setPort(3335)
.add("/ds2", dataService)
.build() ;
server.start() ;
}
```
```
/** Create a Fuseki server with some services on the dataset URL. */
private static void example4() {
DatasetGraph dsg = DatasetGraphFactory.createTxnMem() ;
// A service able to do quads operations and SPARQL query
DataService dataService = new DataService(dsg) ;
dataService.addEndpoint(OperationName.Quads_RW, "");
dataService.addEndpoint(OperationName.Query, "");
FusekiEmbeddedServer server = FusekiEmbeddedServer.create()
.setPort(3336)
.add("/data", dataService)
.build() ;
server.start() ;
}
```
```
/** Create a Fuseki server by reading a configuration file. */
private static void example5() {
FusekiEmbeddedServer server = FusekiEmbeddedServer.create()
.setPort(3337)
// Defines /ds
.parseConfigFile("config.ttl")
.build() ;
server.start() ;
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #162: JENA-1221: Embedded Fuseki server
Github user afs commented on the issue:
https://github.com/apache/jena/pull/162
Example of the application sharing the dataset with Fuseki. This also shows
using java.util.logging for application and embedded Fuseki.
```
public static void example6() {
LogCtl.setJavaLogging();
Logger LOG = LoggerFactory.getLogger("example6") ;
DatasetGraph dsg = DatasetGraphFactory.createTxnMem() ;
FusekiEmbeddedServer server = FusekiEmbeddedServer.make(3330,
"/ds", dsg) ;
server.start() ;
LOG.info("Remote 1") ;
try (QueryExecution qExec =
QueryExecutionFactory.sparqlService("http://localhost:3330/ds/query";, "SELECT *
{ ?s ?p ?o}") ) {
// Empty table
QueryExecUtils.executeQuery(qExec);
}
// Transaction-protected update.
Txn.execWrite(dsg, ()->{
Quad q = SSE.parseQuad("(_ :s :p _:b)") ;
dsg.add(q);
}) ;
LOG.info("Remote 2") ;
try (QueryExecution qExec =
QueryExecutionFactory.sparqlService("http://localhost:3330/ds/query";, "SELECT *
{ ?s ?p ?o}") ) {
// One row.
QueryExecUtils.executeQuery(qExec);
}
server.stop() ;
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #162: JENA-1221: Embedded Fuseki server
Github user afs commented on the issue: https://github.com/apache/jena/pull/162 Yes, that's the idea! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #162: JENA-1221: Embedded Fuseki server
Github user afs commented on the issue: https://github.com/apache/jena/pull/162 Draft documentation: https://gist.github.com/afs/c5e025e55a1d9773f323bb050e4d9d39 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 Eventually, a ``begin()`` would start a transaction in a way that is promotable. This fits the expectation of JDBC (it's even implicit there) and help bring the existing graph-level `TransactionHandler` into line (an existing API so the barrier to change is higher, albeit an API that isn't likely to be used much as it does not automatically fit with dataset transactions). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #161: Plain begin - read transactions promoting to write t...
Github user afs closed the pull request at: https://github.com/apache/jena/pull/161 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 Proposal: the default behaviour is "no read commited" for `begin()` (behavior 2 in the description) with an option to call `setReadCommited` (or some such name). And/or a `readCurrent` which effectively resets a read transaction to see the current (up to last commit) state. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 (mis-closed) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #161: Plain begin - read transactions promoting to write t...
GitHub user afs reopened a pull request: https://github.com/apache/jena/pull/161 Plain begin - read transactions promoting to write transactions. This PR is for discussion and is not yet ready for merging. It adds to TDB the ability for a read transaction to promote to a write transaction. Currently (to avoid general API changes outside TDB) this happens automatically in `DatasetGraphTransaction.getW()`. It needs to be enabled with `DatasetGraphTransaction.promotion = true` If this is useful, we can add new modes to `ReadWrite` and/or add `begin()` (no args). There is a big design point: at the point at which a transaction becomes a writer their are two choices as to what to do if another write transaction happened between this one starting and this one promoting. 1. The transaction can now see changes made by the other writer. A limited form of "read committed" behaviour. Downside: results from actions during the read phase may be invalidated. 1. The transaction is blocked from promoting. This is purer (results are not invalidated; an error is returned) but the code must deal with it. Having it as a choice is possible - one should be the default for `begin()`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena plain-begin Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/161.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #161 commit c3edbd4cd942ea8bac9c4dcda777ae45c594113d Author: Andy Seaborne Date: 2016-07-16T19:51:57Z Split different categories of get() : R, W, G and T. commit 05982a420e06fb99a8607027e42af0f8ad1690e1 Author: Andy Seaborne Date: 2016-07-16T19:53:16Z Typo commit d754ac30a34546b728982ba41437b3989b2e6590 Author: Andy Seaborne Date: 2016-07-16T19:53:50Z Remove pointless private constructor. commit bbdc31f4314718e402eaf0aa4aa3a20d64eef904 Author: Andy Seaborne Date: 2016-07-16T19:55:19Z Typo commit 82ceeb9b52bff7c3b5bd82c6a9980c57569f28e9 Author: Andy Seaborne Date: 2016-07-16T19:55:45Z Make some methods package access. commit 8ba77665fecb6d66e5caa54106b3cf9311786cd1 Author: Andy Seaborne Date: 2016-07-16T19:58:00Z Promotable TDB transactions. commit a041b52e3b9632a7db3149fbc441ae1c24bcdc95 Author: Andy Seaborne Date: 2016-07-17T09:38:24Z Remove unused operations. commit f7ac6e318351a16aa034a8c6462262ce91c7d395 Author: Andy Seaborne Date: 2016-07-17T13:34:25Z Graphs across transaction boundaries. commit 4273ac58700b87d0786124b14a76840204e80663 Author: Andy Seaborne Date: 2016-07-17T16:21:23Z Do not cache default model. Assumes too much about the DatasetGraph. commit 188e860c7ff6bf681fa77c801175e1bcd9936717 Author: Andy Seaborne Date: 2016-07-17T16:22:17Z Refactor GraphTDB. commit ad9f87adf05fb5da0f00b339fbe14312fba93253 Author: Andy Seaborne Date: 2016-07-18T15:24:58Z Switchable promotion. Select txn/non-txn graph versions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 See JENA-1223 for overall discussion of API issues and to record changes in subsystems. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 Squash development commits to a few important ones. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 After that fix, the tests have reasonable coverage of all modes so this is ready to merge from my point-of-view. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 To track whether a promotion is possible for fully serialized, the code now checks a version number. The necessary information is available elsewhere (whether a new transaction would see the same dataset as one wanting promotion) but it's crypt and in TIM, that's in the indexes, not the transaction control in the dataset. Keeping the version is also a nice statistic to keep. The version moves on one when a transaction commits. That leaves one case - what to do when a writer is already active. That writer may commit later (and hence the promotion is not possible) to abort (the promotion may be possible if no other writer comes and goes). The current refuses promotion if there is an active writer on the basis that most writers commit, and abort is unusual. Blocking maybe possible but leads to issues of what happens when another reader that tries to promote at about the same time and of a long running writer (e.g. bulk upload) causing the reader to block for a long time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 Yes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 Version number only tell you about the past (see ``TransactionManager.notifyCommit`` where the version number increases - this is as the writer finishes up). When a writer is active and has not committed or aborted. So how do you wait for the active writer to decide which way it's going to go? (what would be block on to wait for it?) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #161: Plain begin - read transactions promoting to write transact...
Github user afs commented on the issue: https://github.com/apache/jena/pull/161 Let's get this into the codebase first (inactive). JENA-1222 and JENA-1224 are both changes to TDB's ``TransactionManager`` and juggling multiple branches of nearby code changes is not nice. I've tried to write a test for snapshot isolation promotion but it relies on a thread pause which elsewhere has been unstable on a heavily loaded CI server (`TestTransPromote.promote_clash_active_writer`). At the moment, there is no special code for the "active writer then abort" case. It may be possible but if it looks like a major change to TDB1, then it may not be worth and may be a sign that the specific feature is generally something that warps the rest of the system. Until we get the contract details settled, changing TIM seems premature. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #164: JENA-1222 and JENA-1224: Flush queue when large by b...
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/164 JENA-1222 and JENA-1224: Flush queue when large by byte size or by pending commits. This adds additional control over the queue in the journal. If the journal gets large, as measured by the journal file size in bytes (JENA-1222) then try to flush the journal. If the journal becomes excessive long in terms of pending commits (JENA-1224) then after a writer, flip to exclusive mode, flush the journal and flip back to normal mode. You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena tdb-txn Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/164.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #164 commit 6e09208e449ec8b37cd573c0427e31e3fde2017b Author: Andy Seaborne Date: 2016-08-21T19:12:21Z JENA-1222: Try to flush if the journal is large (in bytes). commit 166b6e9ccf1b4d4b0b93b49444a2534a083c09c4 Author: Andy Seaborne Date: 2016-08-21T19:48:37Z JENA-1224: Flush based on trying to keep below a max size. commit 68965e054b5df7e262d377c27bb7a39f08755d6e Author: Andy Seaborne Date: 2016-08-21T19:51:48Z Tests for JENA-1222 and JENA-1224. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #164: JENA-1222 and JENA-1224: Flush queue when large by b...
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/164#discussion_r75635595
--- Diff:
jena-tdb/src/main/java/org/apache/jena/tdb/transaction/TransactionManager.java
---
@@ -613,21 +652,43 @@ public void flush() {
}
// -- The main operations to undertake when a transaction finishes.
-// Called from TSM_WriteBackEndTxn but the worker code is shere so all
+// Called from TSM_WriteBackEndTxn but the worker code is here so all
// related code, including queue flushing is close together.
private void readerFinishesWorker(Transaction txn) {
-if ( queue.size() >= QueueBatchSize )
+if ( checkForJournalFlush() )
processDelayedReplayQueue(txn) ;
}
-
+
private void writerAbortsWorker(Transaction txn) {
-if ( queue.size() >= QueueBatchSize )
+if ( checkForJournalFlush() )
processDelayedReplayQueue(txn) ;
}
+// Whether to try to flush the journal. We may stil find that we are
blocked
+// from doing so by another transaction.
+private boolean checkForJournalFlush() {
+//System.err.printf("checkForJournalFlush: queue size=%d; journal
size = %d\n", queue.size(), journal.size()) ;
+//System.err.printf("checkForJournalFlush: QueueBatchSize=%d;
MaxQueueThreshold=%d; JournalThresholdSize=%d\n",
+// QueueBatchSize, MaxQueueThreshold,
JournalThresholdSize) ;
+if ( QueueBatchSize == 0 )
--- End diff --
True - remains of when JENA-1224 (excessive queue length was handled here
... bit that didn't work out). Tiding up done in the next commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #165: JENA-1228: Expose each optimization transform step i...
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/165 JENA-1228: Expose each optimization transform step in std optimizer. https://issues.apache.org/jira/browse/JENA-1228 You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena optimize-reorg Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/165.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #165 commit 91744594aeb3e55d9e2574b6e23fd24a77fedf2a Author: Andy Seaborne Date: 2016-08-29T11:49:10Z JENA-1228: Expose each optimization transform step as a protected method. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #165: JENA-1228: Expose each optimization transform step i...
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/165#discussion_r76614257
--- Diff:
jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Optimize.java ---
@@ -18,82 +18,58 @@
package org.apache.jena.sparql.algebra.optimize;
-import org.apache.jena.query.ARQ ;
import org.apache.jena.sparql.ARQConstants ;
-import org.apache.jena.sparql.SystemARQ ;
-import org.apache.jena.sparql.algebra.* ;
-import org.apache.jena.sparql.algebra.op.OpLabel ;
+import org.apache.jena.sparql.algebra.Op ;
import org.apache.jena.sparql.engine.ExecutionContext ;
import org.apache.jena.sparql.util.Context ;
-import org.apache.jena.sparql.util.Symbol ;
-import org.slf4j.Logger ;
-import org.slf4j.LoggerFactory ;
-
-public class Optimize implements Rewrite
+/** Optimization of algebra expressions.
+ *
+ * New optimization processes can be installed via a global change:
+ *
+ *Optimize.setFactory((cxt)->new MyOptimize(cxt)) ;
+ * or on a per-context basis:
+ *
+ *Optimize.RewriterFactory f = (cxt)->new MyOptimize(cxt) ;
+ *context.set(ARQConstants.sysOptimizerFactory, f) ;
+ */
+public class Optimize
{
-static private Logger log = LoggerFactory.getLogger(Optimize.class) ;
-
-// A small (one slot) registry to allow plugging in an alternative
optimizer
+/** Factory for rewriters */
--- End diff --
What difference does it make? It's an interface; there is no
static/enclosing distinction.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #139: JSON-LD output
Github user afs commented on the issue:
https://github.com/apache/jena/pull/139
I'm not opposed to adding "Context" variants on principle, just a concern
about warping the API in the application writers view by many operations (not
that there are a few at the moment!). There are 22 write(...) operations
currently. Or maybe only for the 11 `RDFFormat` forms (the Lang/RDFFormat being
one of the causes of so many options)
Adding the PrefixMap variants (4 new operations : 2 each for graph and
datasetgraph writers).
This should work and it's a bit shorter: the machinery for
DatasetGarph/Graph is built-in:
```
private void writeExample(OutputStream out, Model m, RDFFormat f, Context
jenaContext) {
WriterGraphRIOT w = RDFDataMgr.createGraphWriter(f) ;
Graph g = m.getGraph() ;
w.write(out, g, RiotLib.prefixMap(g), null, jenaContext);
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena issue #166: Address Jena-1229, fuseki timeout queries not working prope...
Github user afs commented on the issue: https://github.com/apache/jena/pull/166 You're lucky! The additional first commits c88e008 and 18d28f5 don't contaminate the diff. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #166: Address Jena-1229, fuseki timeout queries not working prope...
Github user afs commented on the issue: https://github.com/apache/jena/pull/166 In the list of commits above, the first commit says "ehedgehog committed on GitHub on 27 Jul" then a merge from jena/master 2 days ago, then the JENA-1229 change. Just a guess: you created a branch, then merged jena/master, rather then getting your local master up to date then branching. But it seems github works it out because if you go to the "166.diff" URL, then there is a short diff, no merge items. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #167: Javadocs
Github user afs commented on the issue: https://github.com/apache/jena/pull/167 There are 5 commits above but only 0eb58ba is for EnhGraph. This has the changes for PR#166 in it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #167: Javadocs
Github user afs commented on the issue: https://github.com/apache/jena/pull/167 I have manually applied the suggested changes - the diff of just EnhGraph didn't apply for me but it's easy enough to read. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #166: Address Jena-1229, fuseki timeout queries not working prope...
Github user afs commented on the issue: https://github.com/apache/jena/pull/166 I have fixed those 3 requestSubCancel instances. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #168: JENA-1231: Modernize executeInTransaction
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/168 JENA-1231: Modernize executeInTransaction You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena graph-txn-upgrade Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/168.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #168 commit 3c736de398862e21b33d577261d6a107c36b16c2 Author: Andy Seaborne Date: 2016-09-08T15:38:02Z JENA-1231: Modernize executeInTransaction --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #168: JENA-1231: Modernize executeInTransaction
Github user afs commented on the issue: https://github.com/apache/jena/pull/168 The second commit, https://github.com/apache/jena/pull/168/commits/1d6a4758de9305d9fe25e08b7594bbda8bd36a83, uses execute/calculate style naming. For discussion (the PR would be tidied up before merging into Jena). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #168: JENA-1231: Modernize executeInTransaction
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/168#discussion_r78088121
--- Diff:
jena-core/src/main/java/org/apache/jena/graph/impl/TransactionHandlerBase.java
---
@@ -36,21 +40,45 @@ public TransactionHandlerBase() {
*/
@Override
public Object executeInTransaction(Command c) {
+return executeRtn( () -> c.execute() ) ;
+}
+
+/* Abort but don't let problems with the transaction system itself
cause loss of the exception */
+private void abort(Throwable e) {
+try { abort() ; }
+catch (Throwable th) { e.addSuppressed(th); }
+}
+
+/**
+ * Execute the runnable action within a transaction. If
it completes normally,
+ * commit the transaction, otherwise abort the transaction.
+ */
+@Override
+public void execute( Runnable action ) {
begin() ;
-Object result ;
try {
-result = c.execute() ;
+action.run();
commit() ;
-return result ;
}
catch (JenaException e) { abort(e) ; throw e ; }
catch (Throwable e) { abort(e) ; throw new JenaException(e) ; }
}
-
-/* Abort but don't let problems with the transaction system itself
cause loss of the exception */
-private void abort(Throwable e) {
-try { abort() ; }
-catch (Throwable th) { e.addSuppressed(th); }
+
+/**
+ * Execute the supplier action within a transaction. If
it completes normally,
+ * commit the transaction and return the result, otherwise abort the
transaction.
+ */
+@Override
+public T executeRtn( Supplier action ) {
+begin() ;
+T result ;
--- End diff --
Yes - that would be better. It was just old stuff, continued.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #169: JENA-1235: Filter placement and OpDisjunction
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/169 JENA-1235: Filter placement and OpDisjunction This provides filter placement for OpDisjunction (a generalization of OpUnion generated internally e.g. for expanding `||` in filters). You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena jena-1235-place-disjunction Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/169.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #169 commit ec02e99e264694817244fa815933f4cc377bb51c Author: Andy Seaborne Date: 2016-09-15T11:34:14Z JENA-1235: Filter placement and OpDisjunction --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #139: JSON-LD output
Github user afs commented on the issue: https://github.com/apache/jena/pull/139 So how many additional 'write+Context' operations? Just the RDFFormat ones for now? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #170: JENA-1223: Txn promotion for TIM (includes patch for...
GitHub user afs opened a pull request: https://github.com/apache/jena/pull/170 JENA-1223: Txn promotion for TIM (includes patch for JENA-1237) Machinery for transaction promotion for the in-memory transactional dataset. You can merge this pull request into a Git repository by running: $ git pull https://github.com/afs/jena promote-mem Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/170.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #170 commit cc2c4bd54c15b958dede6b4e83e6ee440a3157c8 Author: Andy Seaborne Date: 2016-09-17T15:51:25Z JENA-1237: Capture transaction snapshot during begin. commit f9f2870e29fd1ac51c49740a42f6fbb644b7616d Author: Andy Seaborne Date: 2016-09-17T16:02:50Z Clean up. commit 51175d27768955ed1918b79ea7f046723e5c3173 Author: Andy Seaborne Date: 2016-09-17T16:03:31Z JENA-1237: Tests (TDB and TIM) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #170: JENA-1223: Txn promotion for TIM (includes patch for JENA-1...
Github user afs commented on the issue: https://github.com/apache/jena/pull/170 This first part of the PR is for JENA-1237. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #171: Using library code in a few classes
Github user afs commented on the issue: https://github.com/apache/jena/pull/171 Given PR #139 in-progress, can we hold back on the JSON-LD changes for now? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #171: Using library code in a few classes
Github user afs commented on a diff in the pull request:
https://github.com/apache/jena/pull/171#discussion_r80602233
--- Diff: jena-arq/src/main/java/org/apache/jena/atlas/test/Gen.java ---
@@ -95,19 +84,13 @@
}
if ( !found )
System.err.printf("Corrupted permute: [%s] [%s]\n",
strings(x), strings(x2)) ;
-}
+}*/
--- End diff --
Delete commented out code? (we have git!)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request #171: Using library code in a few classes
Github user afs commented on a diff in the pull request: https://github.com/apache/jena/pull/171#discussion_r80601919 --- Diff: jena-arq/src/main/java/org/apache/jena/atlas/test/Gen.java --- @@ -32,52 +38,35 @@ return rand(numRand, low, high, false) ; } -/** Generate a random sequence between low (inclusive) and high (exclusive) - choose whether to have duplicates or not */ +/** + * Generate a random sequence between low (inclusive) and high (exclusive) - with duplicates or no --- End diff -- Truncated javadoc -- was "or not" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #170: JENA-1223: Txn promotion for TIM (includes patch for JENA-1...
Github user afs commented on the issue: https://github.com/apache/jena/pull/170 Is this good to go in the codebase? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #151: JENA-576: Moving away from deprecated HttpCommons Client AP...
Github user afs commented on the issue: https://github.com/apache/jena/pull/151 Any reason not to apply this? It is in danger of becoming a bit of blocker on release 3.1.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #139: JSON-LD output
Github user afs commented on a diff in the pull request: https://github.com/apache/jena/pull/139#discussion_r81471041 --- Diff: jena-arq/pom.xml --- @@ -85,8 +84,9 @@ - com.github.jsonld-java - jsonld-java + com.github.jsonld-java +jsonld-java +0.8.3 --- End diff -- `` not needed - the dependency management in jena-parent controls the version. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #139: JSON-LD output
Github user afs commented on a diff in the pull request: https://github.com/apache/jena/pull/139#discussion_r81471021 --- Diff: jena-arq/pom.xml --- @@ -21,7 +21,7 @@ jena-arq jar Apache Jena - ARQ (SPARQL 1.1 Query Engine) - 3.1.1-SNAPSHOT + 3.1.2-SNAPSHOT --- End diff -- Needs to be 3.1.1-SNAPSHOT --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #139: JSON-LD output
Github user afs commented on a diff in the pull request: https://github.com/apache/jena/pull/139#discussion_r81471051 --- Diff: jena-arq/pom.xml --- @@ -85,8 +84,9 @@ - com.github.jsonld-java - jsonld-java + com.github.jsonld-java --- End diff -- Indentation changed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
