To cite my own tweet from last year: "What I don't like about SHACL is that it mixes instance constructors with constraints. Which is nicely separated in SPIN. These are orthogonal matters IMO." Isn't that the case?
And what is the maturity of the SHACL support? Does Jena have it built-in now? Re. VALUES, yes I guess there will be more complexity, but shouldn't the performance improve compared to the current SPINRDF implementation which loops all instances of the constrained class and executes a query for each of them? https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L604 On Tue, May 26, 2020 at 1:07 PM Andy Seaborne <[email protected]> wrote: > > > > On 24/05/2020 10:45, Martynas Jusevičius wrote: > > Andy, > > > > but Jena itself supports initial bindings in QueryExecutionFactory? > > https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/QueryExecutionFactory.html#create-java.lang.String-org.apache.jena.query.QuerySolution- > > Should that method be avoided or deprecated? > > It depends on the replacement. maybe the API is right even if the corner > cases of the effect changes. > > > Related https://afs.github.io/substitute.html > but not the way forward if we are to cover remote as well as local queries. > > > Any example of how that could be replaced with QueryTransformOps? > > > > What I'm considering is taking only the things we use (constraints and > > constructors) and implementing them using plain SPARQL, ripping out > > stuff like SPIN functions and rules. > > And SHACL? > > Because then micro-checks are fast (in Java) e.g. minCount, maxCount. > The SPARQL overhead would be significant andthe suggestion of VALUE > batching is added complexity. > > > > > I guess I would need to start with a simple test suite. > > > > On Sun, May 24, 2020 at 12:08 AM Andy Seaborne <[email protected]> wrote: > >> > >> Martynas, > >> > >> Two things: > >> > >> 1:: IIRC it is the queries that back various features that need to be > >> more stable. Depending on the internal details of ARQ isn't a good place > >> to be. Note that SpinDRF has it's own function and property function > >> evaluation subsystem with slightly different semantics. > >> > >> This is the problem you are encountering. > >> > >> 2:: The effect is initial binding isn't well defined. > >> Syntax rewriting is going to be more stable (and works remotely which > >> initial bindings can't). > >> > >> Andy > >> > >> On 22/05/2020 19:55, Martynas Jusevičius wrote: > >>> Andy, > >>> > >>> which part needs to be replaced with syntax rewriting -- SPIN > >>> functions or the initial bindings? > >>> > >>> Do you think it's possible to rewrite SPINRDF in a more SPARQL > >>> compliant and less ARQ-dependent way? > >>> > >>> On Thu, May 21, 2020 at 10:54 AM Andy Seaborne <[email protected]> wrote: > >>>> > >>>> > >>>> > >>>> On 20/05/2020 22:11, Martynas Jusevičius wrote: > >>>>> https://github.com/spinrdf/spinrdf/issues/22 > >>>>> > >>>>> On Wed, May 20, 2020 at 10:57 PM Martynas Jusevičius > >>>>> <[email protected]> wrote: > >>>>>> > >>>>>> Andy, > >>>>>> > >>>>>> I was able to isolate a standalone example: > >>>>>> https://github.com/namedgraph/spinrdf-test > >>>>>> > >>>>>> With 3.0.1 it succeeds. Change Jena version to 3.16.0-SNAPSHOT by > >>>>>> uncommenting the other <version> in pom.xml and it will fail: > >>>>>> https://github.com/namedgraph/spinrdf-test/blob/master/pom.xml#L28 > >>>>>> > >>>>>> I don't understand how or where spl:objectCount() and spl:instanceOf() > >>>>>> are executed, but i think it demonstrates clearly that some behaviour > >>>>>> has changed between those versions, even though SpinRDF code hasn't. > >>>> > >>>> Many SpinRDF functions are written with implicit assumptions about how > >>>> ARQ - not SPARQL - executes. They don't work when the optimizer is off, > >>>> reconfigured, or if the reference query engine is used. > >>>> > >>>> In addition, the external injection of initial bindings > >>>> > >>>> (Long term - that should be replaced with syntax rewriting of the query > >>>> - see QueryTransformOps) > >>>> > >>>> The result is that each version of SpinRDF needs to be tuned to the > >>>> version of Jena it uses. > >>>> > >>>> Andy > >>>> > >>>>>> > >>>>>> Will post on https://github.com/spinrdf/spinrdf/issues as well. > >>>>>> > >>>>>> > >>>>>> Martynas > >>>>>> > >>>>>> On Wed, May 20, 2020 at 12:37 PM Martynas Jusevičius > >>>>>> <[email protected]> wrote: > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> I've narrowed down the differences to the counts produced by > >>>>>>> spl:objectCount() SPIN function. > >>>>>>> > >>>>>>> SELECT * > >>>>>>> WHERE > >>>>>>> { ?this a ?TYPE_CLASS > >>>>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, > >>>>>>> ?predicate) > >>>>>>> AS ?objCount) > >>>>>>> FILTER ( bound(?minCount) && ( ?objCount < ?minCount ) ) > >>>>>>> } > >>>>>>> UNION > >>>>>>> { BIND(<http://spinrdf.org/spl#objectCount>(?this, > >>>>>>> ?predicate) > >>>>>>> AS ?objCount) > >>>>>>> FILTER ( bound(?maxCount) && ( ?objCount > ?maxCount ) ) > >>>>>>> } > >>>>>>> UNION > >>>>>>> { FILTER bound(?valueType) > >>>>>>> ?this ?predicate ?value > >>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, > >>>>>>> ?valueType) ) > >>>>>>> } > >>>>>>> } > >>>>>>> > >>>>>>> Bindings: ( ?predicate, spin:body )( ?comment, "the body of the > >>>>>>> Template" )( ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, > >>>>>>> 1 ) > >>>>>>> > >>>>>>> 3.15.0-SNAPSHOT > >>>>>>> > >>>>>>> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > >>>>>>> | this > >>>>>>> | TYPE_CLASS > >>>>>>> | > >>>>>>> objCount | predicate | value | > >>>>>>> ================================================================================================================================================================================================= > >>>>>>> | <http://spinrdf.org/spin#Templates> > >>>>>>> | > >>>>>>> <http://spinrdf.org/spin#Template> | > >>>>>>> 2 | <http://spinrdf.org/spin#body> | | > >>>>>>> | <http://spinrdf.org/spin#ConstructTemplates> > >>>>>>> | > >>>>>>> <http://spinrdf.org/spin#Template> | > >>>>>>> 2 | <http://spinrdf.org/spin#body> | | > >>>>>>> | > >>>>>>> <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectUpdateTemplate> > >>>>>>> | <http://spinrdf.org/spin#Template> | 2 | > >>>>>>> <http://spinrdf.org/spin#body> | | > >>>>>>> | <http://spinrdf.org/spin#AskTemplates> > >>>>>>> | > >>>>>>> <http://spinrdf.org/spin#Template> | > >>>>>>> 2 | <http://spinrdf.org/spin#body> | | > >>>>>>> | > >>>>>>> <https://github.com/AtomGraph/Processor/blob/develop/http-tests/custom#DefaultSubjectQueryTemplate> > >>>>>>> | <http://spinrdf.org/spin#Template> | 2 | > >>>>>>> <http://spinrdf.org/spin#body> | | > >>>>>>> | <http://spinrdf.org/spin#UpdateTemplates> > >>>>>>> | > >>>>>>> <http://spinrdf.org/spin#Template> | > >>>>>>> 2 | <http://spinrdf.org/spin#body> | | > >>>>>>> | <http://spinrdf.org/spl#Argument> > >>>>>>> | > >>>>>>> <http://spinrdf.org/spin#Template> | > >>>>>>> 2 | <http://spinrdf.org/spin#body> | | > >>>>>>> | <http://spinrdf.org/spin#SelectTemplates> > >>>>>>> | > >>>>>>> <http://spinrdf.org/spin#Template> | > >>>>>>> 2 | <http://spinrdf.org/spin#body> | | > >>>>>>> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > >>>>>>> > >>>>>>> 3.0.1 > >>>>>>> > >>>>>>> ---------------------------------------------------- > >>>>>>> | this | TYPE_CLASS | objCount | predicate | value | > >>>>>>> ==================================================== > >>>>>>> ---------------------------------------------------- > >>>>>>> > >>>>>>> I'll try to create an example now. > >>>>>>> > >>>>>>> > >>>>>>> On Tue, May 19, 2020 at 6:14 PM Andy Seaborne <[email protected]> wrote: > >>>>>>>> > >>>>>>>> That's a long jump. > >>>>>>>> > >>>>>>>> (and it's a SpinRDF question) > >>>>>>>> > >>>>>>>> >> Are there some obvious suspects here or do I need to create a > >>>>>>>> >> reproducible example? > >>>>>>>> > >>>>>>>> reproducible example (noting that the query has custom functions > >>>>>>>> which > >>>>>>>> have names that suggest they are not proper "functions" (i.e. their > >>>>>>>> return is not a function of their arguments alone and they access the > >>>>>>>> data graph .. which might be a query). > >>>>>>>> > >>>>>>>> I can't see why it might now have answers when it didn't before. > >>>>>>>> > >>>>>>>> Andy > >>>>>>>> > >>>>>>>> On 19/05/2020 13:04, Martynas Jusevičius wrote: > >>>>>>>>> With 3.0.1, no such internal *violations* are produced. > >>>>>>>>> > >>>>>>>>> On Tue, May 19, 2020 at 2:03 PM Martynas Jusevičius > >>>>>>>>> <[email protected]> wrote: > >>>>>>>>>> > >>>>>>>>>> Hi, > >>>>>>>>>> > >>>>>>>>>> After upgrading our code as well as SPINRDF from 3.0.1 to > >>>>>>>>>> 3.16.0-SNAPSHOT, some constraint violation-related tests are > >>>>>>>>>> failing. > >>>>>>>>>> > >>>>>>>>>> With 3.16.0-SNAPSHOT, it seems that a bunch of constraint > >>>>>>>>>> violations > >>>>>>>>>> are coming from the SPIN vocabulary iself, e.g. "Attribute > >>>>>>>>>> spin:body : > >>>>>>>>>> [0,1]" on spin:ConstructTemplates, spin:SelectTemplates etc. > >>>>>>>>>> With 3.0.1, no such internal validations are produced. > >>>>>>>>>> > >>>>>>>>>> The models being validated use a basic RDFS inference (subclassing > >>>>>>>>>> etc.) > >>>>>>>>>> > >>>>>>>>>> I have tracked the source of the difference to query execution in > >>>>>>>>>> SPINConstraints.runQueryOnClass(): > >>>>>>>>>> https://github.com/spinrdf/spinrdf/blob/master/src/main/java/org/spinrdf/constraints/SPINConstraints.java#L614 > >>>>>>>>>> > >>>>>>>>>> Since SPINRDF code hasn't changed, only the Jena versions, could > >>>>>>>>>> there > >>>>>>>>>> be differences in query execution? The specific query that produces > >>>>>>>>>> different results is this: > >>>>>>>>>> > >>>>>>>>>> CONSTRUCT > >>>>>>>>>> { > >>>>>>>>>> _:c0 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> > >>>>>>>>>> <http://spinrdf.org/spin#ConstraintViolation> . > >>>>>>>>>> _:c0 <http://spinrdf.org/spin#violationRoot> ?this . > >>>>>>>>>> _:c0 <http://spinrdf.org/spin#violationPath> ?predicate . > >>>>>>>>>> } > >>>>>>>>>> WHERE > >>>>>>>>>> { ?this a ?TYPE_CLASS > >>>>>>>>>> { FILTER ( bound(?minCount) && ( > >>>>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) < > >>>>>>>>>> ?minCount ) > >>>>>>>>>> ) } > >>>>>>>>>> UNION > >>>>>>>>>> { FILTER ( bound(?maxCount) && ( > >>>>>>>>>> <http://spinrdf.org/spl#objectCount>(?this, ?predicate) > > >>>>>>>>>> ?maxCount ) > >>>>>>>>>> ) } > >>>>>>>>>> UNION > >>>>>>>>>> { FILTER bound(?valueType) > >>>>>>>>>> ?this ?predicate ?value > >>>>>>>>>> FILTER ( ! <http://spinrdf.org/spl#instanceOf>(?value, > >>>>>>>>>> ?valueType) ) > >>>>>>>>>> } > >>>>>>>>>> } > >>>>>>>>>> > >>>>>>>>>> The query string and initial bindings are the same with both > >>>>>>>>>> versions, > >>>>>>>>>> for example: > >>>>>>>>>> > >>>>>>>>>> ( ?predicate, spin:body )( ?comment, "the body of the Template" )( > >>>>>>>>>> ?minCount, 0 )( ?TYPE_CLASS, spin:Template )( ?maxCount, 1 ) > >>>>>>>>>> > >>>>>>>>>> Are there some obvious suspects here or do I need to create a > >>>>>>>>>> reproducible example? > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> > >>>>>>>>>> Martynas
