[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 Branch created: [JENA-632-before-merge](https://github.com/kinow/jena/tree/JENA-632-before-merge). Won't delete it this year, in case we need to revisit. Followed that by: ``` $ git rebase -i HEAD~18 $ # fixed-up all but the first commit $ git diff JENA-632-before-merge $ # empty response, code is identical even though different commit tree $ git checkout master $ git merge JENA-632-2 $ git commit --amend $ # added This closes #114 message to close this pull request $ mvn clean test install -Pdev $ git push git-at-apache master ``` ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user afs commented on the issue: https://github.com/apache/jena/pull/114 That works for me. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 :tada: should I proceed with the merging? If so, I will squash the 18 commits into a single one before, as said before, but with a backup branch just-in-case. And then start working on the documentation in order to update the ticket in JIRA. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 @afs also removed the `Number()` clause as that was not being used any more. Ready for review again :+1: And big thanks for all the feedback (and patience!) so far. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue:
https://github.com/apache/jena/pull/114
Rebase'd against master. Added `TestJsonEval.java`, with a couple more
tests. Then understood why the projected variables were not working.
Updated the grammar to call the method that populates the `projectedVars`
collection. Now queries like `JSON { 'F' : 'string' } WHERE { }` resolve to `[
{ 'F' : 'string' } ]`.
Still need to work around the other comment about `Double`s.
---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user afs commented on the issue: https://github.com/apache/jena/pull/114 If it does not make the release then - whatever! - there'll be another along soon! I may be able to do some work on it after the release. I think I now understand it better and it makes a useful contribution to developing applications that want JSON simply. Have fun in Japan! ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user afs commented on the issue: https://github.com/apache/jena/pull/114 Eval tests: https://gist.github.com/afs/9e57b705a0d53889606346f6723238ce Framework : needs more tests! ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 @afs had a quick look at your gist and it looks good! I won't be able to have a proper look at it and to commit for the next ~8 hours. But feel free to commit to this branch and update the pull request too if you prefer :-) ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 And only started working again on this because you bumped this PR @ajs6f :-) learning a lot working on this ticket, and with JENA-1488, hope to start contributing back to Jena more now. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 Will apply some of the changes tonight. But I'm afraid I may not be able to fix the serialization issues before this weekend when there might be a new release. This weekend I'll be in Japan for some weeks, so may have to complete working on the other comments by mid-April ð¢ ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user afs commented on the issue: https://github.com/apache/jena/pull/114 Syntax tests: https://gist.github.com/afs/e069478ec6839267256b9eacd30aa2c4 ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 Replied @afs `s last comment, and also rebased the code. Even though it's rebased, there's quite a few commits... let me know if I should squash the commits or if it's preferable to leave the commit history :-) ready to review, take 3? ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114   Screenshots of Fuseki running some JSON queries. Everything seems to be working fine :tada: ready for review, take 2 ! ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user afs commented on the issue: https://github.com/apache/jena/pull/114 Great timing! but as ever, it it makes it, it makes it, and if it doesn't, it doesn't. (I will not be able to build 3.7.0 until next week anyway.) PR #383 merged. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 Most of the tests are done, just need to add at least basic coverage for the SPARQL changes. Then will spend some minutes playing with the queries in Fuseki, before updating the PR. Aiming to push the changes Friday night NZ time, or this weekend. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user afs commented on the issue: https://github.com/apache/jena/pull/114 PR #383 - we can still change it based on experience. BTW The grammar accepts the JSON field names in single quotes to make embedding in strings easier. It's a STRING1, either one `"` or one `'` delimiting (not mixed!). I doubt it will need many tests. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue:
https://github.com/apache/jena/pull/114
FWIW, the output of the second test right now is
```
{
"s" : "s" ,
"p" : "" ,
"o" : "def@en"
}
{
"s" : "s" ,
"p" : "" ,
"o" : "abc"
}
{
"s" : "s" ,
"p" : "" ,
"o" : "123^^http://www.w3.org/2001/XMLSchema#decimal;
}
```
@afs are you planning to send another PR when the draft is ready (it looks
good to me as-is now :-) ) or would you prefer to include in this PR and
Jena-632? It does seem to be related with/necessary for JENA-632. Happy to get
updates to this branch :-)
---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue:
https://github.com/apache/jena/pull/114
+1 for the current draft
I would use it
[here](https://github.com/kinow/jena/blob/a948f300fb40701e49e83db9d03e5d83da8145f6/jena-arq/src/main/java/org/apache/jena/sparql/engine/QueryExecutionBase.java#L358
and
[here](https://github.com/kinow/jena/blob/a948f300fb40701e49e83db9d03e5d83da8145f6/jena-arq/src/main/java/org/apache/jena/sparql/engine/JsonIterator.java#L71).
This way instead of looking only at the literals, we'd have the output from
the table above.
So far I have just two unit tests in my working copy
```java
@Test public void testExecJson() {
// JENA-632
Query query = QueryFactory.create("JSON { \"s\": ?s , \"p\": ?p ,
\"o\" : ?o } "
+ "WHERE { ?s ?p ?o }", Syntax.syntaxARQ);
try ( QueryExecution qexec = QueryExecutionFactory.create(query, m)
) {
JsonArray jsonArray = qexec.execJson();
assertNotNull( jsonArray );
assertEquals(3, jsonArray.size());
}
}
@Test public void testExecJsonItems() {
// JENA-632
Model model = ModelFactory.createDefaultModel();
{
Resource r = model.createResource(AnonId.create("s"));
Property p = model.getProperty("");
RDFNode node = ResourceFactory.createTypedLiteral("123",
XSDDatatype.XSDdecimal);
model.add(r, p, node);
r = model.createResource(AnonId.create("s"));
p = model.getProperty("");
node = ResourceFactory.createTypedLiteral("abc",
XSDDatatype.XSDstring);
model.add(r, p, node);
r = model.createResource(AnonId.create("s"));
p = model.getProperty("");
node = ResourceFactory.createLangLiteral("def", "en");
model.add(r, p, node);
}
Query query = QueryFactory.create("JSON { \"s\": ?s , \"p\": ?p ,
\"o\" : ?o } "
+ "WHERE { ?s ?p ?o }", Syntax.syntaxARQ);
try ( QueryExecution qexec = QueryExecutionFactory.create(query,
model) ) {
Iterator execJsonItems = qexec.execJsonItems();
while(execJsonItems.hasNext()) {
JsonObject next = execJsonItems.next();
System.out.println(next);
}
}
```
Where the second test is doing - more or less - the same example you
provided in your previous comment. Once we have the conversion in place, I will
update it to confirm I get what's expected.
Then after that I was thinking about quickly running with jacoco to see how
much of this PR is being tested.
---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user afs commented on the issue: https://github.com/apache/jena/pull/114 Conversion code (draft): 2dd4418a22 (on a branch in my GH at the moment). The code is in `RDFTerm2Json.fromNode`. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 Going to check how to get the correct values with JSON. Great catch, as I was using just Strings. But shouldn't be too hard. Will also include the initial tests with the next changes, covering both previous features, and the conversion from the table you provided :-) Thanks! ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user afs commented on the issue: https://github.com/apache/jena/pull/114 Tests? ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user afs commented on the issue:
https://github.com/apache/jena/pull/114
I applied 114.patch locally and it integrated just fine.
I tried `JSON { "s": ?s , "p": ?p , "o" : ?o } WHERE { ?s ?p ?o }` on data
```
:s :p 123 .
:s :p "abc" .
:s :p "def"@en .
```
and got:
```
[ { "o" : "123^^http://www.w3.org/2001/XMLSchema#integer; } ,
{ "o" : "abc" } ,
{ "o" : "def@en" }
]
```
I think we should use a more aggressive version of the conversion from RDF
term to JS used in the upcoming [JS custom
Functions](http://jena.staging.apache.org/documentation/query/javascript-functions.html#arguments-and-function-results),
code
https://github.com/apache/jena/blob/master/jena-arq/src/main/java/org/apache/jena/sparql/function/js/NV.java.
That works on `NodeValue`s to make functions more natural to write and is
two-way.
For JSON-SELECT we only need a one way conversion and getting info out even
if it is some string is better IMO.
| RDF | Javascript |
| | |
| String | String |
| XSD numeric | JSON number |
| XSD boolean | boolean |
| String@lang | String, no lang |
| URI | String |
| "lex"^^SomeDatatype | String, no datatype |
| Unbound | JSON null |
If the app wants RDF term details, it can use
`application/result-set+json`. This feature provides provides a simpler, more
JSONy view of the data for consuming RDF-unaware applications.
---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 Done, and ready for review again. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue:
https://github.com/apache/jena/pull/114
Rebased!
Tested the `ARQParser` main method (which is added here, but I find very
handy for quickly testing it, but happy to remove if others prefer). And got
the following:
```
Enter input: JSON { "name": ?name } WHERE { ?name ?a ?b } LIMIT 3
log4j:WARN No appenders could be found for logger (Jena).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
Parsed query successfully!
---
Enter input:
```
Then did a `mvn install -DskipTests -Pdev`, and copied
`apache-jena-fuseki-3.7.0-SNAPSHOT.tar.gz` elsewhere. Unpacked it, and started
`fuseki-server`.
Used an in-memory dataset in Fuseki2, loading the `books.ttl` example
shipped with Jena code.
Finally, used the query found in the JIRA ticket to retrieve the JSON
result:
```
PREFIX purl:
PREFIX :
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 Rebasing again, now manually, testing in Eclipse. Previous branch before the GitHub rebase archived at https://github.com/kinow/jena/tree/JENA-632-3 ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 Rebased using GitHub's interface :tada: (never used it before, hope didn't make any silly mistakes :-) ). Good points @afs ! I noticed the existing SPARQL to JSON result writers, so adjusted a bit the code to use that. Now when I try to rebase against the master branch I get a bunch of conflicts, but GitHub web interface seems to indicate it's all good. I was trying to find a simple way to rebase it, but if I can't fix the conflicts and compile & test it, I'll just revert to what was before and then just manually rebase :) ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user afs commented on the issue: https://github.com/apache/jena/pull/114 The JSON ecosystem moves fast! GraphQL is now significant and how that might be used is also very interesting. There are different ways to use GraphQL and maybe it is not the solution to all needs anyway - this ticket is more about result sets in easy-to-consume JSON than other work in the GraphQL-RDF area and more about JSON services. To move this ticket forward, how about putting the feature in with a big label saying "Experimental - subject to change" and _really_ mean it? By getting it out, it generates feedback, requirements and insight. You can ask for result sets in JSON but the JSON (http://www.w3.org/TR/sparql11-results-json/) or JSON-LD graphs but that isn't what general data consumers would consider to be "easy-to-consume". Is the RDF literal to JSON aligned with: [RDF-JS mapping](http://jena.staging.apache.org/documentation/query/javascript-functions.html#arguments-and-function-results)? (which only came along recently). ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 Hey @ajs6f , I think so. My weak memory seems to tell me that the last time I did that... shortly after, there was some work on JSON (JSON-LD? Or maybe pure JSON) somewhere else in Jena; and when I saw that happening, I wondered if that would perhaps be a replacement for the functionality in this ticket. I wonder if you or @afs would be able to confirm that there isn't already another way to get JSON responses in Jena, and that this feature is still a nice to have feature, please? :) ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user ajs6f commented on the issue: https://github.com/apache/jena/pull/114 Hey, @kinow , is this ready for rebase and review? I can try rebasing it into a separate PR, or you can if you have time... then (hopefully) we can get it in! ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user tommals commented on the issue: https://github.com/apache/jena/pull/114 Relevant to this thread [Return nested JSON results](https://mail-archives.apache.org/mod_mbox/jena-users/201705.mbox/%3Ctrinity-2c35f052-b0a8-4b53-aa95-9963ff11a7a6-1494243601102%403capp-mailcom-lxa06%3E) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user defret5y6u7yhfgtdrfe commented on the issue: https://github.com/apache/jena/pull/114 Relevant to this thread: [Return nested JSON results])https://mail-archives.apache.org/mod_mbox/jena-users/201705.mbox/%3Ctrinity-2c35f052-b0a8-4b53-aa95-9963ff11a7a6-1494243601102%403capp-mailcom-lxa06%3E) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user ajs6f commented on the issue: https://github.com/apache/jena/pull/114 No prob-- good to see you back, @kinow ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on the issue: https://github.com/apache/jena/pull/114 @ajs6f sorry for the delay. Checked out the branch, just need to refresh my memory. Will need a few more days before pinging you guys to check out this PR again. Next month finally will have some time to play with Jena again, then plan to have this PR completed, and check if there's any Fuseki or easy tasks in JIRA :-) Cheers Bruno --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly
Github user ajs6f commented on the issue: https://github.com/apache/jena/pull/114 @kinow is this still live? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
