[GitHub] jena issue #114: JENA-632: Generate JSON from SPARQL directly

[kinow]

Github user kinow commented on the issue:

https://github.com/apache/jena/pull/114
  
Hey @ajs6f , I think so. My weak memory seems to tell me that the last time 
I did that... shortly after, there was some work on JSON (JSON-LD? Or maybe 
pure JSON) somewhere else in Jena; and when I saw that happening, I wondered if 
that would perhaps be a replacement for the functionality in this ticket.

I wonder if you or @afs would be able to confirm that there isn't already 
another way to get JSON responses in Jena, and that this feature is still a 
nice to have feature, please?  :)


---

[jira] [Commented] (JENA-632) Generate JSON from SPARQL directly.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JENA-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390616#comment-16390616
 ] 

ASF GitHub Bot commented on JENA-632:
-

Github user kinow commented on the issue:

https://github.com/apache/jena/pull/114
  
Hey @ajs6f , I think so. My weak memory seems to tell me that the last time 
I did that... shortly after, there was some work on JSON (JSON-LD? Or maybe 
pure JSON) somewhere else in Jena; and when I saw that happening, I wondered if 
that would perhaps be a replacement for the functionality in this ticket.

I wonder if you or @afs would be able to confirm that there isn't already 
another way to get JSON responses in Jena, and that this feature is still a 
nice to have feature, please?  :)


> Generate JSON from SPARQL directly.
> ---
>
> Key: JENA-632
> URL: https://issues.apache.org/jira/browse/JENA-632
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: ARQ, Fuseki
>Reporter: Andy Seaborne
>Assignee: Bruno P. Kinoshita
>Priority: Minor
>  Labels: java, javacc
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The capability to generate JSON directly from a SPARQL (or extended SPARQL) 
> query would enable the creation of JSON data API over published linked data.
> This project would cover:
> # Design and publication of a design.
> # Refinement of design based on community feed
> # Implementation, including testing.
> # Refinement of implementation based on community feed
> Skills required: Java, some parser work, design and discussion with the user 
> community, basic understanding of HTTP and content negotiation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Possible fuseki/memory store bug

[Dave Reynolds]

On 07/03/18 17:23, Andy Seaborne wrote:



On 07/03/18 16:51, Dave Reynolds wrote:

On 07/03/18 16:29, Andy Seaborne wrote:

JENA-1499 may have knock on effects.

Adding a quad and deleting a quad and still listing the graph name 
would be OK but "contains graph" returns false in TIM and true in 
general. That might make a difference - I haven't traced that part of 
the code.


Understood.

As for the :3030, ":" is one of the nasty characters in URIs but the 
Fuseki log looks like it is %-ed correctly.


https://gist.github.com/afs/c5623911293dd19da7f6cc1b2cf64c17

is an updated script to use curl and not Jena tools.

There is variable to set the graph name.

Does this script illustrate the effect?


Yes with one correction. Line 18 should be:

G5X="$(wwwenc $G5)"

rather than:

G5X="$(wwwenc $G)"


My bad - gist corrected.



With that correction that this illustrates the effect for me.


I always get:

---
| s   | p  | o  | G   |
===
| :r4 | rdf:type   | :Resource  | :r4 |
| :r4 | rdfs:label | "r 4 modified" | :r4 |
---

3.4.0, 3.6.0 (I do delete run/ each start-up or use the 3.6.0 basic 
server to ensure no funny stuff).


This is bizarre. No blank nodes either.

openjdk version "1.8.0_151"


Sorry, comparing the gist I also changed the value of G5 to match the 
earlier report so line 16 is:


G5="http://localhost/graph5;

With that [1] I get:

== Setup
 (should be empty)
  
  
== GSP : PUT graph5
G5=http://localhost/graph5
{
  "count" : 2 ,
  "tripleCount" : 2 ,
  "quadCount" : 0
}

  
 {

a    ;

"r 4" .
}
  
== Update
  
 {
}

 {

a    ;

"r 4 modified" .
}
  
== Query
-
| s | p | o | G |
=
-
== Dump remote
  
 {
}

 {

a    ;

"r 4 modified" .
}
  

Same java version.

I also delete the /run area and do a fresh start every time.

Dave

[1] https://gist.github.com/der/ef58cfdb07114d94d1a3d447ecbcf3fc





     Andy



Dave


 Andy


On 07/03/18 14:14, Dave Reynolds wrote:

On 07/03/18 12:59, Andy Seaborne wrote:



On 06/03/18 22:43, Dave Reynolds wrote:

Hi Andy,

Thanks for confirming you seeing something similar, glad I wasn't 
hallucinating!


Not sure any more that I am :-|

There are ghost graphs in a TIM dataset after deletion, JENA-1499, 
but I'm not seeing an empty store and do see the added triples.


OK so this just got weirder. I just tried the script from your gist 
and that works for me. Whereas my own was still failing. The only 
difference is that in your initial s-put you have a slightly 
different graph name (includes the port).


So for me on 3.4.0 and 3.6.0 if I run a trimmed version of your test 
[1], with the same update.ttl [2] and U1.ru [3] scripts. I see 
correct results:


== Step 1a

== Step 1b
 {
 
 a    ;
 
 "r 4" .
}

== Step 2a

 {
 
 a    ;
 
 "r 4 modified" .
}

 {
}

== Step 3a
-- 

| s  | p     | 
o    | G  |
== 

|  | 
 | 
 |  |
|  | 
  | "r 4 modified" 
 |  |
-- 




If I run the same script but without the ":3030" in the graph name 
[4] I see 

Re: Possible fuseki/memory store bug

[ajs6f]

I've got a PR in now for 1499 (TIM remembers graph names even after they are 
empty):

https://github.com/apache/jena/pull/374

Whether or not that's the whole story here, that's a different question!

ajs6f

> On Mar 7, 2018, at 12:23 PM, Andy Seaborne  wrote:
> 
> 
> 
> On 07/03/18 16:51, Dave Reynolds wrote:
>> On 07/03/18 16:29, Andy Seaborne wrote:
>>> JENA-1499 may have knock on effects.
>>> 
>>> Adding a quad and deleting a quad and still listing the graph name would be 
>>> OK but "contains graph" returns false in TIM and true in general. That 
>>> might make a difference - I haven't traced that part of the code.
>> Understood.
>>> As for the :3030, ":" is one of the nasty characters in URIs but the Fuseki 
>>> log looks like it is %-ed correctly.
>>> 
>>> https://gist.github.com/afs/c5623911293dd19da7f6cc1b2cf64c17
>>> 
>>> is an updated script to use curl and not Jena tools.
>>> 
>>> There is variable to set the graph name.
>>> 
>>> Does this script illustrate the effect?
>> Yes with one correction. Line 18 should be:
>> G5X="$(wwwenc $G5)"
>> rather than:
>> G5X="$(wwwenc $G)"
> 
> My bad - gist corrected.
> 
>> With that correction that this illustrates the effect for me.
> 
> I always get:
> 
> ---
> | s   | p  | o  | G   |
> ===
> | :r4 | rdf:type   | :Resource  | :r4 |
> | :r4 | rdfs:label | "r 4 modified" | :r4 |
> ---
> 
> 3.4.0, 3.6.0 (I do delete run/ each start-up or use the 3.6.0 basic server to 
> ensure no funny stuff).
> 
> This is bizarre. No blank nodes either.
> 
> openjdk version "1.8.0_151"
> 
> 
>Andy
> 
>> Dave
>>>  Andy
>>> 
>>> 
>>> On 07/03/18 14:14, Dave Reynolds wrote:
 On 07/03/18 12:59, Andy Seaborne wrote:
> 
> 
> On 06/03/18 22:43, Dave Reynolds wrote:
>> Hi Andy,
>> 
>> Thanks for confirming you seeing something similar, glad I wasn't 
>> hallucinating!
> 
> Not sure any more that I am :-|
> 
> There are ghost graphs in a TIM dataset after deletion, JENA-1499, but 
> I'm not seeing an empty store and do see the added triples.
 
 OK so this just got weirder. I just tried the script from your gist and 
 that works for me. Whereas my own was still failing. The only difference 
 is that in your initial s-put you have a slightly different graph name 
 (includes the port).
 
 So for me on 3.4.0 and 3.6.0 if I run a trimmed version of your test [1], 
 with the same update.ttl [2] and U1.ru [3] scripts. I see correct results:
 
 == Step 1a
 
 == Step 1b
  {
  
  a    ;
  
  "r 4" .
 }
 
 == Step 2a
 
  {
  
  a    ;
  
  "r 4 modified" .
 }
 
  {
 }
 
 == Step 3a
 --
  
 | s  | p | o   
  | G  |
 ==
  
 |  | 
  | 
  |  |
 |  | 
   | "r 4 modified" 
  |  |
 --
  
 
 
 If I run the same script but without the ":3030" in the graph name [4] I 
 see incorrect results from the SPARQL query but correct TRIG:
 
 == Step 1a
 
 == Step 1b
  {
  
  a    ;
  
  "r 4" .
 }
 
 == Step 2a
 
  {
 }
 
  {
  
  a    ;
  
  "r 4 modified" .
 }
 

[jira] [Assigned] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[A. Soroka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/JENA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

A. Soroka reassigned JENA-1499:
---

Assignee: A. Soroka

> The TIM dataset retains a memory of named graphs after deleting all quads.
> --
>
> Key: JENA-1499
> URL: https://issues.apache.org/jira/browse/JENA-1499
> Project: Apache Jena
>  Issue Type: Bug
>Affects Versions: Jena 3.6.0
>Reporter: Andy Seaborne
>Assignee: A. Soroka
>Priority: Minor
>
> Illustration:
> {noformat}
> DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
> Quad q = SSE.parseQuad("(:g :s :p :o)");
> dsg.add(q);
> dsg.delete(q);
> Iter.print(dsg.listGraphNodes());
> {noformat}
> prints {{http://example/g}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jena pull request #374: JENA-1499: Pruning TIM indexes on tuple deletion

[ajs6f]

GitHub user ajs6f opened a pull request:

https://github.com/apache/jena/pull/374

JENA-1499: Pruning TIM indexes on tuple deletion



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajs6f/jena JENA-1499

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/374.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #374


commit d78be3b006a83f4e9e3a83cd76c0c79ee549040c
Author: ajs6f 
Date:   2018-03-07T17:25:05Z

JENA-1499: Pruning TIM indexes on tuple deletion




---

Re: Possible fuseki/memory store bug

[Andy Seaborne]

On 07/03/18 16:51, Dave Reynolds wrote:

On 07/03/18 16:29, Andy Seaborne wrote:

JENA-1499 may have knock on effects.

Adding a quad and deleting a quad and still listing the graph name 
would be OK but "contains graph" returns false in TIM and true in 
general. That might make a difference - I haven't traced that part of 
the code.


Understood.

As for the :3030, ":" is one of the nasty characters in URIs but the 
Fuseki log looks like it is %-ed correctly.


https://gist.github.com/afs/c5623911293dd19da7f6cc1b2cf64c17

is an updated script to use curl and not Jena tools.

There is variable to set the graph name.

Does this script illustrate the effect?


Yes with one correction. Line 18 should be:

G5X="$(wwwenc $G5)"

rather than:

G5X="$(wwwenc $G)"


My bad - gist corrected.



With that correction that this illustrates the effect for me.


I always get:

---
| s   | p  | o  | G   |
===
| :r4 | rdf:type   | :Resource  | :r4 |
| :r4 | rdfs:label | "r 4 modified" | :r4 |
---

3.4.0, 3.6.0 (I do delete run/ each start-up or use the 3.6.0 basic 
server to ensure no funny stuff).


This is bizarre. No blank nodes either.

openjdk version "1.8.0_151"


Andy



Dave


 Andy


On 07/03/18 14:14, Dave Reynolds wrote:

On 07/03/18 12:59, Andy Seaborne wrote:



On 06/03/18 22:43, Dave Reynolds wrote:

Hi Andy,

Thanks for confirming you seeing something similar, glad I wasn't 
hallucinating!


Not sure any more that I am :-|

There are ghost graphs in a TIM dataset after deletion, JENA-1499, 
but I'm not seeing an empty store and do see the added triples.


OK so this just got weirder. I just tried the script from your gist 
and that works for me. Whereas my own was still failing. The only 
difference is that in your initial s-put you have a slightly 
different graph name (includes the port).


So for me on 3.4.0 and 3.6.0 if I run a trimmed version of your test 
[1], with the same update.ttl [2] and U1.ru [3] scripts. I see 
correct results:


== Step 1a

== Step 1b
 {
 
 a    ;
 
 "r 4" .
}

== Step 2a

 {
 
 a    ;
 
 "r 4 modified" .
}

 {
}

== Step 3a
-- 

| s  | p     | 
o    | G  |
== 

|  | 
 | 
 |  |
|  | 
  | "r 4 modified" 
 |  |
-- 




If I run the same script but without the ":3030" in the graph name 
[4] I see incorrect results from the SPARQL query but correct TRIG:


== Step 1a

== Step 1b
 {
 
 a    ;
 
 "r 4" .
}

== Step 2a

 {
}

 {
 
 a    ;
 
 "r 4 modified" .
}

== Step 3a
-
| s | p | o | G |
=
-


The only significant difference I can see is that the change in graph 
names means the graphs list in a different order in the TRIG, which 
suggests they hash in a different order in the store. Why that should 
matter though ... ?


This at least explains why it was so hard to isolate the problem and 
that it seemed non-deterministic. Any attempt to simplify the names 
made the problem go away.


Dave

[1] https://gist.github.com/der/d47602ffbd294b2ed572cbccb02c7778
[2] https://gist.github.com/der/26a2147d19ad495a925374e56428f7e3
[3] https://gist.github.com/der/704363855a7cca35b3f123dff136c56d
[4] https://gist.github.com/der/7e1e711c6341df417e9ff18416bd1a5c




 Andy



I've tried with '' instead of "" in the shell script version of the 
test 

[jira] [Commented] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[A. Soroka (JIRA)]

[ 
https://issues.apache.org/jira/browse/JENA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389790#comment-16389790
 ] 

A. Soroka commented on JENA-1499:
-

No, no-- I just meant in TIM! I'll have a PR in later today to show what I 
mean...

> The TIM dataset retains a memory of named graphs after deleting all quads.
> --
>
> Key: JENA-1499
> URL: https://issues.apache.org/jira/browse/JENA-1499
> Project: Apache Jena
>  Issue Type: Bug
>Affects Versions: Jena 3.6.0
>Reporter: Andy Seaborne
>Priority: Minor
>
> Illustration:
> {noformat}
> DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
> Quad q = SSE.parseQuad("(:g :s :p :o)");
> dsg.add(q);
> dsg.delete(q);
> Iter.print(dsg.listGraphNodes());
> {noformat}
> prints {{http://example/g}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[Andy Seaborne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/JENA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy Seaborne updated JENA-1499:

Priority: Minor  (was: Major)

> The TIM dataset retains a memory of named graphs after deleting all quads.
> --
>
> Key: JENA-1499
> URL: https://issues.apache.org/jira/browse/JENA-1499
> Project: Apache Jena
>  Issue Type: Bug
>Affects Versions: Jena 3.6.0
>Reporter: Andy Seaborne
>Priority: Minor
>
> Illustration:
> {noformat}
> DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
> Quad q = SSE.parseQuad("(:g :s :p :o)");
> dsg.add(q);
> dsg.delete(q);
> Iter.print(dsg.listGraphNodes());
> {noformat}
> prints {{http://example/g}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[Andy Seaborne (JIRA)]

[ 
https://issues.apache.org/jira/browse/JENA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389783#comment-16389783
 ] 

Andy Seaborne commented on JENA-1499:
-

Currently, this ticket is now disconnected from the original report and just 
something we have noticed.

Priority set to "minor".


> The TIM dataset retains a memory of named graphs after deleting all quads.
> --
>
> Key: JENA-1499
> URL: https://issues.apache.org/jira/browse/JENA-1499
> Project: Apache Jena
>  Issue Type: Bug
>Affects Versions: Jena 3.6.0
>Reporter: Andy Seaborne
>Priority: Major
>
> Illustration:
> {noformat}
> DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
> Quad q = SSE.parseQuad("(:g :s :p :o)");
> dsg.add(q);
> dsg.delete(q);
> Iter.print(dsg.listGraphNodes());
> {noformat}
> prints {{http://example/g}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Possible fuseki/memory store bug

[Dave Reynolds]

On 07/03/18 16:29, Andy Seaborne wrote:

JENA-1499 may have knock on effects.

Adding a quad and deleting a quad and still listing the graph name would 
be OK but "contains graph" returns false in TIM and true in general. 
That might make a difference - I haven't traced that part of the code.


Understood.

As for the :3030, ":" is one of the nasty characters in URIs but the 
Fuseki log looks like it is %-ed correctly.


https://gist.github.com/afs/c5623911293dd19da7f6cc1b2cf64c17

is an updated script to use curl and not Jena tools.

There is variable to set the graph name.

Does this script illustrate the effect?


Yes with one correction. Line 18 should be:

G5X="$(wwwenc $G5)"

rather than:

G5X="$(wwwenc $G)"

With that correction that this illustrates the effect for me.

Dave


     Andy


On 07/03/18 14:14, Dave Reynolds wrote:

On 07/03/18 12:59, Andy Seaborne wrote:



On 06/03/18 22:43, Dave Reynolds wrote:

Hi Andy,

Thanks for confirming you seeing something similar, glad I wasn't 
hallucinating!


Not sure any more that I am :-|

There are ghost graphs in a TIM dataset after deletion, JENA-1499, 
but I'm not seeing an empty store and do see the added triples.


OK so this just got weirder. I just tried the script from your gist 
and that works for me. Whereas my own was still failing. The only 
difference is that in your initial s-put you have a slightly different 
graph name (includes the port).


So for me on 3.4.0 and 3.6.0 if I run a trimmed version of your test 
[1], with the same update.ttl [2] and U1.ru [3] scripts. I see correct 
results:


== Step 1a

== Step 1b
 {
 
 a    ;
 
 "r 4" .
}

== Step 2a

 {
 
 a    ;
 
 "r 4 modified" .
}

 {
}

== Step 3a
-- 

| s  | p     | 
o    | G  |
== 

|  | 
 | 
 |  |
|  | 
  | "r 4 modified" 
 |  |
-- 




If I run the same script but without the ":3030" in the graph name [4] 
I see incorrect results from the SPARQL query but correct TRIG:


== Step 1a

== Step 1b
 {
 
 a    ;
 
 "r 4" .
}

== Step 2a

 {
}

 {
 
 a    ;
 
 "r 4 modified" .
}

== Step 3a
-
| s | p | o | G |
=
-


The only significant difference I can see is that the change in graph 
names means the graphs list in a different order in the TRIG, which 
suggests they hash in a different order in the store. Why that should 
matter though ... ?


This at least explains why it was so hard to isolate the problem and 
that it seemed non-deterministic. Any attempt to simplify the names 
made the problem go away.


Dave

[1] https://gist.github.com/der/d47602ffbd294b2ed572cbccb02c7778
[2] https://gist.github.com/der/26a2147d19ad495a925374e56428f7e3
[3] https://gist.github.com/der/704363855a7cca35b3f123dff136c56d
[4] https://gist.github.com/der/7e1e711c6341df417e9ff18416bd1a5c




 Andy



I've tried with '' instead of "" in the shell script version of the 
test with identical results.


Confirmed that using --memTDB the test passes for me.

Dave

On 06/03/18 17:29, Andy Seaborne wrote:

Weird.

I ran this script with Fuseki v3.6.0 "--mem" and also "--memTDB" 
for steps up to and including 6.


https://gist.github.com/afs/cd6953b06985dde37a9581134ec13165

There something going on with TIM because I'm seeing empty graph5 
with TIM but not with TDB.


I may have seen no results once and then I changed:

 > 6. Check the contents of the store:
 >
 >  rsparql --service 

[jira] [Commented] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[Andy Seaborne (JIRA)]

[ 
https://issues.apache.org/jira/browse/JENA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389782#comment-16389782
 ] 

Andy Seaborne commented on JENA-1499:
-

If by a check for empty set, you mean in the TIM code, then that is OK by me 
because that Set is cheap for isEmpty (isn't it?)

In the general stack, no. What is expensive and what is cheap can be hard to 
know and sometimes varies wildly across implementations. The checking would 
have to be in very deep and breaks the "hand out graph" design.

The explicitly link based on ({{createGeneral()}}), it makes sense and output 
in TriG is (a bit) useful.

The implicitly link based on ({{create()}}) it is more like history as it is 
now copy-in semantics, not link. There is a ghost in {{Dataset.listNames()}} 
and {{containsNamedModel}} and the TriG issue is fallout but it is consistent 
for "contains" and "in Trig".

We could add to the javadoc but an actual functional change looks to me like 
something that just as likely to cause the opposite issue. 


> The TIM dataset retains a memory of named graphs after deleting all quads.
> --
>
> Key: JENA-1499
> URL: https://issues.apache.org/jira/browse/JENA-1499
> Project: Apache Jena
>  Issue Type: Bug
>Affects Versions: Jena 3.6.0
>Reporter: Andy Seaborne
>Priority: Major
>
> Illustration:
> {noformat}
> DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
> Quad q = SSE.parseQuad("(:g :s :p :o)");
> dsg.add(q);
> dsg.delete(q);
> Iter.print(dsg.listGraphNodes());
> {noformat}
> prints {{http://example/g}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[Andy Seaborne (JIRA)]

[ 
https://issues.apache.org/jira/browse/JENA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389764#comment-16389764
 ] 

Andy Seaborne commented on JENA-1499:
-

This shows an inconsistency with {{dsg.containsGraph}}.
This _might_ have knock on effects.

{noformat}
public static void main(String... args) throws Exception {
dwim("general", DatasetGraphFactory.createGeneral());
System.out.println();
dwim("TIM", DatasetGraphFactory.createTxnMem());
System.out.println("-- --");
System.out.println("DONE");
System.exit(0);
}

public static void dwim(String label, DatasetGraph dsg) throws Exception {
System.out.println("== "+label);
Node gn = NodeFactory.createURI("http://example/g;) ;
Quad q = SSE.parseQuad("(:g :s :p :o)");
dsg.add(q);
dsg.delete(q);
//UpdateAction.parseExecute("CREATE GRAPH ", dsg);
System.out.println("Contains: "+dsg.containsGraph(gn));
System.out.println("-- listGraphNodes");
Iter.print(dsg.listGraphNodes());
System.out.println("-- dump");
RDFDataMgr.write(System.out, dsg, Lang.TRIG);
return;
}
{noformat}

> The TIM dataset retains a memory of named graphs after deleting all quads.
> --
>
> Key: JENA-1499
> URL: https://issues.apache.org/jira/browse/JENA-1499
> Project: Apache Jena
>  Issue Type: Bug
>Affects Versions: Jena 3.6.0
>Reporter: Andy Seaborne
>Priority: Major
>
> Illustration:
> {noformat}
> DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
> Quad q = SSE.parseQuad("(:g :s :p :o)");
> dsg.add(q);
> dsg.delete(q);
> Iter.print(dsg.listGraphNodes());
> {noformat}
> prints {{http://example/g}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [jira] [Commented] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[ajs6f]

+1!

ajs6f

> On Mar 7, 2018, at 11:29 AM, Claude Warren  wrote:
> 
> This looks like a prime candidate for a contract test (as soon as we agree
> on what the action should be)
> 
> Claude
> 
> On Wed, Mar 7, 2018 at 4:26 PM, A. Soroka (JIRA)  wrote:
> 
>> 
>>[ https://issues.apache.org/jira/browse/JENA-1499?page=
>> com.atlassian.jira.plugin.system.issuetabpanels:comment-
>> tabpanel=16389757#comment-16389757 ]
>> 
>> A. Soroka commented on JENA-1499:
>> -
>> 
>> [~andy.seaborne], 
>> {{org.apache.jena.sparql.core.mem.QuadTableForm.GSPO.listGraphNodes()}}
>> is just taking the keys of the GSPO index and returning a stream of them.
>> That's why (see above) if there are mappings fro{{m }}{{Node => Node =>
>> Set}} with that {{Set}} empty, we're still seeing names of graphs
>> appear at that exact method.
>> 
>>> The TIM dataset retains a memory of named graphs after deleting all
>> quads.
>>> 
>> --
>>> 
>>>Key: JENA-1499
>>>URL: https://issues.apache.org/jira/browse/JENA-1499
>>>Project: Apache Jena
>>> Issue Type: Bug
>>>   Affects Versions: Jena 3.6.0
>>>   Reporter: Andy Seaborne
>>>   Priority: Major
>>> 
>>> Illustration:
>>> {noformat}
>>>DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
>>>Quad q = SSE.parseQuad("(:g :s :p :o)");
>>>dsg.add(q);
>>>dsg.delete(q);
>>>Iter.print(dsg.listGraphNodes());
>>> {noformat}
>>> prints {{http://example/g}}.
>> 
>> 
>> 
>> --
>> This message was sent by Atlassian JIRA
>> (v7.6.3#76005)
>> 
> 
> 
> 
> -- 
> I like: Like Like - The likeliest place on the web
> 
> LinkedIn: http://www.linkedin.com/in/claudewarren

Re: Possible fuseki/memory store bug

[Andy Seaborne]

JENA-1499 may have knock on effects.

Adding a quad and deleting a quad and still listing the graph name would 
be OK but "contains graph" returns false in TIM and true in general. 
That might make a difference - I haven't traced that part of the code.


As for the :3030, ":" is one of the nasty characters in URIs but the 
Fuseki log looks like it is %-ed correctly.


https://gist.github.com/afs/c5623911293dd19da7f6cc1b2cf64c17

is an updated script to use curl and not Jena tools.

There is variable to set the graph name.

Does this script illustrate the effect?

Andy


On 07/03/18 14:14, Dave Reynolds wrote:

On 07/03/18 12:59, Andy Seaborne wrote:



On 06/03/18 22:43, Dave Reynolds wrote:

Hi Andy,

Thanks for confirming you seeing something similar, glad I wasn't 
hallucinating!


Not sure any more that I am :-|

There are ghost graphs in a TIM dataset after deletion, JENA-1499, but 
I'm not seeing an empty store and do see the added triples.


OK so this just got weirder. I just tried the script from your gist and 
that works for me. Whereas my own was still failing. The only difference 
is that in your initial s-put you have a slightly different graph name 
(includes the port).


So for me on 3.4.0 and 3.6.0 if I run a trimmed version of your test 
[1], with the same update.ttl [2] and U1.ru [3] scripts. I see correct 
results:


== Step 1a

== Step 1b
 {
     
     a    ;
     
     "r 4" .
}

== Step 2a

 {
     
     a    ;
     
     "r 4 modified" .
}

 {
}

== Step 3a
-- 

| s  | p     | 
o    | G  |
== 

|  | 
 | 
 |  |
|  | 
  | "r 4 modified" 
     |  |
-- 




If I run the same script but without the ":3030" in the graph name [4] I 
see incorrect results from the SPARQL query but correct TRIG:


== Step 1a

== Step 1b
 {
     
     a    ;
     
     "r 4" .
}

== Step 2a

 {
}

 {
     
     a    ;
     
     "r 4 modified" .
}

== Step 3a
-
| s | p | o | G |
=
-


The only significant difference I can see is that the change in graph 
names means the graphs list in a different order in the TRIG, which 
suggests they hash in a different order in the store. Why that should 
matter though ... ?


This at least explains why it was so hard to isolate the problem and 
that it seemed non-deterministic. Any attempt to simplify the names made 
the problem go away.


Dave

[1] https://gist.github.com/der/d47602ffbd294b2ed572cbccb02c7778
[2] https://gist.github.com/der/26a2147d19ad495a925374e56428f7e3
[3] https://gist.github.com/der/704363855a7cca35b3f123dff136c56d
[4] https://gist.github.com/der/7e1e711c6341df417e9ff18416bd1a5c




 Andy



I've tried with '' instead of "" in the shell script version of the 
test with identical results.


Confirmed that using --memTDB the test passes for me.

Dave

On 06/03/18 17:29, Andy Seaborne wrote:

Weird.

I ran this script with Fuseki v3.6.0 "--mem" and also "--memTDB" for 
steps up to and including 6.


https://gist.github.com/afs/cd6953b06985dde37a9581134ec13165

There something going on with TIM because I'm seeing empty graph5 
with TIM but not with TDB.


I may have seen no results once and then I changed:

 > 6. Check the contents of the store:
 >
 >  rsparql --service http://localhost:3030/ds/query "SELECT * 
WHERE

 > {Graph ?G {?s ?p ?o}} ORDER BY ?G"

There are "" quotes around a "*"  and it's a script.  Could you try 
''-quotes please?


I'll try step 7.

 Andy

On 05/03/18 23:56, Dave Reynolds wrote:
I've 

Re: [jira] [Commented] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[Claude Warren]

This looks like a prime candidate for a contract test (as soon as we agree
on what the action should be)

Claude

On Wed, Mar 7, 2018 at 4:26 PM, A. Soroka (JIRA)  wrote:

>
> [ https://issues.apache.org/jira/browse/JENA-1499?page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel=16389757#comment-16389757 ]
>
> A. Soroka commented on JENA-1499:
> -
>
> [~andy.seaborne], 
> {{org.apache.jena.sparql.core.mem.QuadTableForm.GSPO.listGraphNodes()}}
> is just taking the keys of the GSPO index and returning a stream of them.
> That's why (see above) if there are mappings fro{{m }}{{Node => Node =>
> Set}} with that {{Set}} empty, we're still seeing names of graphs
> appear at that exact method.
>
> > The TIM dataset retains a memory of named graphs after deleting all
> quads.
> > 
> --
> >
> > Key: JENA-1499
> > URL: https://issues.apache.org/jira/browse/JENA-1499
> > Project: Apache Jena
> >  Issue Type: Bug
> >Affects Versions: Jena 3.6.0
> >Reporter: Andy Seaborne
> >Priority: Major
> >
> > Illustration:
> > {noformat}
> > DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
> > Quad q = SSE.parseQuad("(:g :s :p :o)");
> > dsg.add(q);
> > dsg.delete(q);
> > Iter.print(dsg.listGraphNodes());
> > {noformat}
> > prints {{http://example/g}}.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)
>



-- 
I like: Like Like - The likeliest place on the web

LinkedIn: http://www.linkedin.com/in/claudewarren

[jira] [Commented] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[A. Soroka (JIRA)]

[ 
https://issues.apache.org/jira/browse/JENA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389757#comment-16389757
 ] 

A. Soroka commented on JENA-1499:
-

[~andy.seaborne], 
{{org.apache.jena.sparql.core.mem.QuadTableForm.GSPO.listGraphNodes()}} is just 
taking the keys of the GSPO index and returning a stream of them. That's why 
(see above) if there are mappings fro{{m }}{{Node => Node => Set}} with 
that {{Set}} empty, we're still seeing names of graphs appear at that exact 
method.

> The TIM dataset retains a memory of named graphs after deleting all quads.
> --
>
> Key: JENA-1499
> URL: https://issues.apache.org/jira/browse/JENA-1499
> Project: Apache Jena
>  Issue Type: Bug
>Affects Versions: Jena 3.6.0
>Reporter: Andy Seaborne
>Priority: Major
>
> Illustration:
> {noformat}
> DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
> Quad q = SSE.parseQuad("(:g :s :p :o)");
> dsg.add(q);
> dsg.delete(q);
> Iter.print(dsg.listGraphNodes());
> {noformat}
> prints {{http://example/g}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (JENA-1499) The TIM dataset retains a memory of named graphs after deleting all quads.

[A. Soroka (JIRA)]

[ 
https://issues.apache.org/jira/browse/JENA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389708#comment-16389708
 ] 

A. Soroka commented on JENA-1499:
-

So, I'm fairly sure why this is happening: all the indexes in TIM are of the 
form {{Map>}}. When a quad is deleted, there is 
(currently) no check to see if that {{Set}} ends up empty. An empty {{Set}} 
means that there will still be a named graph present with no actual quads-- 
just as we see. We could add a check in the {{DatasetGraph.delete()}} call 
stack so that if the last quad out of a graph is deleted, the {{Node}} => 
(now-empty) {{Set}} mapping is removed from its {{Map}} and if that {{Map}} is 
now empty the {{Node}} => {{Map}} mapping is removed from the outermost 
{{Map}}. Does that sound like a good move?

> The TIM dataset retains a memory of named graphs after deleting all quads.
> --
>
> Key: JENA-1499
> URL: https://issues.apache.org/jira/browse/JENA-1499
> Project: Apache Jena
>  Issue Type: Bug
>Affects Versions: Jena 3.6.0
>Reporter: Andy Seaborne
>Priority: Major
>
> Illustration:
> {noformat}
> DatasetGraph dsg = DatasetGraphFactory.createTxnMem();
> Quad q = SSE.parseQuad("(:g :s :p :o)");
> dsg.add(q);
> dsg.delete(q);
> Iter.print(dsg.listGraphNodes());
> {noformat}
> prints {{http://example/g}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Possible fuseki/memory store bug

[Dave Reynolds]

On 07/03/18 12:59, Andy Seaborne wrote:



On 06/03/18 22:43, Dave Reynolds wrote:

Hi Andy,

Thanks for confirming you seeing something similar, glad I wasn't 
hallucinating!


Not sure any more that I am :-|

There are ghost graphs in a TIM dataset after deletion, JENA-1499, but 
I'm not seeing an empty store and do see the added triples.


OK so this just got weirder. I just tried the script from your gist and 
that works for me. Whereas my own was still failing. The only difference 
is that in your initial s-put you have a slightly different graph name 
(includes the port).


So for me on 3.4.0 and 3.6.0 if I run a trimmed version of your test 
[1], with the same update.ttl [2] and U1.ru [3] scripts. I see correct 
results:


== Step 1a

== Step 1b
 {

a    ;

"r 4" .
}

== Step 2a

 {

a    ;

"r 4 modified" .
}

 {
}

== Step 3a
--
| s  | p 
| o| G  |

==
|  | 
 | 
 |  |
|  | 
  | "r 4 modified" 
|  |

--


If I run the same script but without the ":3030" in the graph name [4] I 
see incorrect results from the SPARQL query but correct TRIG:


== Step 1a

== Step 1b
 {

a    ;

"r 4" .
}

== Step 2a

 {
}

 {

a    ;

"r 4 modified" .
}

== Step 3a
-
| s | p | o | G |
=
-


The only significant difference I can see is that the change in graph 
names means the graphs list in a different order in the TRIG, which 
suggests they hash in a different order in the store. Why that should 
matter though ... ?


This at least explains why it was so hard to isolate the problem and 
that it seemed non-deterministic. Any attempt to simplify the names made 
the problem go away.


Dave

[1] https://gist.github.com/der/d47602ffbd294b2ed572cbccb02c7778
[2] https://gist.github.com/der/26a2147d19ad495a925374e56428f7e3
[3] https://gist.github.com/der/704363855a7cca35b3f123dff136c56d
[4] https://gist.github.com/der/7e1e711c6341df417e9ff18416bd1a5c




     Andy



I've tried with '' instead of "" in the shell script version of the 
test with identical results.


Confirmed that using --memTDB the test passes for me.

Dave

On 06/03/18 17:29, Andy Seaborne wrote:

Weird.

I ran this script with Fuseki v3.6.0 "--mem" and also "--memTDB" for 
steps up to and including 6.


https://gist.github.com/afs/cd6953b06985dde37a9581134ec13165

There something going on with TIM because I'm seeing empty graph5 
with TIM but not with TDB.


I may have seen no results once and then I changed:

 > 6. Check the contents of the store:
 >
 >  rsparql --service http://localhost:3030/ds/query "SELECT * WHERE
 > {Graph ?G {?s ?p ?o}} ORDER BY ?G"

There are "" quotes around a "*"  and it's a script.  Could you try 
''-quotes please?


I'll try step 7.

 Andy

On 05/03/18 23:56, Dave Reynolds wrote:
I've been trying to debug some weird behaviour in my test cases and 
think they are due to a bug in memory-backed fuseki stores.


However, the behaviour is so odd and hard to reproduce I'd like some 
confirmation that someone else sees the same effect before opening a 
JIRA.


# Steps to reproduce

[Sorry this is convoluted but all my attempts to simplify fail to 
show the suspected bug.]


1. Fresh download of fuseki 3.6.0.

2. Start an in memory server in one shell:

 fuseki-server --mem --update /ds

3. Create test data with two statements about one resource:

 echo 'prefix rdfs:  
prefix eg:  

[jira] [Commented] (JENA-1454) Configurable ResultSet reading and writing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JENA-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389513#comment-16389513
 ] 

ASF GitHub Bot commented on JENA-1454:
--

Github user asfgit closed the pull request at:

https://github.com/apache/jena/pull/373


> Configurable ResultSet reading and writing
> --
>
> Key: JENA-1454
> URL: https://issues.apache.org/jira/browse/JENA-1454
> Project: Apache Jena
>  Issue Type: Improvement
>Reporter: Andy Seaborne
>Assignee: Andy Seaborne
>Priority: Minor
> Fix For: Jena 3.7.0
>
>
> Same patterns as {{RDFParser}}, {{RDFWriter}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] jena pull request #373: JENA-1454: Fix to handle text output format.

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/jena/pull/373


---

Re: Possible fuseki/memory store bug

[Andy Seaborne]

On 06/03/18 22:43, Dave Reynolds wrote:

Hi Andy,

Thanks for confirming you seeing something similar, glad I wasn't 
hallucinating!


Not sure any more that I am :-|

There are ghost graphs in a TIM dataset after deletion, JENA-1499, but 
I'm not seeing an empty store and do see the added triples.


Andy



I've tried with '' instead of "" in the shell script version of the test 
with identical results.


Confirmed that using --memTDB the test passes for me.

Dave

On 06/03/18 17:29, Andy Seaborne wrote:

Weird.

I ran this script with Fuseki v3.6.0 "--mem" and also "--memTDB" for 
steps up to and including 6.


https://gist.github.com/afs/cd6953b06985dde37a9581134ec13165

There something going on with TIM because I'm seeing empty graph5 with 
TIM but not with TDB.


I may have seen no results once and then I changed:

 > 6. Check the contents of the store:
 >
 >  rsparql --service http://localhost:3030/ds/query "SELECT * WHERE
 > {Graph ?G {?s ?p ?o}} ORDER BY ?G"

There are "" quotes around a "*"  and it's a script.  Could you try 
''-quotes please?


I'll try step 7.

 Andy

On 05/03/18 23:56, Dave Reynolds wrote:
I've been trying to debug some weird behaviour in my test cases and 
think they are due to a bug in memory-backed fuseki stores.


However, the behaviour is so odd and hard to reproduce I'd like some 
confirmation that someone else sees the same effect before opening a 
JIRA.


# Steps to reproduce

[Sorry this is convoluted but all my attempts to simplify fail to 
show the suspected bug.]


1. Fresh download of fuseki 3.6.0.

2. Start an in memory server in one shell:

 fuseki-server --mem --update /ds

3. Create test data with two statements about one resource:

 echo 'prefix rdfs:  
prefix eg:    
eg:r4 a eg:Resource; rdfs:label "r 4" .' > update.ttl

4. Use the graph REST API to put the data into a named graph:

 s-put http://localhost:3030/ds/data http://localhost/graph5 
update.ttl


5. Run a sparql update which will delete the original statements from 
the graph and add some replacement statements to a new graph:


 rupdate --service=http://localhost:3030/ds/update '
 DELETE {GRAPH ?G { ?p ?o}} WHERE 
{GRAPH ?G { ?p ?o}};

 INSERT DATA { GRAPH  {
  
 "r 4 modified" .
  
 
 .

 } }'

6. Check the contents of the store:

 rsparql --service http://localhost:3030/ds/query "SELECT * WHERE 
{Graph ?G {?s ?p ?o}} ORDER BY ?G"


At this point the store *should* contain two statements in graph 
http://localhost/test/r4. With a TDB-backed fuseki that's what I see. 
With the memory backed fuseki I see an apparently empty store.


If other named graphs are populated with other unrelated data they 
will seem to have disappeared as well.


7. Now reinsert the original data, running step 4 again and check by 
running step 6 again. At this point both the "missing" statements 
from graph http://localhost/test/r4 reappear, as does the reinserted 
original statements in http://localhost/graph5.


# Simplifying the test case

So far ...

   - I've failed to reproduce this outside of fuseki.

   - I've failed to reproduce this without mixing graph operations 
and update operations.


   - If I reduce the inserted/updated data to a single statement 
instead of a pair of statements it passes.


   - If I try with at empty TDB store it passes.

   - I get the same behaviour from 3.4.0 as from 3.6.0.

Any of these failures may be user error on my part, it has been so 
hard turning an apparently non-deterministic error into something 
reproducible that I'm no longer sure of anything :(


Am I going mad or does anyone else see the same behaviour?

Dave