[jira] [Commented] (JENA-1314) Update Using Jena with Eclipse page

2017-03-31 Thread Bruno P. Kinoshita (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951794#comment-15951794
 ] 

Bruno P. Kinoshita commented on JENA-1314:
--

Done. Committed to trunk on SVN so that is easier to review. We can easily 
revert to the previous version too.

http://jena.staging.apache.org/tutorials/using_jena_with_eclipse.html

Leaving this issue open for review, but if there are no objections will close 
it later, and update the Portuguese version too 
(http://jena.staging.apache.org/tutorials/using_jena_with_eclipse_pt.html). 
Looks like this is the only translation and I am able to update it, so we do 
not have to remove the page and  look for broken links.

Cheers
Bruno

> Update Using Jena with Eclipse page
> ---
>
> Key: JENA-1314
> URL: https://issues.apache.org/jira/browse/JENA-1314
> Project: Apache Jena
>  Issue Type: Documentation
>  Components: Documentation
>Reporter: Bruno P. Kinoshita
>Assignee: Bruno P. Kinoshita
>Priority: Minor
>  Labels: documentation, website
>
> The Using Jena with Eclipse page in our web site is empty, with a message 
> "Revised version coming soon. In the meantime, please see the old version.".
> The old version is quite old, and would be nice to address the issues with 
> shaded Guava for now.
> I have screen shots to update the page in the next days. Once we fix the 
> Guava shade issues, it will be a matter of removing a few paragraphs and 
> images.
> [1] http://jena.apache.org/tutorials/using_jena_with_eclipse.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (JENA-1314) Update Using Jena with Eclipse page

2017-03-31 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951775#comment-15951775
 ] 

ASF subversion and git services commented on JENA-1314:
---

Commit 1789762 from [~kinow] in branch 'site/trunk'
[ https://svn.apache.org/r1789762 ]

JENA-1314 updating Eclipse build instructions

> Update Using Jena with Eclipse page
> ---
>
> Key: JENA-1314
> URL: https://issues.apache.org/jira/browse/JENA-1314
> Project: Apache Jena
>  Issue Type: Documentation
>  Components: Documentation
>Reporter: Bruno P. Kinoshita
>Assignee: Bruno P. Kinoshita
>Priority: Minor
>  Labels: documentation, website
>
> The Using Jena with Eclipse page in our web site is empty, with a message 
> "Revised version coming soon. In the meantime, please see the old version.".
> The old version is quite old, and would be nice to address the issues with 
> shaded Guava for now.
> I have screen shots to update the page in the next days. Once we fix the 
> Guava shade issues, it will be a matter of removing a few paragraphs and 
> images.
> [1] http://jena.apache.org/tutorials/using_jena_with_eclipse.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


Re: Bug in SortedDataBag?

2017-03-31 Thread Chris Dollin
Hi Andy

Yes, the cancel thread doesn't call open(), I understand
that. The warnings are intermittent because the iterator
in question (the "source" interator of QueryIterSort)
is only left open if the cancel request is noticed while
the sort databag is filling up.

QueryIterSort never explicitly closes the source
iterator. But if the sort databag pulls all of the
source bindings, the source iterator self-closes.
So there's a window while the bag is filling when
a detected cancellation will leave the source
iterator open.

A fix is to ensure that the source iterator
is closed when QueryIterSort closes:

@Override public void closeIterator() {
embeddedIterator.close();
super.closeIterator();
}

"embeddedIterator" is the source iterator.

This fix eliminates the presenting problem and
the tests continue to pass /except/ that
CallBackIterator's close() method needs to
be made a no-op rather than failing, since
that open() is now actually called.

[It looks like QueryIterTopN has the same problem]

Chris



On 20 March 2017 at 15:48, Chris Dollin 
wrote:

> On 17 March 2017 at 17:08, Andy Seaborne  wrote:
>
>>
>> SortedDataBag.cancel does not call close.
>>
>
> You're quite right, am having another look at the problem.
>
> Chris
>



-- 
"What I don't understand is this ..."   Trevor Chaplin, /The Beiderbeck
Affair/

Epimorphics Ltd, http://www.epimorphics.com
Registered address: Court Lodge, 105 High Street, Portishead, Bristol BS20
6PT
Epimorphics Ltd. is a limited company registered in England (number 7016688)


[GitHub] jena issue #230: JENA-733 add documentation for narrower data type case

2017-03-31 Thread rvesse
Github user rvesse commented on the issue:

https://github.com/apache/jena/pull/230
  
+1 LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (JENA-733) Long typed literals report class java.lang.Long but return java.lang.Integer

2017-03-31 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951133#comment-15951133
 ] 

ASF GitHub Bot commented on JENA-733:
-

GitHub user kinow opened a pull request:

https://github.com/apache/jena/pull/230

JENA-733 add documentation for narrower data type case

Tried to clarify what happens when users provide Long values that fit into 
Integers, for [JENA-733](https://issues.apache.org/jira/browse/JENA-733).

I think if the docs are good to be merged, then the issue in JIRA can be 
closed too.

Cheers
Bruno

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kinow/jena JENA-733-javadocs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/jena/pull/230.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #230


commit 58cd30308e36dda3e2686953257ec771d72f681b
Author: Bruno P. Kinoshita 
Date:   2017-03-31T15:28:41Z

JENA-733 add documentation for narrower data type case




> Long typed literals report class java.lang.Long but return java.lang.Integer
> 
>
> Key: JENA-733
> URL: https://issues.apache.org/jira/browse/JENA-733
> Project: Apache Jena
>  Issue Type: Bug
>  Components: RDF API
>Affects Versions: Jena 2.11.2
>Reporter: Benno Willoweit
>Priority: Minor
>
> This is my first bug report ever so I hope I get this right. When retrieving 
> query results with ARQ, long values are returned as Integers:
> {{QuerySulution qs;}}
> {{Literal l = qs.getLiteral("myVar");}}
> {{l.getDatatype().getJavaClass(); //returns java.lang.Long which is correct}}
> {{l.getValue(); //returns a java.lang.Integer object but it should return a 
> java.lang.Long object according to the docs}}
> I have this problem with ARQ 2.11.2 which does not exist in the "affects 
> version" drop down.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Created] (JENA-1314) Update Using Jena with Eclipse page

2017-03-31 Thread Bruno P. Kinoshita (JIRA)
Bruno P. Kinoshita created JENA-1314:


 Summary: Update Using Jena with Eclipse page
 Key: JENA-1314
 URL: https://issues.apache.org/jira/browse/JENA-1314
 Project: Apache Jena
  Issue Type: Documentation
  Components: Documentation
Reporter: Bruno P. Kinoshita
Assignee: Bruno P. Kinoshita
Priority: Minor


The Using Jena with Eclipse page in our web site is empty, with a message 
"Revised version coming soon. In the meantime, please see the old version.".

The old version is quite old, and would be nice to address the issues with 
shaded Guava for now.

I have screen shots to update the page in the next days. Once we fix the Guava 
shade issues, it will be a matter of removing a few paragraphs and images.

[1] http://jena.apache.org/tutorials/using_jena_with_eclipse.html



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (JENA-1313) Language-specific collation in ARQ

2017-03-31 Thread Andy Seaborne (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951009#comment-15951009
 ] 

Andy Seaborne commented on JENA-1313:
-

We should not need ICU4J - the JDK should it all by now. jena used to use ICU4J 
and it was removed because Java acquired sufficient capabilities. And it's 
large.

If {{collate:collate(?label, 'fi')}} works, the reverse sort is {{- 
collate:collate(?label, 'fi')}}. ASC and DESC would work.

> Language-specific collation in ARQ
> --
>
> Key: JENA-1313
> URL: https://issues.apache.org/jira/browse/JENA-1313
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: ARQ
>Affects Versions: Jena 3.2.0
>Reporter: Osma Suominen
>
> As [discussed|http://markmail.org/message/v2bvsnsza5ksl2cv] on the users 
> mailing list in October 2016, I would like to change ARQ collation of literal 
> values to be language-aware and respect language-specific collation rules.
> This would probably involve changing at least the 
> [NodeUtils.compareLiteralsBySyntax|https://github.com/apache/jena/blob/master/jena-arq/src/main/java/org/apache/jena/sparql/util/NodeUtils.java#L199]
>  method.
> It currently sorts by lexical value first, then by language tag. Since the 
> collation order needs to be stable across all possible literal values, I 
> think the safest way would be to sort by language tag first, then by lexical 
> value according to the collation rules for that language.
> But what about subtags like {{@en-US}} or {{@pt-BR}}? Can they have different 
> collation rules than the main language? It would be a bit strange if all 
> {{@en-US}} literals sorted after {{@en}} literals...
> It would be good to check how Dydra does this and possibly take the same 
> approach. See the message linked above for further backgound.
> I've been talking with [~kinow] about this and he may be interested in 
> implementing it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (JENA-1313) Language-specific collation in ARQ

2017-03-31 Thread Bruno P. Kinoshita (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950983#comment-15950983
 ] 

Bruno P. Kinoshita commented on JENA-1313:
--

Hi Osma,

>> return values in es, then pt, then fi
>Shouldn't this be es, fi, pt?

Ops, my mistake. Fixed my comment, also attaching a screen shot 
(collate-result-no-filter-fullpage.png) with the output in Fuseki.

>Dydra also has a test suite for collation that is published using the 
>Unlicense i.e. placed in the public domain. There I found an example of a 
>proper Danish language collation sequence that could perhaps also be used as a 
>test case. There are other test cases in that directory that may also be 
>relevant.

Used a few words from Dydra Danish test case: 

- Broager
- Brædstrup
- Børkop
- Wandsbek
- Ærøskøbing
- Åkirkeby

The last two get switched in Jena. ICU gives me the same order as in the Dydra 
test for the two Danish collation schemes available.

>For the extension function, I suggest defining it just as a single function 
>e.g. collate:collate that takes up to two parameters: the literal value and 
>the language/locale (which may be omitted, and then the language is extracted 
>from the language tag)

Good points. Would we need a third argument, to define the sorting order as ASC 
or DESC too? Or would we use ASC(collate:collate(?label, 'fi'))?

> Language-specific collation in ARQ
> --
>
> Key: JENA-1313
> URL: https://issues.apache.org/jira/browse/JENA-1313
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: ARQ
>Affects Versions: Jena 3.2.0
>Reporter: Osma Suominen
>
> As [discussed|http://markmail.org/message/v2bvsnsza5ksl2cv] on the users 
> mailing list in October 2016, I would like to change ARQ collation of literal 
> values to be language-aware and respect language-specific collation rules.
> This would probably involve changing at least the 
> [NodeUtils.compareLiteralsBySyntax|https://github.com/apache/jena/blob/master/jena-arq/src/main/java/org/apache/jena/sparql/util/NodeUtils.java#L199]
>  method.
> It currently sorts by lexical value first, then by language tag. Since the 
> collation order needs to be stable across all possible literal values, I 
> think the safest way would be to sort by language tag first, then by lexical 
> value according to the collation rules for that language.
> But what about subtags like {{@en-US}} or {{@pt-BR}}? Can they have different 
> collation rules than the main language? It would be a bit strange if all 
> {{@en-US}} literals sorted after {{@en}} literals...
> It would be good to check how Dydra does this and possibly take the same 
> approach. See the message linked above for further backgound.
> I've been talking with [~kinow] about this and he may be interested in 
> implementing it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (JENA-1302) Intermittent Lucene prepareCommit exception during heavy read/write load

2017-03-31 Thread Andy Seaborne (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950896#comment-15950896
 ] 

Andy Seaborne commented on JENA-1302:
-

Update from my side: I have written a java-based test driver that does not 
cause the build up of WAIT by using a more managed set of reused persistent 
connections. It does exhibit the lockup. It is caused by the journal overload 
code being invoked which takes a while to happen. The code is taking locks in 
the wrong order and this causes a deadlock then then builds up.

> Intermittent Lucene prepareCommit exception during heavy read/write load
> 
>
> Key: JENA-1302
> URL: https://issues.apache.org/jira/browse/JENA-1302
> Project: Apache Jena
>  Issue Type: Bug
>  Components: Fuseki, TDB, Text
>Affects Versions: Jena 3.3.0
> Environment: CentOS 7.3 inside VM, 2 cores, OpenJDK 1.8.0_121 
> (64-bit), 1GB JVM heap, Fuseki running as service
>Reporter: Vilnis Termanis (Iotic Labs)
>Assignee: Andy Seaborne
> Fix For: Jena 3.3.0
>
> Attachments: netstat.log, prepareCommit_error.tgz, 
> visualvm_thread_dump.log
>
>
> (*Note*: Could not add Fuseki 2.6.0 as affected version although I assume 
> this is a TDB/text rather than Fuseki-related issue.)
> *Steps:*
> # Start with plain Fuseki + given configuration (TDB store + lucene text 
> indexing)
> # Import 1k.ttl
> # Run lockup.py (same host, mix of multiple parallel updates & single select)
> *Result:*
> Every now and again one of the update queries fails with a 500 server error 
> which is logged as:
> {{DatasetGraphText ERROR Exception in prepareCommit: prepareCommit was 
> already called with no corresponding call to commit}}
> *Notes:*
> - Not reproducible without fix for JENA-1296 (and discovered during fix 
> verification for said issue)
> - Run with 
> [apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz|https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.6.0-SNAPSHOT/apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz]
> - From cold-start of Fuseki, occurs within 30s of script run
> - Supplied test script required Python 3, {{rdflib>=4.2.2}} & 
> {{SPARQLWrapper>=1.8.0}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version

2017-03-31 Thread Rob Vesse
Yes that would be a good idea

 You should just be able to change a single property and see if it Will 
compile. See the instructions on the website:

http://jena.apache.org/documentation/hadoop/artifacts.html

Rob

On 31/03/2017 13:57, "A. Soroka"  wrote:

So for next steps, should someone (aka me) try to build against a 3.X 
Hadoop to see where the jolts are? Does that make sense as a way to gather more 
info? I would just go ahead and do it, but lacking (as I do) any Hadoop dev 
experience, I want to make sure that it's a good idea before investing time.

---
A. Soroka
The University of Virginia Library

> On Mar 31, 2017, at 5:11 AM, Rob Vesse  wrote:
> 
> My impression is that 3.0 is still a long way off a generally available 
release since they are still in Alpha at the moment.
> 
> I would expect that 2.x would remain the primary distribution for a while 
yet.
> 
> I don’t know how much, If at all, map/reduce has changed so it may 
already be possible to build Elephas against 3.X without any changes but then 
again it might not.
> 
> I’m not sure that it is necessarily safe to stop shading Guava. It is a 
widely used library with poor compatibility between versions and our users may 
experience Version conflicts in other environments.
> 
> Rob
> 
> On 29/03/2017 00:18, "Bruno P. Kinoshita"  wrote:
> 
>> I would think we could go with a minor version for this, no? Or is 
>Elephas such a big part of our offering that we need to make a major 
>release to move with Hadoop...?
> 
> 
>I would think the same. Unless we have other important features to be 
released, maybe something that breaks backward compatibility, etc.
> 
> 
> 
>
>From: A. Soroka 
>To: dev@jena.apache.org 
>Sent: Wednesday, 29 March 2017 3:20 AM
>Subject: Re: [jira] [Updated] (HADOOP-10101) Update guava dependency 
to the latest version
> 
> 
> 
>> * Jena would have to move in step with Hadoop and drop support for older 
versions.
> 
>Maybe we can see what Hadoop itself ends up doing for this problem... 
they are targeting their own major release 3 for this. I would think we could 
go with a minor version for this, no? Or is Elephas such a big part of our 
offering that we need to make a major release to move with Hadoop...?
> 
>---
>A. Soroka
>The University of Virginia Library
> 
> 
>> On Mar 25, 2017, at 7:15 AM, Andy Seaborne  wrote:
>> 
>> 
>> Step 1: upgrade jena-shaded-guava to v21.0
>> 
>> and see if it works.
>> 
>> 
>> There are at least two considerations:
>> 
>> * Jena would have to move in step with Hadoop
>> and drop support for older versions.
>> * Our downstream users may have guava version choices.
>> 
>> and we have to wait until a Hadoop release to remove it.
>> 
>>   Andy
>> 
>> On 25/03/17 01:22, A. Soroka wrote:
>>> Looks like Hadoop has successfully updated their Guava dependency-- 
this might provide us the opportunity to stop shading Guava. (yay!)
>>> 
>>> ---
>>> A. Soroka
>>> The University of Virginia Library
>>> 
 Begin forwarded message:
 
 From: "Tsuyoshi Ozawa (JIRA)" 
 Subject: [jira] [Updated] (HADOOP-10101) Update guava dependency to 
the latest version
 Date: March 24, 2017 at 9:18:42 PM EDT
 To: aj...@apache.org
 
 
   [ 
https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]
 
 Tsuyoshi Ozawa updated HADOOP-10101:
 
 Resolution: Fixed
  Fix Version/s: 3.0.0-alpha3
 Status: Resolved  (was: Patch Available)
 
 Committed this to trunk. Thanks Nicholas and Steve for your review, 
and thanks people who joined this issue for your comments.
 
> Update guava dependency to the latest version
> -
> 
>  Key: HADOOP-10101
>  URL: https://issues.apache.org/jira/browse/HADOOP-10101
>  Project: Hadoop Common
>   Issue Type: Improvement
> Affects Versions: 3.0.0-alpha2
> Reporter: Rakesh R
> Assignee: Tsuyoshi Ozawa
>   Labels: BB2015-05-TBR
>  Fix For: 3.0.0-alpha3
> 
>  Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, 
HADOOP-10101-005.patch, HADOOP-10101-006.patch, HADOOP-10101-007.patch, 
HADOOP-10101-008.patch, HADOOP-10101-009.patch, 

[jira] [Commented] (JENA-632) Generate JSON from SPARQL directly.

2017-03-31 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950874#comment-15950874
 ] 

ASF GitHub Bot commented on JENA-632:
-

Github user kinow commented on a diff in the pull request:

https://github.com/apache/jena/pull/114#discussion_r109147739
  
--- Diff: jena-arq/Grammar/master.jj ---
@@ -100,6 +100,38 @@ import org.apache.jena.sparql.core.Quad ;
 public class CLASS extends PARSERBASE
--- End diff --

Done. First executed jj2tokens, compared the output (digressing: IRIRef 
seems to have been updated I guess) and added the missing JSON token.


> Generate JSON from SPARQL directly.
> ---
>
> Key: JENA-632
> URL: https://issues.apache.org/jira/browse/JENA-632
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: ARQ, Fuseki
>Reporter: Andy Seaborne
>Assignee: Bruno P. Kinoshita
>Priority: Minor
>  Labels: java, javacc
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The capability to generate JSON directly from a SPARQL (or extended SPARQL) 
> query would enable the creation of JSON data API over published linked data.
> This project would cover:
> # Design and publication of a design.
> # Refinement of design based on community feed
> # Implementation, including testing.
> # Refinement of implementation based on community feed
> Skills required: Java, some parser work, design and discussion with the user 
> community, basic understanding of HTTP and content negotiation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (JENA-632) Generate JSON from SPARQL directly.

2017-03-31 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950875#comment-15950875
 ] 

ASF GitHub Bot commented on JENA-632:
-

Github user kinow commented on a diff in the pull request:

https://github.com/apache/jena/pull/114#discussion_r109143318
  
--- Diff: jena-arq/Grammar/master.jj ---
@@ -327,6 +359,35 @@ void AskQuery() : {}
   SolutionModifier()
 }
 
+void JsonQuery() : {}
+{
+  JsonClause()
+  ( DatasetClause() )*
+  WhereClause()
+  SolutionModifier()
+}
+
+void JsonClause() : { Object o ; String s ; }
+{
+   { getQuery().setQueryJsonType() ; }
+  
+  s = String() < PNAME_NS >
+  (
--- End diff --

Replying to my previous comment. The reason why we are not using COLON, is 
that we get this error:

```
[2017-04-01 00:44:24] Fuseki INFO  [1] Query = JSON { "name" : ?name } 
WHERE { ?name ?a ?b } LIMIT 2
[2017-04-01 00:44:24] Fuseki INFO  [1] 400 Parse error: 
JSON { "name" : ?name } WHERE { ?name ?a ?b } LIMIT 2

Encountered "  ": "" at line 1, column 15.
Was expecting:
":" ...
 (27 ms)
```

The PNAME_NS (which is )? ":" >) matches the colon, 
probably for the default namespace declaration. And probably because PNAME_NS 
is declared before COLON.

Maybe moving COLON before PNAME_NS, and using COLON in the PNAME_NS rule 
declaration? Not sure if that could introduce other unwanted bugs.

For the time being going with Andy's suggestion to indicate when we were 
expecting a COLON, and actually have something matching PNAME_NS.


> Generate JSON from SPARQL directly.
> ---
>
> Key: JENA-632
> URL: https://issues.apache.org/jira/browse/JENA-632
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: ARQ, Fuseki
>Reporter: Andy Seaborne
>Assignee: Bruno P. Kinoshita
>Priority: Minor
>  Labels: java, javacc
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The capability to generate JSON directly from a SPARQL (or extended SPARQL) 
> query would enable the creation of JSON data API over published linked data.
> This project would cover:
> # Design and publication of a design.
> # Refinement of design based on community feed
> # Implementation, including testing.
> # Refinement of implementation based on community feed
> Skills required: Java, some parser work, design and discussion with the user 
> community, basic understanding of HTTP and content negotiation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[GitHub] jena pull request #114: JENA-632: Generate JSON from SPARQL directly

2017-03-31 Thread kinow
Github user kinow commented on a diff in the pull request:

https://github.com/apache/jena/pull/114#discussion_r109147739
  
--- Diff: jena-arq/Grammar/master.jj ---
@@ -100,6 +100,38 @@ import org.apache.jena.sparql.core.Quad ;
 public class CLASS extends PARSERBASE
--- End diff --

Done. First executed jj2tokens, compared the output (digressing: IRIRef 
seems to have been updated I guess) and added the missing JSON token.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] jena pull request #114: JENA-632: Generate JSON from SPARQL directly

2017-03-31 Thread kinow
Github user kinow commented on a diff in the pull request:

https://github.com/apache/jena/pull/114#discussion_r109143318
  
--- Diff: jena-arq/Grammar/master.jj ---
@@ -327,6 +359,35 @@ void AskQuery() : {}
   SolutionModifier()
 }
 
+void JsonQuery() : {}
+{
+  JsonClause()
+  ( DatasetClause() )*
+  WhereClause()
+  SolutionModifier()
+}
+
+void JsonClause() : { Object o ; String s ; }
+{
+   { getQuery().setQueryJsonType() ; }
+  
+  s = String() < PNAME_NS >
+  (
--- End diff --

Replying to my previous comment. The reason why we are not using COLON, is 
that we get this error:

```
[2017-04-01 00:44:24] Fuseki INFO  [1] Query = JSON { "name" : ?name } 
WHERE { ?name ?a ?b } LIMIT 2
[2017-04-01 00:44:24] Fuseki INFO  [1] 400 Parse error: 
JSON { "name" : ?name } WHERE { ?name ?a ?b } LIMIT 2

Encountered "  ": "" at line 1, column 15.
Was expecting:
":" ...
 (27 ms)
```

The PNAME_NS (which is )? ":" >) matches the colon, 
probably for the default namespace declaration. And probably because PNAME_NS 
is declared before COLON.

Maybe moving COLON before PNAME_NS, and using COLON in the PNAME_NS rule 
declaration? Not sure if that could introduce other unwanted bugs.

For the time being going with Andy's suggestion to indicate when we were 
expecting a COLON, and actually have something matching PNAME_NS.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (JENA-1313) Language-specific collation in ARQ

2017-03-31 Thread A. Soroka (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950836#comment-15950836
 ] 

A. Soroka commented on JENA-1313:
-

Agreed on the better value of a two-param function. Or maybe go [completely 
over the top|https://link.springer.com/chapter/10.1007/978-3-319-11915-1_26]? 
:grin: j/k

> Language-specific collation in ARQ
> --
>
> Key: JENA-1313
> URL: https://issues.apache.org/jira/browse/JENA-1313
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: ARQ
>Affects Versions: Jena 3.2.0
>Reporter: Osma Suominen
>
> As [discussed|http://markmail.org/message/v2bvsnsza5ksl2cv] on the users 
> mailing list in October 2016, I would like to change ARQ collation of literal 
> values to be language-aware and respect language-specific collation rules.
> This would probably involve changing at least the 
> [NodeUtils.compareLiteralsBySyntax|https://github.com/apache/jena/blob/master/jena-arq/src/main/java/org/apache/jena/sparql/util/NodeUtils.java#L199]
>  method.
> It currently sorts by lexical value first, then by language tag. Since the 
> collation order needs to be stable across all possible literal values, I 
> think the safest way would be to sort by language tag first, then by lexical 
> value according to the collation rules for that language.
> But what about subtags like {{@en-US}} or {{@pt-BR}}? Can they have different 
> collation rules than the main language? It would be a bit strange if all 
> {{@en-US}} literals sorted after {{@en}} literals...
> It would be good to check how Dydra does this and possibly take the same 
> approach. See the message linked above for further backgound.
> I've been talking with [~kinow] about this and he may be interested in 
> implementing it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version

2017-03-31 Thread A. Soroka
So for next steps, should someone (aka me) try to build against a 3.X Hadoop to 
see where the jolts are? Does that make sense as a way to gather more info? I 
would just go ahead and do it, but lacking (as I do) any Hadoop dev experience, 
I want to make sure that it's a good idea before investing time.

---
A. Soroka
The University of Virginia Library

> On Mar 31, 2017, at 5:11 AM, Rob Vesse  wrote:
> 
> My impression is that 3.0 is still a long way off a generally available 
> release since they are still in Alpha at the moment.
> 
> I would expect that 2.x would remain the primary distribution for a while yet.
> 
> I don’t know how much, If at all, map/reduce has changed so it may already be 
> possible to build Elephas against 3.X without any changes but then again it 
> might not.
> 
> I’m not sure that it is necessarily safe to stop shading Guava. It is a 
> widely used library with poor compatibility between versions and our users 
> may experience Version conflicts in other environments.
> 
> Rob
> 
> On 29/03/2017 00:18, "Bruno P. Kinoshita"  wrote:
> 
>> I would think we could go with a minor version for this, no? Or is 
>Elephas such a big part of our offering that we need to make a major 
>release to move with Hadoop...?
> 
> 
>I would think the same. Unless we have other important features to be 
> released, maybe something that breaks backward compatibility, etc.
> 
> 
> 
>
>From: A. Soroka 
>To: dev@jena.apache.org 
>Sent: Wednesday, 29 March 2017 3:20 AM
>Subject: Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to 
> the latest version
> 
> 
> 
>> * Jena would have to move in step with Hadoop and drop support for older 
>> versions.
> 
>Maybe we can see what Hadoop itself ends up doing for this problem... they 
> are targeting their own major release 3 for this. I would think we could go 
> with a minor version for this, no? Or is Elephas such a big part of our 
> offering that we need to make a major release to move with Hadoop...?
> 
>---
>A. Soroka
>The University of Virginia Library
> 
> 
>> On Mar 25, 2017, at 7:15 AM, Andy Seaborne  wrote:
>> 
>> 
>> Step 1: upgrade jena-shaded-guava to v21.0
>> 
>> and see if it works.
>> 
>> 
>> There are at least two considerations:
>> 
>> * Jena would have to move in step with Hadoop
>> and drop support for older versions.
>> * Our downstream users may have guava version choices.
>> 
>> and we have to wait until a Hadoop release to remove it.
>> 
>>   Andy
>> 
>> On 25/03/17 01:22, A. Soroka wrote:
>>> Looks like Hadoop has successfully updated their Guava dependency-- this 
>>> might provide us the opportunity to stop shading Guava. (yay!)
>>> 
>>> ---
>>> A. Soroka
>>> The University of Virginia Library
>>> 
 Begin forwarded message:
 
 From: "Tsuyoshi Ozawa (JIRA)" 
 Subject: [jira] [Updated] (HADOOP-10101) Update guava dependency to the 
 latest version
 Date: March 24, 2017 at 9:18:42 PM EDT
 To: aj...@apache.org
 
 
   [ 
 https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
  ]
 
 Tsuyoshi Ozawa updated HADOOP-10101:
 
 Resolution: Fixed
  Fix Version/s: 3.0.0-alpha3
 Status: Resolved  (was: Patch Available)
 
 Committed this to trunk. Thanks Nicholas and Steve for your review, and 
 thanks people who joined this issue for your comments.
 
> Update guava dependency to the latest version
> -
> 
>  Key: HADOOP-10101
>  URL: https://issues.apache.org/jira/browse/HADOOP-10101
>  Project: Hadoop Common
>   Issue Type: Improvement
> Affects Versions: 3.0.0-alpha2
> Reporter: Rakesh R
> Assignee: Tsuyoshi Ozawa
>   Labels: BB2015-05-TBR
>  Fix For: 3.0.0-alpha3
> 
>  Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, 
> HADOOP-10101-005.patch, HADOOP-10101-006.patch, HADOOP-10101-007.patch, 
> HADOOP-10101-008.patch, HADOOP-10101-009.patch, HADOOP-10101-009.patch, 
> HADOOP-10101-010.patch, HADOOP-10101-010.patch, HADOOP-10101-011.patch, 
> HADOOP-10101.012.patch, HADOOP-10101.013.patch, HADOOP-10101.014.patch, 
> HADOOP-10101.015.patch, HADOOP-10101.016.patch, HADOOP-10101.017.patch, 
> HADOOP-10101.018.patch, HADOOP-10101.patch, HADOOP-10101.patch
> 
> 
> The existing guava version is 11.0.2 which is quite old. This issue tries 
> to update the version to as latest version as possible.
 
 
 
 --
 This message was sent by Atlassian JIRA
 (v6.3.15#6346)
>>> 
> 
> 
> 
> 
> 



[jira] [Commented] (JENA-1313) Language-specific collation in ARQ

2017-03-31 Thread Osma Suominen (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950717#comment-15950717
 ] 

Osma Suominen commented on JENA-1313:
-

> return values in es, then pt, then fi
Shouldn't this be es, fi, pt?

Anyway, people seem to be agreeing on making an extension function rather than 
changing the default sorting order of literals.

I re-read the [description of the Dydra implementation of 
collation|http://blog.dydra.com/2015/05/06/collation], but they don't say what 
the ordering is between literals with differing language tags - it remains 
explicitly undefined, as in SPARQL. I don't know what ORDER BY will in practice 
do in Dydra when the language tags differ.

Dydra also has a [test 
suite|https://github.com/dydra/http-api-tests/tree/master/extensions/sparql-protocol/collation]
 for collation that is published using the Unlicense i.e. placed in the public 
domain. There I found [an 
example|https://github.com/dydra/http-api-tests/blob/master/extensions/sparql-protocol/collation/ordered-locations-da.txt]
 of a proper Danish language collation sequence that could perhaps also be used 
as a test case. There are other test cases in that directory that may also be 
relevant.

For the extension function, I suggest defining it just as a single function 
e.g. {{collate:collate}} that takes up to two parameters: the literal value and 
the language/locale (which may be omitted, and then the language is extracted 
from the language tag). So you can say e.g. {{ORDER BY collate:collate(?label, 
'fi')}} to get the labels sorted according to Finnish collation rules, 
regardless of their language tags. I think this is better than the 
{{collate:fi}} example given by [~kinow] above, because it requires just a 
single extension function and makes the locale/language a parameter that may 
come from a SPARQL variable instead of being hardwired into the query. A 
special parameter value such as {{unicode}} or the empty string could be used 
to force Unicode collation rules. (even a one-parameter version of 
{{collate:collate}} could be used in this way by passing {{STRLANG(?label, 
'fi')}} as the parameter, but that seems unnecessarily complicated to me)

> Language-specific collation in ARQ
> --
>
> Key: JENA-1313
> URL: https://issues.apache.org/jira/browse/JENA-1313
> Project: Apache Jena
>  Issue Type: Improvement
>  Components: ARQ
>Affects Versions: Jena 3.2.0
>Reporter: Osma Suominen
>
> As [discussed|http://markmail.org/message/v2bvsnsza5ksl2cv] on the users 
> mailing list in October 2016, I would like to change ARQ collation of literal 
> values to be language-aware and respect language-specific collation rules.
> This would probably involve changing at least the 
> [NodeUtils.compareLiteralsBySyntax|https://github.com/apache/jena/blob/master/jena-arq/src/main/java/org/apache/jena/sparql/util/NodeUtils.java#L199]
>  method.
> It currently sorts by lexical value first, then by language tag. Since the 
> collation order needs to be stable across all possible literal values, I 
> think the safest way would be to sort by language tag first, then by lexical 
> value according to the collation rules for that language.
> But what about subtags like {{@en-US}} or {{@pt-BR}}? Can they have different 
> collation rules than the main language? It would be a bit strange if all 
> {{@en-US}} literals sorted after {{@en}} literals...
> It would be good to check how Dydra does this and possibly take the same 
> approach. See the message linked above for further backgound.
> I've been talking with [~kinow] about this and he may be interested in 
> implementing it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Updated] (JENA-1302) Intermittent Lucene prepareCommit exception during heavy read/write load

2017-03-31 Thread Vilnis Termanis (Iotic Labs) (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vilnis Termanis (Iotic Labs) updated JENA-1302:
---
Attachment: netstat.log

netstat socket state counts during lockup.py run (with annotations)

> Intermittent Lucene prepareCommit exception during heavy read/write load
> 
>
> Key: JENA-1302
> URL: https://issues.apache.org/jira/browse/JENA-1302
> Project: Apache Jena
>  Issue Type: Bug
>  Components: Fuseki, TDB, Text
>Affects Versions: Jena 3.3.0
> Environment: CentOS 7.3 inside VM, 2 cores, OpenJDK 1.8.0_121 
> (64-bit), 1GB JVM heap, Fuseki running as service
>Reporter: Vilnis Termanis (Iotic Labs)
>Assignee: Andy Seaborne
> Fix For: Jena 3.3.0
>
> Attachments: netstat.log, prepareCommit_error.tgz, 
> visualvm_thread_dump.log
>
>
> (*Note*: Could not add Fuseki 2.6.0 as affected version although I assume 
> this is a TDB/text rather than Fuseki-related issue.)
> *Steps:*
> # Start with plain Fuseki + given configuration (TDB store + lucene text 
> indexing)
> # Import 1k.ttl
> # Run lockup.py (same host, mix of multiple parallel updates & single select)
> *Result:*
> Every now and again one of the update queries fails with a 500 server error 
> which is logged as:
> {{DatasetGraphText ERROR Exception in prepareCommit: prepareCommit was 
> already called with no corresponding call to commit}}
> *Notes:*
> - Not reproducible without fix for JENA-1296 (and discovered during fix 
> verification for said issue)
> - Run with 
> [apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz|https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.6.0-SNAPSHOT/apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz]
> - From cold-start of Fuseki, occurs within 30s of script run
> - Supplied test script required Python 3, {{rdflib>=4.2.2}} & 
> {{SPARQLWrapper>=1.8.0}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (JENA-1302) Intermittent Lucene prepareCommit exception during heavy read/write load

2017-03-31 Thread Vilnis Termanis (Iotic Labs) (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950653#comment-15950653
 ] 

Vilnis Termanis (Iotic Labs) commented on JENA-1302:


>From your description I think we're seeing the same thing.I tracked states for 
>fuseki sockets during the script run (see annotated netstat.log attached).
{code}
while true; do date && netstat -np | grep 3030 | awk '{print $6}' | sort | uniq 
-c && sleep 2; done
{code}

First of all from my understanding, CLOSE_WAIT indicates that the client 
(lockup.py) has closed but the server (fuseki/jetty) hasn't (quoting netstat 
man page):
bq. CLOSE_WAIT: The remote end has shut down, waiting for the socket to close. 


# Initially TIME_WAITs build up (indicating successful finished requests)
# Once the lockup occurs (CPU usage drops) and after at least one request made 
by lockup.py times outs (30s), at least one CLOSE_WAIT can be seen (indicating 
that, although the client has finished the request the server has not yet 
closed the connection.)
# As expected, TIME_WAITs gradually disappear due to lack of new requests being 
completed
# Any further requests made thereafter (e.g. "selection of triples" query 
example via UI or running lockup.py again) produce more connections in 
CLOSE_WAIT once the client has timed out due to lack of server response (or 
when one e.g. closes the browser tab/window).

*Conclusion*: The eventual CLOSE_WAIT state is the symptom of Fuseki (via 
Jetty) no longer being able to handle additional requests due there still being 
a lock held, preventing any other request from being worked on in parallel. 
(I.e. same sort of external behaviour as observed in JENA-1296.)

> Intermittent Lucene prepareCommit exception during heavy read/write load
> 
>
> Key: JENA-1302
> URL: https://issues.apache.org/jira/browse/JENA-1302
> Project: Apache Jena
>  Issue Type: Bug
>  Components: Fuseki, TDB, Text
>Affects Versions: Jena 3.3.0
> Environment: CentOS 7.3 inside VM, 2 cores, OpenJDK 1.8.0_121 
> (64-bit), 1GB JVM heap, Fuseki running as service
>Reporter: Vilnis Termanis (Iotic Labs)
>Assignee: Andy Seaborne
> Fix For: Jena 3.3.0
>
> Attachments: prepareCommit_error.tgz, visualvm_thread_dump.log
>
>
> (*Note*: Could not add Fuseki 2.6.0 as affected version although I assume 
> this is a TDB/text rather than Fuseki-related issue.)
> *Steps:*
> # Start with plain Fuseki + given configuration (TDB store + lucene text 
> indexing)
> # Import 1k.ttl
> # Run lockup.py (same host, mix of multiple parallel updates & single select)
> *Result:*
> Every now and again one of the update queries fails with a 500 server error 
> which is logged as:
> {{DatasetGraphText ERROR Exception in prepareCommit: prepareCommit was 
> already called with no corresponding call to commit}}
> *Notes:*
> - Not reproducible without fix for JENA-1296 (and discovered during fix 
> verification for said issue)
> - Run with 
> [apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz|https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.6.0-SNAPSHOT/apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz]
> - From cold-start of Fuseki, occurs within 30s of script run
> - Supplied test script required Python 3, {{rdflib>=4.2.2}} & 
> {{SPARQLWrapper>=1.8.0}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version

2017-03-31 Thread Rob Vesse
My impression is that 3.0 is still a long way off a generally available release 
since they are still in Alpha at the moment.

 I would expect that 2.x would remain the primary distribution for a while yet.

 I don’t know how much, If at all, map/reduce has changed so it may already be 
possible to build Elephas against 3.X without any changes but then again it 
might not.

 I’m not sure that it is necessarily safe to stop shading Guava. It is a widely 
used library with poor compatibility between versions and our users may 
experience Version conflicts in other environments.

 Rob

On 29/03/2017 00:18, "Bruno P. Kinoshita"  wrote:

>I would think we could go with a minor version for this, no? Or is 
Elephas such a big part of our offering that we need to make a major 
release to move with Hadoop...?


I would think the same. Unless we have other important features to be 
released, maybe something that breaks backward compatibility, etc.




From: A. Soroka 
To: dev@jena.apache.org 
Sent: Wednesday, 29 March 2017 3:20 AM
Subject: Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to the 
latest version



> * Jena would have to move in step with Hadoop and drop support for older 
versions.

Maybe we can see what Hadoop itself ends up doing for this problem... they 
are targeting their own major release 3 for this. I would think we could go 
with a minor version for this, no? Or is Elephas such a big part of our 
offering that we need to make a major release to move with Hadoop...?

---
A. Soroka
The University of Virginia Library


> On Mar 25, 2017, at 7:15 AM, Andy Seaborne  wrote:
> 
> 
> Step 1: upgrade jena-shaded-guava to v21.0
> 
> and see if it works.
> 
> 
> There are at least two considerations:
> 
> * Jena would have to move in step with Hadoop
>  and drop support for older versions.
> * Our downstream users may have guava version choices.
> 
> and we have to wait until a Hadoop release to remove it.
> 
>Andy
> 
> On 25/03/17 01:22, A. Soroka wrote:
>> Looks like Hadoop has successfully updated their Guava dependency-- this 
might provide us the opportunity to stop shading Guava. (yay!)
>> 
>> ---
>> A. Soroka
>> The University of Virginia Library
>> 
>>> Begin forwarded message:
>>> 
>>> From: "Tsuyoshi Ozawa (JIRA)" 
>>> Subject: [jira] [Updated] (HADOOP-10101) Update guava dependency to the 
latest version
>>> Date: March 24, 2017 at 9:18:42 PM EDT
>>> To: aj...@apache.org
>>> 
>>> 
>>>[ 
https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]
>>> 
>>> Tsuyoshi Ozawa updated HADOOP-10101:
>>> 
>>>  Resolution: Fixed
>>>   Fix Version/s: 3.0.0-alpha3
>>>  Status: Resolved  (was: Patch Available)
>>> 
>>> Committed this to trunk. Thanks Nicholas and Steve for your review, and 
thanks people who joined this issue for your comments.
>>> 
 Update guava dependency to the latest version
 -
 
   Key: HADOOP-10101
   URL: https://issues.apache.org/jira/browse/HADOOP-10101
   Project: Hadoop Common
Issue Type: Improvement
  Affects Versions: 3.0.0-alpha2
  Reporter: Rakesh R
  Assignee: Tsuyoshi Ozawa
Labels: BB2015-05-TBR
   Fix For: 3.0.0-alpha3
 
   Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, 
HADOOP-10101-005.patch, HADOOP-10101-006.patch, HADOOP-10101-007.patch, 
HADOOP-10101-008.patch, HADOOP-10101-009.patch, HADOOP-10101-009.patch, 
HADOOP-10101-010.patch, HADOOP-10101-010.patch, HADOOP-10101-011.patch, 
HADOOP-10101.012.patch, HADOOP-10101.013.patch, HADOOP-10101.014.patch, 
HADOOP-10101.015.patch, HADOOP-10101.016.patch, HADOOP-10101.017.patch, 
HADOOP-10101.018.patch, HADOOP-10101.patch, HADOOP-10101.patch
 
 
 The existing guava version is 11.0.2 which is quite old. This issue 
tries to update the version to as latest version as possible.
>>> 
>>> 
>>> 
>>> --
>>> This message was sent by Atlassian JIRA
>>> (v6.3.15#6346)
>>