[jira] [Commented] (JENA-1314) Update Using Jena with Eclipse page
[ https://issues.apache.org/jira/browse/JENA-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951794#comment-15951794 ] Bruno P. Kinoshita commented on JENA-1314: -- Done. Committed to trunk on SVN so that is easier to review. We can easily revert to the previous version too. http://jena.staging.apache.org/tutorials/using_jena_with_eclipse.html Leaving this issue open for review, but if there are no objections will close it later, and update the Portuguese version too (http://jena.staging.apache.org/tutorials/using_jena_with_eclipse_pt.html). Looks like this is the only translation and I am able to update it, so we do not have to remove the page and look for broken links. Cheers Bruno > Update Using Jena with Eclipse page > --- > > Key: JENA-1314 > URL: https://issues.apache.org/jira/browse/JENA-1314 > Project: Apache Jena > Issue Type: Documentation > Components: Documentation >Reporter: Bruno P. Kinoshita >Assignee: Bruno P. Kinoshita >Priority: Minor > Labels: documentation, website > > The Using Jena with Eclipse page in our web site is empty, with a message > "Revised version coming soon. In the meantime, please see the old version.". > The old version is quite old, and would be nice to address the issues with > shaded Guava for now. > I have screen shots to update the page in the next days. Once we fix the > Guava shade issues, it will be a matter of removing a few paragraphs and > images. > [1] http://jena.apache.org/tutorials/using_jena_with_eclipse.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (JENA-1314) Update Using Jena with Eclipse page
[ https://issues.apache.org/jira/browse/JENA-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951775#comment-15951775 ] ASF subversion and git services commented on JENA-1314: --- Commit 1789762 from [~kinow] in branch 'site/trunk' [ https://svn.apache.org/r1789762 ] JENA-1314 updating Eclipse build instructions > Update Using Jena with Eclipse page > --- > > Key: JENA-1314 > URL: https://issues.apache.org/jira/browse/JENA-1314 > Project: Apache Jena > Issue Type: Documentation > Components: Documentation >Reporter: Bruno P. Kinoshita >Assignee: Bruno P. Kinoshita >Priority: Minor > Labels: documentation, website > > The Using Jena with Eclipse page in our web site is empty, with a message > "Revised version coming soon. In the meantime, please see the old version.". > The old version is quite old, and would be nice to address the issues with > shaded Guava for now. > I have screen shots to update the page in the next days. Once we fix the > Guava shade issues, it will be a matter of removing a few paragraphs and > images. > [1] http://jena.apache.org/tutorials/using_jena_with_eclipse.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Bug in SortedDataBag?
Hi Andy Yes, the cancel thread doesn't call open(), I understand that. The warnings are intermittent because the iterator in question (the "source" interator of QueryIterSort) is only left open if the cancel request is noticed while the sort databag is filling up. QueryIterSort never explicitly closes the source iterator. But if the sort databag pulls all of the source bindings, the source iterator self-closes. So there's a window while the bag is filling when a detected cancellation will leave the source iterator open. A fix is to ensure that the source iterator is closed when QueryIterSort closes: @Override public void closeIterator() { embeddedIterator.close(); super.closeIterator(); } "embeddedIterator" is the source iterator. This fix eliminates the presenting problem and the tests continue to pass /except/ that CallBackIterator's close() method needs to be made a no-op rather than failing, since that open() is now actually called. [It looks like QueryIterTopN has the same problem] Chris On 20 March 2017 at 15:48, Chris Dollinwrote: > On 17 March 2017 at 17:08, Andy Seaborne wrote: > >> >> SortedDataBag.cancel does not call close. >> > > You're quite right, am having another look at the problem. > > Chris > -- "What I don't understand is this ..." Trevor Chaplin, /The Beiderbeck Affair/ Epimorphics Ltd, http://www.epimorphics.com Registered address: Court Lodge, 105 High Street, Portishead, Bristol BS20 6PT Epimorphics Ltd. is a limited company registered in England (number 7016688)
[GitHub] jena issue #230: JENA-733 add documentation for narrower data type case
Github user rvesse commented on the issue: https://github.com/apache/jena/pull/230 +1 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (JENA-733) Long typed literals report class java.lang.Long but return java.lang.Integer
[ https://issues.apache.org/jira/browse/JENA-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951133#comment-15951133 ] ASF GitHub Bot commented on JENA-733: - GitHub user kinow opened a pull request: https://github.com/apache/jena/pull/230 JENA-733 add documentation for narrower data type case Tried to clarify what happens when users provide Long values that fit into Integers, for [JENA-733](https://issues.apache.org/jira/browse/JENA-733). I think if the docs are good to be merged, then the issue in JIRA can be closed too. Cheers Bruno You can merge this pull request into a Git repository by running: $ git pull https://github.com/kinow/jena JENA-733-javadocs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/jena/pull/230.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #230 commit 58cd30308e36dda3e2686953257ec771d72f681b Author: Bruno P. KinoshitaDate: 2017-03-31T15:28:41Z JENA-733 add documentation for narrower data type case > Long typed literals report class java.lang.Long but return java.lang.Integer > > > Key: JENA-733 > URL: https://issues.apache.org/jira/browse/JENA-733 > Project: Apache Jena > Issue Type: Bug > Components: RDF API >Affects Versions: Jena 2.11.2 >Reporter: Benno Willoweit >Priority: Minor > > This is my first bug report ever so I hope I get this right. When retrieving > query results with ARQ, long values are returned as Integers: > {{QuerySulution qs;}} > {{Literal l = qs.getLiteral("myVar");}} > {{l.getDatatype().getJavaClass(); //returns java.lang.Long which is correct}} > {{l.getValue(); //returns a java.lang.Integer object but it should return a > java.lang.Long object according to the docs}} > I have this problem with ARQ 2.11.2 which does not exist in the "affects > version" drop down. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (JENA-1314) Update Using Jena with Eclipse page
Bruno P. Kinoshita created JENA-1314: Summary: Update Using Jena with Eclipse page Key: JENA-1314 URL: https://issues.apache.org/jira/browse/JENA-1314 Project: Apache Jena Issue Type: Documentation Components: Documentation Reporter: Bruno P. Kinoshita Assignee: Bruno P. Kinoshita Priority: Minor The Using Jena with Eclipse page in our web site is empty, with a message "Revised version coming soon. In the meantime, please see the old version.". The old version is quite old, and would be nice to address the issues with shaded Guava for now. I have screen shots to update the page in the next days. Once we fix the Guava shade issues, it will be a matter of removing a few paragraphs and images. [1] http://jena.apache.org/tutorials/using_jena_with_eclipse.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (JENA-1313) Language-specific collation in ARQ
[ https://issues.apache.org/jira/browse/JENA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951009#comment-15951009 ] Andy Seaborne commented on JENA-1313: - We should not need ICU4J - the JDK should it all by now. jena used to use ICU4J and it was removed because Java acquired sufficient capabilities. And it's large. If {{collate:collate(?label, 'fi')}} works, the reverse sort is {{- collate:collate(?label, 'fi')}}. ASC and DESC would work. > Language-specific collation in ARQ > -- > > Key: JENA-1313 > URL: https://issues.apache.org/jira/browse/JENA-1313 > Project: Apache Jena > Issue Type: Improvement > Components: ARQ >Affects Versions: Jena 3.2.0 >Reporter: Osma Suominen > > As [discussed|http://markmail.org/message/v2bvsnsza5ksl2cv] on the users > mailing list in October 2016, I would like to change ARQ collation of literal > values to be language-aware and respect language-specific collation rules. > This would probably involve changing at least the > [NodeUtils.compareLiteralsBySyntax|https://github.com/apache/jena/blob/master/jena-arq/src/main/java/org/apache/jena/sparql/util/NodeUtils.java#L199] > method. > It currently sorts by lexical value first, then by language tag. Since the > collation order needs to be stable across all possible literal values, I > think the safest way would be to sort by language tag first, then by lexical > value according to the collation rules for that language. > But what about subtags like {{@en-US}} or {{@pt-BR}}? Can they have different > collation rules than the main language? It would be a bit strange if all > {{@en-US}} literals sorted after {{@en}} literals... > It would be good to check how Dydra does this and possibly take the same > approach. See the message linked above for further backgound. > I've been talking with [~kinow] about this and he may be interested in > implementing it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (JENA-1313) Language-specific collation in ARQ
[ https://issues.apache.org/jira/browse/JENA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950983#comment-15950983 ] Bruno P. Kinoshita commented on JENA-1313: -- Hi Osma, >> return values in es, then pt, then fi >Shouldn't this be es, fi, pt? Ops, my mistake. Fixed my comment, also attaching a screen shot (collate-result-no-filter-fullpage.png) with the output in Fuseki. >Dydra also has a test suite for collation that is published using the >Unlicense i.e. placed in the public domain. There I found an example of a >proper Danish language collation sequence that could perhaps also be used as a >test case. There are other test cases in that directory that may also be >relevant. Used a few words from Dydra Danish test case: - Broager - Brædstrup - Børkop - Wandsbek - Ærøskøbing - Åkirkeby The last two get switched in Jena. ICU gives me the same order as in the Dydra test for the two Danish collation schemes available. >For the extension function, I suggest defining it just as a single function >e.g. collate:collate that takes up to two parameters: the literal value and >the language/locale (which may be omitted, and then the language is extracted >from the language tag) Good points. Would we need a third argument, to define the sorting order as ASC or DESC too? Or would we use ASC(collate:collate(?label, 'fi'))? > Language-specific collation in ARQ > -- > > Key: JENA-1313 > URL: https://issues.apache.org/jira/browse/JENA-1313 > Project: Apache Jena > Issue Type: Improvement > Components: ARQ >Affects Versions: Jena 3.2.0 >Reporter: Osma Suominen > > As [discussed|http://markmail.org/message/v2bvsnsza5ksl2cv] on the users > mailing list in October 2016, I would like to change ARQ collation of literal > values to be language-aware and respect language-specific collation rules. > This would probably involve changing at least the > [NodeUtils.compareLiteralsBySyntax|https://github.com/apache/jena/blob/master/jena-arq/src/main/java/org/apache/jena/sparql/util/NodeUtils.java#L199] > method. > It currently sorts by lexical value first, then by language tag. Since the > collation order needs to be stable across all possible literal values, I > think the safest way would be to sort by language tag first, then by lexical > value according to the collation rules for that language. > But what about subtags like {{@en-US}} or {{@pt-BR}}? Can they have different > collation rules than the main language? It would be a bit strange if all > {{@en-US}} literals sorted after {{@en}} literals... > It would be good to check how Dydra does this and possibly take the same > approach. See the message linked above for further backgound. > I've been talking with [~kinow] about this and he may be interested in > implementing it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (JENA-1302) Intermittent Lucene prepareCommit exception during heavy read/write load
[ https://issues.apache.org/jira/browse/JENA-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950896#comment-15950896 ] Andy Seaborne commented on JENA-1302: - Update from my side: I have written a java-based test driver that does not cause the build up of WAIT by using a more managed set of reused persistent connections. It does exhibit the lockup. It is caused by the journal overload code being invoked which takes a while to happen. The code is taking locks in the wrong order and this causes a deadlock then then builds up. > Intermittent Lucene prepareCommit exception during heavy read/write load > > > Key: JENA-1302 > URL: https://issues.apache.org/jira/browse/JENA-1302 > Project: Apache Jena > Issue Type: Bug > Components: Fuseki, TDB, Text >Affects Versions: Jena 3.3.0 > Environment: CentOS 7.3 inside VM, 2 cores, OpenJDK 1.8.0_121 > (64-bit), 1GB JVM heap, Fuseki running as service >Reporter: Vilnis Termanis (Iotic Labs) >Assignee: Andy Seaborne > Fix For: Jena 3.3.0 > > Attachments: netstat.log, prepareCommit_error.tgz, > visualvm_thread_dump.log > > > (*Note*: Could not add Fuseki 2.6.0 as affected version although I assume > this is a TDB/text rather than Fuseki-related issue.) > *Steps:* > # Start with plain Fuseki + given configuration (TDB store + lucene text > indexing) > # Import 1k.ttl > # Run lockup.py (same host, mix of multiple parallel updates & single select) > *Result:* > Every now and again one of the update queries fails with a 500 server error > which is logged as: > {{DatasetGraphText ERROR Exception in prepareCommit: prepareCommit was > already called with no corresponding call to commit}} > *Notes:* > - Not reproducible without fix for JENA-1296 (and discovered during fix > verification for said issue) > - Run with > [apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz|https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.6.0-SNAPSHOT/apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz] > - From cold-start of Fuseki, occurs within 30s of script run > - Supplied test script required Python 3, {{rdflib>=4.2.2}} & > {{SPARQLWrapper>=1.8.0}} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version
Yes that would be a good idea You should just be able to change a single property and see if it Will compile. See the instructions on the website: http://jena.apache.org/documentation/hadoop/artifacts.html Rob On 31/03/2017 13:57, "A. Soroka"wrote: So for next steps, should someone (aka me) try to build against a 3.X Hadoop to see where the jolts are? Does that make sense as a way to gather more info? I would just go ahead and do it, but lacking (as I do) any Hadoop dev experience, I want to make sure that it's a good idea before investing time. --- A. Soroka The University of Virginia Library > On Mar 31, 2017, at 5:11 AM, Rob Vesse wrote: > > My impression is that 3.0 is still a long way off a generally available release since they are still in Alpha at the moment. > > I would expect that 2.x would remain the primary distribution for a while yet. > > I don’t know how much, If at all, map/reduce has changed so it may already be possible to build Elephas against 3.X without any changes but then again it might not. > > I’m not sure that it is necessarily safe to stop shading Guava. It is a widely used library with poor compatibility between versions and our users may experience Version conflicts in other environments. > > Rob > > On 29/03/2017 00:18, "Bruno P. Kinoshita" wrote: > >> I would think we could go with a minor version for this, no? Or is >Elephas such a big part of our offering that we need to make a major >release to move with Hadoop...? > > >I would think the same. Unless we have other important features to be released, maybe something that breaks backward compatibility, etc. > > > > >From: A. Soroka >To: dev@jena.apache.org >Sent: Wednesday, 29 March 2017 3:20 AM >Subject: Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version > > > >> * Jena would have to move in step with Hadoop and drop support for older versions. > >Maybe we can see what Hadoop itself ends up doing for this problem... they are targeting their own major release 3 for this. I would think we could go with a minor version for this, no? Or is Elephas such a big part of our offering that we need to make a major release to move with Hadoop...? > >--- >A. Soroka >The University of Virginia Library > > >> On Mar 25, 2017, at 7:15 AM, Andy Seaborne wrote: >> >> >> Step 1: upgrade jena-shaded-guava to v21.0 >> >> and see if it works. >> >> >> There are at least two considerations: >> >> * Jena would have to move in step with Hadoop >> and drop support for older versions. >> * Our downstream users may have guava version choices. >> >> and we have to wait until a Hadoop release to remove it. >> >> Andy >> >> On 25/03/17 01:22, A. Soroka wrote: >>> Looks like Hadoop has successfully updated their Guava dependency-- this might provide us the opportunity to stop shading Guava. (yay!) >>> >>> --- >>> A. Soroka >>> The University of Virginia Library >>> Begin forwarded message: From: "Tsuyoshi Ozawa (JIRA)" Subject: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version Date: March 24, 2017 at 9:18:42 PM EDT To: aj...@apache.org [ https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated HADOOP-10101: Resolution: Fixed Fix Version/s: 3.0.0-alpha3 Status: Resolved (was: Patch Available) Committed this to trunk. Thanks Nicholas and Steve for your review, and thanks people who joined this issue for your comments. > Update guava dependency to the latest version > - > > Key: HADOOP-10101 > URL: https://issues.apache.org/jira/browse/HADOOP-10101 > Project: Hadoop Common > Issue Type: Improvement > Affects Versions: 3.0.0-alpha2 > Reporter: Rakesh R > Assignee: Tsuyoshi Ozawa > Labels: BB2015-05-TBR > Fix For: 3.0.0-alpha3 > > Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, HADOOP-10101-005.patch, HADOOP-10101-006.patch, HADOOP-10101-007.patch, HADOOP-10101-008.patch, HADOOP-10101-009.patch,
[jira] [Commented] (JENA-632) Generate JSON from SPARQL directly.
[ https://issues.apache.org/jira/browse/JENA-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950874#comment-15950874 ] ASF GitHub Bot commented on JENA-632: - Github user kinow commented on a diff in the pull request: https://github.com/apache/jena/pull/114#discussion_r109147739 --- Diff: jena-arq/Grammar/master.jj --- @@ -100,6 +100,38 @@ import org.apache.jena.sparql.core.Quad ; public class CLASS extends PARSERBASE --- End diff -- Done. First executed jj2tokens, compared the output (digressing: IRIRef seems to have been updated I guess) and added the missing JSON token. > Generate JSON from SPARQL directly. > --- > > Key: JENA-632 > URL: https://issues.apache.org/jira/browse/JENA-632 > Project: Apache Jena > Issue Type: Improvement > Components: ARQ, Fuseki >Reporter: Andy Seaborne >Assignee: Bruno P. Kinoshita >Priority: Minor > Labels: java, javacc > Time Spent: 0.5h > Remaining Estimate: 0h > > The capability to generate JSON directly from a SPARQL (or extended SPARQL) > query would enable the creation of JSON data API over published linked data. > This project would cover: > # Design and publication of a design. > # Refinement of design based on community feed > # Implementation, including testing. > # Refinement of implementation based on community feed > Skills required: Java, some parser work, design and discussion with the user > community, basic understanding of HTTP and content negotiation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (JENA-632) Generate JSON from SPARQL directly.
[ https://issues.apache.org/jira/browse/JENA-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950875#comment-15950875 ] ASF GitHub Bot commented on JENA-632: - Github user kinow commented on a diff in the pull request: https://github.com/apache/jena/pull/114#discussion_r109143318 --- Diff: jena-arq/Grammar/master.jj --- @@ -327,6 +359,35 @@ void AskQuery() : {} SolutionModifier() } +void JsonQuery() : {} +{ + JsonClause() + ( DatasetClause() )* + WhereClause() + SolutionModifier() +} + +void JsonClause() : { Object o ; String s ; } +{ + { getQuery().setQueryJsonType() ; } + + s = String() < PNAME_NS > + ( --- End diff -- Replying to my previous comment. The reason why we are not using COLON, is that we get this error: ``` [2017-04-01 00:44:24] Fuseki INFO [1] Query = JSON { "name" : ?name } WHERE { ?name ?a ?b } LIMIT 2 [2017-04-01 00:44:24] Fuseki INFO [1] 400 Parse error: JSON { "name" : ?name } WHERE { ?name ?a ?b } LIMIT 2 Encountered " ": "" at line 1, column 15. Was expecting: ":" ... (27 ms) ``` The PNAME_NS (which is )? ":" >) matches the colon, probably for the default namespace declaration. And probably because PNAME_NS is declared before COLON. Maybe moving COLON before PNAME_NS, and using COLON in the PNAME_NS rule declaration? Not sure if that could introduce other unwanted bugs. For the time being going with Andy's suggestion to indicate when we were expecting a COLON, and actually have something matching PNAME_NS. > Generate JSON from SPARQL directly. > --- > > Key: JENA-632 > URL: https://issues.apache.org/jira/browse/JENA-632 > Project: Apache Jena > Issue Type: Improvement > Components: ARQ, Fuseki >Reporter: Andy Seaborne >Assignee: Bruno P. Kinoshita >Priority: Minor > Labels: java, javacc > Time Spent: 0.5h > Remaining Estimate: 0h > > The capability to generate JSON directly from a SPARQL (or extended SPARQL) > query would enable the creation of JSON data API over published linked data. > This project would cover: > # Design and publication of a design. > # Refinement of design based on community feed > # Implementation, including testing. > # Refinement of implementation based on community feed > Skills required: Java, some parser work, design and discussion with the user > community, basic understanding of HTTP and content negotiation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] jena pull request #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on a diff in the pull request: https://github.com/apache/jena/pull/114#discussion_r109147739 --- Diff: jena-arq/Grammar/master.jj --- @@ -100,6 +100,38 @@ import org.apache.jena.sparql.core.Quad ; public class CLASS extends PARSERBASE --- End diff -- Done. First executed jj2tokens, compared the output (digressing: IRIRef seems to have been updated I guess) and added the missing JSON token. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] jena pull request #114: JENA-632: Generate JSON from SPARQL directly
Github user kinow commented on a diff in the pull request: https://github.com/apache/jena/pull/114#discussion_r109143318 --- Diff: jena-arq/Grammar/master.jj --- @@ -327,6 +359,35 @@ void AskQuery() : {} SolutionModifier() } +void JsonQuery() : {} +{ + JsonClause() + ( DatasetClause() )* + WhereClause() + SolutionModifier() +} + +void JsonClause() : { Object o ; String s ; } +{ + { getQuery().setQueryJsonType() ; } + + s = String() < PNAME_NS > + ( --- End diff -- Replying to my previous comment. The reason why we are not using COLON, is that we get this error: ``` [2017-04-01 00:44:24] Fuseki INFO [1] Query = JSON { "name" : ?name } WHERE { ?name ?a ?b } LIMIT 2 [2017-04-01 00:44:24] Fuseki INFO [1] 400 Parse error: JSON { "name" : ?name } WHERE { ?name ?a ?b } LIMIT 2 Encountered " ": "" at line 1, column 15. Was expecting: ":" ... (27 ms) ``` The PNAME_NS (which is )? ":" >) matches the colon, probably for the default namespace declaration. And probably because PNAME_NS is declared before COLON. Maybe moving COLON before PNAME_NS, and using COLON in the PNAME_NS rule declaration? Not sure if that could introduce other unwanted bugs. For the time being going with Andy's suggestion to indicate when we were expecting a COLON, and actually have something matching PNAME_NS. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (JENA-1313) Language-specific collation in ARQ
[ https://issues.apache.org/jira/browse/JENA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950836#comment-15950836 ] A. Soroka commented on JENA-1313: - Agreed on the better value of a two-param function. Or maybe go [completely over the top|https://link.springer.com/chapter/10.1007/978-3-319-11915-1_26]? :grin: j/k > Language-specific collation in ARQ > -- > > Key: JENA-1313 > URL: https://issues.apache.org/jira/browse/JENA-1313 > Project: Apache Jena > Issue Type: Improvement > Components: ARQ >Affects Versions: Jena 3.2.0 >Reporter: Osma Suominen > > As [discussed|http://markmail.org/message/v2bvsnsza5ksl2cv] on the users > mailing list in October 2016, I would like to change ARQ collation of literal > values to be language-aware and respect language-specific collation rules. > This would probably involve changing at least the > [NodeUtils.compareLiteralsBySyntax|https://github.com/apache/jena/blob/master/jena-arq/src/main/java/org/apache/jena/sparql/util/NodeUtils.java#L199] > method. > It currently sorts by lexical value first, then by language tag. Since the > collation order needs to be stable across all possible literal values, I > think the safest way would be to sort by language tag first, then by lexical > value according to the collation rules for that language. > But what about subtags like {{@en-US}} or {{@pt-BR}}? Can they have different > collation rules than the main language? It would be a bit strange if all > {{@en-US}} literals sorted after {{@en}} literals... > It would be good to check how Dydra does this and possibly take the same > approach. See the message linked above for further backgound. > I've been talking with [~kinow] about this and he may be interested in > implementing it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version
So for next steps, should someone (aka me) try to build against a 3.X Hadoop to see where the jolts are? Does that make sense as a way to gather more info? I would just go ahead and do it, but lacking (as I do) any Hadoop dev experience, I want to make sure that it's a good idea before investing time. --- A. Soroka The University of Virginia Library > On Mar 31, 2017, at 5:11 AM, Rob Vessewrote: > > My impression is that 3.0 is still a long way off a generally available > release since they are still in Alpha at the moment. > > I would expect that 2.x would remain the primary distribution for a while yet. > > I don’t know how much, If at all, map/reduce has changed so it may already be > possible to build Elephas against 3.X without any changes but then again it > might not. > > I’m not sure that it is necessarily safe to stop shading Guava. It is a > widely used library with poor compatibility between versions and our users > may experience Version conflicts in other environments. > > Rob > > On 29/03/2017 00:18, "Bruno P. Kinoshita" wrote: > >> I would think we could go with a minor version for this, no? Or is >Elephas such a big part of our offering that we need to make a major >release to move with Hadoop...? > > >I would think the same. Unless we have other important features to be > released, maybe something that breaks backward compatibility, etc. > > > > >From: A. Soroka >To: dev@jena.apache.org >Sent: Wednesday, 29 March 2017 3:20 AM >Subject: Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to > the latest version > > > >> * Jena would have to move in step with Hadoop and drop support for older >> versions. > >Maybe we can see what Hadoop itself ends up doing for this problem... they > are targeting their own major release 3 for this. I would think we could go > with a minor version for this, no? Or is Elephas such a big part of our > offering that we need to make a major release to move with Hadoop...? > >--- >A. Soroka >The University of Virginia Library > > >> On Mar 25, 2017, at 7:15 AM, Andy Seaborne wrote: >> >> >> Step 1: upgrade jena-shaded-guava to v21.0 >> >> and see if it works. >> >> >> There are at least two considerations: >> >> * Jena would have to move in step with Hadoop >> and drop support for older versions. >> * Our downstream users may have guava version choices. >> >> and we have to wait until a Hadoop release to remove it. >> >> Andy >> >> On 25/03/17 01:22, A. Soroka wrote: >>> Looks like Hadoop has successfully updated their Guava dependency-- this >>> might provide us the opportunity to stop shading Guava. (yay!) >>> >>> --- >>> A. Soroka >>> The University of Virginia Library >>> Begin forwarded message: From: "Tsuyoshi Ozawa (JIRA)" Subject: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version Date: March 24, 2017 at 9:18:42 PM EDT To: aj...@apache.org [ https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated HADOOP-10101: Resolution: Fixed Fix Version/s: 3.0.0-alpha3 Status: Resolved (was: Patch Available) Committed this to trunk. Thanks Nicholas and Steve for your review, and thanks people who joined this issue for your comments. > Update guava dependency to the latest version > - > > Key: HADOOP-10101 > URL: https://issues.apache.org/jira/browse/HADOOP-10101 > Project: Hadoop Common > Issue Type: Improvement > Affects Versions: 3.0.0-alpha2 > Reporter: Rakesh R > Assignee: Tsuyoshi Ozawa > Labels: BB2015-05-TBR > Fix For: 3.0.0-alpha3 > > Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, > HADOOP-10101-005.patch, HADOOP-10101-006.patch, HADOOP-10101-007.patch, > HADOOP-10101-008.patch, HADOOP-10101-009.patch, HADOOP-10101-009.patch, > HADOOP-10101-010.patch, HADOOP-10101-010.patch, HADOOP-10101-011.patch, > HADOOP-10101.012.patch, HADOOP-10101.013.patch, HADOOP-10101.014.patch, > HADOOP-10101.015.patch, HADOOP-10101.016.patch, HADOOP-10101.017.patch, > HADOOP-10101.018.patch, HADOOP-10101.patch, HADOOP-10101.patch > > > The existing guava version is 11.0.2 which is quite old. This issue tries > to update the version to as latest version as possible. -- This message was sent by Atlassian JIRA (v6.3.15#6346) >>> > > > > >
[jira] [Commented] (JENA-1313) Language-specific collation in ARQ
[ https://issues.apache.org/jira/browse/JENA-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950717#comment-15950717 ] Osma Suominen commented on JENA-1313: - > return values in es, then pt, then fi Shouldn't this be es, fi, pt? Anyway, people seem to be agreeing on making an extension function rather than changing the default sorting order of literals. I re-read the [description of the Dydra implementation of collation|http://blog.dydra.com/2015/05/06/collation], but they don't say what the ordering is between literals with differing language tags - it remains explicitly undefined, as in SPARQL. I don't know what ORDER BY will in practice do in Dydra when the language tags differ. Dydra also has a [test suite|https://github.com/dydra/http-api-tests/tree/master/extensions/sparql-protocol/collation] for collation that is published using the Unlicense i.e. placed in the public domain. There I found [an example|https://github.com/dydra/http-api-tests/blob/master/extensions/sparql-protocol/collation/ordered-locations-da.txt] of a proper Danish language collation sequence that could perhaps also be used as a test case. There are other test cases in that directory that may also be relevant. For the extension function, I suggest defining it just as a single function e.g. {{collate:collate}} that takes up to two parameters: the literal value and the language/locale (which may be omitted, and then the language is extracted from the language tag). So you can say e.g. {{ORDER BY collate:collate(?label, 'fi')}} to get the labels sorted according to Finnish collation rules, regardless of their language tags. I think this is better than the {{collate:fi}} example given by [~kinow] above, because it requires just a single extension function and makes the locale/language a parameter that may come from a SPARQL variable instead of being hardwired into the query. A special parameter value such as {{unicode}} or the empty string could be used to force Unicode collation rules. (even a one-parameter version of {{collate:collate}} could be used in this way by passing {{STRLANG(?label, 'fi')}} as the parameter, but that seems unnecessarily complicated to me) > Language-specific collation in ARQ > -- > > Key: JENA-1313 > URL: https://issues.apache.org/jira/browse/JENA-1313 > Project: Apache Jena > Issue Type: Improvement > Components: ARQ >Affects Versions: Jena 3.2.0 >Reporter: Osma Suominen > > As [discussed|http://markmail.org/message/v2bvsnsza5ksl2cv] on the users > mailing list in October 2016, I would like to change ARQ collation of literal > values to be language-aware and respect language-specific collation rules. > This would probably involve changing at least the > [NodeUtils.compareLiteralsBySyntax|https://github.com/apache/jena/blob/master/jena-arq/src/main/java/org/apache/jena/sparql/util/NodeUtils.java#L199] > method. > It currently sorts by lexical value first, then by language tag. Since the > collation order needs to be stable across all possible literal values, I > think the safest way would be to sort by language tag first, then by lexical > value according to the collation rules for that language. > But what about subtags like {{@en-US}} or {{@pt-BR}}? Can they have different > collation rules than the main language? It would be a bit strange if all > {{@en-US}} literals sorted after {{@en}} literals... > It would be good to check how Dydra does this and possibly take the same > approach. See the message linked above for further backgound. > I've been talking with [~kinow] about this and he may be interested in > implementing it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (JENA-1302) Intermittent Lucene prepareCommit exception during heavy read/write load
[ https://issues.apache.org/jira/browse/JENA-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vilnis Termanis (Iotic Labs) updated JENA-1302: --- Attachment: netstat.log netstat socket state counts during lockup.py run (with annotations) > Intermittent Lucene prepareCommit exception during heavy read/write load > > > Key: JENA-1302 > URL: https://issues.apache.org/jira/browse/JENA-1302 > Project: Apache Jena > Issue Type: Bug > Components: Fuseki, TDB, Text >Affects Versions: Jena 3.3.0 > Environment: CentOS 7.3 inside VM, 2 cores, OpenJDK 1.8.0_121 > (64-bit), 1GB JVM heap, Fuseki running as service >Reporter: Vilnis Termanis (Iotic Labs) >Assignee: Andy Seaborne > Fix For: Jena 3.3.0 > > Attachments: netstat.log, prepareCommit_error.tgz, > visualvm_thread_dump.log > > > (*Note*: Could not add Fuseki 2.6.0 as affected version although I assume > this is a TDB/text rather than Fuseki-related issue.) > *Steps:* > # Start with plain Fuseki + given configuration (TDB store + lucene text > indexing) > # Import 1k.ttl > # Run lockup.py (same host, mix of multiple parallel updates & single select) > *Result:* > Every now and again one of the update queries fails with a 500 server error > which is logged as: > {{DatasetGraphText ERROR Exception in prepareCommit: prepareCommit was > already called with no corresponding call to commit}} > *Notes:* > - Not reproducible without fix for JENA-1296 (and discovered during fix > verification for said issue) > - Run with > [apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz|https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.6.0-SNAPSHOT/apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz] > - From cold-start of Fuseki, occurs within 30s of script run > - Supplied test script required Python 3, {{rdflib>=4.2.2}} & > {{SPARQLWrapper>=1.8.0}} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (JENA-1302) Intermittent Lucene prepareCommit exception during heavy read/write load
[ https://issues.apache.org/jira/browse/JENA-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950653#comment-15950653 ] Vilnis Termanis (Iotic Labs) commented on JENA-1302: >From your description I think we're seeing the same thing.I tracked states for >fuseki sockets during the script run (see annotated netstat.log attached). {code} while true; do date && netstat -np | grep 3030 | awk '{print $6}' | sort | uniq -c && sleep 2; done {code} First of all from my understanding, CLOSE_WAIT indicates that the client (lockup.py) has closed but the server (fuseki/jetty) hasn't (quoting netstat man page): bq. CLOSE_WAIT: The remote end has shut down, waiting for the socket to close. # Initially TIME_WAITs build up (indicating successful finished requests) # Once the lockup occurs (CPU usage drops) and after at least one request made by lockup.py times outs (30s), at least one CLOSE_WAIT can be seen (indicating that, although the client has finished the request the server has not yet closed the connection.) # As expected, TIME_WAITs gradually disappear due to lack of new requests being completed # Any further requests made thereafter (e.g. "selection of triples" query example via UI or running lockup.py again) produce more connections in CLOSE_WAIT once the client has timed out due to lack of server response (or when one e.g. closes the browser tab/window). *Conclusion*: The eventual CLOSE_WAIT state is the symptom of Fuseki (via Jetty) no longer being able to handle additional requests due there still being a lock held, preventing any other request from being worked on in parallel. (I.e. same sort of external behaviour as observed in JENA-1296.) > Intermittent Lucene prepareCommit exception during heavy read/write load > > > Key: JENA-1302 > URL: https://issues.apache.org/jira/browse/JENA-1302 > Project: Apache Jena > Issue Type: Bug > Components: Fuseki, TDB, Text >Affects Versions: Jena 3.3.0 > Environment: CentOS 7.3 inside VM, 2 cores, OpenJDK 1.8.0_121 > (64-bit), 1GB JVM heap, Fuseki running as service >Reporter: Vilnis Termanis (Iotic Labs) >Assignee: Andy Seaborne > Fix For: Jena 3.3.0 > > Attachments: prepareCommit_error.tgz, visualvm_thread_dump.log > > > (*Note*: Could not add Fuseki 2.6.0 as affected version although I assume > this is a TDB/text rather than Fuseki-related issue.) > *Steps:* > # Start with plain Fuseki + given configuration (TDB store + lucene text > indexing) > # Import 1k.ttl > # Run lockup.py (same host, mix of multiple parallel updates & single select) > *Result:* > Every now and again one of the update queries fails with a 500 server error > which is logged as: > {{DatasetGraphText ERROR Exception in prepareCommit: prepareCommit was > already called with no corresponding call to commit}} > *Notes:* > - Not reproducible without fix for JENA-1296 (and discovered during fix > verification for said issue) > - Run with > [apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz|https://repository.apache.org/content/repositories/snapshots/org/apache/jena/apache-jena-fuseki/2.6.0-SNAPSHOT/apache-jena-fuseki-2.6.0-20170228.233930-10.tar.gz] > - From cold-start of Fuseki, occurs within 30s of script run > - Supplied test script required Python 3, {{rdflib>=4.2.2}} & > {{SPARQLWrapper>=1.8.0}} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version
My impression is that 3.0 is still a long way off a generally available release since they are still in Alpha at the moment. I would expect that 2.x would remain the primary distribution for a while yet. I don’t know how much, If at all, map/reduce has changed so it may already be possible to build Elephas against 3.X without any changes but then again it might not. I’m not sure that it is necessarily safe to stop shading Guava. It is a widely used library with poor compatibility between versions and our users may experience Version conflicts in other environments. Rob On 29/03/2017 00:18, "Bruno P. Kinoshita"wrote: >I would think we could go with a minor version for this, no? Or is Elephas such a big part of our offering that we need to make a major release to move with Hadoop...? I would think the same. Unless we have other important features to be released, maybe something that breaks backward compatibility, etc. From: A. Soroka To: dev@jena.apache.org Sent: Wednesday, 29 March 2017 3:20 AM Subject: Re: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version > * Jena would have to move in step with Hadoop and drop support for older versions. Maybe we can see what Hadoop itself ends up doing for this problem... they are targeting their own major release 3 for this. I would think we could go with a minor version for this, no? Or is Elephas such a big part of our offering that we need to make a major release to move with Hadoop...? --- A. Soroka The University of Virginia Library > On Mar 25, 2017, at 7:15 AM, Andy Seaborne wrote: > > > Step 1: upgrade jena-shaded-guava to v21.0 > > and see if it works. > > > There are at least two considerations: > > * Jena would have to move in step with Hadoop > and drop support for older versions. > * Our downstream users may have guava version choices. > > and we have to wait until a Hadoop release to remove it. > >Andy > > On 25/03/17 01:22, A. Soroka wrote: >> Looks like Hadoop has successfully updated their Guava dependency-- this might provide us the opportunity to stop shading Guava. (yay!) >> >> --- >> A. Soroka >> The University of Virginia Library >> >>> Begin forwarded message: >>> >>> From: "Tsuyoshi Ozawa (JIRA)" >>> Subject: [jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version >>> Date: March 24, 2017 at 9:18:42 PM EDT >>> To: aj...@apache.org >>> >>> >>>[ https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] >>> >>> Tsuyoshi Ozawa updated HADOOP-10101: >>> >>> Resolution: Fixed >>> Fix Version/s: 3.0.0-alpha3 >>> Status: Resolved (was: Patch Available) >>> >>> Committed this to trunk. Thanks Nicholas and Steve for your review, and thanks people who joined this issue for your comments. >>> Update guava dependency to the latest version - Key: HADOOP-10101 URL: https://issues.apache.org/jira/browse/HADOOP-10101 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0-alpha2 Reporter: Rakesh R Assignee: Tsuyoshi Ozawa Labels: BB2015-05-TBR Fix For: 3.0.0-alpha3 Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, HADOOP-10101-005.patch, HADOOP-10101-006.patch, HADOOP-10101-007.patch, HADOOP-10101-008.patch, HADOOP-10101-009.patch, HADOOP-10101-009.patch, HADOOP-10101-010.patch, HADOOP-10101-010.patch, HADOOP-10101-011.patch, HADOOP-10101.012.patch, HADOOP-10101.013.patch, HADOOP-10101.014.patch, HADOOP-10101.015.patch, HADOOP-10101.016.patch, HADOOP-10101.017.patch, HADOOP-10101.018.patch, HADOOP-10101.patch, HADOOP-10101.patch The existing guava version is 11.0.2 which is quite old. This issue tries to update the version to as latest version as possible. >>> >>> >>> >>> -- >>> This message was sent by Atlassian JIRA >>> (v6.3.15#6346) >>