More on TransformEliminateAssignments

2015-07-12 Thread Andy Seaborne

Rob,

I've fixed the merge conflict; pelase check it because it did not quite 
agree with the imports so that might indicate I got it wrong.  Maybe not 
everything got pushed?  (It does at least build now and pass tests.)


A few comments:

1/ TransformEliminateAssignments  TransformRemoveAssignment

Is TransformRemoveAssignment just there to support 
TransformEliminateAssignments?


2/ What's the current status (thinking of jena3 timescales here)

I see TODO unclear

Andy

On 11/07/15 17:51, Andy Seaborne wrote:

Rob,

There is a conflict in TransformEliminateAssignments:

I'm not quite sure what the correct resolution is.

 Andy

(Extract):

 
http://git-wip-us.apache.org/repos/asf/jena/blob/a25c72c9/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java

  --
  diff --git
a/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java
b/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java

  index 77ba124..3ce4198 100644
  ---
a/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java

  +++
b/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java


...

// See if we can inline anything
for (Var var : vars) {
  @@ -532,5 +535,75 @@ public class TransformEliminateAssignments
extends TransformCopy {
this.tracker.decrementDepth();
}
 
  +
HEAD:jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java

  +===
  +@Override







On 08/07/15 15:12, rve...@apache.org wrote:

Additional tests and fixes for assignment inlining (JENA-780)

- Don't inline into EXISTS/NOT EXISTS since those are n-ary operators
- Permit inlining within an EXISTS/NOT EXISTS only if the assignments
   are subject to all the normal restrictions
- Don't remove projection when inlining/eliminating through projections
   since doing so could cause previously hidden variables to become
   visible
- Various additional test cases for these

Conflicts:

jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java



Project: http://git-wip-us.apache.org/repos/asf/jena/repo
Commit: http://git-wip-us.apache.org/repos/asf/jena/commit/a25c72c9
Tree: http://git-wip-us.apache.org/repos/asf/jena/tree/a25c72c9
Diff: http://git-wip-us.apache.org/repos/asf/jena/diff/a25c72c9

Branch: refs/heads/master
Commit: a25c72c94ecd433fbc4cca78d87f9a7c539841df
Parents: 2031b78
Author: Rob Vesse rve...@apache.org
Authored: Wed Jul 8 12:01:28 2015 +0100
Committer: Rob Vesse rve...@apache.org
Committed: Wed Jul 8 15:08:47 2015 +0100

--
  .../optimize/TransformEliminateAssignments.java | 83 -
  .../optimize/TransformRemoveAssignment.java |  7 +-
  .../org/apache/jena/sparql/expr/ExprVars.java   | 27 ++
  .../TestTransformEliminateAssignments.java  | 97
+++-
  4 files changed, 202 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/jena/blob/a25c72c9/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java

--
diff --git
a/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java
b/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java

index 77ba124..3ce4198 100644
---
a/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java

+++
b/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java

@@ -21,9 +21,11 @@ package org.apache.jena.sparql.algebra.optimize;
  import java.util.ArrayList;
  import java.util.Collection;
  import java.util.HashMap;
+import java.util.HashSet;
  import java.util.Iterator;
  import java.util.List;
  import java.util.Map;
+import java.util.Set;

  import org.apache.jena.atlas.lib.CollectionUtils;

@@ -99,8 +101,9 @@ public class TransformEliminateAssignments extends
TransformCopy {
  AssignmentPusher pusher = new AssignmentPusher(tracker);
  AssignmentPopper popper = new AssignmentPopper(tracker);
  Transform transform = new
TransformEliminateAssignments(tracker, pusher, popper, aggressive);
+ExprTransform exprTransform = new
ExprTransformEliminateAssignments(aggressive);

-return Transformer.transformSkipService(transform, op,
pusher, popper);
+return Transformer.transformSkipService(transform,
exprTransform, op, pusher, popper);
  }

  private final OpVisitor before, after;
@@ 

[jira] [Closed] (JENA-966) LazyIterator

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren closed JENA-966.
--

Jenkins build successful after removal of 3 iterators.  Closing issue as 
completed.

 LazyIterator
 

 Key: JENA-966
 URL: https://issues.apache.org/jira/browse/JENA-966
 Project: Apache Jena
  Issue Type: Bug
  Components: Core
Affects Versions: Jena 3.0.0
Reporter: Claude Warren
Assignee: Claude Warren
 Fix For: Jena 3.0.0


 LazyIterator is an abstract class.  The documentation indicates that the 
 create() method needs to be overridden to create an instance.  From this I 
 would expect that 
 now LazyIterator(){
 @Override
 public ExtendedIteratorModel create() {
   ...
 }};
 Would work however LazyIterator does not override:
 remoteNext(), andThen(), toList(), and toSet().
 I believe these should be implemented in the class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (JENA-859) Add logging to jena-security

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren closed JENA-859.
--

Closed as fixed

 Add logging to jena-security
 

 Key: JENA-859
 URL: https://issues.apache.org/jira/browse/JENA-859
 Project: Apache Jena
  Issue Type: Improvement
  Components: Security
Affects Versions: Jena 2.13.0
Reporter: Claude Warren
Assignee: Claude Warren
Priority: Trivial
 Fix For: Fuseki 2.3.0


 There is no logging in the jena-security layer.  this makes debugging 
 difficult. This improvement is to add logging to facilitate debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (JENA-859) Add logging to jena-security

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren resolved JENA-859.

   Resolution: Fixed
Fix Version/s: Fuseki 2.3.0

added logging to OpRewrite.

Other logging should be applied to implementer supplied SecurityEvaluator.

 Add logging to jena-security
 

 Key: JENA-859
 URL: https://issues.apache.org/jira/browse/JENA-859
 Project: Apache Jena
  Issue Type: Improvement
  Components: Security
Affects Versions: Jena 2.13.0
Reporter: Claude Warren
Assignee: Claude Warren
Priority: Trivial
 Fix For: Fuseki 2.3.0


 There is no logging in the jena-security layer.  this makes debugging 
 difficult. This improvement is to add logging to facilitate debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (JENA-861) org.apache.jena.security.impl.SecuredItemInvoker uses wrong method

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren closed JENA-861.
--

Closing

 org.apache.jena.security.impl.SecuredItemInvoker uses wrong method
 --

 Key: JENA-861
 URL: https://issues.apache.org/jira/browse/JENA-861
 Project: Apache Jena
  Issue Type: Bug
  Components: Security
Affects Versions: Jena 2.13.0
Reporter: Claude Warren
Assignee: Claude Warren
 Fix For: Jena 2.13.1, Jena 3.0.0


 When a secured Model returns a secured Graph from getGraph() and that graph 
 is closed an InvocationTargetException is thrown.
  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (JENA-854) The generic assembler for SecurityEvaluators does not handle non string args correctly

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren closed JENA-854.
--

closing

 The generic assembler for SecurityEvaluators does not handle non string args 
 correctly
 --

 Key: JENA-854
 URL: https://issues.apache.org/jira/browse/JENA-854
 Project: Apache Jena
  Issue Type: Bug
  Components: Security
 Environment: all
Reporter: Claude Warren
Assignee: Claude Warren
 Fix For: Jena 2.13.1, Jena 3.0.0


 When a non-string argument is passed to the generic Evaluator assembler it 
 does not correctly detect it and uses the 0 arg constructor instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (JENA-854) The generic assembler for SecurityEvaluators does not handle non string args correctly

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren resolved JENA-854.

   Resolution: Fixed
Fix Version/s: Jena 3.0.0
   Jena 2.13.1

Fixed by previous changes

 The generic assembler for SecurityEvaluators does not handle non string args 
 correctly
 --

 Key: JENA-854
 URL: https://issues.apache.org/jira/browse/JENA-854
 Project: Apache Jena
  Issue Type: Bug
  Components: Security
 Environment: all
Reporter: Claude Warren
Assignee: Claude Warren
 Fix For: Jena 2.13.1, Jena 3.0.0


 When a non-string argument is passed to the generic Evaluator assembler it 
 does not correctly detect it and uses the 0 arg constructor instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (JENA-861) org.apache.jena.security.impl.SecuredItemInvoker uses wrong method

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren resolved JENA-861.

   Resolution: Fixed
Fix Version/s: Jena 3.0.0
   Jena 2.13.1

Fixed by previous checkin

 org.apache.jena.security.impl.SecuredItemInvoker uses wrong method
 --

 Key: JENA-861
 URL: https://issues.apache.org/jira/browse/JENA-861
 Project: Apache Jena
  Issue Type: Bug
  Components: Security
Affects Versions: Jena 2.13.0
Reporter: Claude Warren
Assignee: Claude Warren
 Fix For: Jena 2.13.1, Jena 3.0.0


 When a secured Model returns a secured Graph from getGraph() and that graph 
 is closed an InvocationTargetException is thrown.
  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (JENA-850) The return type of the SecurityEvaluator.getPrincipal() is to restrictive.

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren resolved JENA-850.

   Resolution: Fixed
Fix Version/s: Jena 3.0.0
   Jena 2.13.1

Fixed by previous changes

 The return type of the SecurityEvaluator.getPrincipal() is to restrictive.
 --

 Key: JENA-850
 URL: https://issues.apache.org/jira/browse/JENA-850
 Project: Apache Jena
  Issue Type: Bug
  Components: Security
Affects Versions: Jena 2.12.1
 Environment: All
Reporter: Claude Warren
Assignee: Claude Warren
  Labels: permissions, security
 Fix For: Jena 2.13.1, Jena 3.0.0


 SecurityEvaluator.getPrincipal() is requried to be used in all other 
 SecurityEvaluator methods, however, the return type Principal is to 
 restrictive for some implementations.  
 The Principal is to be retrieved from the Authorization system, and in 
 general may be a Principal, but in some cases, as in when integrating with 
 Apache Shiro, it needs to be a differnt object type.
 The probable fix is to have getPrincipal() return a generic Object.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (JENA-984) Fuseki script does not correctly run when start-stop-daemon is not available

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren closed JENA-984.
--

Fixed by previous checkin

 Fuseki script does not correctly run when start-stop-daemon is not available
 

 Key: JENA-984
 URL: https://issues.apache.org/jira/browse/JENA-984
 Project: Apache Jena
  Issue Type: Improvement
  Components: Fuseki
Affects Versions: Fuseki 2.3.0
 Environment: EC2 - Amazon Linux
Reporter: Claude Warren
Assignee: Claude Warren
Priority: Minor
 Fix For: Fuseki 2.3.0


 When running on an Amazon Linux EC2 instance the  start-stop-daemon is not 
 available. 
 When the fuseki script is run from /etc/init.d it does not find the 
 FUSEKI_HOME directory and so fails to start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (JENA-983) Fuseki script does not support adding classes to the command line.

2015-07-12 Thread Andy Seaborne (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623761#comment-14623761
 ] 

Andy Seaborne commented on JENA-983:


Status?

 Fuseki script does not support adding classes to the command line.
 --

 Key: JENA-983
 URL: https://issues.apache.org/jira/browse/JENA-983
 Project: Apache Jena
  Issue Type: Improvement
  Components: Fuseki
Affects Versions: Fuseki 2.3.0
Reporter: Claude Warren
Assignee: Claude Warren
Priority: Minor

 There is no simple way to add additional classes to the fuseki start script 
 via the configuration file.
 For example to add the security jar one has to hand edit the fuseki start 
 script.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (JENA-983) Fuseki script does not support adding classes to the command line.

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren closed JENA-983.
--

Closing

 Fuseki script does not support adding classes to the command line.
 --

 Key: JENA-983
 URL: https://issues.apache.org/jira/browse/JENA-983
 Project: Apache Jena
  Issue Type: Improvement
  Components: Fuseki
Affects Versions: Fuseki 2.3.0
Reporter: Claude Warren
Assignee: Claude Warren
Priority: Minor
 Fix For: Fuseki 2.3.0


 There is no simple way to add additional classes to the fuseki start script 
 via the configuration file.
 For example to add the security jar one has to hand edit the fuseki start 
 script.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (JENA-984) Fuseki script does not correctly run when start-stop-daemon is not available

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren resolved JENA-984.

   Resolution: Fixed
Fix Version/s: Fuseki 2.3.0

Fixed with previous commit

 Fuseki script does not correctly run when start-stop-daemon is not available
 

 Key: JENA-984
 URL: https://issues.apache.org/jira/browse/JENA-984
 Project: Apache Jena
  Issue Type: Improvement
  Components: Fuseki
Affects Versions: Fuseki 2.3.0
 Environment: EC2 - Amazon Linux
Reporter: Claude Warren
Assignee: Claude Warren
Priority: Minor
 Fix For: Fuseki 2.3.0


 When running on an Amazon Linux EC2 instance the  start-stop-daemon is not 
 available. 
 When the fuseki script is run from /etc/init.d it does not find the 
 FUSEKI_HOME directory and so fails to start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (JENA-850) The return type of the SecurityEvaluator.getPrincipal() is to restrictive.

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren closed JENA-850.
--

closing

 The return type of the SecurityEvaluator.getPrincipal() is to restrictive.
 --

 Key: JENA-850
 URL: https://issues.apache.org/jira/browse/JENA-850
 Project: Apache Jena
  Issue Type: Bug
  Components: Security
Affects Versions: Jena 2.12.1
 Environment: All
Reporter: Claude Warren
Assignee: Claude Warren
  Labels: permissions, security
 Fix For: Jena 2.13.1, Jena 3.0.0


 SecurityEvaluator.getPrincipal() is requried to be used in all other 
 SecurityEvaluator methods, however, the return type Principal is to 
 restrictive for some implementations.  
 The Principal is to be retrieved from the Authorization system, and in 
 general may be a Principal, but in some cases, as in when integrating with 
 Apache Shiro, it needs to be a differnt object type.
 The probable fix is to have getPrincipal() return a generic Object.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (JENA-971) QueryBuilder does not properly construct UNIONs

2015-07-12 Thread Andy Seaborne (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623556#comment-14623556
 ] 

Andy Seaborne edited comment on JENA-971 at 7/12/15 9:08 AM:
-

But I now get this illegal query;

{noformat}
SELECT  *
WHERE
  { ?s  ?p  ?o
UNION
  { SELECT  ?s ?p ?o
WHERE
  { ?s  ?p  ?o }
  }
  }
{noformat}
which should be:
{noformat}
SELECT  *
WHERE
  { { ?s  ?p  ?o }
UNION
  { SELECT  ?s ?p ?o
WHERE
  { ?s  ?p  ?o }
  }
  }
{noformat}
each side of the UNION is a  ElementGroup.


was (Author: claudenw):
But I now get this illegal query;

SELECT  *
WHERE
  { ?s  ?p  ?o
UNION
  { SELECT  ?s ?p ?o
WHERE
  { ?s  ?p  ?o }
  }
  }

which should be:

SELECT  *
WHERE
  { { ?s  ?p  ?o }
UNION
  { SELECT  ?s ?p ?o
WHERE
  { ?s  ?p  ?o }
  }
  }

each side of the UNION is a  ElementGroup.

 QueryBuilder does not properly construct UNIONs
 ---

 Key: JENA-971
 URL: https://issues.apache.org/jira/browse/JENA-971
 Project: Apache Jena
  Issue Type: Bug
  Components: Jena
Affects Versions: Jena 2.13.1, Jena 3.0.0
Reporter: Claude Warren
Assignee: Claude Warren
 Fix For: Jena 2.13.1, Jena 3.0.0


 The QueryBuilder method addUnion() always adds a new union to the query 
 pattern.  It should only add one if the last element in the query pattern is 
 not a union.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (JENA-983) Fuseki script does not support adding classes to the command line.

2015-07-12 Thread Claude Warren (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claude Warren resolved JENA-983.

   Resolution: Fixed
Fix Version/s: Fuseki 2.3.0

Completed with earlier checkin.  Resolving and closing this issue

 Fuseki script does not support adding classes to the command line.
 --

 Key: JENA-983
 URL: https://issues.apache.org/jira/browse/JENA-983
 Project: Apache Jena
  Issue Type: Improvement
  Components: Fuseki
Affects Versions: Fuseki 2.3.0
Reporter: Claude Warren
Assignee: Claude Warren
Priority: Minor
 Fix For: Fuseki 2.3.0


 There is no simple way to add additional classes to the fuseki start script 
 via the configuration file.
 For example to add the security jar one has to hand edit the fuseki start 
 script.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (JENA-780) Single use extend expressions could be substituted directly for their later usage

2015-07-12 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623883#comment-14623883
 ] 

ASF subversion and git services commented on JENA-780:
--

Commit 310b0968d3c8a19dee4e1c28c96d8a4fbe08794e in jena's branch 
refs/heads/master from [~rvesse]
[ https://git-wip-us.apache.org/repos/asf?p=jena.git;h=310b096 ]

Don't inline assignments in some unsafe cases (JENA-780)

There are some cases where inlining is unsafe around moving an
assignment from within an n-ary expression e.g. join, leftjoin, union,
minus etc because the assignments value could be dependent on the branch
of the query it occurs in and moving it outside the branching operator
could change the semantics of its evaluation.

This commit disables inlining in those cases and adds extra test cases
to ensure that those cases are not subject to inlining.

Conflicts:

jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignments.java


 Single use extend expressions could be substituted directly for their later 
 usage
 -

 Key: JENA-780
 URL: https://issues.apache.org/jira/browse/JENA-780
 Project: Apache Jena
  Issue Type: Improvement
  Components: ARQ, Optimizer
Affects Versions: Jena 2.12.0
Reporter: Rob Vesse
Assignee: Rob Vesse
Priority: Minor
 Attachments: JENA-780.patch


 This RFE is a follow on from JENA-779, the query with a sub-optimal plan 
 there uses a {{BIND}} to create a value which is then only used once in a 
 subsequent filter.
 Actually that query uses it twice but I think the general approach I am 
 trying to describe in this RFE bears consideration.  In this case it seems 
 like it would be possible to substitute the extend expression for the bound 
 variable in the filter expression.
 Simplified variant of original query such that the bound value is only used 
 once:
 {noformat}
 SELECT DISTINCT ?uri
 {
   { ?uri ?p ?o }
   UNION
   {
 ?sub ?p ?uri
 FILTER(isIRI(?uri))
   }
   BIND(str(?uri) as ?s)
   FILTER(STRSTARTS(?s, http://;))
 }
 {noformat}
 Rewritten query:
 {noformat}
 SELECT DISTINCT ?uri
 {
   { ?uri ?p ?o }
   UNION
   {
 ?sub ?p ?uri
 FILTER(isIRI(?uri))
   }
   FILTER(STRSTARTS(str(?uri), http://;))
 }
 {noformat}
 Which avoids an extend expression whose value is only used once and will 
 ultimately be thrown away.
 From a {{Transform}} standpoint this is likely awkward to implement in a pure 
 transform since it requires knowledge about the query structure above the 
 {{FILTER}} i.e. whether the bound variable is used elsewhere and so would 
 need to use before and after visitors to track that additional state but I 
 think this is a feasible optimisation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (JENA-966) LazyIterator

2015-07-12 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/JENA-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623937#comment-14623937
 ] 

ASF subversion and git services commented on JENA-966:
--

Commit 764cf529257f3b180e5b6f456a5464813fe47628 in jena's branch 
refs/heads/master from [~cla...@xenei.org]
[ https://git-wip-us.apache.org/repos/asf?p=jena.git;h=764cf52 ]

Removed EarlyBindingIterator, UniqueExtendedIterator, and LateBindingIterator.
Part of a fix for JENA-966
Should complete the fix.


 LazyIterator
 

 Key: JENA-966
 URL: https://issues.apache.org/jira/browse/JENA-966
 Project: Apache Jena
  Issue Type: Bug
  Components: Core
Affects Versions: Jena 3.0.0
Reporter: Claude Warren
Assignee: Claude Warren
 Fix For: Jena 3.0.0


 LazyIterator is an abstract class.  The documentation indicates that the 
 create() method needs to be overridden to create an instance.  From this I 
 would expect that 
 now LazyIterator(){
 @Override
 public ExtendedIteratorModel create() {
   ...
 }};
 Would work however LazyIterator does not override:
 remoteNext(), andThen(), toList(), and toSet().
 I believe these should be implemented in the class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (JENA-980) Store UI-created datasets in configuration/ not the system database.

2015-07-12 Thread Andy Seaborne (JIRA)

 [ 
https://issues.apache.org/jira/browse/JENA-980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy Seaborne resolved JENA-980.

   Resolution: Fixed
Fix Version/s: Fuseki 2.3.0

 Store UI-created datasets in configuration/ not the system database.
 

 Key: JENA-980
 URL: https://issues.apache.org/jira/browse/JENA-980
 Project: Apache Jena
  Issue Type: Improvement
  Components: Fuseki
Reporter: Andy Seaborne
Assignee: Andy Seaborne
 Fix For: Fuseki 2.3.0


 Fuseki2 current stores UI-created datasets in the system database. This means 
 users can't easily edit the configuration.
 Instead, put the UI-created datasets in the configuration/ directory as 
 assembler files.  They are then conveniently accessible to users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: More on TransformEliminateAssignments

2015-07-12 Thread Rob Vesse
Looks like I missed one commit off of the master branch

I've been developing this primarily on the Jena2 branch (we're still on
Java 7 and thus Jena 2 for the time being) and have been cherry-picking
across to master and managed to miss one commit off

Missing commit should be pushed shortly

TransformRemoveAssignment exists to support TransformEliminateAssignment
and serves to separate the logic of determining which assignments can be
inlined/eliminated from the logic of actually removing the original
assignment (which may be deep in the algebra tree relative to the point
where we inline it)

I think it is pretty much there now and should only inline/eliminate
things where it is safe to do so, however it may not be perfect and so I
have left this optimisation disabled by default so users can opt in if
they wish at their own risk

Rob

On 12/07/2015 12:13, Andy Seaborne a...@apache.org wrote:

Rob,

I've fixed the merge conflict; pelase check it because it did not quite
agree with the imports so that might indicate I got it wrong.  Maybe not
everything got pushed?  (It does at least build now and pass tests.)

A few comments:

1/ TransformEliminateAssignments  TransformRemoveAssignment

Is TransformRemoveAssignment just there to support
TransformEliminateAssignments?

2/ What's the current status (thinking of jena3 timescales here)

I see TODO unclear

   Andy

On 11/07/15 17:51, Andy Seaborne wrote:
 Rob,

 There is a conflict in TransformEliminateAssignments:

 I'm not quite sure what the correct resolution is.

  Andy

 (Extract):

  
 
http://git-wip-us.apache.org/repos/asf/jena/blob/a25c72c9/jena-arq/src/ma
in/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssignm
ents.java

   
--
   diff --git
 
a/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Transfor
mEliminateAssignments.java
 
b/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Transfor
mEliminateAssignments.java

   index 77ba124..3ce4198 100644
   ---
 
a/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Transfor
mEliminateAssignments.java

   +++
 
b/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Transfor
mEliminateAssignments.java


 ...

 // See if we can inline anything
 for (Var var : vars) {
   @@ -532,5 +535,75 @@ public class TransformEliminateAssignments
 extends TransformCopy {
 this.tracker.decrementDepth();
 }
  
   +
 
HEAD:jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Trans
formEliminateAssignments.java

   +===
   +@Override







 On 08/07/15 15:12, rve...@apache.org wrote:
 Additional tests and fixes for assignment inlining (JENA-780)

 - Don't inline into EXISTS/NOT EXISTS since those are n-ary operators
 - Permit inlining within an EXISTS/NOT EXISTS only if the assignments
are subject to all the normal restrictions
 - Don't remove projection when inlining/eliminating through projections
since doing so could cause previously hidden variables to become
visible
 - Various additional test cases for these

 Conflicts:
 
jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Transform
EliminateAssignments.java



 Project: http://git-wip-us.apache.org/repos/asf/jena/repo
 Commit: http://git-wip-us.apache.org/repos/asf/jena/commit/a25c72c9
 Tree: http://git-wip-us.apache.org/repos/asf/jena/tree/a25c72c9
 Diff: http://git-wip-us.apache.org/repos/asf/jena/diff/a25c72c9

 Branch: refs/heads/master
 Commit: a25c72c94ecd433fbc4cca78d87f9a7c539841df
 Parents: 2031b78
 Author: Rob Vesse rve...@apache.org
 Authored: Wed Jul 8 12:01:28 2015 +0100
 Committer: Rob Vesse rve...@apache.org
 Committed: Wed Jul 8 15:08:47 2015 +0100

 --
   .../optimize/TransformEliminateAssignments.java | 83
-
   .../optimize/TransformRemoveAssignment.java |  7 +-
   .../org/apache/jena/sparql/expr/ExprVars.java   | 27 ++
   .../TestTransformEliminateAssignments.java  | 97
 +++-
   4 files changed, 202 insertions(+), 12 deletions(-)
 --


 
http://git-wip-us.apache.org/repos/asf/jena/blob/a25c72c9/jena-arq/src/m
ain/java/org/apache/jena/sparql/algebra/optimize/TransformEliminateAssig
nments.java

 --
 diff --git
 
a/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Transfo
rmEliminateAssignments.java
 
b/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Transfo
rmEliminateAssignments.java

 index 77ba124..3ce4198 100644
 ---
 
a/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Transfo
rmEliminateAssignments.java

 +++
 
b/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/Transfo
rmEliminateAssignments.java