[jira] [Commented] (RYA-73) Add insert time to mongo backed rya

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313053#comment-15313053
 ] 

ASF GitHub Bot commented on RYA-73:
---

Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/43
  
looks good.  will merge tomorrow.  good job adding tests.


> Add insert time to mongo backed rya
> ---
>
> Key: RYA-73
> URL: https://issues.apache.org/jira/browse/RYA-73
> Project: Rya
>  Issue Type: Improvement
>Reporter: Andrew Smith
>
> Some features of accumulo backed Rya require db insert time.  This is not 
> given for free in mongo.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-73) Add insert time to mongo backed rya

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313054#comment-15313054
 ] 

ASF GitHub Bot commented on RYA-73:
---

Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/43
  
oh kind of not related to this pr, but should we consider adding context to 
the mongo composite indices?


> Add insert time to mongo backed rya
> ---
>
> Key: RYA-73
> URL: https://issues.apache.org/jira/browse/RYA-73
> Project: Rya
>  Issue Type: Improvement
>Reporter: Andrew Smith
>
> Some features of accumulo backed Rya require db insert time.  This is not 
> given for free in mongo.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-77) Remove usages of non Accumulo public API classes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314287#comment-15314287
 ] 

ASF GitHub Bot commented on RYA-77:
---

Github user adinancr commented on the issue:

https://github.com/apache/incubator-rya/pull/47
  
I noticed you upgraded Accumulo version to 1.7.1 - does Rya compile against 
that version? A user asked about it on the mailing list - he tried it and had 
problems - there is a  Jira ticket RYA-77 for that. I would suggest staying 
with 1.6 for now


> Remove usages of non Accumulo public API classes
> 
>
> Key: RYA-77
> URL: https://issues.apache.org/jira/browse/RYA-77
> Project: Rya
>  Issue Type: Bug
>Reporter: Josh Elser
>
> Kiet Ly asked on the mailing list about compiling Rya against Accumulo 1.7. 
> It looks like Rya is presently referencing (at least) two non public API 
> classes (which are not guaranteed to be stable -- thus, should not be used 
> downstream)
> * ArgumentChecker
> * AbstractRecordReader
> The former is easily replaced. The latter should probably extend the Hadoop 
> RecordReader baseclass/interface.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-49) Update vagrant file to use accumulo version 1.6.5

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-49?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319484#comment-15319484
 ] 

ASF GitHub Bot commented on RYA-49:
---

Github user DLotts commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/47#discussion_r66157256
  
--- Diff: extras/vagrantExample/src/main/vagrant/Vagrantfile ---
@@ -200,74 +247,93 @@ EOF
 
gc.port.client0
 
 EOF
-cat > accumulo-${ACCUMULO_VERSION}/conf/masters < ${ACCUMULO_HOME}/conf/masters < accumulo-${ACCUMULO_VERSION}/conf/slaves < ${ACCUMULO_HOME}/conf/slaves 

[jira] [Commented] (RYA-51) Temporal indexing support for MongoDB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314352#comment-15314352
 ] 

ASF GitHub Bot commented on RYA-51:
---

GitHub user isper3at opened a pull request:

https://github.com/apache/incubator-rya/pull/48

RYA-51 Temporal Indexing mongo support

Added support for mongo backed rya temporal indexing.
Added integration tests
Repackaged mongo secondary indexing
Cleaned up the storage strategies.
Indexing is now consistent across all secondary indexers (mongo only).
renamed StatementContraints to be spelled correctly.

[Jira](https://issues.apache.org/jira/browse/RYA-51)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/isper3at/incubator-rya RYA-51_temporal

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/48.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #48


commit e9a227bb2173b3f92216a4b04126182c2bd3dda0
Author: isper3at 
Date:   2016-04-05T19:42:13Z

RYA-51 Temporal Indexing mongo support

Added support for mongo backed rya temporal
indexing.

Added integration tests

Repackaged mongo secondary indexing

Cleaned up the storage strategies.
Indexing is now consistent across all
secondary indexers (mongo only).

renamed StatementContraints to be spelled correctly.




> Temporal indexing support for MongoDB
> -
>
> Key: RYA-51
> URL: https://issues.apache.org/jira/browse/RYA-51
> Project: Rya
>  Issue Type: New Feature
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Puja Valiyil 
>
> There should be temporal indexing support for Mongo DB backed Rya.  i imagine 
> this would be implemented using another collection, similar to how temporal 
> indexing is implemented in Rya with a supplementary table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-51) Temporal indexing support for MongoDB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336389#comment-15336389
 ] 

ASF GitHub Bot commented on RYA-51:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/48#discussion_r67537519
  
--- Diff: 
common/rya.api/src/main/java/mvm/rya/api/persist/index/RyaSecondaryIndexer.java 
---
@@ -42,4 +44,14 @@
 
 public void dropGraph(RyaURI... graphs);
 
+/**
+ * @return the set of predicates indexed by the indexer.
+ */
+public abstract Set getIndexablePredicates();
--- End diff --

All secondary indexers have one, PCJ's just return an empty HashSet


> Temporal indexing support for MongoDB
> -
>
> Key: RYA-51
> URL: https://issues.apache.org/jira/browse/RYA-51
> Project: Rya
>  Issue Type: New Feature
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Puja Valiyil 
>
> There should be temporal indexing support for Mongo DB backed Rya.  i imagine 
> this would be implemented using another collection, similar to how temporal 
> indexing is implemented in Rya with a supplementary table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-51) Temporal indexing support for MongoDB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336392#comment-15336392
 ] 

ASF GitHub Bot commented on RYA-51:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/48#discussion_r67537827
  
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoDBRyaDAO.java 
---
@@ -179,85 +145,85 @@ public void destroy() throws RyaDAOException {
 IOUtils.closeQuietly(queryEngine);
 }
 
-   public void add(RyaStatement statement) throws RyaDAOException {

-   // add it to the collection
-   try {
-   coll.insert(storageStrategy.serialize(statement));
-   for(RyaSecondaryIndexer index: secondaryIndexers) {
-   index.storeStatement(statement);
-   }
-   }
-   catch (com.mongodb.MongoException.DuplicateKey exception){
-   // ignore
-   }
-   catch (com.mongodb.DuplicateKeyException exception){
-   // ignore
-   }
-   catch (Exception ex){
-   // ignore single exceptions
-   ex.printStackTrace();
-   }
-   }
-   
-   public void add(Iterator statement) throws 
RyaDAOException {
-   List dbInserts = new ArrayList();
-   while (statement.hasNext()){
-   RyaStatement ryaStatement = statement.next();
-   DBObject insert = 
storageStrategy.serialize(ryaStatement);
-   dbInserts.add(insert);
-   
+@Override
+public void add(final RyaStatement statement) throws RyaDAOException {
+// add it to the collection
+try {
+coll.insert(storageStrategy.serialize(statement));
+for(final RyaSecondaryIndexer index: secondaryIndexers) {
+index.storeStatement(statement);
+}
+} catch (final IOException e) {
--- End diff --

since both are just ignored and the actual exception that is caught is a 
type of IOException it makes it more clear where the exception is coming from.  
Its dangerous to catch exceptions just to ignore...


> Temporal indexing support for MongoDB
> -
>
> Key: RYA-51
> URL: https://issues.apache.org/jira/browse/RYA-51
> Project: Rya
>  Issue Type: New Feature
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Puja Valiyil 
>
> There should be temporal indexing support for Mongo DB backed Rya.  i imagine 
> this would be implemented using another collection, similar to how temporal 
> indexing is implemented in Rya with a supplementary table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-51) Temporal indexing support for MongoDB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336294#comment-15336294
 ] 

ASF GitHub Bot commented on RYA-51:
---

Github user pujav65 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/48#discussion_r67527603
  
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoDBRyaDAO.java 
---
@@ -179,85 +145,85 @@ public void destroy() throws RyaDAOException {
 IOUtils.closeQuietly(queryEngine);
 }
 
-   public void add(RyaStatement statement) throws RyaDAOException {

-   // add it to the collection
-   try {
-   coll.insert(storageStrategy.serialize(statement));
-   for(RyaSecondaryIndexer index: secondaryIndexers) {
-   index.storeStatement(statement);
-   }
-   }
-   catch (com.mongodb.MongoException.DuplicateKey exception){
-   // ignore
-   }
-   catch (com.mongodb.DuplicateKeyException exception){
-   // ignore
-   }
-   catch (Exception ex){
-   // ignore single exceptions
-   ex.printStackTrace();
-   }
-   }
-   
-   public void add(Iterator statement) throws 
RyaDAOException {
-   List dbInserts = new ArrayList();
-   while (statement.hasNext()){
-   RyaStatement ryaStatement = statement.next();
-   DBObject insert = 
storageStrategy.serialize(ryaStatement);
-   dbInserts.add(insert);
-   
+@Override
+public void add(final RyaStatement statement) throws RyaDAOException {
+// add it to the collection
+try {
+coll.insert(storageStrategy.serialize(statement));
+for(final RyaSecondaryIndexer index: secondaryIndexers) {
+index.storeStatement(statement);
+}
+} catch (final IOException e) {
--- End diff --

what happened to the duplicate key exceptions?


> Temporal indexing support for MongoDB
> -
>
> Key: RYA-51
> URL: https://issues.apache.org/jira/browse/RYA-51
> Project: Rya
>  Issue Type: New Feature
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Puja Valiyil 
>
> There should be temporal indexing support for Mongo DB backed Rya.  i imagine 
> this would be implemented using another collection, similar to how temporal 
> indexing is implemented in Rya with a supplementary table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-51) Temporal indexing support for MongoDB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336299#comment-15336299
 ] 

ASF GitHub Bot commented on RYA-51:
---

Github user pujav65 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/48#discussion_r67527705
  
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoDBRyaDAO.java 
---
@@ -179,85 +145,85 @@ public void destroy() throws RyaDAOException {
 IOUtils.closeQuietly(queryEngine);
 }
 
-   public void add(RyaStatement statement) throws RyaDAOException {

-   // add it to the collection
-   try {
-   coll.insert(storageStrategy.serialize(statement));
-   for(RyaSecondaryIndexer index: secondaryIndexers) {
-   index.storeStatement(statement);
-   }
-   }
-   catch (com.mongodb.MongoException.DuplicateKey exception){
-   // ignore
-   }
-   catch (com.mongodb.DuplicateKeyException exception){
-   // ignore
-   }
-   catch (Exception ex){
-   // ignore single exceptions
-   ex.printStackTrace();
-   }
-   }
-   
-   public void add(Iterator statement) throws 
RyaDAOException {
-   List dbInserts = new ArrayList();
-   while (statement.hasNext()){
-   RyaStatement ryaStatement = statement.next();
-   DBObject insert = 
storageStrategy.serialize(ryaStatement);
-   dbInserts.add(insert);
-   
+@Override
+public void add(final RyaStatement statement) throws RyaDAOException {
+// add it to the collection
+try {
+coll.insert(storageStrategy.serialize(statement));
+for(final RyaSecondaryIndexer index: secondaryIndexers) {
+index.storeStatement(statement);
+}
+} catch (final IOException e) {
+log.error("Unable to add: " + statement.toString());
+throw new RyaDAOException(e);
+}
+}
+
+@Override
+public void add(final Iterator statement) throws 
RyaDAOException {
+final List dbInserts = new ArrayList();
+while (statement.hasNext()){
+final RyaStatement ryaStatement = statement.next();
+final DBObject insert = 
storageStrategy.serialize(ryaStatement);
+dbInserts.add(insert);
+
 try {
-for (RyaSecondaryIndexer index : secondaryIndexers) {
+for (final RyaSecondaryIndexer index : secondaryIndexers) {
 index.storeStatement(ryaStatement);
 }
-} catch (IOException e) {
-throw new RyaDAOException(e);
+} catch (final IOException e) {
+log.error("Failed to add: " + ryaStatement.toString() + " 
to the indexer");
 }
-
-   }
-   coll.insert(dbInserts, new 
InsertOptions().continueOnError(true));
-   }
-
-   public void delete(RyaStatement statement, MongoDBRdfConfiguration conf)
-   throws RyaDAOException {
-   DBObject obj = storageStrategy.getQuery(statement);
-   coll.remove(obj);
-   }
-
-   public void dropGraph(MongoDBRdfConfiguration conf, RyaURI... graphs)
-   throws RyaDAOException {
-   
-   }
-
-   public void delete(Iterator statements,
-   MongoDBRdfConfiguration conf) throws RyaDAOException {
-   while (statements.hasNext()){
-   RyaStatement ryaStatement = statements.next();
-   coll.remove(storageStrategy.getQuery(ryaStatement));
-   }
-   
-   }
-
-   public String getVersion() throws RyaDAOException {
-   return "1.0";
-   }
-
-   public RyaQueryEngine getQueryEngine() {
-   return queryEngine;
-   }
-
-   public RyaNamespaceManager 
getNamespaceManager() {
-   return nameSpaceManager;
-   }
-
-   public void purge(RdfCloudTripleStoreConfiguration configuration) {
-   // TODO Auto-generated method stub
-   
-   }
-
-   public void dropAndDestroy() throws RyaDAOException {
-   db.dropDatabase(); // this is dangerous!
-   }
 
+}
+coll.insert(dbInserts, new InsertOptions().continueOnError(true));
--- End diff --

ok does the continue on error catch them?


> 

[jira] [Commented] (RYA-51) Temporal indexing support for MongoDB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336318#comment-15336318
 ] 

ASF GitHub Bot commented on RYA-51:
---

Github user pujav65 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/48#discussion_r67529573
  
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoDBRyaDAO.java 
---
@@ -179,85 +145,85 @@ public void destroy() throws RyaDAOException {
 IOUtils.closeQuietly(queryEngine);
 }
 
-   public void add(RyaStatement statement) throws RyaDAOException {

-   // add it to the collection
-   try {
-   coll.insert(storageStrategy.serialize(statement));
-   for(RyaSecondaryIndexer index: secondaryIndexers) {
-   index.storeStatement(statement);
-   }
-   }
-   catch (com.mongodb.MongoException.DuplicateKey exception){
-   // ignore
-   }
-   catch (com.mongodb.DuplicateKeyException exception){
-   // ignore
-   }
-   catch (Exception ex){
-   // ignore single exceptions
-   ex.printStackTrace();
-   }
-   }
-   
-   public void add(Iterator statement) throws 
RyaDAOException {
-   List dbInserts = new ArrayList();
-   while (statement.hasNext()){
-   RyaStatement ryaStatement = statement.next();
-   DBObject insert = 
storageStrategy.serialize(ryaStatement);
-   dbInserts.add(insert);
-   
+@Override
+public void add(final RyaStatement statement) throws RyaDAOException {
+// add it to the collection
+try {
+coll.insert(storageStrategy.serialize(statement));
+for(final RyaSecondaryIndexer index: secondaryIndexers) {
+index.storeStatement(statement);
+}
+} catch (final IOException e) {
+log.error("Unable to add: " + statement.toString());
+throw new RyaDAOException(e);
+}
+}
+
+@Override
+public void add(final Iterator statement) throws 
RyaDAOException {
+final List dbInserts = new ArrayList();
+while (statement.hasNext()){
+final RyaStatement ryaStatement = statement.next();
+final DBObject insert = 
storageStrategy.serialize(ryaStatement);
+dbInserts.add(insert);
+
 try {
-for (RyaSecondaryIndexer index : secondaryIndexers) {
+for (final RyaSecondaryIndexer index : secondaryIndexers) {
 index.storeStatement(ryaStatement);
 }
-} catch (IOException e) {
-throw new RyaDAOException(e);
+} catch (final IOException e) {
+log.error("Failed to add: " + ryaStatement.toString() + " 
to the indexer");
 }
-
-   }
-   coll.insert(dbInserts, new 
InsertOptions().continueOnError(true));
-   }
-
-   public void delete(RyaStatement statement, MongoDBRdfConfiguration conf)
-   throws RyaDAOException {
-   DBObject obj = storageStrategy.getQuery(statement);
-   coll.remove(obj);
-   }
-
-   public void dropGraph(MongoDBRdfConfiguration conf, RyaURI... graphs)
-   throws RyaDAOException {
-   
-   }
-
-   public void delete(Iterator statements,
-   MongoDBRdfConfiguration conf) throws RyaDAOException {
-   while (statements.hasNext()){
-   RyaStatement ryaStatement = statements.next();
-   coll.remove(storageStrategy.getQuery(ryaStatement));
-   }
-   
-   }
-
-   public String getVersion() throws RyaDAOException {
-   return "1.0";
-   }
-
-   public RyaQueryEngine getQueryEngine() {
-   return queryEngine;
-   }
-
-   public RyaNamespaceManager 
getNamespaceManager() {
-   return nameSpaceManager;
-   }
-
-   public void purge(RdfCloudTripleStoreConfiguration configuration) {
-   // TODO Auto-generated method stub
-   
-   }
-
-   public void dropAndDestroy() throws RyaDAOException {
-   db.dropDatabase(); // this is dangerous!
-   }
 
+}
+coll.insert(dbInserts, new InsertOptions().continueOnError(true));
+}
+
+@Override
+public void delete(final 

[jira] [Commented] (RYA-51) Temporal indexing support for MongoDB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336319#comment-15336319
 ] 

ASF GitHub Bot commented on RYA-51:
---

Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/48
  
ok other than needing to put in the delete call in the dao, it looks good.  
please adjust your eclipse settings so you're not auto-adding the final 
statements


> Temporal indexing support for MongoDB
> -
>
> Key: RYA-51
> URL: https://issues.apache.org/jira/browse/RYA-51
> Project: Rya
>  Issue Type: New Feature
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Puja Valiyil 
>
> There should be temporal indexing support for Mongo DB backed Rya.  i imagine 
> this would be implemented using another collection, similar to how temporal 
> indexing is implemented in Rya with a supplementary table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-51) Temporal indexing support for MongoDB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336426#comment-15336426
 ] 

ASF GitHub Bot commented on RYA-51:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/48#discussion_r67539468
  
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoDBRyaDAO.java 
---
@@ -179,85 +145,85 @@ public void destroy() throws RyaDAOException {
 IOUtils.closeQuietly(queryEngine);
 }
 
-   public void add(RyaStatement statement) throws RyaDAOException {

-   // add it to the collection
-   try {
-   coll.insert(storageStrategy.serialize(statement));
-   for(RyaSecondaryIndexer index: secondaryIndexers) {
-   index.storeStatement(statement);
-   }
-   }
-   catch (com.mongodb.MongoException.DuplicateKey exception){
-   // ignore
-   }
-   catch (com.mongodb.DuplicateKeyException exception){
-   // ignore
-   }
-   catch (Exception ex){
-   // ignore single exceptions
-   ex.printStackTrace();
-   }
-   }
-   
-   public void add(Iterator statement) throws 
RyaDAOException {
-   List dbInserts = new ArrayList();
-   while (statement.hasNext()){
-   RyaStatement ryaStatement = statement.next();
-   DBObject insert = 
storageStrategy.serialize(ryaStatement);
-   dbInserts.add(insert);
-   
+@Override
+public void add(final RyaStatement statement) throws RyaDAOException {
+// add it to the collection
+try {
+coll.insert(storageStrategy.serialize(statement));
+for(final RyaSecondaryIndexer index: secondaryIndexers) {
+index.storeStatement(statement);
+}
+} catch (final IOException e) {
+log.error("Unable to add: " + statement.toString());
+throw new RyaDAOException(e);
+}
+}
+
+@Override
+public void add(final Iterator statement) throws 
RyaDAOException {
+final List dbInserts = new ArrayList();
+while (statement.hasNext()){
+final RyaStatement ryaStatement = statement.next();
+final DBObject insert = 
storageStrategy.serialize(ryaStatement);
+dbInserts.add(insert);
+
 try {
-for (RyaSecondaryIndexer index : secondaryIndexers) {
+for (final RyaSecondaryIndexer index : secondaryIndexers) {
 index.storeStatement(ryaStatement);
 }
-} catch (IOException e) {
-throw new RyaDAOException(e);
+} catch (final IOException e) {
+log.error("Failed to add: " + ryaStatement.toString() + " 
to the indexer");
 }
-
-   }
-   coll.insert(dbInserts, new 
InsertOptions().continueOnError(true));
-   }
-
-   public void delete(RyaStatement statement, MongoDBRdfConfiguration conf)
-   throws RyaDAOException {
-   DBObject obj = storageStrategy.getQuery(statement);
-   coll.remove(obj);
-   }
-
-   public void dropGraph(MongoDBRdfConfiguration conf, RyaURI... graphs)
-   throws RyaDAOException {
-   
-   }
-
-   public void delete(Iterator statements,
-   MongoDBRdfConfiguration conf) throws RyaDAOException {
-   while (statements.hasNext()){
-   RyaStatement ryaStatement = statements.next();
-   coll.remove(storageStrategy.getQuery(ryaStatement));
-   }
-   
-   }
-
-   public String getVersion() throws RyaDAOException {
-   return "1.0";
-   }
-
-   public RyaQueryEngine getQueryEngine() {
-   return queryEngine;
-   }
-
-   public RyaNamespaceManager 
getNamespaceManager() {
-   return nameSpaceManager;
-   }
-
-   public void purge(RdfCloudTripleStoreConfiguration configuration) {
-   // TODO Auto-generated method stub
-   
-   }
-
-   public void dropAndDestroy() throws RyaDAOException {
-   db.dropDatabase(); // this is dangerous!
-   }
 
+}
+coll.insert(dbInserts, new InsertOptions().continueOnError(true));
--- End diff --

yeah, I was following how the mongo client 

[jira] [Commented] (RYA-51) Temporal indexing support for MongoDB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336441#comment-15336441
 ] 

ASF GitHub Bot commented on RYA-51:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/48#discussion_r67540581
  
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoDBRyaDAO.java 
---
@@ -179,85 +145,85 @@ public void destroy() throws RyaDAOException {
 IOUtils.closeQuietly(queryEngine);
 }
 
-   public void add(RyaStatement statement) throws RyaDAOException {

-   // add it to the collection
-   try {
-   coll.insert(storageStrategy.serialize(statement));
-   for(RyaSecondaryIndexer index: secondaryIndexers) {
-   index.storeStatement(statement);
-   }
-   }
-   catch (com.mongodb.MongoException.DuplicateKey exception){
-   // ignore
-   }
-   catch (com.mongodb.DuplicateKeyException exception){
-   // ignore
-   }
-   catch (Exception ex){
-   // ignore single exceptions
-   ex.printStackTrace();
-   }
-   }
-   
-   public void add(Iterator statement) throws 
RyaDAOException {
-   List dbInserts = new ArrayList();
-   while (statement.hasNext()){
-   RyaStatement ryaStatement = statement.next();
-   DBObject insert = 
storageStrategy.serialize(ryaStatement);
-   dbInserts.add(insert);
-   
+@Override
+public void add(final RyaStatement statement) throws RyaDAOException {
+// add it to the collection
+try {
+coll.insert(storageStrategy.serialize(statement));
+for(final RyaSecondaryIndexer index: secondaryIndexers) {
+index.storeStatement(statement);
+}
+} catch (final IOException e) {
+log.error("Unable to add: " + statement.toString());
+throw new RyaDAOException(e);
+}
+}
+
+@Override
+public void add(final Iterator statement) throws 
RyaDAOException {
+final List dbInserts = new ArrayList();
+while (statement.hasNext()){
+final RyaStatement ryaStatement = statement.next();
+final DBObject insert = 
storageStrategy.serialize(ryaStatement);
+dbInserts.add(insert);
+
 try {
-for (RyaSecondaryIndexer index : secondaryIndexers) {
+for (final RyaSecondaryIndexer index : secondaryIndexers) {
 index.storeStatement(ryaStatement);
 }
-} catch (IOException e) {
-throw new RyaDAOException(e);
+} catch (final IOException e) {
+log.error("Failed to add: " + ryaStatement.toString() + " 
to the indexer");
 }
-
-   }
-   coll.insert(dbInserts, new 
InsertOptions().continueOnError(true));
-   }
-
-   public void delete(RyaStatement statement, MongoDBRdfConfiguration conf)
-   throws RyaDAOException {
-   DBObject obj = storageStrategy.getQuery(statement);
-   coll.remove(obj);
-   }
-
-   public void dropGraph(MongoDBRdfConfiguration conf, RyaURI... graphs)
-   throws RyaDAOException {
-   
-   }
-
-   public void delete(Iterator statements,
-   MongoDBRdfConfiguration conf) throws RyaDAOException {
-   while (statements.hasNext()){
-   RyaStatement ryaStatement = statements.next();
-   coll.remove(storageStrategy.getQuery(ryaStatement));
-   }
-   
-   }
-
-   public String getVersion() throws RyaDAOException {
-   return "1.0";
-   }
-
-   public RyaQueryEngine getQueryEngine() {
-   return queryEngine;
-   }
-
-   public RyaNamespaceManager 
getNamespaceManager() {
-   return nameSpaceManager;
-   }
-
-   public void purge(RdfCloudTripleStoreConfiguration configuration) {
-   // TODO Auto-generated method stub
-   
-   }
-
-   public void dropAndDestroy() throws RyaDAOException {
-   db.dropDatabase(); // this is dangerous!
-   }
 
+}
+coll.insert(dbInserts, new InsertOptions().continueOnError(true));
+}
+
+@Override
+public void delete(final 

[jira] [Commented] (RYA-96) Geo Indexer zookeeper dependency conflict.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-96?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348975#comment-15348975
 ] 

ASF GitHub Bot commented on RYA-96:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/52


> Geo Indexer zookeeper dependency conflict.
> --
>
> Key: RYA-96
> URL: https://issues.apache.org/jira/browse/RYA-96
> Project: Rya
>  Issue Type: Bug
> Environment: Integration test connected to an Accumulo instance 
> running on a cluster.
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
>Priority: Critical
>
> The Accumulo implementation of the Geo Index does not work because of 
> conflicting zookeeper versions. It looks like Accumulo wants to use 3.3.6, so 
> that's what our build pulls in. However, GeoMesa depends on Curator which 
> uses zookeeper 3.4.6. This throws an exception when you try to initialize 
> Sail using a Geo Index.
> Stack Trace:
> java.lang.NoSuchMethodError: 
> org.apache.zookeeper.ZooKeeper.(Ljava/lang/String;ILorg/apache/zookeeper/Watcher;Z)V
>   at 
> org.apache.curator.utils.DefaultZookeeperFactory.newZooKeeper(DefaultZookeeperFactory.java:29)
>   at 
> org.apache.curator.framework.imps.CuratorFrameworkImpl$2.newZooKeeper(CuratorFrameworkImpl.java:146)
>   at org.apache.curator.HandleHolder$1.getZooKeeper(HandleHolder.java:94)
>   at org.apache.curator.HandleHolder.getZooKeeper(HandleHolder.java:55)
>   at org.apache.curator.ConnectionState.reset(ConnectionState.java:218)
>   at org.apache.curator.ConnectionState.start(ConnectionState.java:102)
>   at 
> org.apache.curator.CuratorZookeeperClient.start(CuratorZookeeperClient.java:189)
>   at 
> org.apache.curator.framework.imps.CuratorFrameworkImpl.start(CuratorFrameworkImpl.java:248)
>   at 
> org.locationtech.geomesa.accumulo.data.AccumuloDataStore.acquireDistributedLock(AccumuloDataStore.scala:806)
>   at 
> org.locationtech.geomesa.accumulo.data.AccumuloDataStore.createSchema(AccumuloDataStore.scala:264)
>   at 
> org.locationtech.geomesa.accumulo.data.AccumuloDataStore.createSchema(AccumuloDataStore.scala:65)
>   at 
> mvm.rya.indexing.accumulo.geo.GeoMesaGeoIndexer.getStatementFeatureType(GeoMesaGeoIndexer.java:226)
>   at 
> mvm.rya.indexing.accumulo.geo.GeoMesaGeoIndexer.initInternal(GeoMesaGeoIndexer.java:170)
>   at 
> mvm.rya.indexing.accumulo.geo.GeoMesaGeoIndexer.setConf(GeoMesaGeoIndexer.java:149)
>   at 
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
>   at 
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
>   at 
> org.apache.hadoop.conf.Configuration.getInstances(Configuration.java:1961)
>   at 
> mvm.rya.accumulo.AccumuloRdfConfiguration.getAdditionalIndexers(AccumuloRdfConfiguration.java:101)
>   at mvm.rya.accumulo.AccumuloRyaDAO.init(AccumuloRyaDAO.java:131)
>   at mvm.rya.sail.config.RyaSailFactory.getRyaSail(RyaSailFactory.java:82)
>   at 
> mvm.rya.sail.config.RyaSailFactory.getInstance(RyaSailFactory.java:50)
>   at 
> mvm.rya.indexing.accumulo.geo.GeoIndexerTest.test(GeoIndexerTest.java:112)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at 
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
>   at 
> 

[jira] [Commented] (RYA-32) Improve how metadata and values are written to Accumulo PCJ tables.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123921#comment-15123921
 ] 

ASF GitHub Bot commented on RYA-32:
---

Github user meiercaleb closed the pull request at:

https://github.com/apache/incubator-rya/pull/15


> Improve how metadata and values are written to Accumulo PCJ tables.
> ---
>
> Key: RYA-32
> URL: https://issues.apache.org/jira/browse/RYA-32
> Project: Rya
>  Issue Type: Improvement
>Reporter: Kevin Chilton
>
> Refactor how Precomputed Joins (PCJs) are stored in Accumulo.
> 1. Change how PCJ results are written to an Accumulo PCJ table so that they 
> include all information required to deserialize them.
> 2. Change how the PCJ metadata is stored within the Accumulo table to allow 
> Cardinalities to be updated.
> 3. Create an API for interacting with PCJ tables.
> 4. Update the accmulo.pig project to write values using the new formats.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-32) Improve how metadata and values are written to Accumulo PCJ tables.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15124097#comment-15124097
 ] 

ASF GitHub Bot commented on RYA-32:
---

GitHub user meiercaleb opened a pull request:

https://github.com/apache/incubator-rya/pull/17

RYA-32 Improve how metadata and values are written to Accumulo PCJ tables

Created a class AccumuloPcjSerializer within the rya.indexing project for 
serializing and deserializing binding sets for storing and reading solutions in 
precomputed join tables. Also developed the class PcjTables to encapsulate the 
logic for creating a PCJ table. Refactored some of the core precomputed join 
classes to utilize these updates and included a number of comments to make the 
code clearer. Updated a large number of tests within the packages 
mvm.rya.indexing.external and mvm.rya.indexing.external.tupleSet to reflect 
these updates.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/meiercaleb/incubator-rya 
RYA-32-Precomputed-Join-Refactor

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/17.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #17


commit c12f58f46be0aed3c88eff7504d4bfd4e4f295e9
Author: Caleb Meier 
Date:   2016-01-29T16:40:59Z

RYA-32 Improve how metadata and values are written to Accumulo PCJ tables




> Improve how metadata and values are written to Accumulo PCJ tables.
> ---
>
> Key: RYA-32
> URL: https://issues.apache.org/jira/browse/RYA-32
> Project: Rya
>  Issue Type: Improvement
>Reporter: Kevin Chilton
>
> Refactor how Precomputed Joins (PCJs) are stored in Accumulo.
> 1. Change how PCJ results are written to an Accumulo PCJ table so that they 
> include all information required to deserialize them.
> 2. Change how the PCJ metadata is stored within the Accumulo table to allow 
> Cardinalities to be updated.
> 3. Create an API for interacting with PCJ tables.
> 4. Update the accmulo.pig project to write values using the new formats.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-41) SPARQL queries with the DISTINCT keyword throws an exception when performing a PCJ enabled query.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157577#comment-15157577
 ] 

ASF GitHub Bot commented on RYA-41:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/20


> SPARQL queries with the DISTINCT keyword throws an exception when performing 
> a PCJ enabled query.
> -
>
> Key: RYA-41
> URL: https://issues.apache.org/jira/browse/RYA-41
> Project: Rya
>  Issue Type: Bug
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
>
> Steps to reproduce:
> 1. Load some data into the core Rya tables within Accumulo.
> 2. Create a PCJ for a query using those Rya tables.
> 3. Run a query where PCJ Inexing is enabled that uses the "DISTINCT" keyword.
> The application will throw a ClassCastException because it expects the top 
> most node of a ParsedQuery to be a Projection node, but it is a Distinct node.
> The fix is to navigate the query's structure until you find the Projection 
> node instead of assuming it is the top most node.
> Stack Trace:
> 1875 [main] WARN org.apache.accumulo.core.client.ClientConfiguration  - Found 
> no client.conf in default paths. Using default client configuration values.
> org.openrdf.query.QueryEvaluationException: java.lang.ClassCastException: 
> org.openrdf.query.algebra.Distinct cannot be cast to 
> org.openrdf.query.algebra.Projection
>   at 
> org.openrdf.repository.sail.SailTupleQuery.evaluate(SailTupleQuery.java:67)
>   at 
> org.openrdf.repository.sail.SailTupleQuery.evaluate(SailTupleQuery.java:75)
>   at EntityDirectExample.testQuery(EntityDirectExample.java:235)
>   at EntityDirectExample.runQuery(EntityDirectExample.java:259)
>   at EntityDirectExample.runBenchmarks(EntityDirectExample.java:193)
>   at EntityDirectExample.main(EntityDirectExample.java:102)
> Caused by: org.openrdf.sail.SailException: java.lang.ClassCastException: 
> org.openrdf.query.algebra.Distinct cannot be cast to 
> org.openrdf.query.algebra.Projection
>   at 
> mvm.rya.rdftriplestore.RdfCloudTripleStoreConnection.evaluateInternal(RdfCloudTripleStoreConnection.java:418)
>   at 
> org.openrdf.sail.helpers.SailConnectionBase.evaluate(SailConnectionBase.java:295)
>   at 
> org.openrdf.repository.sail.SailTupleQuery.evaluate(SailTupleQuery.java:60)
>   ... 5 more
> Caused by: java.lang.ClassCastException: org.openrdf.query.algebra.Distinct 
> cannot be cast to org.openrdf.query.algebra.Projection
>   at 
> mvm.rya.indexing.external.tupleSet.AccumuloIndexSet.(AccumuloIndexSet.java:126)
>   at 
> mvm.rya.indexing.external.PrecompJoinOptimizer.getAccIndices(PrecompJoinOptimizer.java:858)
>   at 
> mvm.rya.indexing.external.PrecompJoinOptimizer.setConf(PrecompJoinOptimizer.java:134)
>   at 
> mvm.rya.rdftriplestore.RdfCloudTripleStoreConnection.evaluateInternal(RdfCloudTripleStoreConnection.java:291)
>   ... 7 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-37) Bump GeoMesa version to 1.2.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157576#comment-15157576
 ] 

ASF GitHub Bot commented on RYA-37:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/19


> Bump GeoMesa version to 1.2.0
> -
>
> Key: RYA-37
> URL: https://issues.apache.org/jira/browse/RYA-37
> Project: Rya
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.2.10
>Reporter: Mike Pfaffenberger
> Fix For: 3.2.10
>
>
> GeoMesa is released officially. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-11) Free Text search support for Mongo DB backed Rya

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172116#comment-15172116
 ] 

ASF GitHub Bot commented on RYA-11:
---

GitHub user pujav65 opened a pull request:

https://github.com/apache/incubator-rya/pull/21

RYA-11 initial implementation of free text for mongo backend

Initial implementation.  Mongo backed free text query does not match 
Accumulo, but I don't have the cycles to finish that.  Updating this so that it 
matches would be a good future task for someone to pick up.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pujav65/incubator-rya RYA-11

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/21.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #21


commit 59073a3dcc4e2737d678255921c27e363a600b1f
Author: pujav65 
Date:   2016-01-13T16:21:52Z

RYA-11 initial implementation of free text for mongo backend




> Free Text search support for Mongo DB backed Rya
> 
>
> Key: RYA-11
> URL: https://issues.apache.org/jira/browse/RYA-11
> Project: Rya
>  Issue Type: New Feature
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Puja Valiyil 
>
> The secondary indexing support for Mongo DB should be extended to also 
> include support for free text search (which the Accumulo back end already 
> provides).  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-41) SPARQL queries with the DISTINCT keyword throws an exception when performing a PCJ enabled query.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155061#comment-15155061
 ] 

ASF GitHub Bot commented on RYA-41:
---

Github user amihalik commented on the pull request:

https://github.com/apache/incubator-rya/pull/20#issuecomment-186440469
  
Looks good.  I'll merge this on Sunday unless there are other concerns.


> SPARQL queries with the DISTINCT keyword throws an exception when performing 
> a PCJ enabled query.
> -
>
> Key: RYA-41
> URL: https://issues.apache.org/jira/browse/RYA-41
> Project: Rya
>  Issue Type: Bug
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
>
> Steps to reproduce:
> 1. Load some data into the core Rya tables within Accumulo.
> 2. Create a PCJ for a query using those Rya tables.
> 3. Run a query where PCJ Inexing is enabled that uses the "DISTINCT" keyword.
> The application will throw a ClassCastException because it expects the top 
> most node of a ParsedQuery to be a Projection node, but it is a Distinct node.
> The fix is to navigate the query's structure until you find the Projection 
> node instead of assuming it is the top most node.
> Stack Trace:
> 1875 [main] WARN org.apache.accumulo.core.client.ClientConfiguration  - Found 
> no client.conf in default paths. Using default client configuration values.
> org.openrdf.query.QueryEvaluationException: java.lang.ClassCastException: 
> org.openrdf.query.algebra.Distinct cannot be cast to 
> org.openrdf.query.algebra.Projection
>   at 
> org.openrdf.repository.sail.SailTupleQuery.evaluate(SailTupleQuery.java:67)
>   at 
> org.openrdf.repository.sail.SailTupleQuery.evaluate(SailTupleQuery.java:75)
>   at EntityDirectExample.testQuery(EntityDirectExample.java:235)
>   at EntityDirectExample.runQuery(EntityDirectExample.java:259)
>   at EntityDirectExample.runBenchmarks(EntityDirectExample.java:193)
>   at EntityDirectExample.main(EntityDirectExample.java:102)
> Caused by: org.openrdf.sail.SailException: java.lang.ClassCastException: 
> org.openrdf.query.algebra.Distinct cannot be cast to 
> org.openrdf.query.algebra.Projection
>   at 
> mvm.rya.rdftriplestore.RdfCloudTripleStoreConnection.evaluateInternal(RdfCloudTripleStoreConnection.java:418)
>   at 
> org.openrdf.sail.helpers.SailConnectionBase.evaluate(SailConnectionBase.java:295)
>   at 
> org.openrdf.repository.sail.SailTupleQuery.evaluate(SailTupleQuery.java:60)
>   ... 5 more
> Caused by: java.lang.ClassCastException: org.openrdf.query.algebra.Distinct 
> cannot be cast to org.openrdf.query.algebra.Projection
>   at 
> mvm.rya.indexing.external.tupleSet.AccumuloIndexSet.(AccumuloIndexSet.java:126)
>   at 
> mvm.rya.indexing.external.PrecompJoinOptimizer.getAccIndices(PrecompJoinOptimizer.java:858)
>   at 
> mvm.rya.indexing.external.PrecompJoinOptimizer.setConf(PrecompJoinOptimizer.java:134)
>   at 
> mvm.rya.rdftriplestore.RdfCloudTripleStoreConnection.evaluateInternal(RdfCloudTripleStoreConnection.java:291)
>   ... 7 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-55) Incrementally update PCJ secondary indicies.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202359#comment-15202359
 ] 

ASF GitHub Bot commented on RYA-55:
---

GitHub user kchilton2 opened a pull request:

https://github.com/apache/incubator-rya/pull/31

RYA-55 Implemented a series of projects that use Fluo to incrementall…

…y update the results of a Rya Precomputed Join secondary index.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kchilton2/incubator-rya RYA-55

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/31.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #31


commit 6f89a5d756eb000b0dd6f3919d63ce94d9b3c7f2
Author: Kevin Chilton 
Date:   2016-03-18T23:13:57Z

RYA-55 Implemented a series of projects that use Fluo to incrementally 
update the results of a Rya Precomputed Join secondary index.




> Incrementally update PCJ secondary indicies.
> 
>
> Key: RYA-55
> URL: https://issues.apache.org/jira/browse/RYA-55
> Project: Rya
>  Issue Type: New Feature
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230635#comment-15230635
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r58911417
  
--- Diff: 
extras/indexing/src/test/java/mvm/rya/indexing/external/tupleSet/PcjTablesIntegrationTests.java
 ---
@@ -151,35 +151,39 @@ public void addResults() throws PcjException, 
TableNotFoundException, BindingSet
 
 // Create a PCJ table in the Mini Accumulo.
 final String pcjTableName = new 
PcjTableNameFactory().makeTableName(RYA_TABLE_PREFIX, "testPcj");
-Set varOrders = new 
ShiftVarOrderFactory().makeVarOrders(new VariableOrder("name;age"));
-PcjTables pcjs = new PcjTables();
+final Set varOrders = new 
ShiftVarOrderFactory().makeVarOrders(new VariableOrder("name;age"));
+final PcjTables pcjs = new PcjTables();
 pcjs.createPcjTable(accumuloConn, pcjTableName, varOrders, sparql);
 
 // Add a few results to the PCJ table.
-MapBindingSet alice = new MapBindingSet();
+final MapBindingSet alice = new MapBindingSet();
 alice.addBinding("name", new URIImpl("http://Alice;));
 alice.addBinding("age", new NumericLiteralImpl(14, 
XMLSchema.INTEGER));
 
-MapBindingSet bob = new MapBindingSet();
+final MapBindingSet bob = new MapBindingSet();
 bob.addBinding("name", new URIImpl("http://Bob;));
 bob.addBinding("age", new NumericLiteralImpl(16, 
XMLSchema.INTEGER));
 
-MapBindingSet charlie = new MapBindingSet();
+final MapBindingSet charlie = new MapBindingSet();
 charlie.addBinding("name", new URIImpl("http://Charlie;));
 charlie.addBinding("age", new NumericLiteralImpl(12, 
XMLSchema.INTEGER));
 
-Set results = Sets.newHashSet(alice, bob, 
charlie);
+final Set results = 
Sets.newHashSet(
+new VisibilityBindingSet(alice),
--- End diff --

whitespace


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230630#comment-15230630
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r58911304
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/accumulo/VisibilityBindingSet.java
 ---
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.indexing.accumulo;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import org.openrdf.query.BindingSet;
+
+import mvm.rya.indexing.external.BindingSetDecorator;
+
+/**
+ * Decorates a {@link BindingSet} with a collection of visibilities.
+ */
--- End diff --

add parametersAreNonNullableByDefault


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230625#comment-15230625
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r58911177
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/external/tupleSet/FullBindingSetStringConverter.java
 ---
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.indexing.external.tupleSet;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.annotation.ParametersAreNonnullByDefault;
+
+import org.openrdf.model.Value;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+
+import com.google.common.base.Joiner;
+
+import mvm.rya.api.domain.RyaType;
+import mvm.rya.api.resolver.RdfToRyaConversions;
+import mvm.rya.indexing.external.tupleSet.PcjTables.VariableOrder;
+
+/**
+ * Converts {@link BindingSet}s to Strings and back again. The Strings do 
not
+ * include the binding names and are ordered with a {@link VariableOrder}.
+ */
+@ParametersAreNonnullByDefault
+public class FullBindingSetStringConverter extends 
BindingSetStringConverter {
--- End diff --

Rename?


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230657#comment-15230657
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r58911855
  
--- Diff: 
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/observers/QueryResultObserver.java
 ---
@@ -48,8 +50,8 @@
 private static final Logger log = 
Logger.getLogger(QueryResultObserver.class);
 
 private final FluoQueryMetadataDAO queryDao = new 
FluoQueryMetadataDAO();
-
-private final BindingSetStringConverter converter = new 
BindingSetStringConverter();
+private final Encoder encoder = new StringEncoder();
+private final VisibilityBindingSetStringConverter converter = new 
VisibilityBindingSetStringConverter();
--- End diff --

static, CONVERTER, ENCODER, might as well update QUERY_DAO?


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230624#comment-15230624
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r58911089
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/external/tupleSet/BindingSetConverter.java
 ---
@@ -1,5 +1,5 @@
 /*
- * Licensed to the Apache Software Foundation (ASF) under one
+* Licensed to the Apache Software Foundation (ASF) under one
--- End diff --

remove this


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230641#comment-15230641
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r58911571
  
--- Diff: 
extras/rya.pcj.fluo/pcj.fluo.api/src/main/java/org/apache/rya/indexing/pcj/fluo/api/InsertTriples.java
 ---
@@ -53,30 +55,35 @@
  */
 private static final WholeRowTripleResolver TRIPLE_RESOLVER = new 
WholeRowTripleResolver();
 
+private static final Encoder encoder = new StringEncoder();
--- End diff --

ENCODER


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235172#comment-15235172
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59214092
  
--- Diff: 
extras/indexing/src/test/java/mvm/rya/indexing/external/tupleSet/VisibilityBindingSetStringConverterTest.java
 ---
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.indexing.external.tupleSet;
+
+import static org.junit.Assert.assertEquals;
+
+import org.junit.Test;
+import org.openrdf.model.impl.URIImpl;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.impl.MapBindingSet;
+
+import mvm.rya.indexing.accumulo.VisibilityBindingSet;
+import 
mvm.rya.indexing.external.tupleSet.BindingSetConverter.BindingSetConversionException;
+import mvm.rya.indexing.external.tupleSet.PcjTables.VariableOrder;
+
+/**
+ * Tests the methods of {@link BindingSetStringConverter}.
+ */
+public class VisibilityBindingSetStringConverterTest {
+@Test
+public void toString_URIs() throws BindingSetConversionException {
+// Setup the binding set that will be converted.
+final MapBindingSet originalBindingSet = new MapBindingSet();
+originalBindingSet.addBinding("x", new URIImpl("http://a;));
+originalBindingSet.addBinding("y", new URIImpl("http://b;));
+originalBindingSet.addBinding("z", new URIImpl("http://c;));
+
+final VisibilityBindingSet visiSet = new 
VisibilityBindingSet(originalBindingSet, "A");
+
+// Convert it to a String.
+final VariableOrder varOrder = new VariableOrder("y", "z", "x");
+final VisibilityBindingSetStringConverter converter = new 
VisibilityBindingSetStringConverter();
+final String bindingSetString = converter.convert(visiSet, 
varOrder);
+
+// Ensure it converted to the expected result.l
+final String expected =
+"http://b<<~>>http://www.w3.org/2001/XMLSchema#anyURI:::; +
+"http://c<<~>>http://www.w3.org/2001/XMLSchema#anyURI:::; +
+"http://a<<~>>http://www.w3.org/2001/XMLSchema#anyURI; +
+(char)1 + "A";
--- End diff --

Might want to store the (char)1 delimiter as a constant somewhere and 
reference that here.


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235130#comment-15235130
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59209572
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/external/BindingSetDecorator.java
 ---
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.indexing.external;
+
+import java.util.Iterator;
+import java.util.Set;
+
+import org.openrdf.model.Value;
+import org.openrdf.query.Binding;
+import org.openrdf.query.BindingSet;
+
+/**
+ * Abstracts out the decoration of a {@link BindingSet}.
+ */
+public abstract class BindingSetDecorator implements BindingSet {
+private static final long serialVersionUID = 1L;
+protected final BindingSet set;
+private volatile int hashCode;
+
+public BindingSetDecorator(final BindingSet set) {
--- End diff --

What is the nullness contract here? Also missing docs.


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235142#comment-15235142
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59210679
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/external/tupleSet/FullBindingSetStringConverter.java
 ---
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.indexing.external.tupleSet;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.annotation.ParametersAreNonnullByDefault;
+
+import org.openrdf.model.Value;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+
+import com.google.common.base.Joiner;
+
+import mvm.rya.api.domain.RyaType;
+import mvm.rya.api.resolver.RdfToRyaConversions;
+import mvm.rya.indexing.external.tupleSet.PcjTables.VariableOrder;
+
+/**
+ * Converts {@link BindingSet}s to Strings and back again. The Strings do 
not
+ * include the binding names and are ordered with a {@link VariableOrder}.
+ */
+@ParametersAreNonnullByDefault
+public class FullBindingSetStringConverter extends 
BindingSetStringConverter {
--- End diff --

Because this implementation totally converts binding sets (including the 
binding names), you don't need to use variable order objects to keep track of 
the bindings.


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235175#comment-15235175
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59214884
  
--- Diff: 
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/FilterResultUpdater.java
 ---
@@ -126,11 +129,19 @@ public void updateFilterResults(
 filterBindingSet.addBinding(binding);
 }
 }
-final String filterBindingSetString = 
converter.convert(filterBindingSet, filterVarOrder);
 
-final Bytes row = encoder.encode( filterMetadata.getNodeId() + 
NODEID_BS_DELIM + filterBindingSetString );
+final String filterBindingSetIdString = 
ID_CONVERTER.convert(filterBindingSet, filterVarOrder);
+String filterBindingSetValueString = "";
+if(childBindingSet instanceof VisibilityBindingSet) {
--- End diff --

We shouldn't do the type checking here. Assume it's always a 
VisibilityBindingSet once it's within the Fluo app. Lets update the BindingSet 
object that is passed into the updaters to require a VisibilityBindingSet.


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237754#comment-15237754
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59431693
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/external/tupleSet/VisibilityBindingSetStringConverter.java
 ---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.indexing.external.tupleSet;
+
+import javax.annotation.ParametersAreNonnullByDefault;
+
+import org.openrdf.query.BindingSet;
+
+import com.google.common.base.Strings;
+
+import mvm.rya.indexing.accumulo.VisibilityBindingSet;
+import mvm.rya.indexing.external.tupleSet.PcjTables.VariableOrder;
+
+/**
+ * Converts {@link BindingSet}s to Strings and back again. The Strings do 
not
+ * include the binding names and are ordered with a {@link VariableOrder}.
+ */
+@ParametersAreNonnullByDefault
+public class VisibilityBindingSetStringConverter extends 
BindingSetStringConverter {
+public static final char VISIBILITY_DELIM = 1;
+
+@Override
+public String convert(final BindingSet bindingSet, final VariableOrder 
varOrder) {
+String visibility = "";
+if(bindingSet instanceof VisibilityBindingSet) {
--- End diff --

Using instanceof here is kind of an abstraction smell, but I'm not sure of 
how to fix this other than maybe having each converter take a generic type that 
extends BindingSet.


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237859#comment-15237859
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59442756
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/external/tupleSet/VisibilityBindingSetStringConverter.java
 ---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.indexing.external.tupleSet;
+
+import javax.annotation.ParametersAreNonnullByDefault;
+
+import org.openrdf.query.BindingSet;
+
+import com.google.common.base.Strings;
+
+import mvm.rya.indexing.accumulo.VisibilityBindingSet;
+import mvm.rya.indexing.external.tupleSet.PcjTables.VariableOrder;
+
+/**
+ * Converts {@link BindingSet}s to Strings and back again. The Strings do 
not
+ * include the binding names and are ordered with a {@link VariableOrder}.
+ */
+@ParametersAreNonnullByDefault
+public class VisibilityBindingSetStringConverter extends 
BindingSetStringConverter {
+public static final char VISIBILITY_DELIM = 1;
+
+@Override
+public String convert(final BindingSet bindingSet, final VariableOrder 
varOrder) {
+String visibility = "";
+if(bindingSet instanceof VisibilityBindingSet) {
--- End diff --

each converter already takes a generic that is what the binding set is 
being converted to/from


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237702#comment-15237702
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59428283
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/external/tupleSet/FullBindingSetStringConverter.java
 ---
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.indexing.external.tupleSet;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.annotation.ParametersAreNonnullByDefault;
+
+import org.openrdf.model.Value;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+
+import com.google.common.base.Joiner;
+
+import mvm.rya.api.domain.RyaType;
+import mvm.rya.api.resolver.RdfToRyaConversions;
+import mvm.rya.indexing.external.tupleSet.PcjTables.VariableOrder;
+
+/**
+ * Converts {@link BindingSet}s to Strings and back again. The Strings do 
not
+ * include the binding names and are ordered with a {@link VariableOrder}.
+ */
+@ParametersAreNonnullByDefault
+public class FullBindingSetStringConverter extends 
BindingSetStringConverter {
--- End diff --

The original idea was to store the binding set + visibility within the 
value section so that you would no longer need to unmarshal the row id using 
the variable order, but I don't think you're using that approach.


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237688#comment-15237688
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59427232
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/external/BindingSetDecorator.java
 ---
@@ -71,7 +78,6 @@ public int size() {
 public int hashCode() {
 int result = hashCode;
--- End diff --

Why not just return the set's hash code?


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235316#comment-15235316
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59226766
  
--- Diff: 
extras/rya.pcj.fluo/pcj.fluo.app/src/test/java/org/apache/rya/indexing/pcj/fluo/app/NaturalJoinTest.java
 ---
@@ -46,120 +48,132 @@
 
 @Test
 public void newLeftResult_noRightMatches() {
-IterativeJoin naturalJoin = new NaturalJoin();
+final IterativeJoin naturalJoin = new NaturalJoin();
 
 // There is a new left result.
-MapBindingSet newLeftResult = new MapBindingSet();
+final MapBindingSet newLeftResult = new MapBindingSet();
 newLeftResult.addBinding("name", vf.createLiteral("Bob"));
 
 // There are no right results that join with the left result.
-Iterator rightResults= new 
ArrayList().iterator();
+final Iterator rightResults= new 
ArrayList().iterator();
 
 // Therefore, the left result is a new join result.
-Iterator newJoinResultsIt = 
naturalJoin.newLeftResult(newLeftResult, rightResults);
+final Iterator newJoinResultsIt = 
naturalJoin.newLeftResult(newLeftResult, rightResults);
 assertFalse( newJoinResultsIt.hasNext() );
 }
 
 @Test
 public void newLeftResult_joinsWithRightResults() {
-IterativeJoin naturalJoin = new NaturalJoin();
+final IterativeJoin naturalJoin = new NaturalJoin();
 
 // There is a new left result.
-MapBindingSet newLeftResult = new MapBindingSet();
+final MapBindingSet newLeftResult = new MapBindingSet();
 newLeftResult.addBinding("name", vf.createLiteral("Bob"));
 newLeftResult.addBinding("height", vf.createLiteral("5'9\""));
 
 // There are a few right results that join with the left result.
-MapBindingSet nameAge = new MapBindingSet();
+final MapBindingSet nameAge = new MapBindingSet();
 nameAge.addBinding("name", vf.createLiteral("Bob"));
 nameAge.addBinding("age", vf.createLiteral(56));
 
-MapBindingSet nameHair = new MapBindingSet();
+final MapBindingSet nameHair = new MapBindingSet();
 nameHair.addBinding("name", vf.createLiteral("Bob"));
 nameHair.addBinding("hairColor", vf.createLiteral("Brown"));
 
-Iterator rightResults = 
Lists.newArrayList(nameAge, nameHair).iterator();
+final Iterator rightResults = 
Lists.newArrayList(nameAge, nameHair).iterator();
 
 // Therefore, there are a few new join results that mix the two 
together.
-Iterator newJoinResultsIt = 
naturalJoin.newLeftResult(newLeftResult, rightResults);
+final Iterator newJoinResultsIt = 
naturalJoin.newLeftResult(newLeftResult, rightResults);
 
-Set newJoinResults = new HashSet<>();
+final Set newJoinResults = new HashSet<>();
 while(newJoinResultsIt.hasNext()) {
 newJoinResults.add( newJoinResultsIt.next() );
 }
 
-Set expected = Sets.newHashSet();
-MapBindingSet nameHeightAge = new MapBindingSet();
+final Set expected = Sets.newHashSet();
+final MapBindingSet nameHeightAge = new MapBindingSet();
 nameHeightAge.addBinding("name", vf.createLiteral("Bob"));
 nameHeightAge.addBinding("height", vf.createLiteral("5'9\""));
 nameHeightAge.addBinding("age", vf.createLiteral(56));
-expected.add(nameHeightAge);
+expected.add(new VisibilityBindingSet(nameHeightAge));
 
-MapBindingSet nameHeightHair = new MapBindingSet();
+final MapBindingSet nameHeightHair = new MapBindingSet();
 nameHeightHair.addBinding("name", vf.createLiteral("Bob"));
 nameHeightHair.addBinding("height", vf.createLiteral("5'9\""));
 nameHeightHair.addBinding("hairColor", vf.createLiteral("Brown"));
-expected.add(nameHeightHair);
+expected.add(new VisibilityBindingSet(nameHeightHair));
 
 assertEquals(expected, newJoinResults);
 }
 
 @Test
 public void newRightResult_noLeftMatches() {
-IterativeJoin naturalJoin = new NaturalJoin();
+final IterativeJoin naturalJoin = new NaturalJoin();
 
 // There are no left results.
-Iterator leftResults= new 
ArrayList().iterator();
+final Iterator leftResults= new 
ArrayList().iterator();
 
 // There is a new right result.
-MapBindingSet newRightResult 

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235333#comment-15235333
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59228867
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/accumulo/VisibilityBindingSet.java
 ---
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.indexing.accumulo;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import javax.annotation.ParametersAreNonnullByDefault;
+
+import org.openrdf.query.BindingSet;
+
+import mvm.rya.indexing.external.BindingSetDecorator;
+
+/**
+ * Decorates a {@link BindingSet} with a collection of visibilities.
+ */
+@ParametersAreNonnullByDefault
+public class VisibilityBindingSet extends BindingSetDecorator {
+private static final long serialVersionUID = 1L;
+private final String visibility;
+private volatile int hashCode;
+
+/**
+ * @param set - Decorates the {@link BindingSet} with no visibilities.
+ */
+public VisibilityBindingSet(final BindingSet set) {
+this(set, "");
+}
+
+/**
+ * Creates a new {@link VisibilityBindingSet}
+ * @param set - The {@link BindingSet} to decorate
+ * @param visibility - The visibilities on the {@link BindingSet} (not 
null)
+ */
+public VisibilityBindingSet(final BindingSet set, final String 
visibility) {
+super(set);
+this.visibility = checkNotNull(visibility);
+}
+
+/**
+ * @return - The Visibilities on the {@link BindingSet}
+ */
+public String getVisibility() {
+return visibility;
+}
+
+@Override
+public boolean equals(final Object o) {
+if (this == o) {
+return true;
+} else if(o instanceof VisibilityBindingSet) {
+final VisibilityBindingSet other = (VisibilityBindingSet) o;
+return set.equals(other) && 
visibility.equals(other.getVisibility());
+}
+return false;
+}
+
+@Override
+public int hashCode() {
--- End diff --

probably not a bad call to cache it anyway


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235431#comment-15235431
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59236564
  
--- Diff: 
extras/rya.pcj.fluo/pcj.fluo.api/src/main/java/org/apache/rya/indexing/pcj/fluo/api/CreatePcj.java
 ---
@@ -139,6 +141,7 @@ public CreatePcj(final int spInsertBatchSize) {
  * @throws QueryEvaluationException Historic results could not be 
added to the initialized PCJ because of
  *   a problem with the Rya connection.
  */
+@VisibleForTesting
--- End diff --

artifact left over from when I added the withRyaIntegration() that returned 
the generated table name


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-53) Add Visibility to Rya PCJs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235422#comment-15235422
 ] 

ASF GitHub Bot commented on RYA-53:
---

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/36#discussion_r59236187
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/external/tupleSet/FullBindingSetStringConverter.java
 ---
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.indexing.external.tupleSet;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.annotation.ParametersAreNonnullByDefault;
+
+import org.openrdf.model.Value;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+
+import com.google.common.base.Joiner;
+
+import mvm.rya.api.domain.RyaType;
+import mvm.rya.api.resolver.RdfToRyaConversions;
+import mvm.rya.indexing.external.tupleSet.PcjTables.VariableOrder;
+
+/**
+ * Converts {@link BindingSet}s to Strings and back again. The Strings do 
not
+ * include the binding names and are ordered with a {@link VariableOrder}.
+ */
+@ParametersAreNonnullByDefault
+public class FullBindingSetStringConverter extends 
BindingSetStringConverter {
--- End diff --

huhI was using it...I deleted it and everything still works 
sohuh...


> Add Visibility to Rya PCJs
> --
>
> Key: RYA-53
> URL: https://issues.apache.org/jira/browse/RYA-53
> Project: Rya
>  Issue Type: New Feature
>Reporter: Andrew Smith
>
> Update the PCJ api to support visibility attached to binding sets



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-11) Free Text search support for Mongo DB backed Rya

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179704#comment-15179704
 ] 

ASF GitHub Bot commented on RYA-11:
---

Github user amihalik commented on the pull request:

https://github.com/apache/incubator-rya/pull/21#issuecomment-192227049
  
If you fix the conflicts below, I'll merge your PR:

```
CONFLICT (content): Merge conflict in 
extras/indexingExample/src/main/java/MongoRyaDirectExample.java
CONFLICT (content): Merge conflict in 
extras/indexing/src/test/java/mvm/rya/indexing/external/PrecompJoinOptimizerIntegrationTest.java
CONFLICT (modify/delete): 
extras/indexing/src/test/java/mvm/rya/indexing/external/AccumuloIndexSetTest2.java
 deleted in 13de8a5645391b65ba0a6753eedbb51bff4d7bee and modified 
```


> Free Text search support for Mongo DB backed Rya
> 
>
> Key: RYA-11
> URL: https://issues.apache.org/jira/browse/RYA-11
> Project: Rya
>  Issue Type: New Feature
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Puja Valiyil 
>
> The secondary indexing support for Mongo DB should be extended to also 
> include support for free text search (which the Accumulo back end already 
> provides).  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-7) Various pom cleanups for the Apache move

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177611#comment-15177611
 ] 

ASF GitHub Bot commented on RYA-7:
--

Github user amihalik commented on the pull request:

https://github.com/apache/incubator-rya/pull/25#issuecomment-191694535
  
Thanks @joshelser, but I need a bit more help.

I don't quite understand the apache-rat behavior.  We have those exclusions 
in the parent pom, we have apache-rat check enabled by default, and everything 
works as expected if we do a `mvn clean install` (i.e. the exclusions are 
propagagted to the submodules).  However those exclusions aren't propagated if 
we do a `mvn apache-rat:check`.  Should I manually cut-and-paste those 
exclusions to the submodules?

Secondly, I don't understand why apache-rat didn't catch the  "Copyright 
Rya" in these two files.  (And I really don't understand how you caught that 
either :) ).  Is there a more stringent setting for apache-rat?

Thanks for your help!


> Various pom cleanups for the Apache move
> 
>
> Key: RYA-7
> URL: https://issues.apache.org/jira/browse/RYA-7
> Project: Rya
>  Issue Type: Task
>Reporter: Josh Elser
>Assignee: Aaron Mihalik
> Fix For: 3.2.10
>
>
> In working on RYA-5, I noticed a few things in the pom that I believe should 
> be taken care of before a release.
> * Make the apache pom the parent (see https://maven.apache.org/pom/asf/)
> * Depend on Apache releases for Hadoop and ZooKeeper (and groovy?)
> * Make sure we're depending on the most recent bug-fix release for 
> dependencies (e.g. upgrade from 1.6.2 to 1.6.4 for Accumulo)
> * Change groupId from {{mvn.rya}} to {{org.apache.rya}} for consistency with 
> other Maven-based ASF projects
> * Set proper names for each maven module (instead of groupid.artifact). This 
> isn't as critical to change for a release, but just is an easy thing to 
> improve.
> * Fix SCM block in parent pom (points to incorrect repository).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-75) Add Merge Tool to Rya

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-75?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278734#comment-15278734
 ] 

ASF GitHub Bot commented on RYA-75:
---

GitHub user ejwhite922 opened a pull request:

https://github.com/apache/incubator-rya/pull/40

Develop

RYA-75
Added Merge Tool which merges a child's Accumulo instance to its parent 
instance after the child has collected/modified its data set. The Copy Tool 
allows the creation of a child instance from a parent Accumulo instance and the 
data that is copied can be filtered by time.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ejwhite922/incubator-rya develop

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/40.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #40


commit 6f7b53404ee8c74316aa53ecc3c32edd777b5f33
Author: eric.white 
Date:   2016-03-10T21:20:02Z

RYA-49 Updated Accumulo version from 1.6.4 to 1.6.5 in the Vagrant file.  
1.6.4 is no longer available on any mirror sites.  The script now stores the 
version in a variable so it only needs to be updated in one spot if the version 
changes again.

commit 3bd2d222481b22315fc521b7da20399485f7e5a6
Author: eric.white 
Date:   2016-05-10T19:16:43Z

Revert "RYA-49 Updated Accumulo version from 1.6.4 to 1.6.5 in the Vagrant 
file.  1.6.4 is no longer available on any mirror sites.  The script now stores 
the version in a variable so it only needs to be updated in one spot if the 
version changes again."

This reverts commit 6f7b53404ee8c74316aa53ecc3c32edd777b5f33.

commit 159b90d7549ed66f7ce46484bb1243e0a750ae64
Author: eric.white 
Date:   2016-05-10T19:22:36Z

Merge branch 'develop' of https://github.com/apache/incubator-rya into 
develop

commit 216b73dbb0d6950dc33baba9024d126d6d02296e
Author: eric.white 
Date:   2016-05-10T19:31:57Z

RYA-75
Added Merge Tool which merges a child's Accumulo instance to its parent 
instance after the child has collected/modified its data set. The Copy Tool 
allows the creation of a child instance from a parent Accumulo instance and the 
data that is copied can be filtered by time.




> Add Merge Tool to Rya
> -
>
> Key: RYA-75
> URL: https://issues.apache.org/jira/browse/RYA-75
> Project: Rya
>  Issue Type: New Feature
>Affects Versions: 3.2.10
>Reporter: Eric White
>
> The Merge Tool feature merges a child's Accumulo instance to its parent 
> instance after the child has collected/modified its data set.  The Copy Tool 
> allows the creation of a child instance from a parent Accumulo instance and 
> the data that is copied can be filtered by time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-73) Add insert time to mongo backed rya

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287064#comment-15287064
 ] 

ASF GitHub Bot commented on RYA-73:
---

GitHub user isper3at opened a pull request:

https://github.com/apache/incubator-rya/pull/43

RYA-73 added timestamp to mongoDB DAO

Changed the mongo storage strategy to now
include timestamp in the [de]serialization
of RyaStatements.

The timestamp is added at insertion timein the DAO

Added test for [de]serialize
Added test for timestamp in the DAO
Added failsafe to mongo rya for integration test framework

[Jira](https://issues.apache.org/jira/browse/RYA-73)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/isper3at/incubator-rya mongoTimestamp

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/43.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #43


commit f6d2db3557c093dde24f445711bec4211824233b
Author: isper3at 
Date:   2016-05-09T18:29:18Z

RYA-73 added timestamp to mongoDB DAO

Changed the mongo storage strategy to now
include timestamp in the [de]serialization
of RyaStatements.

The timestamp is added at insertion timein the DAO

Added test for [de]serialize
Added test for timestamp in the DAO




> Add insert time to mongo backed rya
> ---
>
> Key: RYA-73
> URL: https://issues.apache.org/jira/browse/RYA-73
> Project: Rya
>  Issue Type: Improvement
>Reporter: Andrew Smith
>
> Some features of accumulo backed Rya require db insert time.  This is not 
> given for free in mongo.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-100) Update the build to us Maven animal-sniffer plugin.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388253#comment-15388253
 ] 

ASF GitHub Bot commented on RYA-100:


Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/54#discussion_r71769754
  
--- Diff: pom.xml ---
@@ -562,6 +562,29 @@ under the License.
 1.7
 
 
+
+
+
+org.codehaus.mojo
+animal-sniffer-maven-plugin
+1.15
+
+
+org.codehaus.mojo.signature
+java17
--- End diff --

I'll update the signature to be for Java 1.8.


> Update the build to us Maven animal-sniffer plugin.
> ---
>
> Key: RYA-100
> URL: https://issues.apache.org/jira/browse/RYA-100
> Project: Rya
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.2.10
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
> Fix For: 3.2.10
>
>
> Rya uses Java 1.7. If you are compiling on a machine using a 1.8 JDK, though, 
> the build succeeds when Java 1.8 features are used within the source. This is 
> a problem the animal-sniffer plugin fixes. Hook animal-sniffer:check into the 
> test phase of the build so that it will list where Java 1.8+ features are 
> being used and fail the build.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-148) Normalize Column Visibilities in Fluo Table

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417556#comment-15417556
 ] 

ASF GitHub Bot commented on RYA-148:


Github user isper3at commented on the issue:

https://github.com/apache/incubator-rya/pull/65
  
:+1: 


> Normalize Column Visibilities in Fluo Table
> ---
>
> Key: RYA-148
> URL: https://issues.apache.org/jira/browse/RYA-148
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Kevin Chilton
>Priority: Critical
> Fix For: 3.2.10
>
>
> There is currently a bug in how the Rya Fluo Application handles column 
> visibilities which leads to duplicate results in the Precomputed Join Tables. 
>  For example, consider a query that consists of a join which has 
> StatementsPattern SP1 as a left arg and StatementPattern SP2 as a right arg.  
> Suppose that a triple T1 comes in that matches SP1 and a triple T2 comes in 
> that matches SP2.  Also assume that T1 has visibility v1 and T2 has 
> visibility v2.  When the StatementPatternObserver processes T1, it joins the 
> binding set associated with T1 (with visibility v1) with the binding set 
> associated with T2 (with visibility v2).  The result of the join is a binding 
> set with visibility v1  On the other hand, when the 
> StatementPatternObserver processes T2, it produces the same binding set BS, 
> but with visibility v2  That is, the visibility of the StatementPattern 
> bindingset being processed always comes first in the composite visibility 
> formed from the join.  This results in two copies of BS with the same 
> visibility (just ordered differently).  As a proposed fix, the visibilities 
> should be combined in a manner consistent with how the StatementPatterns are 
> ordered in the query.  For example, always put the visibility of the left 
> argument of the join first when forming the combined visibility.  This will 
> generate a default ordering of the composite visibility that won't lead to 
> duplicates.  In the above example, BS will have visibility v1 regardless 
> of whether T1 or T2 is processed first and the second copy that gets 
> generated will overwrite the first, leading to no duplicate values. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-148) Normalize Column Visibilities in Fluo Table

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417554#comment-15417554
 ] 

ASF GitHub Bot commented on RYA-148:


Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/65#discussion_r74458400
  
--- Diff: 
extras/rya.pcj.fluo/pcj.fluo.app/src/test/java/org/apache/rya/indexing/pcj/fluo/app/IterativeJoinTest.java
 ---
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app;
+
+import static org.junit.Assert.assertEquals;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Iterator;
+
+import 
org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.IterativeJoin;
+import 
org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.LeftOuterJoin;
+import org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.NaturalJoin;
+import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+import org.openrdf.model.ValueFactory;
+import org.openrdf.model.impl.ValueFactoryImpl;
+import org.openrdf.query.impl.MapBindingSet;
+
+/**
+ * Tests the methods of {@link IterativeJoin}.
+ */
+@RunWith(Parameterized.class)
+public class IterativeJoinTest {
+
+@Parameters
+public static Collection data() {
+return Arrays.asList(new Object[][] {
--- End diff --

this just feels.odd


> Normalize Column Visibilities in Fluo Table
> ---
>
> Key: RYA-148
> URL: https://issues.apache.org/jira/browse/RYA-148
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Kevin Chilton
>Priority: Critical
> Fix For: 3.2.10
>
>
> There is currently a bug in how the Rya Fluo Application handles column 
> visibilities which leads to duplicate results in the Precomputed Join Tables. 
>  For example, consider a query that consists of a join which has 
> StatementsPattern SP1 as a left arg and StatementPattern SP2 as a right arg.  
> Suppose that a triple T1 comes in that matches SP1 and a triple T2 comes in 
> that matches SP2.  Also assume that T1 has visibility v1 and T2 has 
> visibility v2.  When the StatementPatternObserver processes T1, it joins the 
> binding set associated with T1 (with visibility v1) with the binding set 
> associated with T2 (with visibility v2).  The result of the join is a binding 
> set with visibility v1  On the other hand, when the 
> StatementPatternObserver processes T2, it produces the same binding set BS, 
> but with visibility v2  That is, the visibility of the StatementPattern 
> bindingset being processed always comes first in the composite visibility 
> formed from the join.  This results in two copies of BS with the same 
> visibility (just ordered differently).  As a proposed fix, the visibilities 
> should be combined in a manner consistent with how the StatementPatterns are 
> ordered in the query.  For example, always put the visibility of the left 
> argument of the join first when forming the combined visibility.  This will 
> generate a default ordering of the composite visibility that won't lead to 
> duplicates.  In the above example, BS will have visibility v1 regardless 
> of whether T1 or T2 is processed first and the second copy that gets 
> generated will overwrite the first, leading to no duplicate values. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-103) Upgrade Rya back to java 8

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417551#comment-15417551
 ] 

ASF GitHub Bot commented on RYA-103:


Github user isper3at commented on the issue:

https://github.com/apache/incubator-rya/pull/63
  
:shipit: 


> Upgrade Rya back to java 8
> --
>
> Key: RYA-103
> URL: https://issues.apache.org/jira/browse/RYA-103
> Project: Rya
>  Issue Type: Bug
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> At some point during the port to apache, we lost the upgrade of Rya to Java 
> 8.  This should involve just updating the pom to reference 1.8 for the 
> compiler level instead of 1.7.  When we did this before, no code changes had 
> to take place.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-130) Rya Mongo Fail on Multi Bindingset Joins

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387657#comment-15387657
 ] 

ASF GitHub Bot commented on RYA-130:


Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/57
  
merged


> Rya Mongo Fail on Multi Bindingset Joins
> 
>
> Key: RYA-130
> URL: https://issues.apache.org/jira/browse/RYA-130
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Affects Versions: 3.2.9
>Reporter: Aaron Mihalik
>Assignee: Aaron Mihalik
>
> Rya Mongo fails on a fairly straightforward query:
> {code}
> DATA = 
>  .
>  .
>  .
> QUERY = 
>   select ?s ?foo where { 
>   ?s  ?foo . 
>   ?s   }
> {code}
> This is a bug in how Mongo Rya solves joins.  The first Statement Pattern 
> will return multiple binding sets, and try to join it with a single 
> statement.  The Mongo Rya Query Engine can't handle this case.
> Full Code at: 
> https://gist.github.com/amihalik/61cf870650b7443389e7be7600f8502c



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-105) Fluo Integration with Rya

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387655#comment-15387655
 ] 

ASF GitHub Bot commented on RYA-105:


Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/58
  
merged


> Fluo Integration with Rya
> -
>
> Key: RYA-105
> URL: https://issues.apache.org/jira/browse/RYA-105
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
> Fix For: 3.2.10
>
>
> The Fluo table for incrementally updating precomputed joins is not fully 
> integrated with Rya.  The PrecomputedJoinIndexer (which is an implementation 
> of the AccumuloIndexer interface) is not being added to the list of indexers 
> when the PCJ flag is set.  So the DAO is never adding to Fluo.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-105) Fluo Integration with Rya

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382501#comment-15382501
 ] 

ASF GitHub Bot commented on RYA-105:


GitHub user meiercaleb opened a pull request:

https://github.com/apache/incubator-rya/pull/58

RYA-105: Fluo Integration

Integrated Fluo with Rya so that when triples are inserted into Rya via the 
AccumuloRyaDAO, they are also inserted into Fluo so that any registered 
Precomputed Joins are incrementally updated.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/meiercaleb/incubator-rya RYA-105

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/58.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #58


commit 516ba3e328631f01bbc376667a462ed8bf0586eb
Author: Caleb Meier 
Date:   2016-07-06T16:26:38Z

RYA-105: Fluo Integration




> Fluo Integration with Rya
> -
>
> Key: RYA-105
> URL: https://issues.apache.org/jira/browse/RYA-105
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
> Fix For: 3.2.10
>
>
> The Fluo table for incrementally updating precomputed joins is not fully 
> integrated with Rya.  The PrecomputedJoinIndexer (which is an implementation 
> of the AccumuloIndexer interface) is not being added to the list of indexers 
> when the PCJ flag is set.  So the DAO is never adding to Fluo.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-130) Rya Mongo Fail on Multi Bindingset Joins

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368648#comment-15368648
 ] 

ASF GitHub Bot commented on RYA-130:


GitHub user amihalik opened a pull request:

https://github.com/apache/incubator-rya/pull/57

RYA-130 Fixed Rya Mongo Multi Binding Set Join

Fixed bug in Mongo Query Engine; added test

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/amihalik/incubator-rya mongoMultiBindingSet

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/57.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #57


commit f9d215e381cec39cb2daca7a06a40ab047580d8c
Author: Aaron Mihalik 
Date:   2016-07-08T23:01:19Z

RYA-130 Fixed Rya Mongo Multi Binding Set Join

Fixed bud in Mongo Query Engine; added test




> Rya Mongo Fail on Multi Bindingset Joins
> 
>
> Key: RYA-130
> URL: https://issues.apache.org/jira/browse/RYA-130
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Affects Versions: 3.2.9
>Reporter: Aaron Mihalik
>Assignee: Aaron Mihalik
>
> Rya Mongo fails on a fairly straightforward query:
> {code}
> DATA = 
>  .
>  .
>  .
> QUERY = 
>   select ?s ?foo where { 
>   ?s  ?foo . 
>   ?s   }
> {code}
> This is a bug in how Mongo Rya solves joins.  The first Statement Pattern 
> will return multiple binding sets, and try to join it with a single 
> statement.  The Mongo Rya Query Engine can't handle this case.
> Full Code at: 
> https://gist.github.com/amihalik/61cf870650b7443389e7be7600f8502c



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-103) Upgrade Rya back to java 8

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408710#comment-15408710
 ] 

ASF GitHub Bot commented on RYA-103:


GitHub user kchilton2 opened a pull request:

https://github.com/apache/incubator-rya/pull/63

RYA-103 Updates the Rya build to use Java 1.8



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kchilton2/incubator-rya RYA-103

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/63.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #63


commit 1caa6eafdd082a029c9a413494fc8f6bb3f7bd52
Author: Kevin Chilton 
Date:   2016-08-05T00:48:30Z

RYA-103 Updates the Rya build to use Java 1.8




> Upgrade Rya back to java 8
> --
>
> Key: RYA-103
> URL: https://issues.apache.org/jira/browse/RYA-103
> Project: Rya
>  Issue Type: Bug
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> At some point during the port to apache, we lost the upgrade of Rya to Java 
> 8.  This should involve just updating the pom to reference 1.8 for the 
> compiler level instead of 1.7.  When we did this before, no code changes had 
> to take place.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-143) Upgrade Mongo to 3.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408762#comment-15408762
 ] 

ASF GitHub Bot commented on RYA-143:


GitHub user kchilton2 opened a pull request:

https://github.com/apache/incubator-rya/pull/64

RYA-143 Updated the Mongo DB Java Driver to 3.3.0.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kchilton2/incubator-rya RYA-143

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/64.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #64


commit 8b424a0b90c14e310784eded023f69f6deee6995
Author: Kevin Chilton 
Date:   2016-08-05T01:47:49Z

RYA-143 Updated the Mongo DB Java Driver to 3.3.0.




> Upgrade Mongo to 3.0
> 
>
> Key: RYA-143
> URL: https://issues.apache.org/jira/browse/RYA-143
> Project: Rya
>  Issue Type: New Feature
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
> Fix For: 3.2.10
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-100) Update the build to us Maven animal-sniffer plugin.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15355852#comment-15355852
 ] 

ASF GitHub Bot commented on RYA-100:


GitHub user kchilton2 opened a pull request:

https://github.com/apache/incubator-rya/pull/54

RYA-100 Added Maven animal-sniffer plugin to build.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kchilton2/incubator-rya animal-sniffer

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/54.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #54


commit c92e04daf98c340f16a8939a9fe141388b3bd6d5
Author: Kevin Chilton 
Date:   2016-06-29T21:44:19Z

RYA-100 Added Maven animal-sniffer plugin to build.




> Update the build to us Maven animal-sniffer plugin.
> ---
>
> Key: RYA-100
> URL: https://issues.apache.org/jira/browse/RYA-100
> Project: Rya
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.2.10
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
> Fix For: 3.2.10
>
>
> Rya uses Java 1.7. If you are compiling on a machine using a 1.8 JDK, though, 
> the build succeeds when Java 1.8 features are used within the source. This is 
> a problem the animal-sniffer plugin fixes. Hook animal-sniffer:check into the 
> test phase of the build so that it will list where Java 1.8+ features are 
> being used and fail the build.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-241) Add Geowave support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852139#comment-15852139
 ] 

ASF GitHub Bot commented on RYA-241:


Github user amihalik commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/139#discussion_r99424446
  
--- Diff: extras/indexingExample/src/main/assembly/assembly.xml ---
@@ -33,6 +33,7 @@ under the License.
 
 
org.apache.rya:rya.indexing:*:accumulo-server
 
org.locationtech.geomesa:geomesa-accumulo-distributed-runtime:*
+
mil.nga.giat:geowave-datastore-accumulo:*
--- End diff --

I think the dependency is something like `geowave-deploy-0.9.3-accumulo`?  
I can't find this dependency on their maven repository, though


> Add Geowave support
> ---
>
> Key: RYA-241
> URL: https://issues.apache.org/jira/browse/RYA-241
> Project: Rya
>  Issue Type: New Feature
>Reporter: Eric White
>Assignee: Eric White
>
> Add in Geowave as an alternative to Geomesa



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (RYA-241) Add Geowave support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852054#comment-15852054
 ] 

ASF GitHub Bot commented on RYA-241:


GitHub user ejwhite922 opened a pull request:

https://github.com/apache/incubator-rya/pull/139

RYA-241 Adding GeoWave indexer to rya.geoindexing.

## Description
Added GeoWave indexer to rya.geoindexing.  This is a configurable option 
that can be used in place of GeoMesa.

### Tests
Unit Tests

### Links
[Jira](https://issues.apache.org/jira/browse/RYA-241)

### Checklist
- [ ] Code Review
- [ ] Squash Commits

 People To Review
@pujav65 
@amihalik 
@isper3at 
@DLotts

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ejwhite922/incubator-rya RYA-241_Geowave

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/139.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #139


commit 208b1efb1b62d1877a439cf3f2379313085b13a9
Author: eric.white 
Date:   2017-01-27T15:46:08Z

RYA-241 Adding GeoWave indexer to rya.geoindexing.  This is a configurable 
option that can be used in place of GeoMesa.




> Add Geowave support
> ---
>
> Key: RYA-241
> URL: https://issues.apache.org/jira/browse/RYA-241
> Project: Rya
>  Issue Type: New Feature
>Reporter: Eric White
>Assignee: Eric White
>
> Add in Geowave as an alternative to Geomesa



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (RYA-241) Add Geowave support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852141#comment-15852141
 ] 

ASF GitHub Bot commented on RYA-241:


Github user amihalik commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/139#discussion_r99417564
  
--- Diff: 
extras/rya.geoindexing/src/main/java/org/apache/rya/indexing/accumulo/geo/GeoWaveGeoIndexer.java
 ---
@@ -0,0 +1,661 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo.geo;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+import static java.util.Objects.requireNonNull;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.mock.MockInstance;
+import org.apache.commons.lang.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.log4j.Logger;
+import org.apache.rya.accumulo.experimental.AbstractAccumuloIndexer;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.domain.RyaStatement;
+import org.apache.rya.api.resolver.RyaToRdfConversions;
+import org.apache.rya.indexing.GeoIndexer;
+import org.apache.rya.indexing.Md5Hash;
+import org.apache.rya.indexing.StatementConstraints;
+import org.apache.rya.indexing.StatementSerializer;
+import org.apache.rya.indexing.accumulo.ConfigUtils;
+import org.geotools.data.DataStore;
+import org.geotools.data.DataUtilities;
+import org.geotools.data.FeatureSource;
+import org.geotools.data.FeatureStore;
+import org.geotools.factory.CommonFactoryFinder;
+import org.geotools.factory.Hints;
+import org.geotools.feature.DefaultFeatureCollection;
+import org.geotools.feature.SchemaException;
+import org.geotools.feature.simple.SimpleFeatureBuilder;
+import org.geotools.filter.text.cql2.CQLException;
+import org.geotools.filter.text.ecql.ECQL;
+import org.opengis.feature.simple.SimpleFeature;
+import org.opengis.feature.simple.SimpleFeatureType;
+import org.opengis.filter.Filter;
+import org.opengis.filter.FilterFactory;
+import org.opengis.filter.identity.Identifier;
+import org.openrdf.model.Literal;
+import org.openrdf.model.Statement;
+import org.openrdf.model.URI;
+import org.openrdf.query.QueryEvaluationException;
+
+import com.vividsolutions.jts.geom.Geometry;
+import com.vividsolutions.jts.io.ParseException;
+
+import info.aduna.iteration.CloseableIteration;
+import mil.nga.giat.geowave.adapter.vector.FeatureDataAdapter;
+import mil.nga.giat.geowave.adapter.vector.plugin.GeoWaveGTDataStore;
+import 
mil.nga.giat.geowave.adapter.vector.plugin.GeoWaveGTDataStoreFactory;
+import mil.nga.giat.geowave.adapter.vector.plugin.GeoWavePluginException;
+import mil.nga.giat.geowave.adapter.vector.query.cql.CQLQuery;
+import 
mil.nga.giat.geowave.core.geotime.ingest.SpatialDimensionalityTypeProvider;
+import mil.nga.giat.geowave.core.store.CloseableIterator;
+import mil.nga.giat.geowave.core.store.StoreFactoryFamilySpi;
+import mil.nga.giat.geowave.core.store.index.PrimaryIndex;
+import mil.nga.giat.geowave.core.store.memory.MemoryStoreFactoryFamily;
+import mil.nga.giat.geowave.core.store.query.EverythingQuery;
+import mil.nga.giat.geowave.core.store.query.QueryOptions;
+import mil.nga.giat.geowave.datastore.accumulo.AccumuloDataStore;
+import mil.nga.giat.geowave.datastore.accumulo.AccumuloStoreFactoryFamily;
+
+/**
+ * A {@link 

[jira] [Commented] (RYA-241) Add Geowave support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852140#comment-15852140
 ] 

ASF GitHub Bot commented on RYA-241:


Github user amihalik commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/139#discussion_r99424572
  
--- Diff: pom.xml ---
@@ -80,6 +80,7 @@ under the License.
 0.9.2 
 
 1.3.0-m1 
+0.9.4-SNAPSHOT 
--- End diff --

Can you back this down to a released version (e.g. 0.9.3)?


> Add Geowave support
> ---
>
> Key: RYA-241
> URL: https://issues.apache.org/jira/browse/RYA-241
> Project: Rya
>  Issue Type: New Feature
>Reporter: Eric White
>Assignee: Eric White
>
> Add in Geowave as an alternative to Geomesa



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (RYA-241) Add Geowave support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856699#comment-15856699
 ] 

ASF GitHub Bot commented on RYA-241:


Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/139#discussion_r99916370
  
--- Diff: 
extras/rya.geoindexing/src/main/java/org/apache/rya/indexing/accumulo/geo/GeoWaveGeoIndexer.java
 ---
@@ -0,0 +1,661 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo.geo;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+import static java.util.Objects.requireNonNull;
+
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.mock.MockInstance;
+import org.apache.commons.lang.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.log4j.Logger;
+import org.apache.rya.accumulo.experimental.AbstractAccumuloIndexer;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.domain.RyaStatement;
+import org.apache.rya.api.resolver.RyaToRdfConversions;
+import org.apache.rya.indexing.GeoIndexer;
+import org.apache.rya.indexing.Md5Hash;
+import org.apache.rya.indexing.StatementConstraints;
+import org.apache.rya.indexing.StatementSerializer;
+import org.apache.rya.indexing.accumulo.ConfigUtils;
+import org.geotools.data.DataStore;
+import org.geotools.data.DataUtilities;
+import org.geotools.data.FeatureSource;
+import org.geotools.data.FeatureStore;
+import org.geotools.factory.CommonFactoryFinder;
+import org.geotools.factory.Hints;
+import org.geotools.feature.DefaultFeatureCollection;
+import org.geotools.feature.SchemaException;
+import org.geotools.feature.simple.SimpleFeatureBuilder;
+import org.geotools.filter.text.cql2.CQLException;
+import org.geotools.filter.text.ecql.ECQL;
+import org.opengis.feature.simple.SimpleFeature;
+import org.opengis.feature.simple.SimpleFeatureType;
+import org.opengis.filter.Filter;
+import org.opengis.filter.FilterFactory;
+import org.opengis.filter.identity.Identifier;
+import org.openrdf.model.Literal;
+import org.openrdf.model.Statement;
+import org.openrdf.model.URI;
+import org.openrdf.query.QueryEvaluationException;
+
+import com.vividsolutions.jts.geom.Geometry;
+import com.vividsolutions.jts.io.ParseException;
+
+import info.aduna.iteration.CloseableIteration;
+import mil.nga.giat.geowave.adapter.vector.FeatureDataAdapter;
+import mil.nga.giat.geowave.adapter.vector.plugin.GeoWaveGTDataStore;
+import 
mil.nga.giat.geowave.adapter.vector.plugin.GeoWaveGTDataStoreFactory;
+import mil.nga.giat.geowave.adapter.vector.plugin.GeoWavePluginException;
+import mil.nga.giat.geowave.adapter.vector.query.cql.CQLQuery;
+import 
mil.nga.giat.geowave.core.geotime.ingest.SpatialDimensionalityTypeProvider;
+import mil.nga.giat.geowave.core.store.CloseableIterator;
+import mil.nga.giat.geowave.core.store.StoreFactoryFamilySpi;
+import mil.nga.giat.geowave.core.store.index.PrimaryIndex;
+import mil.nga.giat.geowave.core.store.memory.MemoryStoreFactoryFamily;
+import mil.nga.giat.geowave.core.store.query.EverythingQuery;
+import mil.nga.giat.geowave.core.store.query.QueryOptions;
+import mil.nga.giat.geowave.datastore.accumulo.AccumuloDataStore;
+import mil.nga.giat.geowave.datastore.accumulo.AccumuloStoreFactoryFamily;
+
+/**
+ * A {@link 

[jira] [Commented] (RYA-241) Add Geowave support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856703#comment-15856703
 ] 

ASF GitHub Bot commented on RYA-241:


Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/139#discussion_r99916874
  
--- Diff: extras/indexingExample/src/main/assembly/assembly.xml ---
@@ -33,6 +33,7 @@ under the License.
 
 
org.apache.rya:rya.indexing:*:accumulo-server
 
org.locationtech.geomesa:geomesa-accumulo-distributed-runtime:*
+
mil.nga.giat:geowave-datastore-accumulo:*
--- End diff --

I added each geowave dependency individually.  The accumulo deploy 
dependency isn't available on the maven repo.  We'd have to build it if we 
wanted to use it.  The deploy pom is here: 
https://github.com/ngageoint/geowave/blob/9d6cfa9fb40e6e17650df3b41854d91402a91648/deploy/pom.xml


> Add Geowave support
> ---
>
> Key: RYA-241
> URL: https://issues.apache.org/jira/browse/RYA-241
> Project: Rya
>  Issue Type: New Feature
>Reporter: Eric White
>Assignee: Eric White
>
> Add in Geowave as an alternative to Geomesa



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (RYA-241) Add Geowave support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856704#comment-15856704
 ] 

ASF GitHub Bot commented on RYA-241:


Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/139#discussion_r99916906
  
--- Diff: pom.xml ---
@@ -80,6 +80,7 @@ under the License.
 0.9.2 
 
 1.3.0-m1 
+0.9.4-SNAPSHOT 
--- End diff --

Changed to 0.9.3


> Add Geowave support
> ---
>
> Key: RYA-241
> URL: https://issues.apache.org/jira/browse/RYA-241
> Project: Rya
>  Issue Type: New Feature
>Reporter: Eric White
>Assignee: Eric White
>
> Add in Geowave as an alternative to Geomesa



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (RYA-249) Mongo GeoIndexer assumes polygon when performing within

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881059#comment-15881059
 ] 

ASF GitHub Bot commented on RYA-249:


GitHub user isper3at opened a pull request:

https://github.com/apache/incubator-rya/pull/140

RYA-249 Geo Indexer precondition check

## Description
>What Changed?
Mongo precondition checks to make sure the geometry type
for within and intersects is a Polygon.

### Tests
>Coverage?
There are no existing tests for GeoMongoDBStorageStrategy.

### Links
[Jira](https://issues.apache.org/jira/browse/RYA-249)

### Checklist
- [ ] Code Review
- [x] Squash Commits

 People To Reivew
@pujav65 
@dlotts


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/isper3at/incubator-rya RYA-249

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/140.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #140


commit 9821f6ebf75642513e805779bff5093ce1b3f710
Author: isper3at 
Date:   2017-02-10T19:33:41Z

RYA-249 Geo Indexer precondition check

Mongo precondition checks to make sure the geometry type
for within and intersects is a Polygon.




> Mongo GeoIndexer assumes polygon when performing within
> ---
>
> Key: RYA-249
> URL: https://issues.apache.org/jira/browse/RYA-249
> Project: Rya
>  Issue Type: Bug
>Reporter: Andrew Smith
>Assignee: Andrew Smith
>
> Mongo Storage strategy assumes a polygon object is being used when performing 
> a WITHIN operation.  If a polygon is not provided the operation breaks on 
> mongodb instead of in RYA.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (RYA-236) Geotemporal Indexing Support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828466#comment-15828466
 ] 

ASF GitHub Bot commented on RYA-236:


GitHub user isper3at opened a pull request:

https://github.com/apache/incubator-rya/pull/138

[WIP] Rya 236 - GeoTemporal Indexing

## Description
>What Changed?

Added interface and model framework for geo temporal indexing.
This framework is setup to take advantage of existing geo indexing and 
temporal indexing.

### Tests
>Coverage?

Incomplete integration test, to be filled in as the interface is 
implemented.

### Links
[Jira core](https://issues.apache.org/jira/browse/RYA-236)
[Jira framework](https://issues.apache.org/jira/browse/RYA-240)

### Checklist
- [ ] Code Review
- [ ] Squash Commits

 People To Reivew
currently a WIP.  This will be filled in when its ready for a formal review.
In progress reviews are welcome.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/isper3at/incubator-rya RYA-236

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/138.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #138


commit 59874d2b1a45bd124d4433031c911e8194701507
Author: me 
Date:   2017-01-18T17:43:29Z

RYA-236 GeoTemporal Indexing

poms and project folders for GeoTemporal
Secondary indexer.

commit e6ad3def254ecb6e5ce77363657b453d470a9814
Author: me 
Date:   2017-01-18T17:48:11Z

RYA-240 GeoTemporal framework

Interfaces and integration test for geotemporal indexing.

The Integration test is incomplete as the expected is
currently unknown.




> Geotemporal Indexing Support
> 
>
> Key: RYA-236
> URL: https://issues.apache.org/jira/browse/RYA-236
> Project: Rya
>  Issue Type: New Feature
>Affects Versions: 3.2.10
>Reporter: Andrew Smith
>Assignee: Andrew Smith
> Fix For: 3.2.10
>
>
> Create a secondary indexer for queries with both a geo aspect and temporal 
> aspect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-94) Remove from secondary indexes when deleting from DAO

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427389#comment-15427389
 ] 

ASF GitHub Bot commented on RYA-94:
---

Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/73
  
hey david,
Two things:
1.  Merge to develop not master
2.  Mark the Pull request as WIP if its not ready to be merged


> Remove from secondary indexes when deleting from DAO
> 
>
> Key: RYA-94
> URL: https://issues.apache.org/jira/browse/RYA-94
> Project: Rya
>  Issue Type: Bug
>Reporter: Andrew Smith
>Assignee: David W. Lotts
>Priority: Blocker
>
> Currently, if a delete is made from the DAO, the secondary indexers are not 
> explicitly notified.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-151) Rya Test Bed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426879#comment-15426879
 ] 

ASF GitHub Bot commented on RYA-151:


Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/68#discussion_r75354404
  
--- Diff: 
extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java
 ---
@@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.benchmark.query;
+
+import static org.junit.Assert.assertEquals;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+
+import javax.xml.bind.JAXBException;
+
+import 
org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader;
+import org.junit.Test;
+import org.xml.sax.SAXException;
+
+import com.google.common.base.Charsets;
+
+/**
+ * Tests the methods of {@link BenchmarkQueriesReader}.
+ */
+public class BenchmarkQueriesReaderIT {
+
+@Test
+public void load() throws JAXBException, SAXException {
+// Unmarshal some XML.
+final String xml =
+"\n" +
+"\n" +
+"" +
--- End diff --

What do you want to do with those attributes? If they aren't used within 
the benchmark, that sort of information could easily just be XML comments.

{code}



   

{code}


> Rya Test Bed
> 
>
> Key: RYA-151
> URL: https://issues.apache.org/jira/browse/RYA-151
> Project: Rya
>  Issue Type: Wish
>  Components: clients, dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> It would be nice if there was a test bed checked in with Rya that could be 
> used for benchmark testing.  This test bed should include options for  
> uploading data to a rya instance, then issuing some benchmark queries (and 
> timing them in a way that everyone agrees is sound).  It could be a command 
> line tool.  It should be dao agnostic (work on either mongo, accumulo, etc.). 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-151) Rya Test Bed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426900#comment-15426900
 ] 

ASF GitHub Bot commented on RYA-151:


Github user amihalik commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/68#discussion_r75355985
  
--- Diff: 
extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java
 ---
@@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.benchmark.query;
+
+import static org.junit.Assert.assertEquals;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+
+import javax.xml.bind.JAXBException;
+
+import 
org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader;
+import org.junit.Test;
+import org.xml.sax.SAXException;
+
+import com.google.common.base.Charsets;
+
+/**
+ * Tests the methods of {@link BenchmarkQueriesReader}.
+ */
+public class BenchmarkQueriesReaderIT {
+
+@Test
+public void load() throws JAXBException, SAXException {
+// Unmarshal some XML.
+final String xml =
+"\n" +
+"\n" +
+"" +
--- End diff --

We should have query level granularity for the benchmark (i.e. we should 
have timing results for each query in the benchmark), so that we can have 
pretty graphs like this:


![image](https://cloud.githubusercontent.com/assets/4710489/17784519/c838a508-654a-11e6-8d65-e4043177dc6a.png)


Also, it would be nice if the benchmark printed out the name of the query 
for the results.  So we can have a table like this:


![image](https://cloud.githubusercontent.com/assets/4710489/17784596/14630158-654b-11e6-8dbb-23325619d3d7.png)

(Note: I ripped off these graphics from Roshan's Paper)


> Rya Test Bed
> 
>
> Key: RYA-151
> URL: https://issues.apache.org/jira/browse/RYA-151
> Project: Rya
>  Issue Type: Wish
>  Components: clients, dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> It would be nice if there was a test bed checked in with Rya that could be 
> used for benchmark testing.  This test bed should include options for  
> uploading data to a rya instance, then issuing some benchmark queries (and 
> timing them in a way that everyone agrees is sound).  It could be a command 
> line tool.  It should be dao agnostic (work on either mongo, accumulo, etc.). 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-163) PCJDetails Test Failures on non-EST TimeZones

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435836#comment-15435836
 ] 

ASF GitHub Bot commented on RYA-163:


Github user DLotts commented on the issue:

https://github.com/apache/incubator-rya/pull/80
  
This is complete and ready for review and pull.


> PCJDetails Test Failures on non-EST TimeZones
> -
>
> Key: RYA-163
> URL: https://issues.apache.org/jira/browse/RYA-163
> Project: Rya
>  Issue Type: Bug
> Environment: Apache Jenkins machine (UTC Timezone)
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> There are two tests failing on the build.  I think they are both due to a 
> timezone issue.
> I was able to reproduce these failures by setting my local timezone to 
> something other than eastern.
> https://builds.apache.org/job/incubator-rya-develop/27/testReport/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-163) PCJDetails Test Failures on non-EST TimeZones

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435865#comment-15435865
 ] 

ASF GitHub Bot commented on RYA-163:


Github user amihalik commented on the issue:

https://github.com/apache/incubator-rya/pull/80
  
Nice... super simple.  I tested it out and it passed on my machine.


> PCJDetails Test Failures on non-EST TimeZones
> -
>
> Key: RYA-163
> URL: https://issues.apache.org/jira/browse/RYA-163
> Project: Rya
>  Issue Type: Bug
> Environment: Apache Jenkins machine (UTC Timezone)
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> There are two tests failing on the build.  I think they are both due to a 
> timezone issue.
> I was able to reproduce these failures by setting my local timezone to 
> something other than eastern.
> https://builds.apache.org/job/incubator-rya-develop/27/testReport/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-163) PCJDetails Test Failures on non-EST TimeZones

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435962#comment-15435962
 ] 

ASF GitHub Bot commented on RYA-163:


Github user DLotts commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/80#discussion_r76160440
  
--- Diff: 
extras/rya.console/src/test/java/mvm/rya/shell/RyaAdminCommandsTest.java ---
@@ -123,6 +124,9 @@ public void deletePCJ() throws 
InstanceDoesNotExistException, RyaClientException
 
 @Test
 public void getInstanceDetails() throws InstanceDoesNotExistException, 
RyaClientException {
+// This test is failed if the default timezone was not EST, so now 
it's fixed at EST.
+// If you get assert mismatch of EST!=EDT, try the deprecated 
getTimeZone("EST") instead.
+TimeZone.setDefault(TimeZone.getTimeZone("America/New_York"));
--- End diff --

These are two different classes.  Before will still have an order 
dependence.  
Also every test after will still be affected.
Another solution is to replace it to the original value when the test is 
over.
With that, only concurrent unit tests will have order dependence, in that 
case, a race condition.  Then we need to synchronize on the timezone setting.
But what is wrong with setting it everywhere there is a dependence on 
TimeZone?  All the issues with that all have to do with setting the timezone 
default globally, but not order, nor concurrency -- unless someone sets it to a 
non EST timezone, then the concurrency issue comes back, but not the serial 
order.  

So I think it is best as it is now -- not perfect, but simple.  What do you 
think?


> PCJDetails Test Failures on non-EST TimeZones
> -
>
> Key: RYA-163
> URL: https://issues.apache.org/jira/browse/RYA-163
> Project: Rya
>  Issue Type: Bug
> Environment: Apache Jenkins machine (UTC Timezone)
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> There are two tests failing on the build.  I think they are both due to a 
> timezone issue.
> I was able to reproduce these failures by setting my local timezone to 
> something other than eastern.
> https://builds.apache.org/job/incubator-rya-develop/27/testReport/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-163) PCJDetails Test Failures on non-EST TimeZones

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435964#comment-15435964
 ] 

ASF GitHub Bot commented on RYA-163:


Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/80#discussion_r76160592
  
--- Diff: 
extras/rya.console/src/test/java/mvm/rya/shell/RyaAdminCommandsTest.java ---
@@ -123,6 +124,9 @@ public void deletePCJ() throws 
InstanceDoesNotExistException, RyaClientException
 
 @Test
 public void getInstanceDetails() throws InstanceDoesNotExistException, 
RyaClientException {
+// This test is failed if the default timezone was not EST, so now 
it's fixed at EST.
+// If you get assert mismatch of EST!=EDT, try the deprecated 
getTimeZone("EST") instead.
+TimeZone.setDefault(TimeZone.getTimeZone("America/New_York"));
--- End diff --

Alternatively, couldn't commands.getInstanceDetails(); accept the time zone 
as a parameter?


> PCJDetails Test Failures on non-EST TimeZones
> -
>
> Key: RYA-163
> URL: https://issues.apache.org/jira/browse/RYA-163
> Project: Rya
>  Issue Type: Bug
> Environment: Apache Jenkins machine (UTC Timezone)
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> There are two tests failing on the build.  I think they are both due to a 
> timezone issue.
> I was able to reproduce these failures by setting my local timezone to 
> something other than eastern.
> https://builds.apache.org/job/incubator-rya-develop/27/testReport/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-163) PCJDetails Test Failures on non-EST TimeZones

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435911#comment-15435911
 ] 

ASF GitHub Bot commented on RYA-163:


Github user DLotts commented on the issue:

https://github.com/apache/incubator-rya/pull/80
  
Done again.  Second failed test is fixed using the same method.


> PCJDetails Test Failures on non-EST TimeZones
> -
>
> Key: RYA-163
> URL: https://issues.apache.org/jira/browse/RYA-163
> Project: Rya
>  Issue Type: Bug
> Environment: Apache Jenkins machine (UTC Timezone)
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> There are two tests failing on the build.  I think they are both due to a 
> timezone issue.
> I was able to reproduce these failures by setting my local timezone to 
> something other than eastern.
> https://builds.apache.org/job/incubator-rya-develop/27/testReport/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-151) Rya Test Bed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467957#comment-15467957
 ] 

ASF GitHub Bot commented on RYA-151:


Github user kchilton2 closed the pull request at:

https://github.com/apache/incubator-rya/pull/68


> Rya Test Bed
> 
>
> Key: RYA-151
> URL: https://issues.apache.org/jira/browse/RYA-151
> Project: Rya
>  Issue Type: Wish
>  Components: clients, dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> It would be nice if there was a test bed checked in with Rya that could be 
> used for benchmark testing.  This test bed should include options for  
> uploading data to a rya instance, then issuing some benchmark queries (and 
> timing them in a way that everyone agrees is sound).  It could be a command 
> line tool.  It should be dao agnostic (work on either mongo, accumulo, etc.). 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-169) MongoRya DAO/Examples are currently broken

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468430#comment-15468430
 ] 

ASF GitHub Bot commented on RYA-169:


Github user isper3at commented on the issue:

https://github.com/apache/incubator-rya/pull/87
  
:ship: :it: Looks good


> MongoRya DAO/Examples are currently broken
> --
>
> Key: RYA-169
> URL: https://issues.apache.org/jira/browse/RYA-169
> Project: Rya
>  Issue Type: Bug
>  Components: clients, dao, examples
>Reporter: Puja Valiyil 
>Assignee: David W. Lotts
>Priority: Blocker
>
> With the pull in of RYA-81 or RYA-82, the examples became broken for Mongo.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-169) MongoRya DAO/Examples are currently broken

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468428#comment-15468428
 ] 

ASF GitHub Bot commented on RYA-169:


Github user isper3at commented on the issue:

https://github.com/apache/incubator-rya/pull/87
  
you've got 2 of the same commit here...


> MongoRya DAO/Examples are currently broken
> --
>
> Key: RYA-169
> URL: https://issues.apache.org/jira/browse/RYA-169
> Project: Rya
>  Issue Type: Bug
>  Components: clients, dao, examples
>Reporter: Puja Valiyil 
>Assignee: David W. Lotts
>Priority: Blocker
>
> With the pull in of RYA-81 or RYA-82, the examples became broken for Mongo.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-129) Create integration tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456221#comment-15456221
 ] 

ASF GitHub Bot commented on RYA-129:


Github user isper3at closed the pull request at:

https://github.com/apache/incubator-rya/pull/75


> Create integration tests
> 
>
> Key: RYA-129
> URL: https://issues.apache.org/jira/browse/RYA-129
> Project: Rya
>  Issue Type: Sub-task
>Affects Versions: 3.2.10
>Reporter: Andrew Smith
>Assignee: Andrew Smith
> Fix For: 3.2.10
>
>
> Add more tests to cover the new features



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-168) Support for property chains for Rya query inference

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472093#comment-15472093
 ] 

ASF GitHub Bot commented on RYA-168:


Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/81


> Support for property chains for Rya query inference
> ---
>
> Key: RYA-168
> URL: https://issues.apache.org/jira/browse/RYA-168
> Project: Rya
>  Issue Type: New Feature
>  Components: erence
>Reporter: Puja Valiyil 
>Assignee: Puja Valiyil 
>
> It would be useful to provide propertyChain support for backwards chaining 
> inference.  This is relatively straight forward and can draw on the other 
> query expansion paradigms in Rya's exisitng inference.  Some research needs 
> to be done to determine how property chains look when ingested into Rya 
> (serialized into N-Triples).  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-165) Fluo PCJ maintenance should use ColumnVisibility.flatten() to simplify the visibility expression

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15487719#comment-15487719
 ] 

ASF GitHub Bot commented on RYA-165:


Github user amihalik commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/90#discussion_r78598233
  
--- Diff: 
dao/accumulo.rya/src/main/java/mvm/rya/accumulo/utils/VisibilitySimplifier.java 
---
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.accumulo.utils;
+
+import static java.util.Objects.requireNonNull;
+
+import javax.annotation.ParametersAreNonnullByDefault;
+
+import org.apache.accumulo.core.security.ColumnVisibility;
+
+import com.google.common.base.Charsets;
+
+/**
+ * Simplifies Accumulo visibility expressions.
+ */
+@ParametersAreNonnullByDefault
+public class VisibilitySimplifier {
+
+/**
+ * Simplifies an Accumulo visibility expression.
+ *
+ * @param visibility - The expression to simplify. (not null)
+ * @return A simplified form of {@code visibility}.
+ */
+public String simplify(final String visibility) {
+requireNonNull(visibility);
+
+String last = visibility;
+String simplified = new String(new 
ColumnVisibility(visibility).flatten(), Charsets.UTF_8);
+
+while(!simplified.equals(last)) {
--- End diff --

wow.  Is this really necessary?  I'm disappointed that you have to run this 
multiple times to completely flatten the expression.


> Fluo PCJ maintenance should use ColumnVisibility.flatten() to simplify the 
> visibility expression
> 
>
> Key: RYA-165
> URL: https://issues.apache.org/jira/browse/RYA-165
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>Priority: Critical
>
> The Fluo application that maintains PCJ indices currently creates complex 
> visibility expressions that grow with every join that is part of the SPARQL 
> query.
> For example, if you create a PCJ for the following SPARQL:
> {code}
> SELECT ?customer ?worker ?city {
> ?customer  ?worker.
> ?worker  ?city.
> ?worker  .
> }
> {code}
> And you insert the following Statements:
> {code}
> Statement: "  " Visibility: "u"
> Statement: "  " Visibility: "u"
> Statement: "  " Visibility: "u"
> {code}
> Then the following result will be exported to the PCJ index:
> {code}
> BindingSet: {{"customer", "urn:Alice"}, {"worker","urn:Bob"}, {"city", 
> "urn:Happyville"} Visibility: "(u)"
> {code}
> That visibility expression could also be represented as just "u". 
> ColumnVisibility.flatten() calls may be used to iteratively simplify the 
> logical expression, so update the Fluo application to do so before exporting 
> the result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-165) Fluo PCJ maintenance should use ColumnVisibility.flatten() to simplify the visibility expression

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15487702#comment-15487702
 ] 

ASF GitHub Bot commented on RYA-165:


Github user pujav65 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/90#discussion_r78596890
  
--- Diff: 
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/observers/QueryResultObserver.java
 ---
@@ -54,6 +58,16 @@
 private static final VisibilityBindingSetStringConverter CONVERTER = 
new VisibilityBindingSetStringConverter();
 
 /**
+ * Simplifies Visibility expressions prior to exporting PCJ results.
+ */
+private static final VisibilitySimplifier SIMPLIFIER = new 
VisibilitySimplifier();
--- End diff --

Is there a reason the simplify method isn't static (why are we creating 
this class)?


> Fluo PCJ maintenance should use ColumnVisibility.flatten() to simplify the 
> visibility expression
> 
>
> Key: RYA-165
> URL: https://issues.apache.org/jira/browse/RYA-165
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>Priority: Critical
>
> The Fluo application that maintains PCJ indices currently creates complex 
> visibility expressions that grow with every join that is part of the SPARQL 
> query.
> For example, if you create a PCJ for the following SPARQL:
> {code}
> SELECT ?customer ?worker ?city {
> ?customer  ?worker.
> ?worker  ?city.
> ?worker  .
> }
> {code}
> And you insert the following Statements:
> {code}
> Statement: "  " Visibility: "u"
> Statement: "  " Visibility: "u"
> Statement: "  " Visibility: "u"
> {code}
> Then the following result will be exported to the PCJ index:
> {code}
> BindingSet: {{"customer", "urn:Alice"}, {"worker","urn:Bob"}, {"city", 
> "urn:Happyville"} Visibility: "(u)"
> {code}
> That visibility expression could also be represented as just "u". 
> ColumnVisibility.flatten() calls may be used to iteratively simplify the 
> logical expression, so update the Fluo application to do so before exporting 
> the result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-168) Support for property chains for Rya query inference

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15487716#comment-15487716
 ] 

ASF GitHub Bot commented on RYA-168:


GitHub user pujav65 opened a pull request:

https://github.com/apache/incubator-rya/pull/92

[WIP] RYA-168: property chain inference support


## Description
>What Changed?
Adding support for property chain inference.  

### Tests
>Coverage?
wrote an example to test end to end.  no integration tests written yet.  

### Links
[Jira](https://issues.apache.org/jira/browse/RYA-168)

### Checklist
- [ ] Code Review
- [ ] Squash Commits

 People To Reivew
[Add those who should review this]



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pujav65/incubator-rya f/RYA-168

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/92.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #92


commit bc988c958da55200d1b8b6a8f39622a44143d6b6
Author: pujav65 
Date:   2016-08-25T17:40:23Z

RYA-168 initial checkin




> Support for property chains for Rya query inference
> ---
>
> Key: RYA-168
> URL: https://issues.apache.org/jira/browse/RYA-168
> Project: Rya
>  Issue Type: New Feature
>  Components: erence
>Reporter: Puja Valiyil 
>Assignee: Puja Valiyil 
>
> It would be useful to provide propertyChain support for backwards chaining 
> inference.  This is relatively straight forward and can draw on the other 
> query expansion paradigms in Rya's exisitng inference.  Some research needs 
> to be done to determine how property chains look when ingested into Rya 
> (serialized into N-Triples).  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-151) Rya Test Bed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15490751#comment-15490751
 ] 

ASF GitHub Bot commented on RYA-151:


Github user amihalik commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/85#discussion_r78775502
  
--- Diff: extras/rya.benchmark/pom.xml ---
@@ -0,0 +1,250 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+ 
+
+rya.extras
+org.apache.rya
+3.2.10-SNAPSHOT
+
+
+4.0.0
+rya.benchmark
+
+Apache Rya Benchmarks
+
+
+
+
+org.openjdk.jmh
+jmh-core
+${jmh.version}
+
+
+org.openjdk.jmh
+jmh-generator-annprocess
+${jmh.version}
+provided
+
+
+
+org.apache.rya
+rya.indexing
+
+
+
+com.google.code.findbugs
+jsr305
+3.0.1
+
+
+
+com.google.guava
+guava
+
+
+
+junit
+junit
+test
+
+
+
+org.apache.accumulo
+accumulo-minicluster
+${accumulo.version}
+test
+
+
+
+
+
+UTF-8
+
+
+1.13
+
+
+1.8
+
+
+benchmarks
+
+
+
+
+
+src/main/xsd
+
+
+
+
+
+org.apache.maven.plugins
--- End diff --

Sorry for my terse comment.  These should be defined in the parent POM:

1. Versions for dependencies should be defined as parameters (e.g. like 
you've done with jmh.version).  You should also do this for maven plug-ins 
(e.g. you haven't done this for maven-clean-plugin).

2. Dependencies (and plug-ins) and their versions should be defined in the 
DependenciesManagement (and pluginManagement) sections of the root pom.

3. The root pom should be the only pom to define the java versions.  
Subprojects shouldn't be using the maven-compiler-plugin to define their own 
versions of java.

4. You have some nice plug-ins that should be moved to the root pom as well 
(e.g. license-maven-plugin).  I think @DLotts is working on this, so there's no 
need for you to modify that here.


> Rya Test Bed
> 
>
> Key: RYA-151
> URL: https://issues.apache.org/jira/browse/RYA-151
> Project: Rya
>  Issue Type: Wish
>  Components: clients, dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> It would be nice if there was a test bed checked in with Rya that could be 
> used for benchmark testing.  This test bed should include options for  
> uploading data to a rya instance, then issuing some benchmark queries (and 
> timing them in a way that everyone agrees is sound).  It could be a command 
> line tool.  It should be dao agnostic (work on either mongo, accumulo, etc.). 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-165) Fluo PCJ maintenance should use ColumnVisibility.flatten() to simplify the visibility expression

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15487868#comment-15487868
 ] 

ASF GitHub Bot commented on RYA-165:


Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/90#discussion_r78608207
  
--- Diff: 
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/observers/QueryResultObserver.java
 ---
@@ -54,6 +58,16 @@
 private static final VisibilityBindingSetStringConverter CONVERTER = 
new VisibilityBindingSetStringConverter();
 
 /**
+ * Simplifies Visibility expressions prior to exporting PCJ results.
+ */
+private static final VisibilitySimplifier SIMPLIFIER = new 
VisibilitySimplifier();
--- End diff --

Static code is harder to test so I just shy away from it.


> Fluo PCJ maintenance should use ColumnVisibility.flatten() to simplify the 
> visibility expression
> 
>
> Key: RYA-165
> URL: https://issues.apache.org/jira/browse/RYA-165
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>Priority: Critical
>
> The Fluo application that maintains PCJ indices currently creates complex 
> visibility expressions that grow with every join that is part of the SPARQL 
> query.
> For example, if you create a PCJ for the following SPARQL:
> {code}
> SELECT ?customer ?worker ?city {
> ?customer  ?worker.
> ?worker  ?city.
> ?worker  .
> }
> {code}
> And you insert the following Statements:
> {code}
> Statement: "  " Visibility: "u"
> Statement: "  " Visibility: "u"
> Statement: "  " Visibility: "u"
> {code}
> Then the following result will be exported to the PCJ index:
> {code}
> BindingSet: {{"customer", "urn:Alice"}, {"worker","urn:Bob"}, {"city", 
> "urn:Happyville"} Visibility: "(u)"
> {code}
> That visibility expression could also be represented as just "u". 
> ColumnVisibility.flatten() calls may be used to iteratively simplify the 
> logical expression, so update the Fluo application to do so before exporting 
> the result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-176) Add Apache Rat exclusion for DEPENDENCIES

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15488238#comment-15488238
 ] 

ASF GitHub Bot commented on RYA-176:


GitHub user amihalik opened a pull request:

https://github.com/apache/incubator-rya/pull/93

RYA-176 Adding DEPENDENCIES to RAT Exclusion

## Description
Updated the POMs to ignore `DEPENDENCIES` file contained in release.  Also 
some minor updates (`rya.console` had a log file hanging out, updated 
consistent use of rat exclusions and documented, removed a random java version 
override)

### Tests

No tests written.  Performed a `mvn clean install` to double check build.

### Links
[Jira](https://issues.apache.org/jira/browse/RYA-176)

### Checklist
- [ ] Code Review
- [x] Squash Commits

 People To Reivew
@pujav65 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/amihalik/incubator-rya RYA-176

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/93.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #93


commit 6e99b89fe252bcb9ccf1b2b58d2b945d5521d321
Author: Aaron Mihalik 
Date:   2016-09-13T16:32:25Z

RYA-176 Adding DEPENDENCIES to RAT Exclusion
Also performing minor cleanup




> Add Apache Rat exclusion for DEPENDENCIES
> -
>
> Key: RYA-176
> URL: https://issues.apache.org/jira/browse/RYA-176
> Project: Rya
>  Issue Type: Bug
>Reporter: Aaron Mihalik
>
> When we package Rya, Apache includes a DEPENDENCIES file.  This does not have 
> the Apache License header and fails Apache Rat.  We can add an exclusion for 
> this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-173) rya.merger not creating signed version of bundle.zip

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474909#comment-15474909
 ] 

ASF GitHub Bot commented on RYA-173:


Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/88


> rya.merger not creating signed version of bundle.zip
> 
>
> Key: RYA-173
> URL: https://issues.apache.org/jira/browse/RYA-173
> Project: Rya
>  Issue Type: Bug
>Reporter: Aaron Mihalik
>Assignee: Eric White
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-151) Rya Test Bed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489134#comment-15489134
 ] 

ASF GitHub Bot commented on RYA-151:


Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/85#discussion_r78677753
  
--- Diff: extras/rya.benchmark/pom.xml ---
@@ -0,0 +1,250 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+ 
+
+rya.extras
+org.apache.rya
+3.2.10-SNAPSHOT
+
+
+4.0.0
+rya.benchmark
+
+Apache Rya Benchmarks
+
+
+
+
+org.openjdk.jmh
+jmh-core
+${jmh.version}
+
+
+org.openjdk.jmh
+jmh-generator-annprocess
+${jmh.version}
+provided
+
+
+
+org.apache.rya
+rya.indexing
+
+
+
+com.google.code.findbugs
+jsr305
+3.0.1
+
+
+
+com.google.guava
+guava
+
+
+
+junit
+junit
+test
+
+
+
+org.apache.accumulo
+accumulo-minicluster
+${accumulo.version}
+test
+
+
+
+
+
+UTF-8
+
+
+1.13
+
+
+1.8
+
+
+benchmarks
+
+
+
+
+
+src/main/xsd
+
+
+
+
+
+org.apache.maven.plugins
--- End diff --

Which part specifically?


> Rya Test Bed
> 
>
> Key: RYA-151
> URL: https://issues.apache.org/jira/browse/RYA-151
> Project: Rya
>  Issue Type: Wish
>  Components: clients, dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> It would be nice if there was a test bed checked in with Rya that could be 
> used for benchmark testing.  This test bed should include options for  
> uploading data to a rya instance, then issuing some benchmark queries (and 
> timing them in a way that everyone agrees is sound).  It could be a command 
> line tool.  It should be dao agnostic (work on either mongo, accumulo, etc.). 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-151) Rya Test Bed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489132#comment-15489132
 ] 

ASF GitHub Bot commented on RYA-151:


Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/85#discussion_r78677735
  
--- Diff: 
extras/indexing/src/main/java/mvm/rya/indexing/external/PrecomputedJoinStorageSupplier.java
 ---
@@ -27,6 +27,7 @@
 import com.google.common.base.Optional;
 import com.google.common.base.Supplier;
 
+import cern.colt.Arrays;
--- End diff --

That's likely a mistake. Good catch.


> Rya Test Bed
> 
>
> Key: RYA-151
> URL: https://issues.apache.org/jira/browse/RYA-151
> Project: Rya
>  Issue Type: Wish
>  Components: clients, dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> It would be nice if there was a test bed checked in with Rya that could be 
> used for benchmark testing.  This test bed should include options for  
> uploading data to a rya instance, then issuing some benchmark queries (and 
> timing them in a way that everyone agrees is sound).  It could be a command 
> line tool.  It should be dao agnostic (work on either mongo, accumulo, etc.). 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-151) Rya Test Bed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15488985#comment-15488985
 ] 

ASF GitHub Bot commented on RYA-151:


Github user amihalik commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/85#discussion_r78671939
  
--- Diff: extras/rya.benchmark/pom.xml ---
@@ -0,0 +1,250 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+ 
+
+rya.extras
+org.apache.rya
+3.2.10-SNAPSHOT
+
+
+4.0.0
+rya.benchmark
+
+Apache Rya Benchmarks
+
+
+
+
+org.openjdk.jmh
+jmh-core
+${jmh.version}
--- End diff --

These versions should be defined in the root pom as properties and in the 
root pom's plug-in management section


> Rya Test Bed
> 
>
> Key: RYA-151
> URL: https://issues.apache.org/jira/browse/RYA-151
> Project: Rya
>  Issue Type: Wish
>  Components: clients, dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> It would be nice if there was a test bed checked in with Rya that could be 
> used for benchmark testing.  This test bed should include options for  
> uploading data to a rya instance, then issuing some benchmark queries (and 
> timing them in a way that everyone agrees is sound).  It could be a command 
> line tool.  It should be dao agnostic (work on either mongo, accumulo, etc.). 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-151) Rya Test Bed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15488982#comment-15488982
 ] 

ASF GitHub Bot commented on RYA-151:


Github user amihalik commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/85#discussion_r78671763
  
--- Diff: extras/rya.benchmark/pom.xml ---
@@ -0,0 +1,250 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+ 
+
+rya.extras
+org.apache.rya
+3.2.10-SNAPSHOT
+
+
+4.0.0
+rya.benchmark
+
+Apache Rya Benchmarks
+
+
+
+
+org.openjdk.jmh
+jmh-core
+${jmh.version}
+
+
+org.openjdk.jmh
+jmh-generator-annprocess
+${jmh.version}
+provided
+
+
+
+org.apache.rya
+rya.indexing
+
+
+
+com.google.code.findbugs
+jsr305
+3.0.1
+
+
+
+com.google.guava
+guava
+
+
+
+junit
+junit
+test
+
+
+
+org.apache.accumulo
+accumulo-minicluster
+${accumulo.version}
+test
+
+
+
+
+
+UTF-8
+
+
+1.13
+
+
+1.8
+
+
+benchmarks
+
+
+
+
+
+src/main/xsd
+
+
+
+
+
+org.apache.maven.plugins
--- End diff --

this should be define in the project's root pom


> Rya Test Bed
> 
>
> Key: RYA-151
> URL: https://issues.apache.org/jira/browse/RYA-151
> Project: Rya
>  Issue Type: Wish
>  Components: clients, dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> It would be nice if there was a test bed checked in with Rya that could be 
> used for benchmark testing.  This test bed should include options for  
> uploading data to a rya instance, then issuing some benchmark queries (and 
> timing them in a way that everyone agrees is sound).  It could be a command 
> line tool.  It should be dao agnostic (work on either mongo, accumulo, etc.). 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-165) Fluo PCJ maintenance should use ColumnVisibility.flatten() to simplify the visibility expression

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485467#comment-15485467
 ] 

ASF GitHub Bot commented on RYA-165:


Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/90#discussion_r78465895
  
--- Diff: 
dao/accumulo.rya/src/main/java/mvm/rya/accumulo/utils/VisibilitySimplifier.java 
---
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package mvm.rya.accumulo.utils;
+
+import static java.util.Objects.requireNonNull;
+
+import javax.annotation.ParametersAreNonnullByDefault;
+
+import org.apache.accumulo.core.security.ColumnVisibility;
+
+import com.google.common.base.Charsets;
+
+/**
+ * Simplifies Accumulo visibility expressions.
+ */
+@ParametersAreNonnullByDefault
+public class VisibilitySimplifier {
+
+/**
+ * Simplifies an Accumulo visibility expression.
+ *
+ * @param visibility - The expression to simplify. (not null)
+ * @return A simplified form of {@code visibility}.
+ */
+public String simplify(final String visibility) {
--- End diff --

:+1:  I feel like this is how it should've been implemented in flatten()


> Fluo PCJ maintenance should use ColumnVisibility.flatten() to simplify the 
> visibility expression
> 
>
> Key: RYA-165
> URL: https://issues.apache.org/jira/browse/RYA-165
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>Priority: Critical
>
> The Fluo application that maintains PCJ indices currently creates complex 
> visibility expressions that grow with every join that is part of the SPARQL 
> query.
> For example, if you create a PCJ for the following SPARQL:
> {code}
> SELECT ?customer ?worker ?city {
> ?customer  ?worker.
> ?worker  ?city.
> ?worker  .
> }
> {code}
> And you insert the following Statements:
> {code}
> Statement: "  " Visibility: "u"
> Statement: "  " Visibility: "u"
> Statement: "  " Visibility: "u"
> {code}
> Then the following result will be exported to the PCJ index:
> {code}
> BindingSet: {{"customer", "urn:Alice"}, {"worker","urn:Bob"}, {"city", 
> "urn:Happyville"} Visibility: "(u)"
> {code}
> That visibility expression could also be represented as just "u". 
> ColumnVisibility.flatten() calls may be used to iteratively simplify the 
> logical expression, so update the Fluo application to do so before exporting 
> the result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-172) Rya Merger Test Fail (persists temp data between test)

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15478266#comment-15478266
 ] 

ASF GitHub Bot commented on RYA-172:


GitHub user ejwhite922 opened a pull request:

https://github.com/apache/incubator-rya/pull/89

RYA-172 Fixed CopyTool unit test from not clearing tmp hadoop directo…

…ry between tests.

## Description
Fixed CopyTool unit test so it clears out the hadoop temp directory it uses 
between each test.  Most of the changes are from save actions.  Search for the 
importFilesToChildTable method to see actual changes.

### Tests
Ran unit tests multiple times

### Links
[Jira](https://issues.apache.org/jira/browse/RYA-172)

### Checklist
- [x] Code Review
- [ ] Squash Commits

 People To Review
@amihalik 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ejwhite922/incubator-rya 
RYA-172_RyaMergerTestTempData

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/89.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #89


commit 2e3505eb6993c2df09470b782418b0b8275cc762
Author: ejwhite922 
Date:   2016-09-09T20:34:32Z

RYA-172 Fixed CopyTool unit test from not clearing tmp hadoop directory 
between tests.




> Rya Merger Test Fail (persists temp data between test)
> --
>
> Key: RYA-172
> URL: https://issues.apache.org/jira/browse/RYA-172
> Project: Rya
>  Issue Type: Bug
>Reporter: Aaron Mihalik
>Assignee: Eric White
>
> CopyToolTest.testImportDirectoryTool will fail the second time it's run.  
> There is data hanging around in  /tmp/hadoop-amihalik/ (for me) after the 
> first run of the test and it's causing issues the second time.
> To reproduce:
> in incubator-rya, build rya:
> {code}
> mvn clean install -DskipTests
> {code}
> now run the rya.merger tests a couple times:
> {code}
> cd extras/rya.merger
> mvn clean install
> mvn clean install
> {code}
> The error should be this on the second time through
> {code}
> CopyToolTest.testImportDirectoryTool:355 expected:<20> but was:<0>
> {code}
> This was in my log (in eclispe):
> {code}
> 2016-09-07 18:16:44,752 ERROR [main] merge.CopyTool 
> (CopyTool.java:setupAndRun(860)) - Error running copy tool
> java.io.IOException: Target 
> /tmp/hadoop-mihalik/copy_tool_import/ct_spo/ct_spo is a directory
> {code}
> my work around is to 
> {code}
> rm -rf /tmp/hadoop-mihalik/
> {code}
> between tests



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-94) Remove from secondary indexes when deleting from DAO

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523413#comment-15523413
 ] 

ASF GitHub Bot commented on RYA-94:
---

Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/73
  
hey david, is this still in progress or was it made obsolete by another 
pull request?


> Remove from secondary indexes when deleting from DAO
> 
>
> Key: RYA-94
> URL: https://issues.apache.org/jira/browse/RYA-94
> Project: Rya
>  Issue Type: Bug
>Reporter: Andrew Smith
>Assignee: David W. Lotts
>Priority: Blocker
>
> Currently, if a delete is made from the DAO, the secondary indexers are not 
> explicitly notified.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-177) Review License on Rya Dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1575#comment-1575
 ] 

ASF GitHub Bot commented on RYA-177:


Github user amihalik commented on the issue:

https://github.com/apache/incubator-rya/pull/101
  
Yep, I will missing a commit.  Looks good.  Building and running the tests 
now.


> Review License on Rya Dependencies
> --
>
> Key: RYA-177
> URL: https://issues.apache.org/jira/browse/RYA-177
> Project: Rya
>  Issue Type: Task
>Affects Versions: 3.2.10
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> * jgridshift:jgridshift appears to be LGPL licensed
> (https://github.com/floscher/jGridShift/blob/master/LICENSE). You may
> not use this software. It looks like it was not appropriately marked in
> its pom which is why the configuration from Rya's parent apache.pom did
> not catch it. This is brought in via org.geotools.xsd:gt-xsd-gml3.
> * colt (http://dst.lbl.gov/ACSSoftware/colt/) appears to be another
> brought in by com.tinkerpop.blueprints:blueprints-core
> * com.google.code.findbugs:jsr305 is another example of GPL licensing.
> While the artifact appears to have the ASL tagged on the pom, all
> Findbugs documentation states that the project is GPL.
> I would recommend to make a pass over your dependencies to verify that
> you aren't depending on any projects which are licensed with a license
> on this list: http://www.apache.org/legal/resolved.html#category-x. See
> http://www.apache.org/licenses/GPL-compatibility.html for more details.
> The above three examples were found via a brief glance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-177) Review License on Rya Dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1381#comment-1381
 ] 

ASF GitHub Bot commented on RYA-177:


GitHub user pujav65 opened a pull request:

https://github.com/apache/incubator-rya/pull/101

RYA-177 adding optional geoindexing profile


## Description
>What Changed?
Moved geoindexing to a separate project and added a profile.

### Tests
>Coverage?

N/A

### Links
[Jira](https://issues.apache.org/jira/browse/RYA-177)

### Checklist
- [ ] Code Review
- [ x] Squash Commits

 People To Reivew
@amihalik



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pujav65/incubator-rya RYA-177

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/101.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #101


commit 7727b1651b734d4b06d518032897f0c0dc4f88d5
Author: pujav65 
Date:   2016-10-07T15:26:43Z

RYA-177 adding optional geoindexing profile




> Review License on Rya Dependencies
> --
>
> Key: RYA-177
> URL: https://issues.apache.org/jira/browse/RYA-177
> Project: Rya
>  Issue Type: Task
>Affects Versions: 3.2.10
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> * jgridshift:jgridshift appears to be LGPL licensed
> (https://github.com/floscher/jGridShift/blob/master/LICENSE). You may
> not use this software. It looks like it was not appropriately marked in
> its pom which is why the configuration from Rya's parent apache.pom did
> not catch it. This is brought in via org.geotools.xsd:gt-xsd-gml3.
> * colt (http://dst.lbl.gov/ACSSoftware/colt/) appears to be another
> brought in by com.tinkerpop.blueprints:blueprints-core
> * com.google.code.findbugs:jsr305 is another example of GPL licensing.
> While the artifact appears to have the ASL tagged on the pom, all
> Findbugs documentation states that the project is GPL.
> I would recommend to make a pass over your dependencies to verify that
> you aren't depending on any projects which are licensed with a license
> on this list: http://www.apache.org/legal/resolved.html#category-x. See
> http://www.apache.org/licenses/GPL-compatibility.html for more details.
> The above three examples were found via a brief glance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-177) Review License on Rya Dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1427#comment-1427
 ] 

ASF GitHub Bot commented on RYA-177:


Github user amihalik commented on the issue:

https://github.com/apache/incubator-rya/pull/101
  
I'm still seeing gt libs pulled into the build eg:

```
[INFO] 

[INFO] Building Apache Rya Secondary Indexing 3.2.10-SNAPSHOT
[INFO] 


[...]

[INFO] Including org.geotools.xsd:gt-xsd-core:jar:14.3 in the shaded jar.
[INFO] Including org.geotools:gt-graph:jar:14.3 in the shaded jar.
```

What's the plan to resolve these before a release?


> Review License on Rya Dependencies
> --
>
> Key: RYA-177
> URL: https://issues.apache.org/jira/browse/RYA-177
> Project: Rya
>  Issue Type: Task
>Affects Versions: 3.2.10
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> * jgridshift:jgridshift appears to be LGPL licensed
> (https://github.com/floscher/jGridShift/blob/master/LICENSE). You may
> not use this software. It looks like it was not appropriately marked in
> its pom which is why the configuration from Rya's parent apache.pom did
> not catch it. This is brought in via org.geotools.xsd:gt-xsd-gml3.
> * colt (http://dst.lbl.gov/ACSSoftware/colt/) appears to be another
> brought in by com.tinkerpop.blueprints:blueprints-core
> * com.google.code.findbugs:jsr305 is another example of GPL licensing.
> While the artifact appears to have the ASL tagged on the pom, all
> Findbugs documentation states that the project is GPL.
> I would recommend to make a pass over your dependencies to verify that
> you aren't depending on any projects which are licensed with a license
> on this list: http://www.apache.org/legal/resolved.html#category-x. See
> http://www.apache.org/licenses/GPL-compatibility.html for more details.
> The above three examples were found via a brief glance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-177) Review License on Rya Dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1489#comment-1489
 ] 

ASF GitHub Bot commented on RYA-177:


Github user amihalik commented on the issue:

https://github.com/apache/incubator-rya/pull/101
  
Better.  I'm still seeing gt dependencies.

try running `mvn dependency:tree | grep gt` in your environment.  I get 
this:

```
[INFO] |  |  |  +- org.geotools:gt-referencing:jar:14.1:compile
[INFO] |  |  |  +- org.geotools:gt-grid:jar:14.1:compile
[INFO] |  |  |  \- org.geotools:gt-transform:jar:14.1:compile
[INFO] |  |  +- org.geotools:gt-main:jar:14.1:compile
[INFO] |  |  +- org.geotools:gt-api:jar:14.1:compile
[INFO] |  |  +- org.geotools:gt-data:jar:14.1:compile
[INFO] |  |  +- org.geotools:gt-cql:jar:14.1:compile
[INFO] |  |  +- org.geotools:gt-metadata:jar:14.1:compile
[INFO] |  |  +- org.geotools:gt-render:jar:14.1:compile
[INFO] |  |  |  \- org.geotools:gt-coverage:jar:14.1:compile
[INFO] |  |  +- org.geotools:gt-process-feature:jar:14.1:compile
[INFO] |  |  |  \- org.geotools:gt-process:jar:14.1:compile
[INFO] |  |  | \- org.geotools:gt-shapefile:jar:14.1:compile
[INFO] |  +- org.geotools:gt-opengis:jar:14.1:compile
```


> Review License on Rya Dependencies
> --
>
> Key: RYA-177
> URL: https://issues.apache.org/jira/browse/RYA-177
> Project: Rya
>  Issue Type: Task
>Affects Versions: 3.2.10
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> * jgridshift:jgridshift appears to be LGPL licensed
> (https://github.com/floscher/jGridShift/blob/master/LICENSE). You may
> not use this software. It looks like it was not appropriately marked in
> its pom which is why the configuration from Rya's parent apache.pom did
> not catch it. This is brought in via org.geotools.xsd:gt-xsd-gml3.
> * colt (http://dst.lbl.gov/ACSSoftware/colt/) appears to be another
> brought in by com.tinkerpop.blueprints:blueprints-core
> * com.google.code.findbugs:jsr305 is another example of GPL licensing.
> While the artifact appears to have the ASL tagged on the pom, all
> Findbugs documentation states that the project is GPL.
> I would recommend to make a pass over your dependencies to verify that
> you aren't depending on any projects which are licensed with a license
> on this list: http://www.apache.org/legal/resolved.html#category-x. See
> http://www.apache.org/licenses/GPL-compatibility.html for more details.
> The above three examples were found via a brief glance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-177) Review License on Rya Dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1394#comment-1394
 ] 

ASF GitHub Bot commented on RYA-177:


Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/101
  
This is meant to be a start to get us through a release -- there are at 
least three issues that should accompany this:
1.  Bring back the geo examples
2.  Have Map Reduce delegate to the DAO to write to indexers, not writing 
to them directly
3.  Look at further refactoring to bring back the mongo geoindexing since 
it doesn't implicitly depend on geotools
I feel these are all relatively minor and could be solved later.


> Review License on Rya Dependencies
> --
>
> Key: RYA-177
> URL: https://issues.apache.org/jira/browse/RYA-177
> Project: Rya
>  Issue Type: Task
>Affects Versions: 3.2.10
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> * jgridshift:jgridshift appears to be LGPL licensed
> (https://github.com/floscher/jGridShift/blob/master/LICENSE). You may
> not use this software. It looks like it was not appropriately marked in
> its pom which is why the configuration from Rya's parent apache.pom did
> not catch it. This is brought in via org.geotools.xsd:gt-xsd-gml3.
> * colt (http://dst.lbl.gov/ACSSoftware/colt/) appears to be another
> brought in by com.tinkerpop.blueprints:blueprints-core
> * com.google.code.findbugs:jsr305 is another example of GPL licensing.
> While the artifact appears to have the ASL tagged on the pom, all
> Findbugs documentation states that the project is GPL.
> I would recommend to make a pass over your dependencies to verify that
> you aren't depending on any projects which are licensed with a license
> on this list: http://www.apache.org/legal/resolved.html#category-x. See
> http://www.apache.org/licenses/GPL-compatibility.html for more details.
> The above three examples were found via a brief glance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-177) Review License on Rya Dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1457#comment-1457
 ] 

ASF GitHub Bot commented on RYA-177:


Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/101
  
Sorry missed the jt dependency that was pulling those in.  Let me know if 
you see anything else being pulled in.


> Review License on Rya Dependencies
> --
>
> Key: RYA-177
> URL: https://issues.apache.org/jira/browse/RYA-177
> Project: Rya
>  Issue Type: Task
>Affects Versions: 3.2.10
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> * jgridshift:jgridshift appears to be LGPL licensed
> (https://github.com/floscher/jGridShift/blob/master/LICENSE). You may
> not use this software. It looks like it was not appropriately marked in
> its pom which is why the configuration from Rya's parent apache.pom did
> not catch it. This is brought in via org.geotools.xsd:gt-xsd-gml3.
> * colt (http://dst.lbl.gov/ACSSoftware/colt/) appears to be another
> brought in by com.tinkerpop.blueprints:blueprints-core
> * com.google.code.findbugs:jsr305 is another example of GPL licensing.
> While the artifact appears to have the ASL tagged on the pom, all
> Findbugs documentation states that the project is GPL.
> I would recommend to make a pass over your dependencies to verify that
> you aren't depending on any projects which are licensed with a license
> on this list: http://www.apache.org/legal/resolved.html#category-x. See
> http://www.apache.org/licenses/GPL-compatibility.html for more details.
> The above three examples were found via a brief glance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-177) Review License on Rya Dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553419#comment-15553419
 ] 

ASF GitHub Bot commented on RYA-177:


GitHub user DLotts opened a pull request:

https://github.com/apache/incubator-rya/pull/99

rya-177 WIP; indexing and mapreduce are now optional profiles in Mave…

## Description
>What Changed?

Made indexing and mapreduce modules part of profiles that are not built 
unless using one of these:
mvn ... -P indexing 
mvn ... -P mapreduce 

The purpose is to make geotools related features optional since it has an 
LGPL license.
Also under RYA-177 are removing several other less difficult dependencies.

TODO: remove Geo dependencies from top level POM.xml and find dependencies. 
 They are marked with comments.  Also remove other dependencies that have 
incompatible licenses.

### Tests
>Coverage?

No additional junit tests are needed.  A good test is to run exactly this 
in the project root:
mvn license:aggregate-add-third-party

GeoTools should be missing.  Also look for GPL and LGPL and some other 
incompatible licensed libraries.  look on the rya dev list where I posted the 
subset of bad ones.  It subject has the wrong issue number, here it is:

subject: RYA-179 Review License / Copyright notices on Rya Artifacts
found here: 
https://www.mail-archive.com/dev@rya.incubator.apache.org/msg00969.html

### Links
[Jira](https://issues.apache.org/jira/browse/RYA-177)

### Checklist
- [ ] Code Review
- [ ] Squash Commits

 People To Reivew


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/DLotts/incubator-rya 
RYA-177_incompatLicenseOptional

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/99.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #99


commit 714be87f230ea1da596870910206085b3e3e410d
Author: David W. Lotts 
Date:   2016-10-06T21:54:40Z

rya-177 WIP; indexing and mapreduce are now optional profiles in Maven. 
TODO: remove Geo from top level POM and find dependencies.




> Review License on Rya Dependencies
> --
>
> Key: RYA-177
> URL: https://issues.apache.org/jira/browse/RYA-177
> Project: Rya
>  Issue Type: Task
>Affects Versions: 3.2.10
>Reporter: Aaron Mihalik
>Assignee: David W. Lotts
>Priority: Blocker
>
> * jgridshift:jgridshift appears to be LGPL licensed
> (https://github.com/floscher/jGridShift/blob/master/LICENSE). You may
> not use this software. It looks like it was not appropriately marked in
> its pom which is why the configuration from Rya's parent apache.pom did
> not catch it. This is brought in via org.geotools.xsd:gt-xsd-gml3.
> * colt (http://dst.lbl.gov/ACSSoftware/colt/) appears to be another
> brought in by com.tinkerpop.blueprints:blueprints-core
> * com.google.code.findbugs:jsr305 is another example of GPL licensing.
> While the artifact appears to have the ASL tagged on the pom, all
> Findbugs documentation states that the project is GPL.
> I would recommend to make a pass over your dependencies to verify that
> you aren't depending on any projects which are licensed with a license
> on this list: http://www.apache.org/legal/resolved.html#category-x. See
> http://www.apache.org/licenses/GPL-compatibility.html for more details.
> The above three examples were found via a brief glance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-144) New Secondary Index: Smart URI

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15497602#comment-15497602
 ] 

ASF GitHub Bot commented on RYA-144:


GitHub user kchilton2 opened a pull request:

https://github.com/apache/incubator-rya/pull/96

[WIP] RYA-144 Entity index.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kchilton2/incubator-rya RYA-144

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/96.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #96


commit ffdb0569a9327020242c0a4decab86021e478a08
Author: Kevin Chilton 
Date:   2016-09-16T22:22:54Z

RYA-144 Entity index.




> New Secondary Index: Smart URI
> --
>
> Key: RYA-144
> URL: https://issues.apache.org/jira/browse/RYA-144
> Project: Rya
>  Issue Type: New Feature
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
>
> This is intended primarily for the Mongo implementation of Rya.
> A purpose of this is so that we can discover it though the SPARQL query 
> interface.
> Suppose you have a star entity named 'David':
> David is a person
> david has an SSN
> david has a birthday, 
> So a URI holding all of this information about David could look like this:
> HTTP://Person=david=111-11-=20160101
> This makes it so that instead of using 3 joins it is only a scan.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-151) Rya Test Bed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426397#comment-15426397
 ] 

ASF GitHub Bot commented on RYA-151:


Github user amihalik commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/68#discussion_r75303476
  
--- Diff: 
extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java
 ---
@@ -0,0 +1,55 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.benchmark.query;
+
+import static org.junit.Assert.assertEquals;
+
+import java.io.ByteArrayInputStream;
+import java.io.InputStream;
+
+import javax.xml.bind.JAXBException;
+
+import 
org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader;
+import org.junit.Test;
+import org.xml.sax.SAXException;
+
+import com.google.common.base.Charsets;
+
+/**
+ * Tests the methods of {@link BenchmarkQueriesReader}.
+ */
+public class BenchmarkQueriesReaderIT {
+
+@Test
+public void load() throws JAXBException, SAXException {
+// Unmarshal some XML.
+final String xml =
+"\n" +
+"\n" +
+"" +
--- End diff --

Can we add attributes to this?  for instance

```xml

   


```


> Rya Test Bed
> 
>
> Key: RYA-151
> URL: https://issues.apache.org/jira/browse/RYA-151
> Project: Rya
>  Issue Type: Wish
>  Components: clients, dao
>Reporter: Puja Valiyil 
>Assignee: Kevin Chilton
>
> It would be nice if there was a test bed checked in with Rya that could be 
> used for benchmark testing.  This test bed should include options for  
> uploading data to a rya instance, then issuing some benchmark queries (and 
> timing them in a way that everyone agrees is sound).  It could be a command 
> line tool.  It should be dao agnostic (work on either mongo, accumulo, etc.). 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (RYA-130) Rya Mongo Fail on Multi Bindingset Joins

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426369#comment-15426369
 ] 

ASF GitHub Bot commented on RYA-130:


Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/57
  
@amihalik can you close this pull request?


> Rya Mongo Fail on Multi Bindingset Joins
> 
>
> Key: RYA-130
> URL: https://issues.apache.org/jira/browse/RYA-130
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Affects Versions: 3.2.9
>Reporter: Aaron Mihalik
>Assignee: Aaron Mihalik
>
> Rya Mongo fails on a fairly straightforward query:
> {code}
> DATA = 
>  .
>  .
>  .
> QUERY = 
>   select ?s ?foo where { 
>   ?s  ?foo . 
>   ?s   }
> {code}
> This is a bug in how Mongo Rya solves joins.  The first Statement Pattern 
> will return multiple binding sets, and try to join it with a single 
> statement.  The Mongo Rya Query Engine can't handle this case.
> Full Code at: 
> https://gist.github.com/amihalik/61cf870650b7443389e7be7600f8502c



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)