[jira] [Created] (RYA-442) KafkaStreams: Implement the StartQuery and StopQuery interactors.

2018-01-11 Thread Kevin Chilton (JIRA)
Kevin Chilton created RYA-442:
-

 Summary: KafkaStreams: Implement the StartQuery and StopQuery 
interactors.
 Key: RYA-442
 URL: https://issues.apache.org/jira/browse/RYA-442
 Project: Rya
  Issue Type: Task
Reporter: Kevin Chilton






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[GitHub] incubator-rya issue #172: RYA-303 Mongo PCJ Support

2018-01-11 Thread asfgit
Github user asfgit commented on the issue:

https://github.com/apache/incubator-rya/pull/172
  

Refer to this link for build results (access rights to CI server needed): 

https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/658/



---


[jira] [Commented] (RYA-303) Mongo PCJ indexer support

2018-01-11 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/RYA-303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323296#comment-16323296
 ] 

ASF GitHub Bot commented on RYA-303:


Github user asfgit commented on the issue:

https://github.com/apache/incubator-rya/pull/172
  

Refer to this link for build results (access rights to CI server needed): 

https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/658/



> Mongo PCJ indexer support
> -
>
> Key: RYA-303
> URL: https://issues.apache.org/jira/browse/RYA-303
> Project: Rya
>  Issue Type: Improvement
>Reporter: Andrew Smith
>Assignee: Andrew Smith
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (RYA-303) Mongo PCJ indexer support

2018-01-11 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/RYA-303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323285#comment-16323285
 ] 

ASF GitHub Bot commented on RYA-303:


Github user asfgit commented on the issue:

https://github.com/apache/incubator-rya/pull/172
  

Refer to this link for build results (access rights to CI server needed): 

https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/657/



> Mongo PCJ indexer support
> -
>
> Key: RYA-303
> URL: https://issues.apache.org/jira/browse/RYA-303
> Project: Rya
>  Issue Type: Improvement
>Reporter: Andrew Smith
>Assignee: Andrew Smith
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[GitHub] incubator-rya issue #172: RYA-303 Mongo PCJ Support

2018-01-11 Thread asfgit
Github user asfgit commented on the issue:

https://github.com/apache/incubator-rya/pull/172
  

Refer to this link for build results (access rights to CI server needed): 

https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/657/



---


[jira] [Created] (RYA-441) Mongo PCJ indexers need to be able to support Authoizations

2018-01-11 Thread Andrew Smith (JIRA)
Andrew Smith created RYA-441:


 Summary: Mongo PCJ indexers need to be able to support 
Authoizations
 Key: RYA-441
 URL: https://issues.apache.org/jira/browse/RYA-441
 Project: Rya
  Issue Type: Improvement
Reporter: Andrew Smith






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (RYA-303) Mongo PCJ indexer support

2018-01-11 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/RYA-303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323181#comment-16323181
 ] 

ASF GitHub Bot commented on RYA-303:


Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/172#discussion_r161103080
  
--- Diff: 
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/mongo/MongoPcjDocuments.java
 ---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.mongo;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.api.domain.RyaType;
+import org.apache.rya.api.model.VisibilityBindingSet;
+import org.apache.rya.api.resolver.RdfToRyaConversions;
+import org.apache.rya.api.resolver.RyaToRdfConversions;
+import org.apache.rya.api.utils.CloseableIterator;
+import org.apache.rya.indexing.pcj.storage.PcjMetadata;
+import 
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.PCJStorageException;
+import org.apache.rya.indexing.pcj.storage.accumulo.PcjVarOrderFactory;
+import org.apache.rya.indexing.pcj.storage.accumulo.ShiftVarOrderFactory;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.bson.Document;
+import org.bson.conversions.Bson;
+import org.openrdf.model.URI;
+import org.openrdf.model.Value;
+import org.openrdf.model.impl.URIImpl;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.QueryEvaluationException;
+import org.openrdf.query.QueryLanguage;
+import org.openrdf.query.TupleQuery;
+import org.openrdf.query.TupleQueryResult;
+import org.openrdf.query.impl.MapBindingSet;
+import org.openrdf.repository.RepositoryConnection;
+import org.openrdf.repository.RepositoryException;
+
+import com.mongodb.MongoClient;
+import com.mongodb.client.FindIterable;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.util.JSON;
+
+/**
+ * Creates and modifies PCJs in MongoDB. PCJ's are stored as follows:
+ *
+ * 
+ * 
+ * - PCJ Metadata Doc -
+ * {
+ *   _id: [pcj_ID]_METADATA,
+ *   sparql: [sparql query to match results],
+ *   varOrders: [varOrder1, VarOrder2, ..., VarOrdern]
+ *   cardinality: [number of results]
+ * }
+ *
+ * - PCJ Results Doc -
+ * {
+ *   pcjId: [pcj_ID],
+ *   visibilities: [visibilities]
+ *   [binding_var1]: {
+ * uri: [type_uri],
+ * value: [value]
+ *   }
+ *   .
+ *   .
+ *   .
+ *   [binding_varn]: {
+ * uri: [type_uri],
+ * value: [value]
+ *   }
+ * }
+ * 
+ * 
+ */
+public class MongoPcjDocuments {
+public static final String PCJ_COLLECTION_NAME = "pcjs";
+
+// metadata fields
+public static final String CARDINALITY_FIELD = "cardinality";
+public static final String SPARQL_FIELD = "sparql";
+public static final String PCJ_METADATA_ID = "_id";
+public static final String VAR_ORDER_FIELD = "varOrders";
+
+// pcj results fields
+private static final String BINDING_VALUE = "value";
+private static final String BINDING_TYPE = "rdfType";
+private static final String VISIBILITIES_FIELD = "visibilities";
+private static final String PCJ_ID = "pcjId";
+
+private final MongoCollection pcjCollection;
+private static final PcjVarOrderFactory pcjVarOrderFactory = new 
ShiftVarOrderFactory();
+
+/**
+ * Creates a new {@link MongoPcjDocuments}.
 

[GitHub] incubator-rya pull request #172: RYA-303 Mongo PCJ Support

2018-01-11 Thread isper3at
Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/172#discussion_r161103080
  
--- Diff: 
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/mongo/MongoPcjDocuments.java
 ---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.mongo;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.api.domain.RyaType;
+import org.apache.rya.api.model.VisibilityBindingSet;
+import org.apache.rya.api.resolver.RdfToRyaConversions;
+import org.apache.rya.api.resolver.RyaToRdfConversions;
+import org.apache.rya.api.utils.CloseableIterator;
+import org.apache.rya.indexing.pcj.storage.PcjMetadata;
+import 
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.PCJStorageException;
+import org.apache.rya.indexing.pcj.storage.accumulo.PcjVarOrderFactory;
+import org.apache.rya.indexing.pcj.storage.accumulo.ShiftVarOrderFactory;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.bson.Document;
+import org.bson.conversions.Bson;
+import org.openrdf.model.URI;
+import org.openrdf.model.Value;
+import org.openrdf.model.impl.URIImpl;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.QueryEvaluationException;
+import org.openrdf.query.QueryLanguage;
+import org.openrdf.query.TupleQuery;
+import org.openrdf.query.TupleQueryResult;
+import org.openrdf.query.impl.MapBindingSet;
+import org.openrdf.repository.RepositoryConnection;
+import org.openrdf.repository.RepositoryException;
+
+import com.mongodb.MongoClient;
+import com.mongodb.client.FindIterable;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.util.JSON;
+
+/**
+ * Creates and modifies PCJs in MongoDB. PCJ's are stored as follows:
+ *
+ * 
+ * 
+ * - PCJ Metadata Doc -
+ * {
+ *   _id: [pcj_ID]_METADATA,
+ *   sparql: [sparql query to match results],
+ *   varOrders: [varOrder1, VarOrder2, ..., VarOrdern]
+ *   cardinality: [number of results]
+ * }
+ *
+ * - PCJ Results Doc -
+ * {
+ *   pcjId: [pcj_ID],
+ *   visibilities: [visibilities]
+ *   [binding_var1]: {
+ * uri: [type_uri],
+ * value: [value]
+ *   }
+ *   .
+ *   .
+ *   .
+ *   [binding_varn]: {
+ * uri: [type_uri],
+ * value: [value]
+ *   }
+ * }
+ * 
+ * 
+ */
+public class MongoPcjDocuments {
+public static final String PCJ_COLLECTION_NAME = "pcjs";
+
+// metadata fields
+public static final String CARDINALITY_FIELD = "cardinality";
+public static final String SPARQL_FIELD = "sparql";
+public static final String PCJ_METADATA_ID = "_id";
+public static final String VAR_ORDER_FIELD = "varOrders";
+
+// pcj results fields
+private static final String BINDING_VALUE = "value";
+private static final String BINDING_TYPE = "rdfType";
+private static final String VISIBILITIES_FIELD = "visibilities";
+private static final String PCJ_ID = "pcjId";
+
+private final MongoCollection pcjCollection;
+private static final PcjVarOrderFactory pcjVarOrderFactory = new 
ShiftVarOrderFactory();
+
+/**
+ * Creates a new {@link MongoPcjDocuments}.
+ * @param client - The {@link MongoClient} to use to connect to mongo.
+ * @param ryaInstanceName - The rya instance to connect to.
+ */
+public MongoPcjDocuments(final MongoClient client, final 

[jira] [Updated] (RYA-440) Rya Streams: Integration with the existing Rya Shell.

2018-01-11 Thread Kevin Chilton (JIRA)

 [ 
https://issues.apache.org/jira/browse/RYA-440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Chilton updated RYA-440:
--
Description: 
Once Rya Streams is deployed, we will need a way to interact with it. The shell 
should include the following commands:

* *streams-configure*: Lets you change how the Rya instance interacts with 
backend Rya Streams components. For example, if you are using Kafka Streams, 
this command would let you configure where the Kafka services are found.
* *streams-details*: Prints to the screen the currently configured values.
* *streams-queries-add*: Tells Rya Streams that it needs to maintain a query
* *streams-queries-delete*: Tells Rya Streams that it no longer needs to 
maintain a query.
* *streams-queries-start*: Tells Rya Streams to start maintaining a query.
* *streams-queries-stop*: Tells Rya Streams to stop maintaining a query.
* *streams-queries-list*: Lists the queries that are being maintained by Rya 
Streams as well as their start/stop state.

> Rya Streams: Integration with the existing Rya Shell.
> -
>
> Key: RYA-440
> URL: https://issues.apache.org/jira/browse/RYA-440
> Project: Rya
>  Issue Type: Task
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
>
> Once Rya Streams is deployed, we will need a way to interact with it. The 
> shell should include the following commands:
> * *streams-configure*: Lets you change how the Rya instance interacts with 
> backend Rya Streams components. For example, if you are using Kafka Streams, 
> this command would let you configure where the Kafka services are found.
> * *streams-details*: Prints to the screen the currently configured values.
> * *streams-queries-add*: Tells Rya Streams that it needs to maintain a query
> * *streams-queries-delete*: Tells Rya Streams that it no longer needs to 
> maintain a query.
> * *streams-queries-start*: Tells Rya Streams to start maintaining a query.
> * *streams-queries-stop*: Tells Rya Streams to stop maintaining a query.
> * *streams-queries-list*: Lists the queries that are being maintained by Rya 
> Streams as well as their start/stop state.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (RYA-440) Rya Streams: Integration with the existing Rya Shell.

2018-01-11 Thread Kevin Chilton (JIRA)
Kevin Chilton created RYA-440:
-

 Summary: Rya Streams: Integration with the existing Rya Shell.
 Key: RYA-440
 URL: https://issues.apache.org/jira/browse/RYA-440
 Project: Rya
  Issue Type: Task
Reporter: Kevin Chilton
Assignee: Kevin Chilton






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (RYA-303) Mongo PCJ indexer support

2018-01-11 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/RYA-303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323011#comment-16323011
 ] 

ASF GitHub Bot commented on RYA-303:


Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/172#discussion_r161082144
  
--- Diff: 
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/mongo/MongoPcjDocuments.java
 ---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.mongo;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.api.domain.RyaType;
+import org.apache.rya.api.model.VisibilityBindingSet;
+import org.apache.rya.api.resolver.RdfToRyaConversions;
+import org.apache.rya.api.resolver.RyaToRdfConversions;
+import org.apache.rya.api.utils.CloseableIterator;
+import org.apache.rya.indexing.pcj.storage.PcjMetadata;
+import 
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.PCJStorageException;
+import org.apache.rya.indexing.pcj.storage.accumulo.PcjVarOrderFactory;
+import org.apache.rya.indexing.pcj.storage.accumulo.ShiftVarOrderFactory;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.bson.Document;
+import org.bson.conversions.Bson;
+import org.openrdf.model.URI;
+import org.openrdf.model.Value;
+import org.openrdf.model.impl.URIImpl;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.QueryEvaluationException;
+import org.openrdf.query.QueryLanguage;
+import org.openrdf.query.TupleQuery;
+import org.openrdf.query.TupleQueryResult;
+import org.openrdf.query.impl.MapBindingSet;
+import org.openrdf.repository.RepositoryConnection;
+import org.openrdf.repository.RepositoryException;
+
+import com.mongodb.MongoClient;
+import com.mongodb.client.FindIterable;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.util.JSON;
+
+/**
+ * Creates and modifies PCJs in MongoDB. PCJ's are stored as follows:
+ *
+ * 
+ * 
+ * - PCJ Metadata Doc -
+ * {
+ *   _id: [pcj_ID]_METADATA,
+ *   sparql: [sparql query to match results],
+ *   varOrders: [varOrder1, VarOrder2, ..., VarOrdern]
+ *   cardinality: [number of results]
+ * }
+ *
+ * - PCJ Results Doc -
+ * {
+ *   pcjId: [pcj_ID],
+ *   visibilities: [visibilities]
+ *   [binding_var1]: {
+ * uri: [type_uri],
+ * value: [value]
+ *   }
+ *   .
+ *   .
+ *   .
+ *   [binding_varn]: {
+ * uri: [type_uri],
+ * value: [value]
+ *   }
+ * }
+ * 
+ * 
+ */
+public class MongoPcjDocuments {
+public static final String PCJ_COLLECTION_NAME = "pcjs";
+
+// metadata fields
+public static final String CARDINALITY_FIELD = "cardinality";
+public static final String SPARQL_FIELD = "sparql";
+public static final String PCJ_METADATA_ID = "_id";
+public static final String VAR_ORDER_FIELD = "varOrders";
+
+// pcj results fields
+private static final String BINDING_VALUE = "value";
+private static final String BINDING_TYPE = "rdfType";
+private static final String VISIBILITIES_FIELD = "visibilities";
+private static final String PCJ_ID = "pcjId";
+
+private final MongoCollection pcjCollection;
+private static final PcjVarOrderFactory pcjVarOrderFactory = new 
ShiftVarOrderFactory();
+
+/**
+ * Creates a new {@link MongoPcjDocuments}.

[GitHub] incubator-rya pull request #172: RYA-303 Mongo PCJ Support

2018-01-11 Thread kchilton2
Github user kchilton2 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/172#discussion_r161082144
  
--- Diff: 
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/mongo/MongoPcjDocuments.java
 ---
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.mongo;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.api.domain.RyaType;
+import org.apache.rya.api.model.VisibilityBindingSet;
+import org.apache.rya.api.resolver.RdfToRyaConversions;
+import org.apache.rya.api.resolver.RyaToRdfConversions;
+import org.apache.rya.api.utils.CloseableIterator;
+import org.apache.rya.indexing.pcj.storage.PcjMetadata;
+import 
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.PCJStorageException;
+import org.apache.rya.indexing.pcj.storage.accumulo.PcjVarOrderFactory;
+import org.apache.rya.indexing.pcj.storage.accumulo.ShiftVarOrderFactory;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.bson.Document;
+import org.bson.conversions.Bson;
+import org.openrdf.model.URI;
+import org.openrdf.model.Value;
+import org.openrdf.model.impl.URIImpl;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.QueryEvaluationException;
+import org.openrdf.query.QueryLanguage;
+import org.openrdf.query.TupleQuery;
+import org.openrdf.query.TupleQueryResult;
+import org.openrdf.query.impl.MapBindingSet;
+import org.openrdf.repository.RepositoryConnection;
+import org.openrdf.repository.RepositoryException;
+
+import com.mongodb.MongoClient;
+import com.mongodb.client.FindIterable;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.util.JSON;
+
+/**
+ * Creates and modifies PCJs in MongoDB. PCJ's are stored as follows:
+ *
+ * 
+ * 
+ * - PCJ Metadata Doc -
+ * {
+ *   _id: [pcj_ID]_METADATA,
+ *   sparql: [sparql query to match results],
+ *   varOrders: [varOrder1, VarOrder2, ..., VarOrdern]
+ *   cardinality: [number of results]
+ * }
+ *
+ * - PCJ Results Doc -
+ * {
+ *   pcjId: [pcj_ID],
+ *   visibilities: [visibilities]
+ *   [binding_var1]: {
+ * uri: [type_uri],
+ * value: [value]
+ *   }
+ *   .
+ *   .
+ *   .
+ *   [binding_varn]: {
+ * uri: [type_uri],
+ * value: [value]
+ *   }
+ * }
+ * 
+ * 
+ */
+public class MongoPcjDocuments {
+public static final String PCJ_COLLECTION_NAME = "pcjs";
+
+// metadata fields
+public static final String CARDINALITY_FIELD = "cardinality";
+public static final String SPARQL_FIELD = "sparql";
+public static final String PCJ_METADATA_ID = "_id";
+public static final String VAR_ORDER_FIELD = "varOrders";
+
+// pcj results fields
+private static final String BINDING_VALUE = "value";
+private static final String BINDING_TYPE = "rdfType";
+private static final String VISIBILITIES_FIELD = "visibilities";
+private static final String PCJ_ID = "pcjId";
+
+private final MongoCollection pcjCollection;
+private static final PcjVarOrderFactory pcjVarOrderFactory = new 
ShiftVarOrderFactory();
+
+/**
+ * Creates a new {@link MongoPcjDocuments}.
+ * @param client - The {@link MongoClient} to use to connect to mongo.
+ * @param ryaInstanceName - The rya instance to connect to.
+ */
+public MongoPcjDocuments(final MongoClient client, final 

[jira] [Created] (RYA-439) Update RyaDetails to allow Entity Indexer

2018-01-11 Thread Andrew Smith (JIRA)
Andrew Smith created RYA-439:


 Summary: Update RyaDetails to allow Entity Indexer
 Key: RYA-439
 URL: https://issues.apache.org/jira/browse/RYA-439
 Project: Rya
  Issue Type: Sub-task
Reporter: Andrew Smith
Assignee: Andrew Smith






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (RYA-438) Create a client command to delete an Entity Type

2018-01-11 Thread Andrew Smith (JIRA)
Andrew Smith created RYA-438:


 Summary: Create a client command to delete an Entity Type
 Key: RYA-438
 URL: https://issues.apache.org/jira/browse/RYA-438
 Project: Rya
  Issue Type: Sub-task
Reporter: Andrew Smith






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (RYA-437) Create an interactor for deleting an Entity Type

2018-01-11 Thread Andrew Smith (JIRA)
Andrew Smith created RYA-437:


 Summary: Create an interactor for deleting an Entity Type
 Key: RYA-437
 URL: https://issues.apache.org/jira/browse/RYA-437
 Project: Rya
  Issue Type: Sub-task
Reporter: Andrew Smith






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (RYA-436) Create a client command to Batch Update an Entity Type

2018-01-11 Thread Andrew Smith (JIRA)
Andrew Smith created RYA-436:


 Summary: Create a client command to Batch Update an Entity Type
 Key: RYA-436
 URL: https://issues.apache.org/jira/browse/RYA-436
 Project: Rya
  Issue Type: Sub-task
Reporter: Andrew Smith






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (RYA-435) Create a client command to list existing Entity Types

2018-01-11 Thread Andrew Smith (JIRA)
Andrew Smith created RYA-435:


 Summary: Create a client command to list existing Entity Types
 Key: RYA-435
 URL: https://issues.apache.org/jira/browse/RYA-435
 Project: Rya
  Issue Type: Sub-task
Reporter: Andrew Smith






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (RYA-433) Create an interactor for listing existing Types

2018-01-11 Thread Andrew Smith (JIRA)
Andrew Smith created RYA-433:


 Summary: Create an interactor for listing existing Types
 Key: RYA-433
 URL: https://issues.apache.org/jira/browse/RYA-433
 Project: Rya
  Issue Type: Sub-task
Reporter: Andrew Smith






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (RYA-434) Create an Interactor for Batch Updating for a Type

2018-01-11 Thread Andrew Smith (JIRA)
Andrew Smith created RYA-434:


 Summary: Create an Interactor for Batch Updating for a Type
 Key: RYA-434
 URL: https://issues.apache.org/jira/browse/RYA-434
 Project: Rya
  Issue Type: Sub-task
Reporter: Andrew Smith






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (RYA-428) Mongo Shell: Can not load statements file.

2018-01-11 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/RYA-428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322356#comment-16322356
 ] 

ASF GitHub Bot commented on RYA-428:


Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/262


> Mongo Shell: Can not load statements file.
> --
>
> Key: RYA-428
> URL: https://issues.apache.org/jira/browse/RYA-428
> Project: Rya
>  Issue Type: Bug
>Affects Versions: 3.2.12
> Environment: Executing the shaded shell jar connecting to Mongo DB 
> running on my local machine.
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
> Fix For: 3.2.12
>
> Attachments: test1.nt
>
>
> While using the Rya shell, if you connect to an instance of Mongo DB and try 
> to load a statements file using the following command:
> {code}
> rya/localhost:test> load-data --file ~/test1.nt
> {code}
> Then you will see the following output:
> {code}
> Detected RDF Format: N-Triples (mimeTypes=text/plain; ext=nt)
> Command failed java.lang.RuntimeException: Can not load the RDF Statement 
> data. Reason: Could not load the statements into Rya.
> {code}
> See the attached test1.nt file for the data that was used while executing 
> this test.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (RYA-416) Add the ability use the MongoDB aggregation pipeline to evaluate simple SPARQL expressions

2018-01-11 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/RYA-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322357#comment-16322357
 ] 

ASF GitHub Bot commented on RYA-416:


Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/254


> Add the ability use the MongoDB aggregation pipeline to evaluate simple 
> SPARQL expressions
> --
>
> Key: RYA-416
> URL: https://issues.apache.org/jira/browse/RYA-416
> Project: Rya
>  Issue Type: New Feature
>Reporter: Jesse Hatfield
>Assignee: Jesse Hatfield
>
> MongoDB provides the [aggregation pipeline 
> framework|https://docs.mongodb.com/manual/core/aggregation-pipeline/] for 
> multi-stage data processing. Currently, the query engine invokes this 
> framework to apply individual statement patterns (using a "$match" expression 
> for each and iterating through the results), then applies higher-level query 
> operations (join, filter, select, project, etc) client-side.
> In principle, those high-level query operations could be rewritten as 
> aggregation pipeline stages as well ($group, $match, $project, etc). This 
> would allow more query evaluation logic to be executed by the MongoDB server 
> itself, enabling server-side optimization. This could be used as a general 
> query optimization, but would additionally be useful for any tool that only 
> needed to write query results back to the server: adding a write step to the 
> end of the resulting pipeline could obviate the need to communicate 
> individual results to the client at all.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (RYA-417) Implement a forward-chaining rules engine (SPIN)

2018-01-11 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/RYA-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322358#comment-16322358
 ] 

ASF GitHub Bot commented on RYA-417:


Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/255


> Implement a forward-chaining rules engine (SPIN)
> 
>
> Key: RYA-417
> URL: https://issues.apache.org/jira/browse/RYA-417
> Project: Rya
>  Issue Type: New Feature
>Reporter: Jesse Hatfield
>Assignee: Jesse Hatfield
>
> Implement a forward-chaining reasoner that:
> * Runs as a batch process
> * Operates on user-defined [SPIN|http://spinrdf.org] rules
> * Inserts derived information back into Rya
> * Iterates until no new information can be derived



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[GitHub] incubator-rya pull request #255: RYA-417 Forward-chaining batch rules engine

2018-01-11 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/255


---


[GitHub] incubator-rya pull request #262: RYA-428 Made the Rya Shell's load statement...

2018-01-11 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/262


---


[GitHub] incubator-rya pull request #254: RYA-416 Optionally invoke aggregation pipel...

2018-01-11 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/254


---