[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[meiercaleb]

Github user meiercaleb commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r119368692
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementCursorIterator.java
 ---
@@ -22,83 +40,83 @@
 
 import info.aduna.iteration.CloseableIteration;
 
-import java.util.Iterator;
-import java.util.Map.Entry;
-import java.util.Set;
-
-import org.apache.rya.api.RdfCloudTripleStoreUtils;
-import org.apache.rya.api.domain.RyaStatement;
-import org.apache.rya.api.persist.RyaDAOException;
-import org.apache.rya.mongodb.dao.MongoDBStorageStrategy;
-
-import org.calrissian.mango.collect.CloseableIterable;
-import org.openrdf.query.BindingSet;
-
-import com.mongodb.DBCollection;
-import com.mongodb.DBCursor;
-import com.mongodb.DBObject;
-
 public class RyaStatementCursorIterator implements 
CloseableIteration {
-
-   private DBCollection coll;
-   private Iterator queryIterator;
-   private DBCursor currentCursor;
-   private MongoDBStorageStrategy strategy;
-   private Long maxResults;
-
-   public RyaStatementCursorIterator(DBCollection coll, Set 
queries, MongoDBStorageStrategy strategy) {
-   this.coll = coll;
-   this.queryIterator = queries.iterator();
-   this.strategy = strategy;
-   }
-
-   @Override
-   public boolean hasNext() {
-   if (!currentCursorIsValid()) {
-   findNextValidCursor();
-   }
-   return currentCursorIsValid();
-   }
-
-   @Override
-   public RyaStatement next() {
-   if (!currentCursorIsValid()) {
-   findNextValidCursor();
-   }
-   if (currentCursorIsValid()) {
-   // convert to Rya Statement
-   DBObject queryResult = currentCursor.next();
-   RyaStatement statement = 
strategy.deserializeDBObject(queryResult);
-   return statement;
-   }
-   return null;
-   }
-   
-   private void findNextValidCursor() {
-   while (queryIterator.hasNext()){
-   DBObject currentQuery = queryIterator.next();
-   currentCursor = coll.find(currentQuery);
-   if (currentCursor.hasNext()) break;
-   }
-   }
-   
-   private boolean currentCursorIsValid() {
-   return (currentCursor != null) && currentCursor.hasNext();
-   }
-
-
-   public void setMaxResults(Long maxResults) {
-   this.maxResults = maxResults;
-   }
-
-   @Override
-   public void close() throws RyaDAOException {
-   // TODO don't know what to do here
-   }
-
-   @Override
-   public void remove() throws RyaDAOException {
-   next();
-   }
-
+private static final Logger log = 
Logger.getLogger(RyaStatementCursorIterator.class);
+
+private final DBCollection coll;
+private final Iterator queryIterator;
+private Iterator resultsIterator;
+private final MongoDBStorageStrategy strategy;
+private Long maxResults;
+private final Authorizations auths;
+
+public RyaStatementCursorIterator(final DBCollection coll, final 
Set queries, final MongoDBStorageStrategy strategy, 
final MongoDBRdfConfiguration conf) {
+this.coll = coll;
+this.queryIterator = queries.iterator();
+this.strategy = strategy;
+if (conf != null) {
+this.auths = conf.getAuthorizations();
+} else {
+auths = MongoDbRdfConstants.ALL_AUTHORIZATIONS;
+}
+}
+
+@Override
+public boolean hasNext() {
+if (!currentCursorIsValid()) {
+findNextValidCursor();
+}
+return currentCursorIsValid();
+}
+
+@Override
+public RyaStatement next() {
+if (!currentCursorIsValid()) {
+findNextValidCursor();
+}
+if (currentCursorIsValid()) {
+// convert to Rya Statement
+final DBObject queryResult = resultsIterator.next();
+final RyaStatement statement = 
strategy.deserializeDBObject(queryResult);
+return statement;
+}
+return null;
+}
+
+private void findNextValidCursor() {
+while (queryIterator.hasNext()){
+final DBObject currentQuery = queryIterator.next();
+
+// Executing redact aggregation to only return documents the 
user
+// has access to.
+final List pipeline = new 

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/124


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r94812491
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/DocumentVisibilityAdapter.java
 ---
@@ -0,0 +1,137 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.dao.SimpleMongoDBStorageStrategy;
+import 
org.apache.rya.mongodb.document.util.DocumentVisibilityConversionException;
+import org.apache.rya.mongodb.document.util.DocumentVisibilityUtil;
+
+import com.mongodb.BasicDBList;
+import com.mongodb.BasicDBObject;
+import com.mongodb.BasicDBObjectBuilder;
+import com.mongodb.DBObject;
+
+import edu.umd.cs.findbugs.annotations.DefaultAnnotation;
+import edu.umd.cs.findbugs.annotations.NonNull;
+
+/**
+ * Serializes the document visibility field of a Rya Statement for use in
+ * MongoDB.
+ * The {@link DBObject} will look like:
+ * 
+ * {@code
+ * {
+ *   "documentVisibility": array,
+ * }
+ * 
+ */
+@DefaultAnnotation(NonNull.class)
+public class DocumentVisibilityAdapter {
+private static final Logger log = 
Logger.getLogger(DocumentVisibilityAdapter.class);
+
+public static final String DOCUMENT_VISIBILITY_KEY = 
SimpleMongoDBStorageStrategy.DOCUMENT_VISIBILITY;
+
+/**
+ * Serializes a document visibility expression byte array to a MongoDB
+ * {@link DBObject}.
+ * @param expression the document visibility expression byte array to 
be
+ * serialized.
+ * @return The MongoDB {@link DBObject}.
+ */
+public static BasicDBObject toDBObject(final byte[] expression) {
--- End diff --

generally, we don't make the adapters static.  It makes writing unit tests 
for them a lot easier.  Do tests exist for the adapter?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r94812004
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/operators/aggregation/PipelineOperators.java
 ---
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.operators.aggregation;
+
+import static 
org.apache.rya.mongodb.document.operators.query.ConditionalOperators.cond;
+
+import com.mongodb.BasicDBObject;
+import com.mongodb.BasicDBObjectBuilder;
+
+/**
+ * Utility methods for pipeline operators.
+ */
+public final class PipelineOperators {
--- End diff --

For all the *Operators, I think it would be helpful to make the functions 
return the actual BasicDBObjectBuilder.  That way you can supply each function 
with a builder and it appends the operation to it, making it a lot easier to 
spin up more complicated queries.

If you have time to do this, awesome, if not we can open a ticket


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r94617786
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRyaDAO.java ---
@@ -146,15 +153,18 @@ public void destroy() throws RyaDAOException {
 public void add(final RyaStatement statement) throws RyaDAOException {
 // add it to the collection
 try {
-coll.insert(storageStrategy.serialize(statement));
-for(final RyaSecondaryIndexer index: secondaryIndexers) {
-index.storeStatement(statement);
+final boolean canAdd = 
DocumentVisibilityUtil.doesUserHaveDocumentAccess(auths, 
statement.getColumnVisibility());
--- End diff --

Added exceptions for add and delete if they don't have the required 
authorizations.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90357571
  
--- Diff: 
dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRyaDAOTest.java ---
@@ -28,102 +30,279 @@
 import org.apache.rya.api.domain.RyaStatement.RyaStatementBuilder;
 import org.apache.rya.api.domain.RyaURI;
 import org.apache.rya.api.persist.RyaDAOException;
+import org.apache.rya.api.persist.query.RyaQuery;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.bson.Document;
+import org.calrissian.mango.collect.CloseableIterable;
 import org.junit.Before;
 import org.junit.Test;
 
-import com.mongodb.DB;
-import com.mongodb.DBCollection;
 import com.mongodb.MongoException;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.client.MongoDatabase;
 
 public class MongoDBRyaDAOTest extends MongoRyaTestBase {
 
-   private MongoDBRyaDAO dao;
-   private MongoDBRdfConfiguration configuration;
+private MongoDBRyaDAO dao;
+private MongoDBRdfConfiguration configuration;
 
-   @Before
-   public void setUp() throws IOException, RyaDAOException{
-   final Configuration conf = new Configuration();
+@Before
+public void setUp() throws IOException, RyaDAOException{
+final Configuration conf = new Configuration();
 conf.set(MongoDBRdfConfiguration.MONGO_DB_NAME, "test");
 conf.set(MongoDBRdfConfiguration.MONGO_COLLECTION_PREFIX, "rya_");
 conf.set(RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX, "rya_");
 configuration = new MongoDBRdfConfiguration(conf);
 final int port = 
mongoClient.getServerAddressList().get(0).getPort();
 configuration.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, 
""+port);
-   dao = new MongoDBRyaDAO(configuration, mongoClient);
-   }
+dao = new MongoDBRyaDAO(configuration, mongoClient);
+}
 
 
-   @Test
-   public void testDeleteWildcard() throws RyaDAOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   dao.delete(builder.build(), configuration);
-   }
+@Test
+public void testDeleteWildcard() throws RyaDAOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+dao.delete(builder.build(), configuration);
+}
 
 
-   @Test
-   public void testAdd() throws RyaDAOException, MongoException, 
IOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   builder.setSubject(new RyaURI("http://subject.com;));
-   builder.setObject(new RyaURI("http://object.com;));
+@Test
+public void testAdd() throws RyaDAOException, MongoException, 
IOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+builder.setSubject(new RyaURI("http://subject.com;));
+builder.setObject(new RyaURI("http://object.com;));
 
-   final DB db = 
mongoClient.getDB(configuration.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
-final DBCollection coll = 
db.getCollection(configuration.getTriplesCollectionName());
+final MongoDatabase db = 
mongoClient.getDatabase(configuration.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
+final MongoCollection coll = 
db.getCollection(configuration.getTriplesCollectionName());
 
-   dao.add(builder.build());
+dao.add(builder.build());
 
 assertEquals(coll.count(),1);
 
-   }
+}
 
-   @Test
-   public void testDelete() throws RyaDAOException, MongoException, 
IOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   builder.setSubject(new RyaURI("http://subject.com;));
-   builder.setObject(new RyaURI("http://object.com;));
-   final RyaStatement statement = builder.build();
+@Test
+public void testDelete() throws RyaDAOException, MongoException, 
IOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+builder.setSubject(new RyaURI("http://subject.com;));
+builder.setObject(new RyaURI("http://object.com;));
+final RyaStatement statement = builder.build();
 
-   final DB db 

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90357458
  
--- Diff: 
dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRyaDAOTest.java ---
@@ -28,102 +30,279 @@
 import org.apache.rya.api.domain.RyaStatement.RyaStatementBuilder;
 import org.apache.rya.api.domain.RyaURI;
 import org.apache.rya.api.persist.RyaDAOException;
+import org.apache.rya.api.persist.query.RyaQuery;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.bson.Document;
+import org.calrissian.mango.collect.CloseableIterable;
 import org.junit.Before;
 import org.junit.Test;
 
-import com.mongodb.DB;
-import com.mongodb.DBCollection;
 import com.mongodb.MongoException;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.client.MongoDatabase;
 
 public class MongoDBRyaDAOTest extends MongoRyaTestBase {
 
-   private MongoDBRyaDAO dao;
-   private MongoDBRdfConfiguration configuration;
+private MongoDBRyaDAO dao;
+private MongoDBRdfConfiguration configuration;
 
-   @Before
-   public void setUp() throws IOException, RyaDAOException{
-   final Configuration conf = new Configuration();
+@Before
+public void setUp() throws IOException, RyaDAOException{
+final Configuration conf = new Configuration();
 conf.set(MongoDBRdfConfiguration.MONGO_DB_NAME, "test");
 conf.set(MongoDBRdfConfiguration.MONGO_COLLECTION_PREFIX, "rya_");
 conf.set(RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX, "rya_");
 configuration = new MongoDBRdfConfiguration(conf);
 final int port = 
mongoClient.getServerAddressList().get(0).getPort();
 configuration.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, 
""+port);
-   dao = new MongoDBRyaDAO(configuration, mongoClient);
-   }
+dao = new MongoDBRyaDAO(configuration, mongoClient);
+}
 
 
-   @Test
-   public void testDeleteWildcard() throws RyaDAOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   dao.delete(builder.build(), configuration);
-   }
+@Test
+public void testDeleteWildcard() throws RyaDAOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+dao.delete(builder.build(), configuration);
+}
 
 
-   @Test
-   public void testAdd() throws RyaDAOException, MongoException, 
IOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   builder.setSubject(new RyaURI("http://subject.com;));
-   builder.setObject(new RyaURI("http://object.com;));
+@Test
+public void testAdd() throws RyaDAOException, MongoException, 
IOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+builder.setSubject(new RyaURI("http://subject.com;));
+builder.setObject(new RyaURI("http://object.com;));
 
-   final DB db = 
mongoClient.getDB(configuration.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
-final DBCollection coll = 
db.getCollection(configuration.getTriplesCollectionName());
+final MongoDatabase db = 
mongoClient.getDatabase(configuration.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
+final MongoCollection coll = 
db.getCollection(configuration.getTriplesCollectionName());
 
-   dao.add(builder.build());
+dao.add(builder.build());
 
 assertEquals(coll.count(),1);
 
-   }
+}
 
-   @Test
-   public void testDelete() throws RyaDAOException, MongoException, 
IOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   builder.setSubject(new RyaURI("http://subject.com;));
-   builder.setObject(new RyaURI("http://object.com;));
-   final RyaStatement statement = builder.build();
+@Test
+public void testDelete() throws RyaDAOException, MongoException, 
IOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+builder.setSubject(new RyaURI("http://subject.com;));
+builder.setObject(new RyaURI("http://object.com;));
+final RyaStatement statement = builder.build();
 
-   final DB db 

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90357174
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/AggregationUtil.java 
---
@@ -0,0 +1,322 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.iter;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.LinkedList;
+import java.util.List;
+
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.dao.SimpleMongoDBStorageStrategy;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+
+import com.google.common.collect.Lists;
+import com.mongodb.BasicDBObject;
+import com.mongodb.DBObject;
+
+/**
+ * Utility methods for MongoDB aggregation.
+ */
+public final class AggregationUtil {
+/**
+ * Private constructor to prevent instantiation.
+ */
+private AggregationUtil() {
+}
+
+/**
+ * Creates a MongoDB $redact aggregation pipeline that only include
+ * documents whose document visibility match the provided 
authorizations.
+ * All other documents are excluded.
+ * @param authorizations the {@link Authorization}s to include in the
+ * $redact. Only documents that match the authorizations will be 
returned.
+ * @return the {@link List} of {@link DBObject}s that represents the 
$redact
+ * aggregation pipeline.
+ */
+public static List createRedactPipeline(final Authorizations 
authorizations) {
+if (MongoDbRdfConstants.ALL_AUTHORIZATIONS.equals(authorizations)) 
{
+return Lists.newArrayList();
+}
+final List authAndList = 
authorizations.getAuthorizationsStrings();
+
+// Generate all combinations of the authorization strings without 
repetition.
+final List authOrList = 
createCombinations(authorizations.getAuthorizationsStrings());
+
+final String documentVisibilityField = "$" + 
SimpleMongoDBStorageStrategy.DOCUMENT_VISIBILITY;
+
+final BasicDBObject setIsSubset =
+setIsSubsetNullSafe(
+documentVisibilityField,
+authAndList.toArray()
+);
+
+final BasicDBObject setIntersectionExists =
+gt(
+size(
+setIntersection(
+documentVisibilityField,
+authOrList.toArray()
+)
+),
+0
+);
+
+final BasicDBObject orExpression = or(setIsSubset, 
setIntersectionExists);
+
+final List pipeline = new ArrayList<>();
+pipeline.add(
+redact(
+   orExpression,
+   "$$DESCEND",
+   "$$PRUNE"
+)
+);
+
+return pipeline;
+}
+
+/**
+ * Creates all combinations of the values that are of the size of value
+ * array or smaller without repetition.
+ * @param values the {@link List} of values to create combinations 
from.
+ * @return the {@link List} of combinations.
+ */
+public static  List createCombinations(final List 
values) {
+final List allCombinations = new ArrayList<>();
+for (int i = 1; i <= values.size(); i++) {
+allCombinations.addAll(createCombinations(values, i));
+}
+return allCombinations;
+}
+
+/**
+ * Creates all combinations of the values that are of the specified 
size
+ * without repetition.
+ * @param values the {@link List} of values to create combinations 
from.
+ * @param size the size of the combinations.
+ * @return the {@link List} of combinations.
   

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90343835
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final byte[] auth : authorizations) {
+   auths.add(new ArrayByteSequence(auth));
+   }
+checkAuths();
+  }
+
+  /**
+   * Constructs an authorization object from a list of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does not
+   * verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  list of authorizations, as strings encoded in UTF-8 and 
placed in buffers
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final List authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final ByteBuffer buffer : authorizations) {
+  

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90343779
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final byte[] auth : authorizations) {
+   auths.add(new ArrayByteSequence(auth));
+   }
+checkAuths();
+  }
+
+  /**
+   * Constructs an authorization object from a list of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does not
+   * verify that each encoded string is valid UTF-8.
--- End diff --

same as above


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90343756
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final byte[] auth : authorizations) {
+   auths.add(new ArrayByteSequence(auth));
+   }
+checkAuths();
+  }
+
+  /**
+   * Constructs an authorization object from a list of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does not
+   * verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  list of authorizations, as strings encoded in UTF-8 and 
placed in buffers
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final List authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final ByteBuffer buffer : authorizations) {
+  

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90331288
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/util/DocumentVisibilityUtil.java
 ---
@@ -0,0 +1,316 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.util;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.ByteSequence;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility.Node;
+import 
org.apache.rya.mongodb.document.visibility.DocumentVisibility.NodeType;
+import org.apache.rya.mongodb.document.visibility.VisibilityEvaluator;
+import org.apache.rya.mongodb.document.visibility.VisibilityParseException;
+
+import com.google.common.base.Charsets;
+import com.google.common.collect.Lists;
+import com.mongodb.BasicDBList;
+
+/**
+ * Utility methods for converting boolean expressions between an Accumulo 
column
+ * visibility string style and a multidimensional array that can be used
+ * in MongoDB expressions.
+ */
+public final class DocumentVisibilityUtil {
+private static final Logger log = 
Logger.getLogger(DocumentVisibilityUtil.class);
+
+/**
+ * Private constructor to prevent instantiation.
+ */
+private DocumentVisibilityUtil() {
+}
+
+/**
+ * Converts a boolean string expression into a multidimensional
+ * array representation of the boolean expression.
+ * @param booleanString the boolean string expression.
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final String 
booleanString) {
+final DocumentVisibility dv = new 
DocumentVisibility(booleanString);
+return toMultidimensionalArray(dv);
+}
+
+/**
+ * Converts a {@link DocumentVisibility} object into a multidimensional
+ * array representation of the boolean expression.
+ * @param dv the {@link DocumentVisibility}. (not {@code null})
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final 
DocumentVisibility dv) {
+checkNotNull(dv);
+final byte[] expression = dv.flatten();
+final DocumentVisibility flattenedDv = 
DisjunctiveNormalFormConverter.createDnfDocumentVisibility(expression);
+final Object[] result = 
toMultidimensionalArray(flattenedDv.getParseTree(), expression);
+return result;
+}
+
+/**
+ * Converts a {@link Node} and its corresponding expression into a
+ * multidimensional array representation of the boolean expression.
+ * @param node the {@link Node}. (not {@code null})
+ * @param expression the expression byte array.
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final Node node, final 
byte[] expression) {
+checkNotNull(node);
+final List array = new ArrayList<>();
+
+if (node.getChildren().isEmpty() && node.getType() == 
NodeType.TERM) {
+final String data = getTermNodeData(node);
+array.add(data);
+}
+
+log.trace("Children size: " + node.getChildren().size() + " Type: 
" + node.getType());
+for (final Node child : node.getChildren()) {
+switch (child.getType()) {
+  

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90330643
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/util/DocumentVisibilityUtil.java
 ---
@@ -0,0 +1,316 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.util;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.ByteSequence;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility.Node;
+import 
org.apache.rya.mongodb.document.visibility.DocumentVisibility.NodeType;
+import org.apache.rya.mongodb.document.visibility.VisibilityEvaluator;
+import org.apache.rya.mongodb.document.visibility.VisibilityParseException;
+
+import com.google.common.base.Charsets;
+import com.google.common.collect.Lists;
+import com.mongodb.BasicDBList;
+
+/**
+ * Utility methods for converting boolean expressions between an Accumulo 
column
+ * visibility string style and a multidimensional array that can be used
+ * in MongoDB expressions.
--- End diff --

Done.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90068092
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
--- End diff --

same as above


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90068055
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
--- End diff --

same as above


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90068005
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
--- End diff --

same as above.  The constant class had a empty document visibility.   It 
also referenced this constant to let be known it was for ALL_AUTHORIZATIONS 
like in Accumulo Rya.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90067685
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
--- End diff --

same as above


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90067593
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
--- End diff --

same as above


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90067552
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/AuthorizationContainer.java
 ---
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+/**
+ * An interface for classes that contain a collection of authorizations.
+ */
+public interface AuthorizationContainer {
+  /**
+   * Checks whether this object contains the given authorization.
+   *
+   * @param auth
+   *  authorization, as a string encoded in UTF-8
+   * @return true if authorization is in this collection
+   */
+  boolean contains(ByteSequence auth);
--- End diff --

same as above


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90067506
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/ArrayByteSequence.java
 ---
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+
+public class ArrayByteSequence extends ByteSequence implements 
Serializable {
--- End diff --

Copied from accumulo-core because ColumnVisibility uses it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90067343
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/ArgumentChecker.java
 ---
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+/**
+ * This class provides methods to check arguments of a variable number for 
null values, or anything else that might be required on a routine basis. These
+ * methods should be used for early failures as close to the end user as 
possible, so things do not fail later on the server side, when they are harder 
to
+ * debug.
+ *
+ * Methods are created for a specific number of arguments, due to the poor 
performance of array allocation for varargs methods.
+ */
+public class ArgumentChecker {
--- End diff --

This is a straight copy from accumulo-core so we wouldn't need to add that 
dependency for mongodb components.  All the classes in the 
org.apache.rya.mongodb.document.visibility package are from accumulo-core and I 
didn't really modify any of them beside the ColumnVisibility class.

I could add the accumulo-core dependency and we'd have less classes to 
maintain.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90066459
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/util/DocumentVisibilityUtil.java
 ---
@@ -0,0 +1,316 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.util;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.ByteSequence;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility.Node;
+import 
org.apache.rya.mongodb.document.visibility.DocumentVisibility.NodeType;
+import org.apache.rya.mongodb.document.visibility.VisibilityEvaluator;
+import org.apache.rya.mongodb.document.visibility.VisibilityParseException;
+
+import com.google.common.base.Charsets;
+import com.google.common.collect.Lists;
+import com.mongodb.BasicDBList;
+
+/**
+ * Utility methods for converting boolean expressions between an Accumulo 
column
+ * visibility string style and a multidimensional array that can be used
+ * in MongoDB expressions.
+ */
+public final class DocumentVisibilityUtil {
+private static final Logger log = 
Logger.getLogger(DocumentVisibilityUtil.class);
+
+/**
+ * Private constructor to prevent instantiation.
+ */
+private DocumentVisibilityUtil() {
+}
+
+/**
+ * Converts a boolean string expression into a multidimensional
+ * array representation of the boolean expression.
+ * @param booleanString the boolean string expression.
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final String 
booleanString) {
+final DocumentVisibility dv = new 
DocumentVisibility(booleanString);
+return toMultidimensionalArray(dv);
+}
+
+/**
+ * Converts a {@link DocumentVisibility} object into a multidimensional
+ * array representation of the boolean expression.
+ * @param dv the {@link DocumentVisibility}. (not {@code null})
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final 
DocumentVisibility dv) {
+checkNotNull(dv);
+final byte[] expression = dv.flatten();
+final DocumentVisibility flattenedDv = 
DisjunctiveNormalFormConverter.createDnfDocumentVisibility(expression);
+final Object[] result = 
toMultidimensionalArray(flattenedDv.getParseTree(), expression);
+return result;
+}
+
+/**
+ * Converts a {@link Node} and its corresponding expression into a
+ * multidimensional array representation of the boolean expression.
+ * @param node the {@link Node}. (not {@code null})
+ * @param expression the expression byte array.
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final Node node, final 
byte[] expression) {
+checkNotNull(node);
+final List array = new ArrayList<>();
+
+if (node.getChildren().isEmpty() && node.getType() == 
NodeType.TERM) {
+final String data = getTermNodeData(node);
+array.add(data);
+}
+
+log.trace("Children size: " + node.getChildren().size() + " Type: 
" + node.getType());
+for (final Node child : node.getChildren()) {
+switch (child.getType()) {
+  

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90065742
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/util/DisjunctiveNormalFormConverter.java
 ---
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.util;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Set;
+
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility.Node;
+import 
org.apache.rya.mongodb.document.visibility.DocumentVisibility.NodeType;
+
+import com.google.common.base.Joiner;
+import com.google.common.collect.Lists;
+
+/**
+ * Utility for converting document visibility boolean expressions into
+ * Disjunctive Normal Form.
+ */
+public final class DisjunctiveNormalFormConverter {
+private static final Logger log = 
Logger.getLogger(DisjunctiveNormalFormConverter.class);
+
+/**
+ * Private constructor to prevent instantiation.
+ */
+private DisjunctiveNormalFormConverter() {
--- End diff --

It's not a singleton.  It's a static-method only class.  So, the private 
constructor is to enforce that idea and I added final to the class to prevent 
subclassing.  I usually do both for my utility classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90059667
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRyaDAO.java ---
@@ -164,26 +174,31 @@ public void add(final Iterator 
statement) throws RyaDAOException {
 final List dbInserts = new ArrayList();
 while (statement.hasNext()){
 final RyaStatement ryaStatement = statement.next();
-final DBObject insert = 
storageStrategy.serialize(ryaStatement);
-dbInserts.add(insert);
-
-try {
-for (final RyaSecondaryIndexer index : secondaryIndexers) {
-index.storeStatement(ryaStatement);
+final boolean canAdd = 
DocumentVisibilityUtil.doesUserHaveDocumentAccess(auths, 
ryaStatement.getColumnVisibility());
+if (canAdd) {
+final DBObject insert = 
storageStrategy.serialize(ryaStatement);
+dbInserts.add(insert);
+
+try {
+for (final RyaSecondaryIndexer index : 
secondaryIndexers) {
+index.storeStatement(ryaStatement);
+}
+} catch (final IOException e) {
+log.error("Failed to add: " + ryaStatement.toString() 
+ " to the indexer");
 }
-} catch (final IOException e) {
-log.error("Failed to add: " + ryaStatement.toString() + " 
to the indexer");
 }
-
 }
 coll.insert(dbInserts, new InsertOptions().continueOnError(true));
 }
 
 @Override
 public void delete(final RyaStatement statement, final 
MongoDBRdfConfiguration conf)
 throws RyaDAOException {
-final DBObject obj = storageStrategy.getQuery(statement);
-coll.remove(obj);
+final boolean canDelete = 
DocumentVisibilityUtil.doesUserHaveDocumentAccess(auths, 
statement.getColumnVisibility());
+if (canDelete) {
+final DBObject obj = storageStrategy.getQuery(statement);
+coll.remove(obj);
--- End diff --

it looks like the fix was only implemented for batch remove


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90058015
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/dao/SimpleMongoDBStorageStrategy.java
 ---
@@ -96,12 +103,19 @@ public DBObject getQuery(final RyaStatement stmt) {
 
 @Override
 public RyaStatement deserializeDBObject(final DBObject queryResult) {
-final Map result = queryResult.toMap();
+final Map result = queryResult.toMap();
--- End diff --

edit last sentence above. Map< ?, ?> gets rid of the warning


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90057717
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/dao/SimpleMongoDBStorageStrategy.java
 ---
@@ -96,12 +103,19 @@ public DBObject getQuery(final RyaStatement stmt) {
 
 @Override
 public RyaStatement deserializeDBObject(final DBObject queryResult) {
-final Map result = queryResult.toMap();
+final Map result = queryResult.toMap();
--- End diff --

A Map is returned here but I would need to add 
@SuppressWarnings("unchecked") here which I'd prefer not to do.  Leaving it as 
just Map generates a raw type warning.  Map gets rid of the warning and 
is safe.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90055877
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDbRdfConstants.java 
---
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+
+/**
+ * Interface MongoDbRdfConstants.
+ */
+public interface MongoDbRdfConstants {
+public static final Authorizations ALL_AUTHORIZATIONS = 
Authorizations.EMPTY;
--- End diff --

Added docs


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90055421
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDbRdfConstants.java 
---
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+
+/**
+ * Interface MongoDbRdfConstants.
--- End diff --

I was mimicking how the Rya Accumulo side did things.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90055207
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRyaDAO.java ---
@@ -164,26 +174,31 @@ public void add(final Iterator 
statement) throws RyaDAOException {
 final List dbInserts = new ArrayList();
 while (statement.hasNext()){
 final RyaStatement ryaStatement = statement.next();
-final DBObject insert = 
storageStrategy.serialize(ryaStatement);
-dbInserts.add(insert);
-
-try {
-for (final RyaSecondaryIndexer index : secondaryIndexers) {
-index.storeStatement(ryaStatement);
+final boolean canAdd = 
DocumentVisibilityUtil.doesUserHaveDocumentAccess(auths, 
ryaStatement.getColumnVisibility());
+if (canAdd) {
+final DBObject insert = 
storageStrategy.serialize(ryaStatement);
+dbInserts.add(insert);
+
+try {
+for (final RyaSecondaryIndexer index : 
secondaryIndexers) {
+index.storeStatement(ryaStatement);
+}
+} catch (final IOException e) {
+log.error("Failed to add: " + ryaStatement.toString() 
+ " to the indexer");
 }
-} catch (final IOException e) {
-log.error("Failed to add: " + ryaStatement.toString() + " 
to the indexer");
 }
-
 }
 coll.insert(dbInserts, new InsertOptions().continueOnError(true));
 }
 
 @Override
 public void delete(final RyaStatement statement, final 
MongoDBRdfConfiguration conf)
 throws RyaDAOException {
-final DBObject obj = storageStrategy.getQuery(statement);
-coll.remove(obj);
+final boolean canDelete = 
DocumentVisibilityUtil.doesUserHaveDocumentAccess(auths, 
statement.getColumnVisibility());
+if (canDelete) {
+final DBObject obj = storageStrategy.getQuery(statement);
+coll.remove(obj);
--- End diff --

This PR was rebased off master before posting so I don't think I took out 
anybody's code.  I'll rebase again if I make any more updates.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r90053582
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRyaDAO.java ---
@@ -111,26 +117,27 @@ public MongoDBRdfConfiguration getConf() {
 
 @Override
 public void init() throws RyaDAOException {
-secondaryIndexers = conf.getAdditionalIndexers();
-for(final MongoSecondaryIndex index: secondaryIndexers) {
-index.setConf(conf);
-index.setClient(mongoClient);
-}
+secondaryIndexers = conf.getAdditionalIndexers();
+for(final MongoSecondaryIndex index: secondaryIndexers) {
+index.setConf(conf);
+index.setClient(mongoClient);
+}
 
-db = 
mongoClient.getDB(conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
-coll = db.getCollection(conf.getTriplesCollectionName());
-nameSpaceManager = new 
SimpleMongoDBNamespaceManager(db.getCollection(conf.getNameSpacesCollectionName()));
-queryEngine = new MongoDBQueryEngine(conf, mongoClient);
-storageStrategy = new SimpleMongoDBStorageStrategy();
-storageStrategy.createIndices(coll);
-for(final MongoSecondaryIndex index: secondaryIndexers) {
-index.init();
-}
+db = 
mongoClient.getDB(conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
+coll = db.getCollection(conf.getTriplesCollectionName());
+nameSpaceManager = new 
SimpleMongoDBNamespaceManager(db.getCollection(conf.getNameSpacesCollectionName()));
+queryEngine = new MongoDBQueryEngine(conf, mongoClient);
+storageStrategy = new SimpleMongoDBStorageStrategy();
+storageStrategy.createIndices(coll);
+for(final MongoSecondaryIndex index: secondaryIndexers) {
+index.init();
+}
 }
 
 @Override
 public boolean isInitialized() throws RyaDAOException {
 return true;
+
--- End diff --

Done


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89906381
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/DocumentVisibility.java
 ---
@@ -0,0 +1,584 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.TreeSet;
+
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.io.WritableComparator;
+
+/**
+ * Validate the document visibility is a valid expression and set the 
visibility for a Mutation. See {@link 
DocumentVisibility#DocumentVisibility(byte[])} for the
+ * definition of an expression.
+ *
+ * 
+ * The expression is a sequence of characters from the set [A-Za-z0-9_-.] 
along with the binary operators "" and "|" indicating that both operands 
are
+ * necessary, or the either is necessary. The following are valid 
expressions for visibility:
+ *
+ * 
+ * A
+ * A|B
+ * (A|B)(C|D)
+ * orange|(redyellow)
+ * 
+ *
+ * 
+ * The following are not valid expressions for visibility:
+ *
+ * 
+ * A|BC
+ * A=B
+ * A|B|
+ * A|B
+ * ()
+ * )
+ * dog|!cat
+ * 
+ *
+ * 
+ * In addition to the base set of visibilities, any character can be used 
in the expression if it is quoted. If the quoted term contains '' or '\', 
then
+ * escape the character with '\'. The {@link #quote(String)} method can be 
used to properly quote and escape terms automatically. The following is an 
example of
+ * a quoted term:
+ *
+ * 
+ * A#C  B
+ * 
+ */
+public class DocumentVisibility {
+
+  Node node = null;
+  private byte[] expression;
+
+  /**
+   * Accessor for the underlying byte string.
+   *
+   * @return byte array representation of a visibility expression
+   */
+  public byte[] getExpression() {
+return expression;
+  }
+
+  /**
+   * The node types in a parse tree for a visibility expression.
+   */
+  public static enum NodeType {
+EMPTY, TERM, OR, AND,
+  }
+
+  /**
+   * All empty nodes are equal and represent the same value.
+   */
+  private static final Node EMPTY_NODE = new Node("".getBytes(), 
NodeType.EMPTY, 0);
+
+  /**
+   * A node in the parse tree for a visibility expression.
+   */
+  public static class Node {
+/**
+ * An empty list of nodes.
+ */
+public final static List EMPTY = Collections.emptyList();
+NodeType type;
+int start;
+int end;
+List children = EMPTY;
+byte[] expression;
+
+public Node(final byte[] expression, final NodeType type, final int 
start) {
+  this.type = type;
+  this.start = start;
+  this.end = start + 1;
+  this.expression = expression;
+}
+
+public Node(final byte[] expression, final int start, final int end) {
+  this.type = NodeType.TERM;
+  this.start = start;
+  this.end = end;
+  this.expression = expression;
+}
+
+public void add(final Node child) {
+  if (children == EMPTY) {
+   children = new ArrayList<>();
+   }
+
+  children.add(child);
+}
+
+public NodeType getType() {
+  return type;
+}
+
+public List getChildren() {
+  return children;
+}
+
+public int getTermStart() {
+  return start;
+}
+
+public int getTermEnd() {
+  return end;
+}
+
+public byte[] getExpression() {
+  return expression;
+}
+
+public ByteSequence 

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89902538
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/util/DocumentVisibilityUtil.java
 ---
@@ -0,0 +1,316 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.util;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.ByteSequence;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility.Node;
+import 
org.apache.rya.mongodb.document.visibility.DocumentVisibility.NodeType;
+import org.apache.rya.mongodb.document.visibility.VisibilityEvaluator;
+import org.apache.rya.mongodb.document.visibility.VisibilityParseException;
+
+import com.google.common.base.Charsets;
+import com.google.common.collect.Lists;
+import com.mongodb.BasicDBList;
+
+/**
+ * Utility methods for converting boolean expressions between an Accumulo 
column
+ * visibility string style and a multidimensional array that can be used
+ * in MongoDB expressions.
+ */
+public final class DocumentVisibilityUtil {
+private static final Logger log = 
Logger.getLogger(DocumentVisibilityUtil.class);
+
+/**
+ * Private constructor to prevent instantiation.
+ */
+private DocumentVisibilityUtil() {
+}
+
+/**
+ * Converts a boolean string expression into a multidimensional
+ * array representation of the boolean expression.
+ * @param booleanString the boolean string expression.
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final String 
booleanString) {
+final DocumentVisibility dv = new 
DocumentVisibility(booleanString);
+return toMultidimensionalArray(dv);
+}
+
+/**
+ * Converts a {@link DocumentVisibility} object into a multidimensional
+ * array representation of the boolean expression.
+ * @param dv the {@link DocumentVisibility}. (not {@code null})
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final 
DocumentVisibility dv) {
+checkNotNull(dv);
+final byte[] expression = dv.flatten();
+final DocumentVisibility flattenedDv = 
DisjunctiveNormalFormConverter.createDnfDocumentVisibility(expression);
+final Object[] result = 
toMultidimensionalArray(flattenedDv.getParseTree(), expression);
+return result;
+}
+
+/**
+ * Converts a {@link Node} and its corresponding expression into a
+ * multidimensional array representation of the boolean expression.
+ * @param node the {@link Node}. (not {@code null})
+ * @param expression the expression byte array.
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final Node node, final 
byte[] expression) {
+checkNotNull(node);
+final List array = new ArrayList<>();
+
+if (node.getChildren().isEmpty() && node.getType() == 
NodeType.TERM) {
+final String data = getTermNodeData(node);
+array.add(data);
+}
+
+log.trace("Children size: " + node.getChildren().size() + " Type: 
" + node.getType());
+for (final Node child : node.getChildren()) {
+switch (child.getType()) {
+

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89901044
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/util/DocumentVisibilityUtil.java
 ---
@@ -0,0 +1,316 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.util;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.ByteSequence;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility.Node;
+import 
org.apache.rya.mongodb.document.visibility.DocumentVisibility.NodeType;
+import org.apache.rya.mongodb.document.visibility.VisibilityEvaluator;
+import org.apache.rya.mongodb.document.visibility.VisibilityParseException;
+
+import com.google.common.base.Charsets;
+import com.google.common.collect.Lists;
+import com.mongodb.BasicDBList;
+
+/**
+ * Utility methods for converting boolean expressions between an Accumulo 
column
+ * visibility string style and a multidimensional array that can be used
+ * in MongoDB expressions.
+ */
+public final class DocumentVisibilityUtil {
+private static final Logger log = 
Logger.getLogger(DocumentVisibilityUtil.class);
+
+/**
+ * Private constructor to prevent instantiation.
+ */
+private DocumentVisibilityUtil() {
+}
+
+/**
+ * Converts a boolean string expression into a multidimensional
+ * array representation of the boolean expression.
+ * @param booleanString the boolean string expression.
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final String 
booleanString) {
+final DocumentVisibility dv = new 
DocumentVisibility(booleanString);
+return toMultidimensionalArray(dv);
+}
+
+/**
+ * Converts a {@link DocumentVisibility} object into a multidimensional
+ * array representation of the boolean expression.
+ * @param dv the {@link DocumentVisibility}. (not {@code null})
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final 
DocumentVisibility dv) {
+checkNotNull(dv);
+final byte[] expression = dv.flatten();
+final DocumentVisibility flattenedDv = 
DisjunctiveNormalFormConverter.createDnfDocumentVisibility(expression);
+final Object[] result = 
toMultidimensionalArray(flattenedDv.getParseTree(), expression);
+return result;
+}
+
+/**
+ * Converts a {@link Node} and its corresponding expression into a
+ * multidimensional array representation of the boolean expression.
+ * @param node the {@link Node}. (not {@code null})
+ * @param expression the expression byte array.
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final Node node, final 
byte[] expression) {
+checkNotNull(node);
+final List array = new ArrayList<>();
+
+if (node.getChildren().isEmpty() && node.getType() == 
NodeType.TERM) {
+final String data = getTermNodeData(node);
+array.add(data);
+}
+
+log.trace("Children size: " + node.getChildren().size() + " Type: 
" + node.getType());
+for (final Node child : node.getChildren()) {
+switch (child.getType()) {
+

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89908758
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/AggregationUtil.java 
---
@@ -0,0 +1,322 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.iter;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.LinkedList;
+import java.util.List;
+
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.dao.SimpleMongoDBStorageStrategy;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+
+import com.google.common.collect.Lists;
+import com.mongodb.BasicDBObject;
+import com.mongodb.DBObject;
+
+/**
+ * Utility methods for MongoDB aggregation.
+ */
+public final class AggregationUtil {
+/**
+ * Private constructor to prevent instantiation.
+ */
+private AggregationUtil() {
+}
+
+/**
+ * Creates a MongoDB $redact aggregation pipeline that only include
+ * documents whose document visibility match the provided 
authorizations.
+ * All other documents are excluded.
+ * @param authorizations the {@link Authorization}s to include in the
+ * $redact. Only documents that match the authorizations will be 
returned.
+ * @return the {@link List} of {@link DBObject}s that represents the 
$redact
+ * aggregation pipeline.
+ */
+public static List createRedactPipeline(final Authorizations 
authorizations) {
+if (MongoDbRdfConstants.ALL_AUTHORIZATIONS.equals(authorizations)) 
{
+return Lists.newArrayList();
+}
+final List authAndList = 
authorizations.getAuthorizationsStrings();
+
+// Generate all combinations of the authorization strings without 
repetition.
+final List authOrList = 
createCombinations(authorizations.getAuthorizationsStrings());
+
+final String documentVisibilityField = "$" + 
SimpleMongoDBStorageStrategy.DOCUMENT_VISIBILITY;
+
+final BasicDBObject setIsSubset =
+setIsSubsetNullSafe(
+documentVisibilityField,
+authAndList.toArray()
+);
+
+final BasicDBObject setIntersectionExists =
+gt(
+size(
+setIntersection(
+documentVisibilityField,
+authOrList.toArray()
+)
+),
+0
+);
+
+final BasicDBObject orExpression = or(setIsSubset, 
setIntersectionExists);
+
+final List pipeline = new ArrayList<>();
+pipeline.add(
+redact(
+   orExpression,
+   "$$DESCEND",
+   "$$PRUNE"
+)
+);
+
+return pipeline;
+}
+
+/**
+ * Creates all combinations of the values that are of the size of value
+ * array or smaller without repetition.
+ * @param values the {@link List} of values to create combinations 
from.
+ * @return the {@link List} of combinations.
+ */
+public static  List createCombinations(final List 
values) {
+final List allCombinations = new ArrayList<>();
+for (int i = 1; i <= values.size(); i++) {
+allCombinations.addAll(createCombinations(values, i));
+}
+return allCombinations;
+}
+
+/**
+ * Creates all combinations of the values that are of the specified 
size
+ * without repetition.
+ * @param values the {@link List} of values to create combinations 
from.
+ * @param size the size of the combinations.
+ * @return the {@link List} of combinations.

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89909627
  
--- Diff: 
dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRyaDAOTest.java ---
@@ -28,102 +30,279 @@
 import org.apache.rya.api.domain.RyaStatement.RyaStatementBuilder;
 import org.apache.rya.api.domain.RyaURI;
 import org.apache.rya.api.persist.RyaDAOException;
+import org.apache.rya.api.persist.query.RyaQuery;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.bson.Document;
+import org.calrissian.mango.collect.CloseableIterable;
 import org.junit.Before;
 import org.junit.Test;
 
-import com.mongodb.DB;
-import com.mongodb.DBCollection;
 import com.mongodb.MongoException;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.client.MongoDatabase;
 
 public class MongoDBRyaDAOTest extends MongoRyaTestBase {
 
-   private MongoDBRyaDAO dao;
-   private MongoDBRdfConfiguration configuration;
+private MongoDBRyaDAO dao;
+private MongoDBRdfConfiguration configuration;
 
-   @Before
-   public void setUp() throws IOException, RyaDAOException{
-   final Configuration conf = new Configuration();
+@Before
+public void setUp() throws IOException, RyaDAOException{
+final Configuration conf = new Configuration();
 conf.set(MongoDBRdfConfiguration.MONGO_DB_NAME, "test");
 conf.set(MongoDBRdfConfiguration.MONGO_COLLECTION_PREFIX, "rya_");
 conf.set(RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX, "rya_");
 configuration = new MongoDBRdfConfiguration(conf);
 final int port = 
mongoClient.getServerAddressList().get(0).getPort();
 configuration.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, 
""+port);
-   dao = new MongoDBRyaDAO(configuration, mongoClient);
-   }
+dao = new MongoDBRyaDAO(configuration, mongoClient);
+}
 
 
-   @Test
-   public void testDeleteWildcard() throws RyaDAOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   dao.delete(builder.build(), configuration);
-   }
+@Test
+public void testDeleteWildcard() throws RyaDAOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+dao.delete(builder.build(), configuration);
+}
 
 
-   @Test
-   public void testAdd() throws RyaDAOException, MongoException, 
IOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   builder.setSubject(new RyaURI("http://subject.com;));
-   builder.setObject(new RyaURI("http://object.com;));
+@Test
+public void testAdd() throws RyaDAOException, MongoException, 
IOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+builder.setSubject(new RyaURI("http://subject.com;));
+builder.setObject(new RyaURI("http://object.com;));
 
-   final DB db = 
mongoClient.getDB(configuration.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
-final DBCollection coll = 
db.getCollection(configuration.getTriplesCollectionName());
+final MongoDatabase db = 
mongoClient.getDatabase(configuration.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
+final MongoCollection coll = 
db.getCollection(configuration.getTriplesCollectionName());
 
-   dao.add(builder.build());
+dao.add(builder.build());
 
 assertEquals(coll.count(),1);
 
-   }
+}
 
-   @Test
-   public void testDelete() throws RyaDAOException, MongoException, 
IOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   builder.setSubject(new RyaURI("http://subject.com;));
-   builder.setObject(new RyaURI("http://object.com;));
-   final RyaStatement statement = builder.build();
+@Test
+public void testDelete() throws RyaDAOException, MongoException, 
IOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+builder.setSubject(new RyaURI("http://subject.com;));
+builder.setObject(new RyaURI("http://object.com;));
+final RyaStatement statement = builder.build();
 
-   final DB db = 

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89909295
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementCursorIterator.java
 ---
@@ -22,83 +40,83 @@
 
 import info.aduna.iteration.CloseableIteration;
 
-import java.util.Iterator;
-import java.util.Map.Entry;
-import java.util.Set;
-
-import org.apache.rya.api.RdfCloudTripleStoreUtils;
-import org.apache.rya.api.domain.RyaStatement;
-import org.apache.rya.api.persist.RyaDAOException;
-import org.apache.rya.mongodb.dao.MongoDBStorageStrategy;
-
-import org.calrissian.mango.collect.CloseableIterable;
-import org.openrdf.query.BindingSet;
-
-import com.mongodb.DBCollection;
-import com.mongodb.DBCursor;
-import com.mongodb.DBObject;
-
 public class RyaStatementCursorIterator implements 
CloseableIteration {
-
-   private DBCollection coll;
-   private Iterator queryIterator;
-   private DBCursor currentCursor;
-   private MongoDBStorageStrategy strategy;
-   private Long maxResults;
-
-   public RyaStatementCursorIterator(DBCollection coll, Set 
queries, MongoDBStorageStrategy strategy) {
-   this.coll = coll;
-   this.queryIterator = queries.iterator();
-   this.strategy = strategy;
-   }
-
-   @Override
-   public boolean hasNext() {
-   if (!currentCursorIsValid()) {
-   findNextValidCursor();
-   }
-   return currentCursorIsValid();
-   }
-
-   @Override
-   public RyaStatement next() {
-   if (!currentCursorIsValid()) {
-   findNextValidCursor();
-   }
-   if (currentCursorIsValid()) {
-   // convert to Rya Statement
-   DBObject queryResult = currentCursor.next();
-   RyaStatement statement = 
strategy.deserializeDBObject(queryResult);
-   return statement;
-   }
-   return null;
-   }
-   
-   private void findNextValidCursor() {
-   while (queryIterator.hasNext()){
-   DBObject currentQuery = queryIterator.next();
-   currentCursor = coll.find(currentQuery);
-   if (currentCursor.hasNext()) break;
-   }
-   }
-   
-   private boolean currentCursorIsValid() {
-   return (currentCursor != null) && currentCursor.hasNext();
-   }
-
-
-   public void setMaxResults(Long maxResults) {
-   this.maxResults = maxResults;
-   }
-
-   @Override
-   public void close() throws RyaDAOException {
-   // TODO don't know what to do here
-   }
-
-   @Override
-   public void remove() throws RyaDAOException {
-   next();
-   }
-
+private static final Logger log = 
Logger.getLogger(RyaStatementCursorIterator.class);
+
+private final DBCollection coll;
+private final Iterator queryIterator;
+private Iterator resultsIterator;
+private final MongoDBStorageStrategy strategy;
+private Long maxResults;
+private final Authorizations auths;
+
+public RyaStatementCursorIterator(final DBCollection coll, final 
Set queries, final MongoDBStorageStrategy strategy, 
final MongoDBRdfConfiguration conf) {
+this.coll = coll;
+this.queryIterator = queries.iterator();
+this.strategy = strategy;
+if (conf != null) {
+this.auths = conf.getAuthorizations();
+} else {
+auths = MongoDbRdfConstants.ALL_AUTHORIZATIONS;
+}
+}
+
+@Override
+public boolean hasNext() {
+if (!currentCursorIsValid()) {
+findNextValidCursor();
+}
+return currentCursorIsValid();
+}
+
+@Override
+public RyaStatement next() {
+if (!currentCursorIsValid()) {
+findNextValidCursor();
+}
+if (currentCursorIsValid()) {
+// convert to Rya Statement
+final DBObject queryResult = resultsIterator.next();
+final RyaStatement statement = 
strategy.deserializeDBObject(queryResult);
+return statement;
+}
+return null;
+}
+
+private void findNextValidCursor() {
+while (queryIterator.hasNext()){
+final DBObject currentQuery = queryIterator.next();
+
+// Executing redact aggregation to only return documents the 
user
+// has access to.
+final List pipeline = new ArrayList<>();

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89906785
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/DocumentVisibility.java
 ---
@@ -0,0 +1,584 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.TreeSet;
+
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.io.WritableComparator;
+
+/**
+ * Validate the document visibility is a valid expression and set the 
visibility for a Mutation. See {@link 
DocumentVisibility#DocumentVisibility(byte[])} for the
+ * definition of an expression.
+ *
+ * 
+ * The expression is a sequence of characters from the set [A-Za-z0-9_-.] 
along with the binary operators "" and "|" indicating that both operands 
are
+ * necessary, or the either is necessary. The following are valid 
expressions for visibility:
+ *
+ * 
+ * A
+ * A|B
+ * (A|B)(C|D)
+ * orange|(redyellow)
+ * 
+ *
+ * 
+ * The following are not valid expressions for visibility:
+ *
+ * 
+ * A|BC
+ * A=B
+ * A|B|
+ * A|B
+ * ()
+ * )
+ * dog|!cat
+ * 
+ *
+ * 
+ * In addition to the base set of visibilities, any character can be used 
in the expression if it is quoted. If the quoted term contains '' or '\', 
then
+ * escape the character with '\'. The {@link #quote(String)} method can be 
used to properly quote and escape terms automatically. The following is an 
example of
+ * a quoted term:
+ *
+ * 
+ * A#C  B
+ * 
+ */
+public class DocumentVisibility {
+
+  Node node = null;
+  private byte[] expression;
+
+  /**
+   * Accessor for the underlying byte string.
+   *
+   * @return byte array representation of a visibility expression
+   */
+  public byte[] getExpression() {
+return expression;
+  }
+
+  /**
+   * The node types in a parse tree for a visibility expression.
+   */
+  public static enum NodeType {
+EMPTY, TERM, OR, AND,
+  }
+
+  /**
+   * All empty nodes are equal and represent the same value.
+   */
+  private static final Node EMPTY_NODE = new Node("".getBytes(), 
NodeType.EMPTY, 0);
+
+  /**
+   * A node in the parse tree for a visibility expression.
+   */
+  public static class Node {
+/**
+ * An empty list of nodes.
+ */
+public final static List EMPTY = Collections.emptyList();
+NodeType type;
+int start;
+int end;
+List children = EMPTY;
+byte[] expression;
+
+public Node(final byte[] expression, final NodeType type, final int 
start) {
+  this.type = type;
+  this.start = start;
+  this.end = start + 1;
+  this.expression = expression;
+}
+
+public Node(final byte[] expression, final int start, final int end) {
+  this.type = NodeType.TERM;
+  this.start = start;
+  this.end = end;
+  this.expression = expression;
+}
+
+public void add(final Node child) {
+  if (children == EMPTY) {
+   children = new ArrayList<>();
+   }
+
+  children.add(child);
+}
+
+public NodeType getType() {
+  return type;
+}
+
+public List getChildren() {
+  return children;
+}
+
+public int getTermStart() {
+  return start;
+}
+
+public int getTermEnd() {
+  return end;
+}
+
+public byte[] getExpression() {
+  return expression;
+}
+
+public ByteSequence 

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89909207
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementCursorIterator.java
 ---
@@ -22,83 +40,83 @@
 
 import info.aduna.iteration.CloseableIteration;
 
-import java.util.Iterator;
-import java.util.Map.Entry;
-import java.util.Set;
-
-import org.apache.rya.api.RdfCloudTripleStoreUtils;
-import org.apache.rya.api.domain.RyaStatement;
-import org.apache.rya.api.persist.RyaDAOException;
-import org.apache.rya.mongodb.dao.MongoDBStorageStrategy;
-
-import org.calrissian.mango.collect.CloseableIterable;
-import org.openrdf.query.BindingSet;
-
-import com.mongodb.DBCollection;
-import com.mongodb.DBCursor;
-import com.mongodb.DBObject;
-
 public class RyaStatementCursorIterator implements 
CloseableIteration {
-
-   private DBCollection coll;
-   private Iterator queryIterator;
-   private DBCursor currentCursor;
-   private MongoDBStorageStrategy strategy;
-   private Long maxResults;
-
-   public RyaStatementCursorIterator(DBCollection coll, Set 
queries, MongoDBStorageStrategy strategy) {
-   this.coll = coll;
-   this.queryIterator = queries.iterator();
-   this.strategy = strategy;
-   }
-
-   @Override
-   public boolean hasNext() {
-   if (!currentCursorIsValid()) {
-   findNextValidCursor();
-   }
-   return currentCursorIsValid();
-   }
-
-   @Override
-   public RyaStatement next() {
-   if (!currentCursorIsValid()) {
-   findNextValidCursor();
-   }
-   if (currentCursorIsValid()) {
-   // convert to Rya Statement
-   DBObject queryResult = currentCursor.next();
-   RyaStatement statement = 
strategy.deserializeDBObject(queryResult);
-   return statement;
-   }
-   return null;
-   }
-   
-   private void findNextValidCursor() {
-   while (queryIterator.hasNext()){
-   DBObject currentQuery = queryIterator.next();
-   currentCursor = coll.find(currentQuery);
-   if (currentCursor.hasNext()) break;
-   }
-   }
-   
-   private boolean currentCursorIsValid() {
-   return (currentCursor != null) && currentCursor.hasNext();
-   }
-
-
-   public void setMaxResults(Long maxResults) {
-   this.maxResults = maxResults;
-   }
-
-   @Override
-   public void close() throws RyaDAOException {
-   // TODO don't know what to do here
-   }
-
-   @Override
-   public void remove() throws RyaDAOException {
-   next();
-   }
-
+private static final Logger log = 
Logger.getLogger(RyaStatementCursorIterator.class);
+
+private final DBCollection coll;
+private final Iterator queryIterator;
+private Iterator resultsIterator;
+private final MongoDBStorageStrategy strategy;
+private Long maxResults;
+private final Authorizations auths;
+
+public RyaStatementCursorIterator(final DBCollection coll, final 
Set queries, final MongoDBStorageStrategy strategy, 
final MongoDBRdfConfiguration conf) {
--- End diff --

same, don't pass the config when all you need is auths


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89905453
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/BadArgumentException.java
 ---
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import java.util.regex.PatternSyntaxException;
+
+public final class BadArgumentException extends PatternSyntaxException {
--- End diff --

Doc


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89905111
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final byte[] auth : authorizations) {
+   auths.add(new ArrayByteSequence(auth));
+   }
+checkAuths();
+  }
+
+  /**
+   * Constructs an authorization object from a list of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does not
+   * verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  list of authorizations, as strings encoded in UTF-8 and 
placed in buffers
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final List authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final ByteBuffer buffer : authorizations) {
+  

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89904319
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final byte[] auth : authorizations) {
+   auths.add(new ArrayByteSequence(auth));
+   }
+checkAuths();
+  }
+
+  /**
+   * Constructs an authorization object from a list of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does not
+   * verify that each encoded string is valid UTF-8.
--- End diff --

why can't we check encoding?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89900897
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/util/DocumentVisibilityUtil.java
 ---
@@ -0,0 +1,316 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.util;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.ByteSequence;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility.Node;
+import 
org.apache.rya.mongodb.document.visibility.DocumentVisibility.NodeType;
+import org.apache.rya.mongodb.document.visibility.VisibilityEvaluator;
+import org.apache.rya.mongodb.document.visibility.VisibilityParseException;
+
+import com.google.common.base.Charsets;
+import com.google.common.collect.Lists;
+import com.mongodb.BasicDBList;
+
+/**
+ * Utility methods for converting boolean expressions between an Accumulo 
column
+ * visibility string style and a multidimensional array that can be used
+ * in MongoDB expressions.
+ */
+public final class DocumentVisibilityUtil {
+private static final Logger log = 
Logger.getLogger(DocumentVisibilityUtil.class);
+
+/**
+ * Private constructor to prevent instantiation.
+ */
+private DocumentVisibilityUtil() {
+}
+
+/**
+ * Converts a boolean string expression into a multidimensional
+ * array representation of the boolean expression.
+ * @param booleanString the boolean string expression.
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final String 
booleanString) {
+final DocumentVisibility dv = new 
DocumentVisibility(booleanString);
+return toMultidimensionalArray(dv);
+}
+
+/**
+ * Converts a {@link DocumentVisibility} object into a multidimensional
+ * array representation of the boolean expression.
+ * @param dv the {@link DocumentVisibility}. (not {@code null})
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final 
DocumentVisibility dv) {
+checkNotNull(dv);
+final byte[] expression = dv.flatten();
+final DocumentVisibility flattenedDv = 
DisjunctiveNormalFormConverter.createDnfDocumentVisibility(expression);
+final Object[] result = 
toMultidimensionalArray(flattenedDv.getParseTree(), expression);
+return result;
+}
+
+/**
+ * Converts a {@link Node} and its corresponding expression into a
+ * multidimensional array representation of the boolean expression.
+ * @param node the {@link Node}. (not {@code null})
+ * @param expression the expression byte array.
+ * @return the multidimensional array representation of the boolean
+ * expression.
+ */
+public static Object[] toMultidimensionalArray(final Node node, final 
byte[] expression) {
+checkNotNull(node);
+final List array = new ArrayList<>();
+
+if (node.getChildren().isEmpty() && node.getType() == 
NodeType.TERM) {
+final String data = getTermNodeData(node);
+array.add(data);
+}
+
+log.trace("Children size: " + node.getChildren().size() + " Type: 
" + node.getType());
+for (final Node child : node.getChildren()) {
+switch (child.getType()) {
+

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89905766
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/ByteBufferUtil.java
 ---
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+
+import org.apache.hadoop.io.Text;
+
+public class ByteBufferUtil {
--- End diff --

doc


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89905342
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final byte[] auth : authorizations) {
+   auths.add(new ArrayByteSequence(auth));
+   }
+checkAuths();
+  }
+
+  /**
+   * Constructs an authorization object from a list of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does not
+   * verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  list of authorizations, as strings encoded in UTF-8 and 
placed in buffers
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final List authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final ByteBuffer buffer : authorizations) {
+  

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89904761
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final byte[] auth : authorizations) {
+   auths.add(new ArrayByteSequence(auth));
+   }
+checkAuths();
+  }
+
+  /**
+   * Constructs an authorization object from a list of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does not
+   * verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  list of authorizations, as strings encoded in UTF-8 and 
placed in buffers
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final List authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final ByteBuffer buffer : authorizations) {
+  

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89909687
  
--- Diff: 
dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRyaDAOTest.java ---
@@ -28,102 +30,279 @@
 import org.apache.rya.api.domain.RyaStatement.RyaStatementBuilder;
 import org.apache.rya.api.domain.RyaURI;
 import org.apache.rya.api.persist.RyaDAOException;
+import org.apache.rya.api.persist.query.RyaQuery;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.bson.Document;
+import org.calrissian.mango.collect.CloseableIterable;
 import org.junit.Before;
 import org.junit.Test;
 
-import com.mongodb.DB;
-import com.mongodb.DBCollection;
 import com.mongodb.MongoException;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.client.MongoDatabase;
 
 public class MongoDBRyaDAOTest extends MongoRyaTestBase {
 
-   private MongoDBRyaDAO dao;
-   private MongoDBRdfConfiguration configuration;
+private MongoDBRyaDAO dao;
+private MongoDBRdfConfiguration configuration;
 
-   @Before
-   public void setUp() throws IOException, RyaDAOException{
-   final Configuration conf = new Configuration();
+@Before
+public void setUp() throws IOException, RyaDAOException{
+final Configuration conf = new Configuration();
 conf.set(MongoDBRdfConfiguration.MONGO_DB_NAME, "test");
 conf.set(MongoDBRdfConfiguration.MONGO_COLLECTION_PREFIX, "rya_");
 conf.set(RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX, "rya_");
 configuration = new MongoDBRdfConfiguration(conf);
 final int port = 
mongoClient.getServerAddressList().get(0).getPort();
 configuration.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, 
""+port);
-   dao = new MongoDBRyaDAO(configuration, mongoClient);
-   }
+dao = new MongoDBRyaDAO(configuration, mongoClient);
+}
 
 
-   @Test
-   public void testDeleteWildcard() throws RyaDAOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   dao.delete(builder.build(), configuration);
-   }
+@Test
+public void testDeleteWildcard() throws RyaDAOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+dao.delete(builder.build(), configuration);
+}
 
 
-   @Test
-   public void testAdd() throws RyaDAOException, MongoException, 
IOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   builder.setSubject(new RyaURI("http://subject.com;));
-   builder.setObject(new RyaURI("http://object.com;));
+@Test
+public void testAdd() throws RyaDAOException, MongoException, 
IOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+builder.setSubject(new RyaURI("http://subject.com;));
+builder.setObject(new RyaURI("http://object.com;));
 
-   final DB db = 
mongoClient.getDB(configuration.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
-final DBCollection coll = 
db.getCollection(configuration.getTriplesCollectionName());
+final MongoDatabase db = 
mongoClient.getDatabase(configuration.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
+final MongoCollection coll = 
db.getCollection(configuration.getTriplesCollectionName());
 
-   dao.add(builder.build());
+dao.add(builder.build());
 
 assertEquals(coll.count(),1);
 
-   }
+}
 
-   @Test
-   public void testDelete() throws RyaDAOException, MongoException, 
IOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   builder.setSubject(new RyaURI("http://subject.com;));
-   builder.setObject(new RyaURI("http://object.com;));
-   final RyaStatement statement = builder.build();
+@Test
+public void testDelete() throws RyaDAOException, MongoException, 
IOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+builder.setSubject(new RyaURI("http://subject.com;));
+builder.setObject(new RyaURI("http://object.com;));
+final RyaStatement statement = builder.build();
 
-   final DB db = 

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89906753
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/DocumentVisibility.java
 ---
@@ -0,0 +1,584 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.TreeSet;
+
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.io.WritableComparator;
+
+/**
+ * Validate the document visibility is a valid expression and set the 
visibility for a Mutation. See {@link 
DocumentVisibility#DocumentVisibility(byte[])} for the
+ * definition of an expression.
+ *
+ * 
+ * The expression is a sequence of characters from the set [A-Za-z0-9_-.] 
along with the binary operators "" and "|" indicating that both operands 
are
+ * necessary, or the either is necessary. The following are valid 
expressions for visibility:
+ *
+ * 
+ * A
+ * A|B
+ * (A|B)(C|D)
+ * orange|(redyellow)
+ * 
+ *
+ * 
+ * The following are not valid expressions for visibility:
+ *
+ * 
+ * A|BC
+ * A=B
+ * A|B|
+ * A|B
+ * ()
+ * )
+ * dog|!cat
+ * 
+ *
+ * 
+ * In addition to the base set of visibilities, any character can be used 
in the expression if it is quoted. If the quoted term contains '' or '\', 
then
+ * escape the character with '\'. The {@link #quote(String)} method can be 
used to properly quote and escape terms automatically. The following is an 
example of
+ * a quoted term:
+ *
+ * 
+ * A#C  B
+ * 
+ */
+public class DocumentVisibility {
+
+  Node node = null;
+  private byte[] expression;
+
+  /**
+   * Accessor for the underlying byte string.
+   *
+   * @return byte array representation of a visibility expression
+   */
+  public byte[] getExpression() {
+return expression;
+  }
+
+  /**
+   * The node types in a parse tree for a visibility expression.
+   */
+  public static enum NodeType {
+EMPTY, TERM, OR, AND,
+  }
+
+  /**
+   * All empty nodes are equal and represent the same value.
+   */
+  private static final Node EMPTY_NODE = new Node("".getBytes(), 
NodeType.EMPTY, 0);
+
+  /**
+   * A node in the parse tree for a visibility expression.
+   */
+  public static class Node {
+/**
+ * An empty list of nodes.
+ */
+public final static List EMPTY = Collections.emptyList();
+NodeType type;
+int start;
+int end;
+List children = EMPTY;
+byte[] expression;
+
+public Node(final byte[] expression, final NodeType type, final int 
start) {
+  this.type = type;
+  this.start = start;
+  this.end = start + 1;
+  this.expression = expression;
+}
+
+public Node(final byte[] expression, final int start, final int end) {
+  this.type = NodeType.TERM;
+  this.start = start;
+  this.end = end;
+  this.expression = expression;
+}
+
+public void add(final Node child) {
+  if (children == EMPTY) {
+   children = new ArrayList<>();
+   }
+
+  children.add(child);
+}
+
+public NodeType getType() {
+  return type;
+}
+
+public List getChildren() {
+  return children;
+}
+
+public int getTermStart() {
+  return start;
+}
+
+public int getTermEnd() {
+  return end;
+}
+
+public byte[] getExpression() {
+  return expression;
+}
+
+public ByteSequence 

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89897518
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRyaDAO.java ---
@@ -164,26 +174,31 @@ public void add(final Iterator 
statement) throws RyaDAOException {
 final List dbInserts = new ArrayList();
 while (statement.hasNext()){
 final RyaStatement ryaStatement = statement.next();
-final DBObject insert = 
storageStrategy.serialize(ryaStatement);
-dbInserts.add(insert);
-
-try {
-for (final RyaSecondaryIndexer index : secondaryIndexers) {
-index.storeStatement(ryaStatement);
+final boolean canAdd = 
DocumentVisibilityUtil.doesUserHaveDocumentAccess(auths, 
ryaStatement.getColumnVisibility());
+if (canAdd) {
+final DBObject insert = 
storageStrategy.serialize(ryaStatement);
+dbInserts.add(insert);
+
+try {
+for (final RyaSecondaryIndexer index : 
secondaryIndexers) {
+index.storeStatement(ryaStatement);
+}
+} catch (final IOException e) {
+log.error("Failed to add: " + ryaStatement.toString() 
+ " to the indexer");
 }
-} catch (final IOException e) {
-log.error("Failed to add: " + ryaStatement.toString() + " 
to the indexer");
 }
-
 }
 coll.insert(dbInserts, new InsertOptions().continueOnError(true));
 }
 
 @Override
 public void delete(final RyaStatement statement, final 
MongoDBRdfConfiguration conf)
 throws RyaDAOException {
-final DBObject obj = storageStrategy.getQuery(statement);
-coll.remove(obj);
+final boolean canDelete = 
DocumentVisibilityUtil.doesUserHaveDocumentAccess(auths, 
statement.getColumnVisibility());
+if (canDelete) {
+final DBObject obj = storageStrategy.getQuery(statement);
+coll.remove(obj);
--- End diff --

I think @dlotts added some code here for removing statements from the 
indexers.  make sure that doesn't get erased here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89900078
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/util/DisjunctiveNormalFormConverter.java
 ---
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.util;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Set;
+
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility.Node;
+import 
org.apache.rya.mongodb.document.visibility.DocumentVisibility.NodeType;
+
+import com.google.common.base.Joiner;
+import com.google.common.collect.Lists;
+
+/**
+ * Utility for converting document visibility boolean expressions into
+ * Disjunctive Normal Form.
+ */
+public final class DisjunctiveNormalFormConverter {
+private static final Logger log = 
Logger.getLogger(DisjunctiveNormalFormConverter.class);
+
+/**
+ * Private constructor to prevent instantiation.
+ */
+private DisjunctiveNormalFormConverter() {
--- End diff --

why does this need to be a singleton?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89909939
  
--- Diff: 
dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRyaDAOTest.java ---
@@ -28,102 +30,279 @@
 import org.apache.rya.api.domain.RyaStatement.RyaStatementBuilder;
 import org.apache.rya.api.domain.RyaURI;
 import org.apache.rya.api.persist.RyaDAOException;
+import org.apache.rya.api.persist.query.RyaQuery;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.bson.Document;
+import org.calrissian.mango.collect.CloseableIterable;
 import org.junit.Before;
 import org.junit.Test;
 
-import com.mongodb.DB;
-import com.mongodb.DBCollection;
 import com.mongodb.MongoException;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.client.MongoDatabase;
 
 public class MongoDBRyaDAOTest extends MongoRyaTestBase {
 
-   private MongoDBRyaDAO dao;
-   private MongoDBRdfConfiguration configuration;
+private MongoDBRyaDAO dao;
+private MongoDBRdfConfiguration configuration;
 
-   @Before
-   public void setUp() throws IOException, RyaDAOException{
-   final Configuration conf = new Configuration();
+@Before
+public void setUp() throws IOException, RyaDAOException{
+final Configuration conf = new Configuration();
 conf.set(MongoDBRdfConfiguration.MONGO_DB_NAME, "test");
 conf.set(MongoDBRdfConfiguration.MONGO_COLLECTION_PREFIX, "rya_");
 conf.set(RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX, "rya_");
 configuration = new MongoDBRdfConfiguration(conf);
 final int port = 
mongoClient.getServerAddressList().get(0).getPort();
 configuration.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, 
""+port);
-   dao = new MongoDBRyaDAO(configuration, mongoClient);
-   }
+dao = new MongoDBRyaDAO(configuration, mongoClient);
+}
 
 
-   @Test
-   public void testDeleteWildcard() throws RyaDAOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   dao.delete(builder.build(), configuration);
-   }
+@Test
+public void testDeleteWildcard() throws RyaDAOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+dao.delete(builder.build(), configuration);
+}
 
 
-   @Test
-   public void testAdd() throws RyaDAOException, MongoException, 
IOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   builder.setSubject(new RyaURI("http://subject.com;));
-   builder.setObject(new RyaURI("http://object.com;));
+@Test
+public void testAdd() throws RyaDAOException, MongoException, 
IOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+builder.setSubject(new RyaURI("http://subject.com;));
+builder.setObject(new RyaURI("http://object.com;));
 
-   final DB db = 
mongoClient.getDB(configuration.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
-final DBCollection coll = 
db.getCollection(configuration.getTriplesCollectionName());
+final MongoDatabase db = 
mongoClient.getDatabase(configuration.get(MongoDBRdfConfiguration.MONGO_DB_NAME));
+final MongoCollection coll = 
db.getCollection(configuration.getTriplesCollectionName());
 
-   dao.add(builder.build());
+dao.add(builder.build());
 
 assertEquals(coll.count(),1);
 
-   }
+}
 
-   @Test
-   public void testDelete() throws RyaDAOException, MongoException, 
IOException {
-   final RyaStatementBuilder builder = new RyaStatementBuilder();
-   builder.setPredicate(new RyaURI("http://temp.com;));
-   builder.setSubject(new RyaURI("http://subject.com;));
-   builder.setObject(new RyaURI("http://object.com;));
-   final RyaStatement statement = builder.build();
+@Test
+public void testDelete() throws RyaDAOException, MongoException, 
IOException {
+final RyaStatementBuilder builder = new RyaStatementBuilder();
+builder.setPredicate(new RyaURI("http://temp.com;));
+builder.setSubject(new RyaURI("http://subject.com;));
+builder.setObject(new RyaURI("http://object.com;));
+final RyaStatement statement = builder.build();
 
-   final DB db = 

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89896570
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRyaDAO.java ---
@@ -164,26 +174,31 @@ public void add(final Iterator 
statement) throws RyaDAOException {
 final List dbInserts = new ArrayList();
 while (statement.hasNext()){
 final RyaStatement ryaStatement = statement.next();
--- End diff --

I know this has been here, but can you rename this to statements?  or 
statementIter?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89903996
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
--- End diff --

why not use Preconditions?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89908817
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementBindingSetCursorIterator.java
 ---
@@ -1,5 +1,27 @@
 package org.apache.rya.mongodb.iter;
 
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map.Entry;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.api.RdfCloudTripleStoreUtils;
+import org.apache.rya.api.domain.RyaStatement;
+import org.apache.rya.api.persist.RyaDAOException;
+import org.apache.rya.mongodb.MongoDBRdfConfiguration;
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.dao.MongoDBStorageStrategy;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.openrdf.query.BindingSet;
+
+import com.google.common.collect.Multimap;
+import com.mongodb.AggregationOutput;
+import com.mongodb.BasicDBObject;
+import com.mongodb.DBCollection;
+import com.mongodb.DBObject;
+
 /*
--- End diff --

move this back above the package declaration


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89897692
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDbRdfConstants.java 
---
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+
+/**
+ * Interface MongoDbRdfConstants.
--- End diff --

why do you need a class for just constants?  if its absolutely necessary, 
doc it better


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89904458
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final byte[] auth : authorizations) {
+   auths.add(new ArrayByteSequence(auth));
+   }
+checkAuths();
+  }
+
+  /**
+   * Constructs an authorization object from a list of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does not
+   * verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  list of authorizations, as strings encoded in UTF-8 and 
placed in buffers
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final List authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final ByteBuffer buffer : authorizations) {
+  

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89908376
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/AggregationUtil.java 
---
@@ -0,0 +1,322 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.iter;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.LinkedList;
+import java.util.List;
+
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.dao.SimpleMongoDBStorageStrategy;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+
+import com.google.common.collect.Lists;
+import com.mongodb.BasicDBObject;
+import com.mongodb.DBObject;
+
+/**
+ * Utility methods for MongoDB aggregation.
+ */
+public final class AggregationUtil {
+/**
+ * Private constructor to prevent instantiation.
+ */
+private AggregationUtil() {
+}
+
+/**
+ * Creates a MongoDB $redact aggregation pipeline that only include
+ * documents whose document visibility match the provided 
authorizations.
+ * All other documents are excluded.
+ * @param authorizations the {@link Authorization}s to include in the
+ * $redact. Only documents that match the authorizations will be 
returned.
+ * @return the {@link List} of {@link DBObject}s that represents the 
$redact
+ * aggregation pipeline.
+ */
+public static List createRedactPipeline(final Authorizations 
authorizations) {
+if (MongoDbRdfConstants.ALL_AUTHORIZATIONS.equals(authorizations)) 
{
+return Lists.newArrayList();
+}
+final List authAndList = 
authorizations.getAuthorizationsStrings();
+
+// Generate all combinations of the authorization strings without 
repetition.
+final List authOrList = 
createCombinations(authorizations.getAuthorizationsStrings());
+
+final String documentVisibilityField = "$" + 
SimpleMongoDBStorageStrategy.DOCUMENT_VISIBILITY;
+
+final BasicDBObject setIsSubset =
+setIsSubsetNullSafe(
+documentVisibilityField,
+authAndList.toArray()
+);
+
+final BasicDBObject setIntersectionExists =
+gt(
+size(
+setIntersection(
+documentVisibilityField,
+authOrList.toArray()
+)
+),
+0
+);
+
+final BasicDBObject orExpression = or(setIsSubset, 
setIntersectionExists);
+
+final List pipeline = new ArrayList<>();
+pipeline.add(
+redact(
+   orExpression,
+   "$$DESCEND",
+   "$$PRUNE"
+)
+);
+
+return pipeline;
+}
+
+/**
+ * Creates all combinations of the values that are of the size of value
+ * array or smaller without repetition.
+ * @param values the {@link List} of values to create combinations 
from.
+ * @return the {@link List} of combinations.
+ */
+public static  List createCombinations(final List 
values) {
+final List allCombinations = new ArrayList<>();
+for (int i = 1; i <= values.size(); i++) {
+allCombinations.addAll(createCombinations(values, i));
+}
+return allCombinations;
+}
+
+/**
+ * Creates all combinations of the values that are of the specified 
size
+ * without repetition.
+ * @param values the {@link List} of values to create combinations 
from.
+ * @param size the size of the combinations.
+ * @return the {@link List} of combinations.

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89900455
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/util/DocumentVisibilityUtil.java
 ---
@@ -0,0 +1,316 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb.document.util;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.ByteSequence;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility.Node;
+import 
org.apache.rya.mongodb.document.visibility.DocumentVisibility.NodeType;
+import org.apache.rya.mongodb.document.visibility.VisibilityEvaluator;
+import org.apache.rya.mongodb.document.visibility.VisibilityParseException;
+
+import com.google.common.base.Charsets;
+import com.google.common.collect.Lists;
+import com.mongodb.BasicDBList;
+
+/**
+ * Utility methods for converting boolean expressions between an Accumulo 
column
+ * visibility string style and a multidimensional array that can be used
+ * in MongoDB expressions.
--- End diff --

reword this to make it more db agnostic.  From string representation to a 
mongo-friendly form.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89908937
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementBindingSetCursorIterator.java
 ---
@@ -22,104 +44,104 @@
 
 import info.aduna.iteration.CloseableIteration;
 
-import java.util.Collection;
-import java.util.Iterator;
-import java.util.Map.Entry;
-
-import org.apache.rya.api.RdfCloudTripleStoreUtils;
-import org.apache.rya.api.domain.RyaStatement;
-import org.apache.rya.api.persist.RyaDAOException;
-import org.apache.rya.mongodb.dao.MongoDBStorageStrategy;
-
-import org.openrdf.query.BindingSet;
-
-import com.google.common.collect.Multimap;
-import com.mongodb.DBCollection;
-import com.mongodb.DBCursor;
-import com.mongodb.DBObject;
-
 public class RyaStatementBindingSetCursorIterator implements 
CloseableIteration, RyaDAOException> {
-
-   private DBCollection coll;
-   private Multimap rangeMap;
-   private Iterator queryIterator;
-   private Long maxResults;
-   private DBCursor resultCursor;
-   private RyaStatement currentStatement;
-   private Collection currentBindingSetCollection;
-   private Iterator currentBindingSetIterator;
-   private MongoDBStorageStrategy strategy;
-
-   public RyaStatementBindingSetCursorIterator(DBCollection coll,
-   Multimap rangeMap, 
MongoDBStorageStrategy strategy) {
-   this.coll = coll;
-   this.rangeMap = rangeMap;
-   this.queryIterator = rangeMap.keySet().iterator();
-   this.strategy = strategy;
-   }
-
-   @Override
-   public boolean hasNext() {
-   if (!currentBindingSetIteratorIsValid()) {
-   findNextResult();
-   }
-   return currentBindingSetIteratorIsValid();
-   }
-
-   @Override
-   public Entry next() {
-   if (!currentBindingSetIteratorIsValid()) {
-   findNextResult();
-   }
-   if (currentBindingSetIteratorIsValid()) {
-   BindingSet currentBindingSet = 
currentBindingSetIterator.next();
-   return new 
RdfCloudTripleStoreUtils.CustomEntry(currentStatement, currentBindingSet);
-   }
-   return null;
-   }
-   
-   private boolean currentBindingSetIteratorIsValid() {
-   return (currentBindingSetIterator != null) && 
currentBindingSetIterator.hasNext();
-   }
-
-   private void findNextResult() {
-   if (!currentResultCursorIsValid()) {
-   findNextValidResultCursor();
-   }
-   if (currentResultCursorIsValid()) {
-   // convert to Rya Statement
-   DBObject queryResult = resultCursor.next();
-   currentStatement = 
strategy.deserializeDBObject(queryResult);
-   currentBindingSetIterator = 
currentBindingSetCollection.iterator();
-   }
-   }
-
-   private void findNextValidResultCursor() {
-   while (queryIterator.hasNext()){
-   DBObject currentQuery = queryIterator.next();
-   resultCursor = coll.find(currentQuery);
-   currentBindingSetCollection = 
rangeMap.get(currentQuery);
-   if (resultCursor.hasNext()) return;
-   }
-   }
-   
-   private boolean currentResultCursorIsValid() {
-   return (resultCursor != null) && resultCursor.hasNext();
-   }
-
-
-   public void setMaxResults(Long maxResults) {
-   this.maxResults = maxResults;
-   }
-
-   @Override
-   public void close() throws RyaDAOException {
-   // TODO don't know what to do here
-   }
-
-   @Override
-   public void remove() throws RyaDAOException {
-   next();
-   }
+private static final Logger log = 
Logger.getLogger(RyaStatementBindingSetCursorIterator.class);
+
+private final DBCollection coll;
+private final Multimap rangeMap;
+private final Iterator queryIterator;
+private Long maxResults;
+private Iterator resultsIterator;
+private RyaStatement currentStatement;
+private Collection currentBindingSetCollection;
+private Iterator currentBindingSetIterator;
+private final MongoDBStorageStrategy strategy;
+private final Authorizations auths;
+
+public RyaStatementBindingSetCursorIterator(final DBCollection coll,
--- End diff --

don't pass in 

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89907937
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/VisibilityEvaluator.java
 ---
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import java.util.ArrayList;
+
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility.Node;
+
+/**
+ * A class which evaluates visibility expressions against a set of 
authorizations.
+ */
+public class VisibilityEvaluator {
+  private final AuthorizationContainer auths;
+
+  /**
+   * Creates a new {@link Authorizations} object with escaped forms of the 
authorizations in the given object.
+   *
+   * @param auths
+   *  original authorizations
+   * @return authorizations object with escaped authorization strings
+   * @see #escape(byte[], boolean)
+   */
+  static Authorizations escape(final Authorizations auths) {
+final ArrayList retAuths = new 
ArrayList(auths.getAuthorizations().size());
+
+for (final byte[] auth : auths.getAuthorizations()) {
+   retAuths.add(escape(auth, false));
+   }
+
+return new Authorizations(retAuths);
+  }
+
+  /**
+   * Properly escapes an authorization string. The string can be quoted if 
desired.
+   *
+   * @param auth
+   *  authorization string, as UTF-8 encoded bytes
+   * @param quote
+   *  true to wrap escaped authorization in quotes
+   * @return escaped authorization string
+   */
+  public static byte[] escape(byte[] auth, final boolean quote) {
+int escapeCount = 0;
+
+for (final byte element : auth) {
+   if (element == '"' || element == '\\') {
+   escapeCount++;
+   }
+   }
+
+if (escapeCount > 0 || quote) {
+  final byte[] escapedAuth = new byte[auth.length + escapeCount + 
(quote ? 2 : 0)];
+  int index = quote ? 1 : 0;
+  for (final byte element : auth) {
+if (element == '"' || element == '\\') {
+   escapedAuth[index++] = '\\';
+   }
+escapedAuth[index++] = element;
+  }
+
+  if (quote) {
+escapedAuth[0] = '"';
+escapedAuth[escapedAuth.length - 1] = '"';
+  }
+
+  auth = escapedAuth;
+}
+return auth;
+  }
+
+//  /**
--- End diff --

remove doc'd code


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89903895
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
--- End diff --

replace this with a regex pattern?  then just do a match on it?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89902691
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/ArgumentChecker.java
 ---
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+/**
+ * This class provides methods to check arguments of a variable number for 
null values, or anything else that might be required on a routine basis. These
+ * methods should be used for early failures as close to the end user as 
possible, so things do not fail later on the server side, when they are harder 
to
+ * debug.
+ *
+ * Methods are created for a specific number of arguments, due to the poor 
performance of array allocation for varargs methods.
+ */
+public class ArgumentChecker {
--- End diff --

what is this?  Why not use preconditions?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89909168
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementCursorIterator.java
 ---
@@ -1,5 +1,23 @@
 package org.apache.rya.mongodb.iter;
 
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.api.domain.RyaStatement;
+import org.apache.rya.api.persist.RyaDAOException;
+import org.apache.rya.mongodb.MongoDBRdfConfiguration;
+import org.apache.rya.mongodb.MongoDbRdfConstants;
+import org.apache.rya.mongodb.dao.MongoDBStorageStrategy;
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+
+import com.mongodb.AggregationOutput;
+import com.mongodb.BasicDBObject;
+import com.mongodb.DBCollection;
+import com.mongodb.DBObject;
+
 /*
--- End diff --

same


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89897743
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDbRdfConstants.java 
---
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.rya.mongodb.document.visibility.Authorizations;
+import org.apache.rya.mongodb.document.visibility.DocumentVisibility;
+
+/**
+ * Interface MongoDbRdfConstants.
+ */
+public interface MongoDbRdfConstants {
+public static final Authorizations ALL_AUTHORIZATIONS = 
Authorizations.EMPTY;
--- End diff --

doc what each constant is and what its for


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89905264
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
--- End diff --

AuthorizationContainer only has 1 contains, but you end up with 3 here, 
making it vague which to use for AuthorizationContainer


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89904345
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
+
+  private static final boolean[] validAuthChars = new boolean[256];
+
+  /**
+   * A special header string used when serializing instances of this class.
+   *
+   * @see #serialize()
+   */
+  public static final String HEADER = "!AUTH1:";
+
+  static {
+for (int i = 0; i < 256; i++) {
+  validAuthChars[i] = false;
+}
+
+for (int i = 'a'; i <= 'z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = 'A'; i <= 'Z'; i++) {
+  validAuthChars[i] = true;
+}
+
+for (int i = '0'; i <= '9'; i++) {
+  validAuthChars[i] = true;
+}
+
+validAuthChars['_'] = true;
+validAuthChars['-'] = true;
+validAuthChars[':'] = true;
+validAuthChars['.'] = true;
+validAuthChars['/'] = true;
+  }
+
+  static final boolean isValidAuthChar(final byte b) {
+return validAuthChars[0xff & b];
+  }
+
+  private void checkAuths() {
+final Set sortedAuths = new TreeSet(auths);
+
+for (final ByteSequence bs : sortedAuths) {
+  if (bs.length() == 0) {
+throw new IllegalArgumentException("Empty authorization");
+  }
+
+  authsList.add(bs.toArray());
+}
+  }
+
+  /**
+   * Constructs an authorization object from a collection of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does
+   * not verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  collection of authorizations, as strings encoded in UTF-8
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final Collection authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final byte[] auth : authorizations) {
+   auths.add(new ArrayByteSequence(auth));
+   }
+checkAuths();
+  }
+
+  /**
+   * Constructs an authorization object from a list of string 
authorizations that have each already been encoded as UTF-8 bytes. Warning: 
This method does not
+   * verify that each encoded string is valid UTF-8.
+   *
+   * @param authorizations
+   *  list of authorizations, as strings encoded in UTF-8 and 
placed in buffers
+   * @throws IllegalArgumentException
+   *   if authorizations is null
+   * @see #Authorizations(String...)
+   */
+  public Authorizations(final List authorizations) {
+ArgumentChecker.notNull(authorizations);
+for (final ByteBuffer buffer : authorizations) {
+  

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89895899
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRyaDAO.java ---
@@ -146,15 +153,18 @@ public void destroy() throws RyaDAOException {
 public void add(final RyaStatement statement) throws RyaDAOException {
 // add it to the collection
 try {
-coll.insert(storageStrategy.serialize(statement));
-for(final RyaSecondaryIndexer index: secondaryIndexers) {
-index.storeStatement(statement);
+final boolean canAdd = 
DocumentVisibilityUtil.doesUserHaveDocumentAccess(auths, 
statement.getColumnVisibility());
--- End diff --

I'll check later, but should this be doesUserHaveCollectionAccess?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89903387
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/AuthorizationContainer.java
 ---
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+/**
+ * An interface for classes that contain a collection of authorizations.
+ */
+public interface AuthorizationContainer {
+  /**
+   * Checks whether this object contains the given authorization.
+   *
+   * @param auth
+   *  authorization, as a string encoded in UTF-8
+   * @return true if authorization is in this collection
+   */
+  boolean contains(ByteSequence auth);
--- End diff --

rename to isAuthorized(), or something along those lines.  makes more 
readable over contains()


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89903230
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/ArrayByteSequence.java
 ---
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+
+public class ArrayByteSequence extends ByteSequence implements 
Serializable {
--- End diff --

This looks like Java's ByteBuffer could replace the whole class?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89903542
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
+ */
+public class Authorizations implements Iterable, Serializable, 
AuthorizationContainer {
+
+  private static final long serialVersionUID = 1L;
+
+  private final Set auths = new HashSet();
+  private final List authsList = new ArrayList(); // 
sorted order
+
+  /**
+   * An empty set of authorizations.
+   */
+  public static final Authorizations EMPTY = new Authorizations();
--- End diff --

didn't you already make an empty auth in the constants class?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/124#discussion_r89903566
  
--- Diff: 
dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/document/visibility/Authorizations.java
 ---
@@ -0,0 +1,369 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rya.mongodb.document.visibility;
+
+import static com.google.common.base.Charsets.UTF_8;
+
+import java.io.Serializable;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.TreeSet;
+
+import com.google.common.base.Charsets;
+
+/**
+ * A collection of authorization strings.
--- End diff --

Doc how this is to be used, how to use it, etc.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #124: RYA-119 Added MongoDB Column Visibility (ca...

[ejwhite922]

GitHub user ejwhite922 opened a pull request:

https://github.com/apache/incubator-rya/pull/124

RYA-119 Added MongoDB Column Visibility (called Document Visibility).

## Description
This adds a new field to each document called documentVisibility which
uses a boolean expression to determine if the user can access the document.
The boolean expression is in Disjunctive Normal Formal so that the 
expression's grouping is simplified enough that MongoDB can run Set operations 
on it to determine if
the document is viewable.  The expression is stored as an array in MongoDB.

The classes in the package "org.apache.rya.mongodb.document.visibility" are 
pretty much a copy of the classes that are used by 
"org.apache.accumulo.core.security.ColumnVisibility" so we don't need to have 
an Accumulo dependency in a MongoDB component.  If anyone is against that then 
let me know.

### Tests
Unit Tests

### Links
[Jira](https://issues.apache.org/jira/browse/RYA-119)

### Checklist
- [x] Code Review
- [ ] Squash Commits

 People To Review
@pujav65 
@amihalik 
@isper3at 
@DLotts

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ejwhite922/incubator-rya 
RYA-119_MongoDBColumnVisibility

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rya/pull/124.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #124


commit 2c14d78a82562ea556fdd7e652185605feeb5544
Author: ejwhite922 
Date:   2016-11-02T13:51:32Z

RYA-119 Added MongoDB Column Visibility (called Document Visibility).
This adds a new field to each document called documentVisibility which
uses a boolean expression to determine if the user can access the document.
The boolean expression is in Disjunctive Normal Formal so that it can be
stored as an array that MongoDB can run Set operations on to determine if
the document is viewable.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---