[jira] [Commented] (RYA-250) Smart URI avoid data duplication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122461#comment-16122461
 ] 

ASF GitHub Bot commented on RYA-250:


Github user isper3at commented on the issue:

https://github.com/apache/incubator-rya/pull/153
  
looked good, just docs stuff


> Smart URI avoid data duplication
> 
>
> Key: RYA-250
> URL: https://issues.apache.org/jira/browse/RYA-250
> Project: Rya
>  Issue Type: Task
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Eric White
>Assignee: Eric White
> Fix For: 3.2.10
>
>
> Implement Smart URI methods for avoiding data duplication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RYA-250) Smart URI avoid data duplication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122457#comment-16122457
 ] 

ASF GitHub Bot commented on RYA-250:


Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/153#discussion_r132576212
  
--- Diff: 
extras/indexing/src/main/java/org/apache/rya/indexing/smarturi/duplication/DuplicateDataDetector.java
 ---
@@ -0,0 +1,1059 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.smarturi.duplication;
+
+import static java.util.Objects.requireNonNull;
+
+import java.math.BigDecimal;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.Set;
+import java.util.TreeSet;
+
+import org.apache.commons.configuration.ConfigurationException;
+import org.apache.commons.lang.StringUtils;
+import org.apache.rya.api.domain.RyaType;
+import org.apache.rya.api.domain.RyaURI;
+import org.apache.rya.api.resolver.impl.DateTimeRyaTypeResolver;
+import org.apache.rya.indexing.entity.model.Entity;
+import org.apache.rya.indexing.entity.model.Property;
+import org.apache.rya.indexing.smarturi.SmartUriAdapter;
+import org.apache.rya.indexing.smarturi.SmartUriException;
+import 
org.apache.rya.indexing.smarturi.duplication.conf.DuplicateDataConfig;
+import org.calrissian.mango.types.exception.TypeEncodingException;
+import org.joda.time.DateTime;
+import org.openrdf.model.URI;
+import org.openrdf.model.impl.URIImpl;
+import org.openrdf.model.vocabulary.XMLSchema;
+
+import com.google.common.collect.ImmutableMap;
+
+/**
+ * Detects if two entities contain data that's nearly identical based on a 
set
--- End diff --

nearly identical?  can you define that a bit more in the class docs?


> Smart URI avoid data duplication
> 
>
> Key: RYA-250
> URL: https://issues.apache.org/jira/browse/RYA-250
> Project: Rya
>  Issue Type: Task
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Eric White
>Assignee: Eric White
> Fix For: 3.2.10
>
>
> Implement Smart URI methods for avoiding data duplication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RYA-250) Smart URI avoid data duplication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122458#comment-16122458
 ] 

ASF GitHub Bot commented on RYA-250:


Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/153#discussion_r132557770
  
--- Diff: 
common/rya.api/src/main/java/org/apache/rya/api/domain/RyaTypeUtils.java ---
@@ -24,12 +24,44 @@
 import org.joda.time.DateTimeZone;
 import org.joda.time.format.ISODateTimeFormat;
 import org.openrdf.model.URI;
+import org.openrdf.model.impl.URIImpl;
 import org.openrdf.model.vocabulary.XMLSchema;
 
+import com.google.common.collect.ImmutableMap;
+
 /**
  * Utility methods for using {@link RyaType}.
  */
 public final class RyaTypeUtils {
+private static final ImmutableMap METHOD_MAP =
+ImmutableMap.builder()
+.put(Boolean.class, (v) -> booleanRyaType((Boolean) v))
+.put(Byte.class, (v) -> byteRyaType((Byte) v))
+.put(Date.class, (v) -> dateRyaType((Date) v))
+.put(DateTime.class, (v) -> dateRyaType((DateTime) v))
+.put(Double.class, (v) -> doubleRyaType((Double) v))
+.put(Float.class, (v) -> floatRyaType((Float) v))
+.put(Integer.class, (v) -> intRyaType((Integer) v))
+.put(Long.class, (v) -> longRyaType((Long) v))
+.put(Short.class, (v) -> shortRyaType((Short) v))
+.put(String.class, (v) -> stringRyaType((String) v))
+.put(URI.class, (v) -> uriRyaType((URI) v))
+.put(URIImpl.class, (v) -> uriRyaType((URIImpl) v))
+.build();
+
+/**
+ * Represents a method inside the {@link RyaTypeUtils} class that can 
be
+ * call.
+ */
+private static interface RyaTypeMethod {
--- End diff --

confused.is this for reflection?


> Smart URI avoid data duplication
> 
>
> Key: RYA-250
> URL: https://issues.apache.org/jira/browse/RYA-250
> Project: Rya
>  Issue Type: Task
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Eric White
>Assignee: Eric White
> Fix For: 3.2.10
>
>
> Implement Smart URI methods for avoiding data duplication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RYA-250) Smart URI avoid data duplication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122460#comment-16122460
 ] 

ASF GitHub Bot commented on RYA-250:


Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/153#discussion_r132585736
  
--- Diff: 
common/rya.api/src/main/java/org/apache/rya/api/domain/RyaTypeUtils.java ---
@@ -24,12 +24,44 @@
 import org.joda.time.DateTimeZone;
 import org.joda.time.format.ISODateTimeFormat;
 import org.openrdf.model.URI;
+import org.openrdf.model.impl.URIImpl;
 import org.openrdf.model.vocabulary.XMLSchema;
 
+import com.google.common.collect.ImmutableMap;
+
 /**
  * Utility methods for using {@link RyaType}.
  */
 public final class RyaTypeUtils {
+private static final ImmutableMap METHOD_MAP =
+ImmutableMap.builder()
+.put(Boolean.class, (v) -> booleanRyaType((Boolean) v))
+.put(Byte.class, (v) -> byteRyaType((Byte) v))
+.put(Date.class, (v) -> dateRyaType((Date) v))
+.put(DateTime.class, (v) -> dateRyaType((DateTime) v))
+.put(Double.class, (v) -> doubleRyaType((Double) v))
+.put(Float.class, (v) -> floatRyaType((Float) v))
+.put(Integer.class, (v) -> intRyaType((Integer) v))
+.put(Long.class, (v) -> longRyaType((Long) v))
+.put(Short.class, (v) -> shortRyaType((Short) v))
+.put(String.class, (v) -> stringRyaType((String) v))
+.put(URI.class, (v) -> uriRyaType((URI) v))
+.put(URIImpl.class, (v) -> uriRyaType((URIImpl) v))
+.build();
+
+/**
+ * Represents a method inside the {@link RyaTypeUtils} class that can 
be
+ * call.
+ */
+private static interface RyaTypeMethod {
--- End diff --

ignore this


> Smart URI avoid data duplication
> 
>
> Key: RYA-250
> URL: https://issues.apache.org/jira/browse/RYA-250
> Project: Rya
>  Issue Type: Task
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Eric White
>Assignee: Eric White
> Fix For: 3.2.10
>
>
> Implement Smart URI methods for avoiding data duplication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RYA-250) Smart URI avoid data duplication

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122459#comment-16122459
 ] 

ASF GitHub Bot commented on RYA-250:


Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/153#discussion_r132571187
  
--- Diff: 
extras/indexing/src/main/java/org/apache/rya/indexing/entity/storage/mongo/MongoEntityStorage.java
 ---
@@ -242,4 +282,49 @@ private static Bson makeExplicitTypeFilter(final 
RyaURI typeId) {
 
 return Stream.of(dataTypeFilter, valueFilter);
 }
+
+private boolean detectDuplicates(final Entity entity) throws 
EntityStorageException {
+boolean hasDuplicate = false;
+if (duplicateDataDetector.isDetectionEnabled()) {
+if (mongoTypeStorage == null) {
+mongoTypeStorage = new MongoTypeStorage(mongo, 
ryaInstanceName);
+}
+final Builder builder = new Builder();
+builder.setSubject(entity.getSubject());
+boolean abort = false;
+for (final RyaURI typeRyaUri : entity.getExplicitTypeIds()) {
+final ImmutableMap typePropertyMap = 
entity.getProperties().get(typeRyaUri);
+final Set properties = new 
HashSet<>(typePropertyMap.values());
+Optional type;
+try {
+type = mongoTypeStorage.get(typeRyaUri);
+} catch (final TypeStorageException e) {
+throw new EntityStorageException("Unable to get entity 
type: " + typeRyaUri, e);
+}
+if (type.isPresent()) {
+//final ConvertingCursor cursor = 
search(Optional.empty(), type.get(), properties);
--- End diff --

commented code


> Smart URI avoid data duplication
> 
>
> Key: RYA-250
> URL: https://issues.apache.org/jira/browse/RYA-250
> Project: Rya
>  Issue Type: Task
>  Components: dao
>Affects Versions: 3.2.10
>Reporter: Eric White
>Assignee: Eric White
> Fix For: 3.2.10
>
>
> Implement Smart URI methods for avoiding data duplication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-rya issue #153: RYA-250 Smart URI avoiding data duplication

[isper3at]

Github user isper3at commented on the issue:

https://github.com/apache/incubator-rya/pull/153
  
looked good, just docs stuff


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #153: RYA-250 Smart URI avoiding data duplication

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/153#discussion_r132585736
  
--- Diff: 
common/rya.api/src/main/java/org/apache/rya/api/domain/RyaTypeUtils.java ---
@@ -24,12 +24,44 @@
 import org.joda.time.DateTimeZone;
 import org.joda.time.format.ISODateTimeFormat;
 import org.openrdf.model.URI;
+import org.openrdf.model.impl.URIImpl;
 import org.openrdf.model.vocabulary.XMLSchema;
 
+import com.google.common.collect.ImmutableMap;
+
 /**
  * Utility methods for using {@link RyaType}.
  */
 public final class RyaTypeUtils {
+private static final ImmutableMap METHOD_MAP =
+ImmutableMap.builder()
+.put(Boolean.class, (v) -> booleanRyaType((Boolean) v))
+.put(Byte.class, (v) -> byteRyaType((Byte) v))
+.put(Date.class, (v) -> dateRyaType((Date) v))
+.put(DateTime.class, (v) -> dateRyaType((DateTime) v))
+.put(Double.class, (v) -> doubleRyaType((Double) v))
+.put(Float.class, (v) -> floatRyaType((Float) v))
+.put(Integer.class, (v) -> intRyaType((Integer) v))
+.put(Long.class, (v) -> longRyaType((Long) v))
+.put(Short.class, (v) -> shortRyaType((Short) v))
+.put(String.class, (v) -> stringRyaType((String) v))
+.put(URI.class, (v) -> uriRyaType((URI) v))
+.put(URIImpl.class, (v) -> uriRyaType((URIImpl) v))
+.build();
+
+/**
+ * Represents a method inside the {@link RyaTypeUtils} class that can 
be
+ * call.
+ */
+private static interface RyaTypeMethod {
--- End diff --

ignore this


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #153: RYA-250 Smart URI avoiding data duplication

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/153#discussion_r132576212
  
--- Diff: 
extras/indexing/src/main/java/org/apache/rya/indexing/smarturi/duplication/DuplicateDataDetector.java
 ---
@@ -0,0 +1,1059 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.smarturi.duplication;
+
+import static java.util.Objects.requireNonNull;
+
+import java.math.BigDecimal;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.Set;
+import java.util.TreeSet;
+
+import org.apache.commons.configuration.ConfigurationException;
+import org.apache.commons.lang.StringUtils;
+import org.apache.rya.api.domain.RyaType;
+import org.apache.rya.api.domain.RyaURI;
+import org.apache.rya.api.resolver.impl.DateTimeRyaTypeResolver;
+import org.apache.rya.indexing.entity.model.Entity;
+import org.apache.rya.indexing.entity.model.Property;
+import org.apache.rya.indexing.smarturi.SmartUriAdapter;
+import org.apache.rya.indexing.smarturi.SmartUriException;
+import 
org.apache.rya.indexing.smarturi.duplication.conf.DuplicateDataConfig;
+import org.calrissian.mango.types.exception.TypeEncodingException;
+import org.joda.time.DateTime;
+import org.openrdf.model.URI;
+import org.openrdf.model.impl.URIImpl;
+import org.openrdf.model.vocabulary.XMLSchema;
+
+import com.google.common.collect.ImmutableMap;
+
+/**
+ * Detects if two entities contain data that's nearly identical based on a 
set
--- End diff --

nearly identical?  can you define that a bit more in the class docs?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #153: RYA-250 Smart URI avoiding data duplication

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/153#discussion_r132571187
  
--- Diff: 
extras/indexing/src/main/java/org/apache/rya/indexing/entity/storage/mongo/MongoEntityStorage.java
 ---
@@ -242,4 +282,49 @@ private static Bson makeExplicitTypeFilter(final 
RyaURI typeId) {
 
 return Stream.of(dataTypeFilter, valueFilter);
 }
+
+private boolean detectDuplicates(final Entity entity) throws 
EntityStorageException {
+boolean hasDuplicate = false;
+if (duplicateDataDetector.isDetectionEnabled()) {
+if (mongoTypeStorage == null) {
+mongoTypeStorage = new MongoTypeStorage(mongo, 
ryaInstanceName);
+}
+final Builder builder = new Builder();
+builder.setSubject(entity.getSubject());
+boolean abort = false;
+for (final RyaURI typeRyaUri : entity.getExplicitTypeIds()) {
+final ImmutableMap typePropertyMap = 
entity.getProperties().get(typeRyaUri);
+final Set properties = new 
HashSet<>(typePropertyMap.values());
+Optional type;
+try {
+type = mongoTypeStorage.get(typeRyaUri);
+} catch (final TypeStorageException e) {
+throw new EntityStorageException("Unable to get entity 
type: " + typeRyaUri, e);
+}
+if (type.isPresent()) {
+//final ConvertingCursor cursor = 
search(Optional.empty(), type.get(), properties);
--- End diff --

commented code


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #153: RYA-250 Smart URI avoiding data duplication

[isper3at]

Github user isper3at commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/153#discussion_r132557770
  
--- Diff: 
common/rya.api/src/main/java/org/apache/rya/api/domain/RyaTypeUtils.java ---
@@ -24,12 +24,44 @@
 import org.joda.time.DateTimeZone;
 import org.joda.time.format.ISODateTimeFormat;
 import org.openrdf.model.URI;
+import org.openrdf.model.impl.URIImpl;
 import org.openrdf.model.vocabulary.XMLSchema;
 
+import com.google.common.collect.ImmutableMap;
+
 /**
  * Utility methods for using {@link RyaType}.
  */
 public final class RyaTypeUtils {
+private static final ImmutableMap METHOD_MAP =
+ImmutableMap.builder()
+.put(Boolean.class, (v) -> booleanRyaType((Boolean) v))
+.put(Byte.class, (v) -> byteRyaType((Byte) v))
+.put(Date.class, (v) -> dateRyaType((Date) v))
+.put(DateTime.class, (v) -> dateRyaType((DateTime) v))
+.put(Double.class, (v) -> doubleRyaType((Double) v))
+.put(Float.class, (v) -> floatRyaType((Float) v))
+.put(Integer.class, (v) -> intRyaType((Integer) v))
+.put(Long.class, (v) -> longRyaType((Long) v))
+.put(Short.class, (v) -> shortRyaType((Short) v))
+.put(String.class, (v) -> stringRyaType((String) v))
+.put(URI.class, (v) -> uriRyaType((URI) v))
+.put(URIImpl.class, (v) -> uriRyaType((URIImpl) v))
+.build();
+
+/**
+ * Represents a method inside the {@link RyaTypeUtils} class that can 
be
+ * call.
+ */
+private static interface RyaTypeMethod {
--- End diff --

confused.is this for reflection?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (RYA-316) Long LineStrings break MongoDB ingest

[Andrew Smith (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RYA-316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Smith resolved RYA-316.
--
Resolution: Fixed

> Long LineStrings break MongoDB ingest
> -
>
> Key: RYA-316
> URL: https://issues.apache.org/jira/browse/RYA-316
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Aaron Mihalik
>Assignee: Andrew Smith
>
> MongoDB will reject statements they contain very long linestrings.  
> Basically, the mongodb index key is limited to 1024 chars, so the insert will 
> fail if the literal is longer.
> [Here is some example 
> code|https://github.com/amihalik/rya-mongo-debugging/blob/master/src/main/java/com/github/amihalik/rya/mongo/debugging/linestring/LoadLineString.java].
>   I think the inserts will work if you use 10 points, but fail if you use 
> linestrings with 100 points.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RYA-316) Long LineStrings break MongoDB ingest

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121986#comment-16121986
 ] 

ASF GitHub Bot commented on RYA-316:


Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/199


> Long LineStrings break MongoDB ingest
> -
>
> Key: RYA-316
> URL: https://issues.apache.org/jira/browse/RYA-316
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Aaron Mihalik
>Assignee: Andrew Smith
>
> MongoDB will reject statements they contain very long linestrings.  
> Basically, the mongodb index key is limited to 1024 chars, so the insert will 
> fail if the literal is longer.
> [Here is some example 
> code|https://github.com/amihalik/rya-mongo-debugging/blob/master/src/main/java/com/github/amihalik/rya/mongo/debugging/linestring/LoadLineString.java].
>   I think the inserts will work if you use 10 points, but fail if you use 
> linestrings with 100 points.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-rya pull request #199: RYA-316 Long OBJ string

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-rya/pull/199


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (RYA-316) Long LineStrings break MongoDB ingest

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121954#comment-16121954
 ] 

ASF GitHub Bot commented on RYA-316:


Github user amihalik commented on the issue:

https://github.com/apache/incubator-rya/pull/199
  
@isper3at thanks!  Merging now...


> Long LineStrings break MongoDB ingest
> -
>
> Key: RYA-316
> URL: https://issues.apache.org/jira/browse/RYA-316
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Aaron Mihalik
>Assignee: Andrew Smith
>
> MongoDB will reject statements they contain very long linestrings.  
> Basically, the mongodb index key is limited to 1024 chars, so the insert will 
> fail if the literal is longer.
> [Here is some example 
> code|https://github.com/amihalik/rya-mongo-debugging/blob/master/src/main/java/com/github/amihalik/rya/mongo/debugging/linestring/LoadLineString.java].
>   I think the inserts will work if you use 10 points, but fail if you use 
> linestrings with 100 points.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-rya issue #199: RYA-316 Long OBJ string

[amihalik]

Github user amihalik commented on the issue:

https://github.com/apache/incubator-rya/pull/199
  
@isper3at thanks!  Merging now...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (RYA-316) Long LineStrings break MongoDB ingest

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121950#comment-16121950
 ] 

ASF GitHub Bot commented on RYA-316:


Github user asfgit commented on the issue:

https://github.com/apache/incubator-rya/pull/199
  

Refer to this link for build results (access rights to CI server needed): 

https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/382/



> Long LineStrings break MongoDB ingest
> -
>
> Key: RYA-316
> URL: https://issues.apache.org/jira/browse/RYA-316
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Aaron Mihalik
>Assignee: Andrew Smith
>
> MongoDB will reject statements they contain very long linestrings.  
> Basically, the mongodb index key is limited to 1024 chars, so the insert will 
> fail if the literal is longer.
> [Here is some example 
> code|https://github.com/amihalik/rya-mongo-debugging/blob/master/src/main/java/com/github/amihalik/rya/mongo/debugging/linestring/LoadLineString.java].
>   I think the inserts will work if you use 10 points, but fail if you use 
> linestrings with 100 points.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-rya issue #199: RYA-316 Long OBJ string

[asfgit]

Github user asfgit commented on the issue:

https://github.com/apache/incubator-rya/pull/199
  

Refer to this link for build results (access rights to CI server needed): 

https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/382/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (RYA-316) Long LineStrings break MongoDB ingest

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121736#comment-16121736
 ] 

ASF GitHub Bot commented on RYA-316:


Github user amihalik commented on the issue:

https://github.com/apache/incubator-rya/pull/199
  
@pujav65 totally agree.  That's why I added the "please get some metrics" 
portion.


> Long LineStrings break MongoDB ingest
> -
>
> Key: RYA-316
> URL: https://issues.apache.org/jira/browse/RYA-316
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Aaron Mihalik
>Assignee: Andrew Smith
>
> MongoDB will reject statements they contain very long linestrings.  
> Basically, the mongodb index key is limited to 1024 chars, so the insert will 
> fail if the literal is longer.
> [Here is some example 
> code|https://github.com/amihalik/rya-mongo-debugging/blob/master/src/main/java/com/github/amihalik/rya/mongo/debugging/linestring/LoadLineString.java].
>   I think the inserts will work if you use 10 points, but fail if you use 
> linestrings with 100 points.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-rya issue #199: RYA-316 Long OBJ string

[amihalik]

Github user amihalik commented on the issue:

https://github.com/apache/incubator-rya/pull/199
  
@pujav65 totally agree.  That's why I added the "please get some metrics" 
portion.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (RYA-316) Long LineStrings break MongoDB ingest

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121728#comment-16121728
 ] 

ASF GitHub Bot commented on RYA-316:


Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/199
  
The reason I wanted it broken apart was so this could be merged and more 
time could be spent figuring out if hashing is necessary.  So I say merge this 
and do that on a follow on or.


> Long LineStrings break MongoDB ingest
> -
>
> Key: RYA-316
> URL: https://issues.apache.org/jira/browse/RYA-316
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Aaron Mihalik
>Assignee: Andrew Smith
>
> MongoDB will reject statements they contain very long linestrings.  
> Basically, the mongodb index key is limited to 1024 chars, so the insert will 
> fail if the literal is longer.
> [Here is some example 
> code|https://github.com/amihalik/rya-mongo-debugging/blob/master/src/main/java/com/github/amihalik/rya/mongo/debugging/linestring/LoadLineString.java].
>   I think the inserts will work if you use 10 points, but fail if you use 
> linestrings with 100 points.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-rya issue #199: RYA-316 Long OBJ string

[pujav65]

Github user pujav65 commented on the issue:

https://github.com/apache/incubator-rya/pull/199
  
The reason I wanted it broken apart was so this could be merged and more 
time could be spent figuring out if hashing is necessary.  So I say merge this 
and do that on a follow on or.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (RYA-316) Long LineStrings break MongoDB ingest

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121682#comment-16121682
 ] 

ASF GitHub Bot commented on RYA-316:


Github user amihalik commented on the issue:

https://github.com/apache/incubator-rya/pull/199
  
I added the "please hash everything" task as requested by @pujav65 and we 
can continue the work on that task.  The task is more of a "please get some 
metrics then decide if we want to hash everything."  The task is [RYA-338 - 
Revisit Core indcies in MongoDB](https://issues.apache.org/jira/browse/RYA-338)


> Long LineStrings break MongoDB ingest
> -
>
> Key: RYA-316
> URL: https://issues.apache.org/jira/browse/RYA-316
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Aaron Mihalik
>Assignee: Andrew Smith
>
> MongoDB will reject statements they contain very long linestrings.  
> Basically, the mongodb index key is limited to 1024 chars, so the insert will 
> fail if the literal is longer.
> [Here is some example 
> code|https://github.com/amihalik/rya-mongo-debugging/blob/master/src/main/java/com/github/amihalik/rya/mongo/debugging/linestring/LoadLineString.java].
>   I think the inserts will work if you use 10 points, but fail if you use 
> linestrings with 100 points.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (RYA-338) Revisit Core indcies in MongoDB

[Aaron Mihalik (JIRA)]

Aaron Mihalik created RYA-338:
-

 Summary: Revisit Core indcies in MongoDB
 Key: RYA-338
 URL: https://issues.apache.org/jira/browse/RYA-338
 Project: Rya
  Issue Type: Improvement
  Components: dao
Affects Versions: 3.2.10
Reporter: Aaron Mihalik
Assignee: Aaron Mihalik


This task is to look at the core indices for Rya in MongoDB.  This includes 
both the fields that are being indexed (i.e. S_P, P_O_OT, and O_OT_S) and how 
those fields are being represented (e.g. values, sha256hex, sha256binary).

The aim is two fold: (1) make indices small (to fit into memory) and useful for 
a reasonable number of query patterns and (2) make the calls between the 
client/server relatively small as well.  Another, but less important aim is to 
minimize the size of the datastore.  (It is very important that the datastore 
size grows linearly with the number of triples, but the rate isn't necessarily 
important).

This task should produce a table showing the metrics (ie. size, latency, etc) 
for the various configurations, a recommendation, and changes to the code base 
to implement the recommendation.

This task can leverage the work done here: [Github PR 
199|https://github.com/apache/incubator-rya/pull/199] and is related to RYA-316



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RYA-298) Implement rdfs:domain inference

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121676#comment-16121676
 ] 

ASF GitHub Bot commented on RYA-298:


Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/197#discussion_r132466881
  
--- Diff: 
sail/src/test/java/org/apache/rya/rdftriplestore/inference/DomainRangeVisitorTest.java
 ---
@@ -0,0 +1,140 @@
+package org.apache.rya.rdftriplestore.inference;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.Stack;
+
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.apache.rya.rdftriplestore.utils.FixedStatementPattern;
+import org.junit.Assert;
+import org.junit.Test;
+import org.openrdf.model.Resource;
+import org.openrdf.model.Statement;
+import org.openrdf.model.URI;
+import org.openrdf.model.ValueFactory;
+import org.openrdf.model.impl.ValueFactoryImpl;
+import org.openrdf.model.vocabulary.RDF;
+import org.openrdf.model.vocabulary.RDFS;
+import org.openrdf.query.algebra.Join;
+import org.openrdf.query.algebra.Projection;
+import org.openrdf.query.algebra.ProjectionElem;
+import org.openrdf.query.algebra.ProjectionElemList;
+import org.openrdf.query.algebra.StatementPattern;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.Union;
+import org.openrdf.query.algebra.Var;
+
+public class DomainRangeVisitorTest {
+private final AccumuloRdfConfiguration conf = new 
AccumuloRdfConfiguration();
+private final ValueFactory vf = new ValueFactoryImpl();
+private final URI person = vf.createURI("lubm:Person");
--- End diff --

These URI's can be made static final


> Implement rdfs:domain inference
> ---
>
> Key: RYA-298
> URL: https://issues.apache.org/jira/browse/RYA-298
> Project: Rya
>  Issue Type: Sub-task
>  Components: sail
>Reporter: Jesse Hatfield
>Assignee: Jesse Hatfield
>
> If a predicate has an *{{rdfs:domain}}* of some class, than the subject of 
> any triple including that predicate belongs to the class.
> If the ontology states that {{:advisor}} has the domain of {{:Person}}, then 
> the inference engine should rewrite queries of the form {{?x rdf:type 
> :Person}} to check for resources which have any {{:advisor}} (as well as any 
> specifically stated to have type {{:Person}} ).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RYA-298) Implement rdfs:domain inference

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121677#comment-16121677
 ] 

ASF GitHub Bot commented on RYA-298:


Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/197#discussion_r132466657
  
--- Diff: 
sail/src/main/java/org/apache/rya/rdftriplestore/inference/InferenceEngine.java 
---
@@ -73,6 +74,8 @@
 private Set transitivePropertySet;
 private Map> hasValueByType;
 private Map> hasValueByProperty;
+private Map domainByType;
--- End diff --

We should probably start using ConcurrentHashMaps for all these maps.  The 
InferenceEngine is not very thread safe right now.


> Implement rdfs:domain inference
> ---
>
> Key: RYA-298
> URL: https://issues.apache.org/jira/browse/RYA-298
> Project: Rya
>  Issue Type: Sub-task
>  Components: sail
>Reporter: Jesse Hatfield
>Assignee: Jesse Hatfield
>
> If a predicate has an *{{rdfs:domain}}* of some class, than the subject of 
> any triple including that predicate belongs to the class.
> If the ontology states that {{:advisor}} has the domain of {{:Person}}, then 
> the inference engine should rewrite queries of the form {{?x rdf:type 
> :Person}} to check for resources which have any {{:advisor}} (as well as any 
> specifically stated to have type {{:Person}} ).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RYA-298) Implement rdfs:domain inference

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121675#comment-16121675
 ] 

ASF GitHub Bot commented on RYA-298:


Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/197#discussion_r132463450
  
--- Diff: 
sail/src/test/java/org/apache/rya/rdftriplestore/inference/DomainRangeVisitorTest.java
 ---
@@ -0,0 +1,140 @@
+package org.apache.rya.rdftriplestore.inference;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.Stack;
+
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.apache.rya.rdftriplestore.utils.FixedStatementPattern;
+import org.junit.Assert;
+import org.junit.Test;
+import org.openrdf.model.Resource;
+import org.openrdf.model.Statement;
+import org.openrdf.model.URI;
+import org.openrdf.model.ValueFactory;
+import org.openrdf.model.impl.ValueFactoryImpl;
+import org.openrdf.model.vocabulary.RDF;
+import org.openrdf.model.vocabulary.RDFS;
+import org.openrdf.query.algebra.Join;
+import org.openrdf.query.algebra.Projection;
+import org.openrdf.query.algebra.ProjectionElem;
+import org.openrdf.query.algebra.ProjectionElemList;
+import org.openrdf.query.algebra.StatementPattern;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.Union;
+import org.openrdf.query.algebra.Var;
+
+public class DomainRangeVisitorTest {
+private final AccumuloRdfConfiguration conf = new 
AccumuloRdfConfiguration();
+private final ValueFactory vf = new ValueFactoryImpl();
+private final URI person = vf.createURI("lubm:Person");
+private final URI advisor = vf.createURI("lubm:advisor");
+private final URI takesCourse = vf.createURI("lubm:takesCourse");
+
+@Test
+public void testRewriteTypePattern() throws Exception {
+final InferenceEngine inferenceEngine = 
mock(InferenceEngine.class);
+final Set domainPredicates = new HashSet<>();
+final Set rangePredicates = new HashSet<>();
+domainPredicates.add(advisor);
+domainPredicates.add(takesCourse);
+rangePredicates.add(advisor);
+
when(inferenceEngine.getPropertiesWithDomain(person)).thenReturn(domainPredicates);
+
when(inferenceEngine.getPropertiesWithRange(person)).thenReturn(rangePredicates);
+final Var subjVar = new Var("s");
+final StatementPattern originalSP = new StatementPattern(subjVar, 
new Var("p", RDF.TYPE), new Var("o", person));
+final Projection query = new Projection(originalSP, new 
ProjectionElemList(new ProjectionElem("s", "subject")));
+new DomainRangeVisitor(conf, inferenceEngine).meet(query);
--- End diff --

Should this be query.visit(new DomainRangeVisitor(conf, inferenceEngine))?  
Or does it not matter?


> Implement rdfs:domain inference
> ---
>
> Key: RYA-298
> URL: https://issues.apache.org/jira/browse/RYA-298
> Project: Rya
>  Issue Type: Sub-task
>  Components: sail
>Reporter: Jesse Hatfield
>Assignee: Jesse Hatfield
>
> If a predicate has an *{{rdfs:domain}}* of some class, than the subject of 
> any triple including that predicate belongs to the class.
> If the ontology states that {{:advisor}} has the domain of {{:Person}}, then 
> the inference engine should rewrite queries of the form {{?x rdf:type 
> :Person}} to check for resources which have any {{:advisor}} (as well as any 
> specifically stated to have type {{:Person}} ).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-rya pull request #197: RYA-298, RYA-299 Domain/range inference.

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/197#discussion_r132466657
  
--- Diff: 
sail/src/main/java/org/apache/rya/rdftriplestore/inference/InferenceEngine.java 
---
@@ -73,6 +74,8 @@
 private Set transitivePropertySet;
 private Map> hasValueByType;
 private Map> hasValueByProperty;
+private Map domainByType;
--- End diff --

We should probably start using ConcurrentHashMaps for all these maps.  The 
InferenceEngine is not very thread safe right now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #197: RYA-298, RYA-299 Domain/range inference.

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/197#discussion_r132466881
  
--- Diff: 
sail/src/test/java/org/apache/rya/rdftriplestore/inference/DomainRangeVisitorTest.java
 ---
@@ -0,0 +1,140 @@
+package org.apache.rya.rdftriplestore.inference;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.Stack;
+
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.apache.rya.rdftriplestore.utils.FixedStatementPattern;
+import org.junit.Assert;
+import org.junit.Test;
+import org.openrdf.model.Resource;
+import org.openrdf.model.Statement;
+import org.openrdf.model.URI;
+import org.openrdf.model.ValueFactory;
+import org.openrdf.model.impl.ValueFactoryImpl;
+import org.openrdf.model.vocabulary.RDF;
+import org.openrdf.model.vocabulary.RDFS;
+import org.openrdf.query.algebra.Join;
+import org.openrdf.query.algebra.Projection;
+import org.openrdf.query.algebra.ProjectionElem;
+import org.openrdf.query.algebra.ProjectionElemList;
+import org.openrdf.query.algebra.StatementPattern;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.Union;
+import org.openrdf.query.algebra.Var;
+
+public class DomainRangeVisitorTest {
+private final AccumuloRdfConfiguration conf = new 
AccumuloRdfConfiguration();
+private final ValueFactory vf = new ValueFactoryImpl();
+private final URI person = vf.createURI("lubm:Person");
--- End diff --

These URI's can be made static final


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-rya pull request #197: RYA-298, RYA-299 Domain/range inference.

[ejwhite922]

Github user ejwhite922 commented on a diff in the pull request:

https://github.com/apache/incubator-rya/pull/197#discussion_r132463450
  
--- Diff: 
sail/src/test/java/org/apache/rya/rdftriplestore/inference/DomainRangeVisitorTest.java
 ---
@@ -0,0 +1,140 @@
+package org.apache.rya.rdftriplestore.inference;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.Stack;
+
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.apache.rya.rdftriplestore.utils.FixedStatementPattern;
+import org.junit.Assert;
+import org.junit.Test;
+import org.openrdf.model.Resource;
+import org.openrdf.model.Statement;
+import org.openrdf.model.URI;
+import org.openrdf.model.ValueFactory;
+import org.openrdf.model.impl.ValueFactoryImpl;
+import org.openrdf.model.vocabulary.RDF;
+import org.openrdf.model.vocabulary.RDFS;
+import org.openrdf.query.algebra.Join;
+import org.openrdf.query.algebra.Projection;
+import org.openrdf.query.algebra.ProjectionElem;
+import org.openrdf.query.algebra.ProjectionElemList;
+import org.openrdf.query.algebra.StatementPattern;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.Union;
+import org.openrdf.query.algebra.Var;
+
+public class DomainRangeVisitorTest {
+private final AccumuloRdfConfiguration conf = new 
AccumuloRdfConfiguration();
+private final ValueFactory vf = new ValueFactoryImpl();
+private final URI person = vf.createURI("lubm:Person");
+private final URI advisor = vf.createURI("lubm:advisor");
+private final URI takesCourse = vf.createURI("lubm:takesCourse");
+
+@Test
+public void testRewriteTypePattern() throws Exception {
+final InferenceEngine inferenceEngine = 
mock(InferenceEngine.class);
+final Set domainPredicates = new HashSet<>();
+final Set rangePredicates = new HashSet<>();
+domainPredicates.add(advisor);
+domainPredicates.add(takesCourse);
+rangePredicates.add(advisor);
+
when(inferenceEngine.getPropertiesWithDomain(person)).thenReturn(domainPredicates);
+
when(inferenceEngine.getPropertiesWithRange(person)).thenReturn(rangePredicates);
+final Var subjVar = new Var("s");
+final StatementPattern originalSP = new StatementPattern(subjVar, 
new Var("p", RDF.TYPE), new Var("o", person));
+final Projection query = new Projection(originalSP, new 
ProjectionElemList(new ProjectionElem("s", "subject")));
+new DomainRangeVisitor(conf, inferenceEngine).meet(query);
--- End diff --

Should this be query.visit(new DomainRangeVisitor(conf, inferenceEngine))?  
Or does it not matter?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (RYA-316) Long LineStrings break MongoDB ingest

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/RYA-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121663#comment-16121663
 ] 

ASF GitHub Bot commented on RYA-316:


Github user amihalik commented on the issue:

https://github.com/apache/incubator-rya/pull/199
  
@isper3at Change line 80 from "SUBJECT" to "SUBJECT_HASH" and I'll be 
satisfied and I'll merge it.  Thanks!


> Long LineStrings break MongoDB ingest
> -
>
> Key: RYA-316
> URL: https://issues.apache.org/jira/browse/RYA-316
> Project: Rya
>  Issue Type: Bug
>  Components: dao
>Reporter: Aaron Mihalik
>Assignee: Andrew Smith
>
> MongoDB will reject statements they contain very long linestrings.  
> Basically, the mongodb index key is limited to 1024 chars, so the insert will 
> fail if the literal is longer.
> [Here is some example 
> code|https://github.com/amihalik/rya-mongo-debugging/blob/master/src/main/java/com/github/amihalik/rya/mongo/debugging/linestring/LoadLineString.java].
>   I think the inserts will work if you use 10 points, but fail if you use 
> linestrings with 100 points.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)