[
https://issues.apache.org/jira/browse/ATLAS-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269001#comment-16269001
]
Pierre Padovani edited comment on ATLAS-2270 at 11/28/17 5:24 PM:
------------------------------------------------------------------
[~davidrad] Per the JanusGraph documentation, Cassandra and Elasticsearch are
fully supported.
After some debugging, it seems that the main issue with supporting
Elasticsearch has to do with two lines in
org.apache.atlas.repository.Constants.java:
{code:java}
public static final String VERTEX_TYPE_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type";
public static final String TYPENAME_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type.name";
{code}
Those constants are used in this code:
{code:java}
private void createTypeStoreIndexes(AtlasGraphManagement management) {
//Create unique index on typeName
createIndexes(management, Constants.TYPENAME_PROPERTY_KEY,
String.class, true, AtlasCardinality.SINGLE,
true, true);
//create index on vertex type
createIndexes(management, Constants.VERTEX_TYPE_PROPERTY_KEY,
String.class, false, AtlasCardinality.SINGLE,
true, true);
}
{code}
In Elasticsearch 2.x and above, creating two fields named as above is
disallowed. The reason for this has to do with the treatment of a '.' in the
name as a path within a sub object. So the above code would attempt to create
the following:
1) { "type" : { "name" : "string"}}
2) { "type": "string"}
This fails because the update is attempting to change an object to a field
index. The simple fix would be to change the '.' to '_', however this may cause
backward incompatibilities for existing systems.
*Update:* - Changing the '.' to '_' solves the issues, and I have a fully
operational Atlas with Cassandra and Elasticsearch.
was (Author: ppadovani):
[~davidrad] Per the JanusGraph documentation, Cassandra and Elasticsearch are
fully supported.
After some debugging, it seems that the main issue with supporting
Elasticsearch has to do with two lines in
org.apache.atlas.repository.Constants.java:
{code:java}
public static final String VERTEX_TYPE_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type";
public static final String TYPENAME_PROPERTY_KEY =
INTERNAL_PROPERTY_KEY_PREFIX + "type.name";
{code}
Those constants are used in this code:
{code:java}
private void createTypeStoreIndexes(AtlasGraphManagement management) {
//Create unique index on typeName
createIndexes(management, Constants.TYPENAME_PROPERTY_KEY,
String.class, true, AtlasCardinality.SINGLE,
true, true);
//create index on vertex type
createIndexes(management, Constants.VERTEX_TYPE_PROPERTY_KEY,
String.class, false, AtlasCardinality.SINGLE,
true, true);
}
{code}
In Elasticsearch 2.x and above, creating two fields named as above is
disallowed. The reason for this has to do with the treatment of a '.' in the
name as a path within a sub object. So the above code would attempt to create
the following:
1) { "type" : { "name" : "string"}}
2) { "type": "string"}
This fails because the update is attempting to change an object to a field
index. The simple fix would be to change the '.' to '_', however this may cause
backward incompatibilities for existing systems.
> Supported combinations of persistent store and index backend
> ------------------------------------------------------------
>
> Key: ATLAS-2270
> URL: https://issues.apache.org/jira/browse/ATLAS-2270
> Project: Atlas
> Issue Type: Bug
> Reporter: Graham Wallis
>
> We need to discuss and decide which combinations of persistent store and
> indexing backend Atlas 1.0.0 (master) should support. This includes
> building/running Atlas as a standalone package and running UTs/ITs as part of
> the Atlas build.
> This JIRA focusses on titan0 and janusgraph 0.2.0, as they are the graph
> databases that will be supported in master/1.0.0. This JIRA deliberately
> ignores titan1 and janusgraph 0.1.1 as the former should be
> deprecated/removed and the other is a transient state as we get to janusgraph
> 0.2.0.
> With titan0 as the graph provider, Atlas has supported the following
> combinations of persistent store and indexer. It is suggested that this set
> is kept unchanged:
> {{
> titan0 solr es
> ------------------------------------
> berkeley 0 1
> hbase 1 0
> cassandra 0 0
> }}
> With janusgraph (0.2.0) as the graph provider, Atlas *could* support
> additional combinations. Cassandra is included in this discussion pending
> response to ATLAS-2259.
> {{
> janus 0.2.0 solr es
> ------------------------------------
> berkeley ? 1
> hbase 1 ?
> cassandra ? ?
> }}
> It is suggested that the combinations marked with '1' should be continued and
> the remaining 4 combinations, marked with '?', should be considered. There
> seems to be evidence of people using all 4 of these combinations, although
> not necessarily with Atlas.
> Depending on the decision made above, we need to ensure that it is possible
> to build Atlas as a standalone package with any of the combinations - i.e.
> that they are mutually exclusive and do not interfere with one another. They
> currently interfere which makes it impossible to build Atlas with
> -Pdist,berkeley-elasticsearch because the 'dist' profile will exclude jars
> that are needed by the berkeley-elasticsearch profile - which leads to class
> not found exceptions when the Atlas server is started. The solution to this
> could be very simple, or slightly more sophisticated, depending on how many
> of the combinations we choose to support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
