Anshul Mehta created ATLAS-4358:
-----------------------------------
Summary: Mapping for some internal Atlas attributes ( like
__patch.type , __timestamp, etc) does not exist in Elasticsearch
Key: ATLAS-4358
URL: https://issues.apache.org/jira/browse/ATLAS-4358
Project: Atlas
Issue Type: Bug
Reporter: Anshul Mehta
*Impact of the problem -*
* Atlas pod taking much longer to become active and this time keep increasing
as the assets increase. This basically means a downtime on every Atlas release.
* Not able to filter via basic search on attributes like {{__timestamp}} ,
{{__modificationTimestamp}} , {{createdBy}} and {{modifiedBy}} .
*Issue -*
So just before creating the mapping in the mixed index (ES index) Atlas creates
something called {{propertyKey}} and this propertyKey is used to create the
mapping. The code is written in a way that checks if propertyKey for the
current property is null or not. If it is null it creates the propertyKey and
then adds it to mixed index. If it is not Null it assumes that the property has
already been added to the index and so skips adding it.
Now in our case when Atlas checked the propertyKey it was not null (which
should not have been the case) therefore Atlas skipped adding it to the mixed
index and so these properties never got added to the mixed index. This
basically meant propertyKey for these properties were getting created somewhere
else. We looked into the entire codebase but could not find the use of
makePropertyKey method ( which is used to create propertyKey) or any other
similar method.
Then I saw certain java patch vertices getting created even before these
internal attributes are added to various indices. Though these patches were
applied later once all internal attributes were added to all the indices.
Now, these patch vertices have 9 attributes and we releaized these 9 attributes
are the only attributes missing from ES. So basically when patch vertices got
created and these vertices with their attributes got added to cassandra via
janusgraph, janusgraph automatically created propertyKey for all these
attributes (the janusgraph's makePropertyKey method is not called during this
process anywhere in the Atlas code). And because internal attributes were
getting added to indices in another thread at the same time, when code checked
for propertyKey, it was not null and so it did not add the property to the
mixed index.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
