[
https://issues.apache.org/jira/browse/ATLAS-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Mestry reassigned ATLAS-4358:
--------------------------------------
Assignee: Ashutosh Mestry
> Mapping for some internal Atlas attributes ( like __patch.type , __timestamp,
> etc) does not exist in Elasticsearch
> ------------------------------------------------------------------------------------------------------------------
>
> Key: ATLAS-4358
> URL: https://issues.apache.org/jira/browse/ATLAS-4358
> Project: Atlas
> Issue Type: Bug
> Reporter: Anshul Mehta
> Assignee: Ashutosh Mestry
> Priority: Major
> Attachments: patch-manager-fix.patch
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> *Impact of the problem -*
> * Atlas pod taking much longer to become active and this time keep
> increasing as the assets increase. This basically means a downtime on every
> Atlas release.
> * Not able to filter via basic search on attributes like {{__timestamp}} ,
> {{__modificationTimestamp}} , {{createdBy}} and {{modifiedBy}} .
> *Issue -*
> So just before creating the mapping in the mixed index (ES index) Atlas
> creates something called {{propertyKey}} and this propertyKey is used to
> create the mapping. The code is written in a way that checks if propertyKey
> for the current property is null or not. If it is null it creates the
> propertyKey and then adds it to mixed index. If it is not Null it assumes
> that the property has already been added to the index and so skips adding it.
> Now in our case when Atlas checked the propertyKey it was not null (which
> should not have been the case) therefore Atlas skipped adding it to the mixed
> index and so these properties never got added to the mixed index. This
> basically meant propertyKey for these properties were getting created
> somewhere else. We looked into the entire codebase but could not find the use
> of makePropertyKey method ( which is used to create propertyKey) or any other
> similar method.
> Then I saw certain java patch vertices getting created even before these
> internal attributes are added to various indices. Though these patches were
> applied later once all internal attributes were added to all the indices.
> Now, these patch vertices have 9 attributes and we releaized these 9
> attributes are the only attributes missing from ES. So basically when patch
> vertices got created and these vertices with their attributes got added to
> cassandra via janusgraph, janusgraph automatically created propertyKey for
> all these attributes (the janusgraph's makePropertyKey method is not called
> during this process anywhere in the Atlas code). And because internal
> attributes were getting added to indices in another thread at the same time,
> when code checked for propertyKey, it was not null and so it did not add the
> property to the mixed index.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
