[
https://issues.apache.org/jira/browse/ATLAS-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235354#comment-17235354
]
Yuvraj Singh Shekhawat commented on ATLAS-488:
----------------------------------------------
Do we have any update on running multiple atlas instances [~vmadugun]? We are
running atlas inside kubernetes but we are only able to run it with 1 replica.
We tried using it with HA but in case of new version. It goes down for a while
since it takes around 1 minute to change from PASSIVE to ACTIVE. We are looking
to have a zero downtime releases process in which case having multiple atlas
instances running would be great. There is also a possibility of changing the
algorithm we use for leader election- Take down ACTIVE instance only when
another PASSIVE is about to become ACTIVE. With this downtime can be decreased
to few seconds. Right now it takes down ACTIVE instance then it waits for a
PASSIVE instance to become ACTIVE.
> Atlas service scalability
> -------------------------
>
> Key: ATLAS-488
> URL: https://issues.apache.org/jira/browse/ATLAS-488
> Project: Atlas
> Issue Type: Improvement
> Reporter: venkata madugundu
> Priority: Major
>
> Requirement -- We are looking for ways to have Atlas service running in
> multiple instances for Cloud deployment, and this requirement is kind of
> crucial to the adoption of Atlas.
> On the dev discussion forum, I had discussed earlier (Jan 25th) on the
> requirement to run multiple instances of Atlas service/runtime for horizontal
> scale. Per answers that we got from Hemanth, Atlas has a current limitation
> that only instance can be active at any given point of time for a couple of
> reasons. Few reasons cited were ...
> 1) For performance reasons, the typesystem is cached in-memory. Type lookup
> is needed for query evaluation, instance serialization/deserialization etc.,
> 2) Titan locks for Hbase transactions, causing performance degradation.
> Considering the above reasons, among them,
> #1 can be mitigated in our scenarios. The types that we create are more or
> less design-time application data-models (metamodels) and they do not change
> at runtime. So if I were to pre-register types with Atlas before my cluster
> is up, I am hoping #1 is not an issue. In the sense, how many ever Atlas
> instances come up, all would have the same state, which would mean I am
> guaranteed API correctness.
> I am not so sure how we can avoid #2. I neither understand the persistence
> mapping of metadata to the graph(Atlas) and the mapping of graph to the Hbase
> (Titan) to comment on whether #2 is really a problem for us in terms of
> lock-contention. Atlas as REST API inherently does not offer any
> "transaction" like facility to run a set of API for its end-users. Given
> that, we are more likely to implement some kind of optimistic locking
> strategy to avoid data inconsistencies.
> Are there are any other reasons, that you can think of why multiple Atlas
> runtime instances cannot be talking to a single Hbase cluster for horizontal
> scale ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)