venkata madugundu created ATLAS-683:
---------------------------------------
Summary: Refactor local type-system cache with cache provider
interface
Key: ATLAS-683
URL: https://issues.apache.org/jira/browse/ATLAS-683
Project: Atlas
Issue Type: Sub-task
Affects Versions: 0.7-incubating
Reporter: venkata madugundu
Priority: Critical
Fix For: 0.7-incubating
As noted in ATLAS-488, local type-system cache makes Atlas runtime stateful and
prevents multiple Atlas instances to be active in a cluster. Either the
type-cache should be synched across Atlas instances (on all type create/update
requests) or the type-cache should be moved out of Atlas to something like a
distributed cache.
1. As a first step, the local type-cache code in TypeSystem.java can be
refactored to be carved out as an interface like TypeCacheProvider (whose
default implementation for a standalone Atlas server would just use in-process
local cache). The cache provider implementation itself could be specified as an
optional configuration property. Expert users of Atlas can choose to inject a
custom cache provider which can likely hit a distributed cache. We are
evaluating the use of a distributed cache.
2. As a second step, some more refactoring can be done to minimize/optimize the
calls made to TypeSystem for type lookup queries. Essentially, in a given
transaction/request, once a type lookup is done, it should not be requeried
again. A request scoped variable (guice would probably help with that scoping)
can hold all the lookups made in a request. This might sound like a cache of a
cache, but I think it should help in reducing the hits to cache provider if the
provider is hitting a remote cache.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
