[
https://issues.apache.org/jira/browse/UNOMI-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francois Gerthoffert updated UNOMI-908:
---------------------------------------
Description:
h3. Context
Currently, the ClusterService implementation in Unomi stores each ClusterNode
as a document in Elasticsearch. A scheduled job updates a heartbeat timestamp
every 10 seconds.
h3. Problem
This approach is resource-intensive and not well-suited for the use case of
relying on ElasticCloud, where operational costs are directly impacted by the
number of indexes. The ClusterNode implementation creates an additional index,
unnecessarily increasing costs.
h3. Proposal
Replace the current persistence-based storage of ClusterNode objects with an
InMemory cache (also supporting distributed cache for clustered instances).
Infinispan could be a good candidate as it is available as a Karaf feature and
can be easily integrated.
This change will:
* Reduce operational costs by removing the need for a dedicated Elasticsearch
index for cluster nodes.
* Improve performance and scalability for cluster node management.
* Align with other planned features, such as a distributed entity cache to
avoid intensive polling, which will also leverage the Infinispan Karaf feature.
h3. Tasks
* Refactor ClusterServiceImpl to use distributed cache for storing and updating
ClusterNode information.
* Remove Elasticsearch-based persistence logic for ClusterNode.
* Ensure heartbeat updates are managed via distributed cache ; if not, rely on
the distributed cache underlying cluster management to manage ClusterNode
entities (JGroup for Infinispan)
* Update documentation to reflect the new architecture.
* Validate compatibility with existing and planned features using Infinispan.
h3. Definition of Done
* ClusterNode information is available and updated without Elasticsearch.
* No additional Elasticsearch index is created for cluster nodes.
* Heartbeat mechanism works reliably.
* All relevant documentation is updated.
* Integration tests confirm correct cluster node management and heartbeat
updates.
* No regression in cluster management functionality.
was:
h3. Context
Currently, the ClusterService implementation in Unomi stores each ClusterNode
as a document in Elasticsearch. A scheduled job updates a heartbeat timestamp
every 10 seconds.
h3. Problem
This approach is resource-intensive and not well-suited for the use case of
relying on ElasticCloud, where operational costs are directly impacted by the
number of indexes. The ClusterNode implementation creates an additional index,
unnecessarily increasing costs.
h3. Proposal
Replace the current persistence-based storage of ClusterNode objects with an
InMemory cache (also supporting distributed cache for clustered instances).
Infinispan could be a good candidate as it is available as a Karaf feature and
can be easily integrated. This change will:
Reduce operational costs by removing the need for a dedicated Elasticsearch
index for cluster nodes.
Improve performance and scalability for cluster node management.
Align with other planned features, such as a distributed entity cache to avoid
intensive polling, which will also leverage the Infinispan Karaf feature.
h3. Tasks
Refactor ClusterServiceImpl to use distributed cache for storing and updating
ClusterNode information.
Remove Elasticsearch-based persistence logic for ClusterNode.
Ensure heartbeat updates are managed via distributed cache ; if not, rely on
the distributed cache underlying cluster management to manage ClusterNode
entities (JGroup for Infinispan)
Update documentation to reflect the new architecture.
Validate compatibility with existing and planned features using Infinispan.
h3. Definition of Done
ClusterNode information is available and updated without Elasticsearch.
No additional Elasticsearch index is created for cluster nodes.
Heartbeat mechanism works reliably.
All relevant documentation is updated.
Integration tests confirm correct cluster node management and heartbeat updates.
No regression in cluster management functionality.
> Replace ClusterNode Persistence with Distributed Cache
> ------------------------------------------------------
>
> Key: UNOMI-908
> URL: https://issues.apache.org/jira/browse/UNOMI-908
> Project: Apache Unomi
> Issue Type: Improvement
> Components: unomi(-core)
> Affects Versions: unomi-3.0.0
> Reporter: Jerome Blanchard
> Priority: Major
>
> h3. Context
> Currently, the ClusterService implementation in Unomi stores each ClusterNode
> as a document in Elasticsearch. A scheduled job updates a heartbeat timestamp
> every 10 seconds.
> h3. Problem
> This approach is resource-intensive and not well-suited for the use case of
> relying on ElasticCloud, where operational costs are directly impacted by the
> number of indexes. The ClusterNode implementation creates an additional
> index, unnecessarily increasing costs.
> h3. Proposal
> Replace the current persistence-based storage of ClusterNode objects with an
> InMemory cache (also supporting distributed cache for clustered instances).
> Infinispan could be a good candidate as it is available as a Karaf feature
> and can be easily integrated.
> This change will:
> * Reduce operational costs by removing the need for a dedicated Elasticsearch
> index for cluster nodes.
> * Improve performance and scalability for cluster node management.
> * Align with other planned features, such as a distributed entity cache to
> avoid intensive polling, which will also leverage the Infinispan Karaf
> feature.
> h3. Tasks
> * Refactor ClusterServiceImpl to use distributed cache for storing and
> updating ClusterNode information.
> * Remove Elasticsearch-based persistence logic for ClusterNode.
> * Ensure heartbeat updates are managed via distributed cache ; if not, rely
> on the distributed cache underlying cluster management to manage ClusterNode
> entities (JGroup for Infinispan)
> * Update documentation to reflect the new architecture.
> * Validate compatibility with existing and planned features using Infinispan.
> h3. Definition of Done
> * ClusterNode information is available and updated without Elasticsearch.
> * No additional Elasticsearch index is created for cluster nodes.
> * Heartbeat mechanism works reliably.
> * All relevant documentation is updated.
> * Integration tests confirm correct cluster node management and heartbeat
> updates.
> * No regression in cluster management functionality.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
