[
https://issues.apache.org/jira/browse/UNOMI-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerome Blanchard updated UNOMI-907:
-----------------------------------
Description:
h3. Context
The goal of this improvement is to check ElasticSearch persistence consistency
regarding the one expected by current Unomi version in order to detect any
problem early. Inconsistencies could be a wrong collection mapping or a missed
migration step.
Sometime we face a situation where a migration was not complete or an index
migration or reindexation broke the collection mapping. This is more or less
silent after startup and may lead to loss of data.
h3. Proposal
We could add those checks in the *PersistenceHealthCheckProvider* :
{code:java}
try {
if (service != null) {
builder.up();
//TODO : Improve checks here
if (!service.query("target", "profiles", null,
PropertyType.class).isEmpty()) {
builder.live();
}
}
} catch (Exception e) {
builder.error().withData("error", e.getMessage());
LOGGER.error("Error while checking persistence health", e);
}
{code}
Expected checks are :
- Check if all migrations are OK (this will imply to store an entity with
migrations steps already executed, like in flyway or liquibase)
- Check if collection's mapping is consistent with current Unomi's entities
version. This can be achieve by comparing compatibility (not a equals() but if
mandatory elements are present) between collection's mapping (GET
/index/_mapping) with json files in the persistence module
(/unomi/persistence-elasticsearch/core/src/main/resources/META-INF/cxs/mappings/*.json)
- Maybe also check collection's settings (max fields)
Thus if something is not compliant, the healthcheck should fail with also a
detailed log error message.
was:
The goal of this improvement is to check that expected Unomi's persistence is
consistent with the expected Unomi version that is running in order to detect
any problem early. Inconsistencies could be a wrong collection mapping or a
miss of migration.
Sometime we face a situation where a migration was not complete or an index
migration or reindexation broke the collection mapping. This is more or less
silent after startup and may lead to loss of data.
We could add those checks in the PersistenceHealthCheckProvider :
```java
try {
if (service != null) {
builder.up();
//TODO : Improve checks here
if (!service.query("target", "profiles", null,
PropertyType.class).isEmpty()) {
builder.live();
}
}
} catch (Exception e) {
builder.error().withData("error", e.getMessage());
LOGGER.error("Error while checking persistence health", e);
}
```
Expected checks are :
- Check if all migrations are OK (this will imply to store an entity with
migrations steps already executed, like in flyway or liquibase)
- Check if collection's mapping is consistent with current Unomi's entities
version. This can be achieve by comparing compatibility (not a equals() but if
mandatory elements are present) between collection's mapping (GET
/index/_mapping) with json files in the persistence module
(/unomi/persistence-elasticsearch/core/src/main/resources/META-INF/cxs/mappings/*.json)
- Maybe also check collection's settings (max fields)
Thus if something is not compliant, the healthcheck should fail with also a
detailed log error message.
> Add ES Persistence Consistency Check in HealthCheck
> ---------------------------------------------------
>
> Key: UNOMI-907
> URL: https://issues.apache.org/jira/browse/UNOMI-907
> Project: Apache Unomi
> Issue Type: Improvement
> Components: unomi(-core)
> Reporter: Jerome Blanchard
> Priority: Major
>
> h3. Context
> The goal of this improvement is to check ElasticSearch persistence
> consistency regarding the one expected by current Unomi version in order to
> detect any problem early. Inconsistencies could be a wrong collection mapping
> or a missed migration step.
> Sometime we face a situation where a migration was not complete or an index
> migration or reindexation broke the collection mapping. This is more or less
> silent after startup and may lead to loss of data.
> h3. Proposal
> We could add those checks in the *PersistenceHealthCheckProvider* :
> {code:java}
> try {
> if (service != null) {
> builder.up();
> //TODO : Improve checks here
> if (!service.query("target", "profiles", null,
> PropertyType.class).isEmpty()) {
> builder.live();
> }
> }
> } catch (Exception e) {
> builder.error().withData("error", e.getMessage());
> LOGGER.error("Error while checking persistence health", e);
> }
> {code}
> Expected checks are :
> - Check if all migrations are OK (this will imply to store an entity with
> migrations steps already executed, like in flyway or liquibase)
> - Check if collection's mapping is consistent with current Unomi's entities
> version. This can be achieve by comparing compatibility (not a equals() but
> if mandatory elements are present) between collection's mapping (GET
> /index/_mapping) with json files in the persistence module
> (/unomi/persistence-elasticsearch/core/src/main/resources/META-INF/cxs/mappings/*.json)
>
> - Maybe also check collection's settings (max fields)
> Thus if something is not compliant, the healthcheck should fail with also a
> detailed log error message.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
