[
https://issues.apache.org/jira/browse/UNOMI-739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Griffon updated UNOMI-739:
--------------------------------
Description:
Currently the configuration is set by:
{code}
# Purge all monthly indexes (sessions/events) that have been created for a
specific number of months
org.apache.unomi.monthly.index.purge.existTime=${env:UNOMI_MONTHLY_INDEX_PURGE_EXISTTIME:-12}
{code}
it has to be changed to allow a purge by type: session and event
{code}
org.apache.unomi.sessions.purge.existTime=${env:UNOMI_SESSIONS_PURGE_EXISTTIME:-}
org.apache.unomi.events.purge.existTime=${env:UNOMI_EVENTS_PURGE_EXISTTIME:-}
{code}
*Old value should be deprecated and use as default if the new one are not set.*
h2. Implementation
2 mechanisms need to be implemented in the purge of sessions / events:
h3. Deleting old documents:
There is a job that currently purge monthly indices and profiles, the goal is
to replace the [monthly indices
purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L402]
by a dedicated method to remove events and sessions.
This method will work as the [profile
purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L333]
calling {{persistenceService.removeByQuery(..)}}.
Like in here:
https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L365
*A log need to be added to mention the number of document removed.*
h3. Removing empty indices.
When documents are deleted, it will happen that some indices contain no
document, we need to have a dedicated execution to remove the empty indices.
*A log need to be added to mention the deleted indices and the remaining
indices with the number of documents in each.*
h2. Handle TODOs
- Remove index deletion from the rollup configuration.
- Enable back and adapt integration test related to the purge.
was:
Currently the configuration is set by:
{code}
# Purge all monthly indexes (sessions/events) that have been created for a
specific number of months
org.apache.unomi.monthly.index.purge.existTime=${env:UNOMI_MONTHLY_INDEX_PURGE_EXISTTIME:-12}
{code}
it has to be changed to allow a purge by type: session and event
{code}
org.apache.unomi.sessions.purge.existTime=${env:UNOMI_SESSIONS_PURGE_EXISTTIME:-}
org.apache.unomi.events.purge.existTime=${env:UNOMI_EVENTS_PURGE_EXISTTIME:-}
{code}
Old value should be deprecated and use as default if the new one are not set.
h2. Implementation
2 mechanisms need to be implemented in the purge of sessions / events:
h3. Deleting old documents:
There is a job that currently purge monthly indices and profiles, the goal is
to replace the [monthly indices
purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L402]
by a dedicated method to remove events and sessions.
This method will work as the [profile
purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L333]
calling {{persistenceService.removeByQuery(..)}}.
Like in here:
https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L365
*A log need to be added to mention the number of document removed.*
h3. Removing empty indices.
When documents are deleted, it will happen that some indices contain no
document, we need to have a dedicated execution to remove the empty indices.
*A log need to be added to mention the deleted indices and the remaining
indices with the number of documents in each.*
h2. Handle TODOs
- Remove index deletion from the rollup configuration.
- Enable back and adapt integration test related to the purge.
> Implemenation of purge of sessions / events following rollover usage.
> ---------------------------------------------------------------------
>
> Key: UNOMI-739
> URL: https://issues.apache.org/jira/browse/UNOMI-739
> Project: Apache Unomi
> Issue Type: New Feature
> Reporter: David Griffon
> Priority: Major
>
> Currently the configuration is set by:
> {code}
> # Purge all monthly indexes (sessions/events) that have been created for a
> specific number of months
> org.apache.unomi.monthly.index.purge.existTime=${env:UNOMI_MONTHLY_INDEX_PURGE_EXISTTIME:-12}
> {code}
> it has to be changed to allow a purge by type: session and event
> {code}
> org.apache.unomi.sessions.purge.existTime=${env:UNOMI_SESSIONS_PURGE_EXISTTIME:-}
> org.apache.unomi.events.purge.existTime=${env:UNOMI_EVENTS_PURGE_EXISTTIME:-}
> {code}
> *Old value should be deprecated and use as default if the new one are not
> set.*
> h2. Implementation
> 2 mechanisms need to be implemented in the purge of sessions / events:
> h3. Deleting old documents:
> There is a job that currently purge monthly indices and profiles, the goal is
> to replace the [monthly indices
> purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L402]
> by a dedicated method to remove events and sessions.
> This method will work as the [profile
> purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L333]
> calling {{persistenceService.removeByQuery(..)}}.
> Like in here:
> https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L365
> *A log need to be added to mention the number of document removed.*
> h3. Removing empty indices.
> When documents are deleted, it will happen that some indices contain no
> document, we need to have a dedicated execution to remove the empty indices.
> *A log need to be added to mention the deleted indices and the remaining
> indices with the number of documents in each.*
> h2. Handle TODOs
> - Remove index deletion from the rollup configuration.
> - Enable back and adapt integration test related to the purge.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
