Hi all,
I'd like to improve the efficiency of Brooklyn persistence - depending
on your entities, a lot of files/objects can be written a lot of times.
This results in overly high traffic to the persistence directory (be it
an object store like S3, or the file system).
Below are three big improvements I plan to make - any objections/comments?
*
1. When an entity changes and is persisted, its 'adjuncts' (e.g.
policies and enrichers) are also re-persisted.*
I've created a PR to persist only the entity when it changes:
https://github.com/apache/brooklyn-server/pull/964.
The old behaviour was for legacy reasons. A long long time ago, some
policies/locations had been written such that they did not notify any
persistence-listeners when they changed. They piggybacked on the
entity's persistence.
There is a risk that some users still have such legacy
entities/policies/locations in use. I've therefore included a
feature-flag (`referencedObjectsPersistence`) to re-enable the old
behaviour.
---
*2. When an attribute is re-published with the same value, don't re-persist*
Currently, when `entity.sensors().set(attribute, val)` is called with
the same value as it already has, it still calls `onAttributeChange`
which causes the entity to be re-persisted.
We could guard this, to only trigger re-persistence when the value has
actually changed.
Note you can currently prevent an enricher re-publishing the attribute
by using the config `enricher.suppressDuplicates`. You can can do the
same for a sensor feed using `suppressDuplicates`.
---
*3. When any of the entity's attributes change, it is re-persisted
(capped at max once per second, configurable).*
We should mark some attributes as non-persisted. If their value changes,
it will not cause the entity to be re-persisted.
This would be especially useful for attributes like cpu-usage,
requests-per-second, etc.
In the Java code, there is already support for marking an attribute as
non-persisted. It will require some work to wire this up properly (e.g.
currently an attribute-change still triggers re-persistence, but that
attribute is then excluded).
Aled
