GitHub user ahgittin opened a pull request:
https://github.com/apache/incubator-brooklyn/pull/303
Persistence better logging and optimization
see commits for details
logging is now quite elegant (and code more elegant too). at `MASTER` we
log this for all persistence:
```
2014-11-06 01:53:24,484 DEBUG b.e.r.PeriodicDeltaChangeListener
[brooklyn-execmanager-cPHPyZLD-49]: Checkpointing delta of memento: updating
entities=[NginxControllerImpl{id=yj1CnqgS}, TomcatServerImpl{id=jFLN539t},
DynamicWebAppClusterImpl{id=mOp5gagc},
ControlledDynamicWebAppClusterImpl{id=cuPk22K9}], locations=[], policies=[],
enrichers=[], catalog items=[]; removing entities=[], locations=[],
policies=[], enrichers=[], catalog items=[]
2014-11-06 01:53:24,510 DEBUG b.e.r.p.BrooklynMementoPersisterToObjectStore
[brooklyn-execmanager-cPHPyZLD-49]: Checkpointed delta of memento in 15ms:
updated 4 entities, 3 locations, 2 policies, 35 enrichers, 0 catalog items;
removed 0 entities, 0 locations, 0 policies, 0 enrichers, 0 catalog items
```
at `HOT_STANDBY` you see the following most of the time (much more detail
on first 5 rebinds and every 1000th, which is about every ~15m):
```
2014-11-06 01:53:24,221 DEBUG b.e.rebind.RebindManagerImpl
[brooklyn-execmanager-IEUSHnfF-1]: Rebinding (HOT_STANDBY, iteration 12) from
/tmp/mms-amp-data/master...
2014-11-06 01:53:24,223 DEBUG b.e.r.p.BrooklynMementoPersisterToObjectStore
[brooklyn-execmanager-IEUSHnfF-1]: Loaded rebind lists; took 1ms: 7 entities, 3
locations, 2 policies, 48 enrichers, 0 feeds, 0 catalog items; from
/tmp/mms-amp-data/master
2014-11-06 01:53:24,292 DEBUG b.e.r.p.BrooklynMementoPersisterToObjectStore
[brooklyn-execmanager-IEUSHnfF-1]: Loaded rebind raw data; took 69ms; 7
entities, 3 locations, 2 policies, 48 enrichers, 0 feeds, 0 catalog items, from
/tmp/mms-amp-data/master
2014-11-06 01:53:24,467 DEBUG b.e.r.p.BrooklynMementoPersisterToObjectStore
[brooklyn-execmanager-IEUSHnfF-1]: Loaded rebind manifests; took 174ms: 7
entities, 3 locations, 2 policies, 48 enrichers, 0 feeds, 0 catalog items; from
/tmp/mms-amp-data/master
2014-11-06 01:53:24,533 DEBUG b.e.r.p.BrooklynMementoPersisterToObjectStore
[brooklyn-execmanager-IEUSHnfF-1]: Loaded rebind mementos; took 5ms: 7
entities, 3 locations, 2 policies, 48 enrichers, 0 feeds, 0 catalog items, from
/tmp/mms-amp-data/master
2014-11-06 01:53:24,570 DEBUG b.e.rebind.RebindManagerImpl
[brooklyn-execmanager-IEUSHnfF-1]: Rebind complete (HOT_STANDBY, iteration 12)
in 349ms: 1 app, 7 entities, 3 locations, 2 policies, 48 enrichers, 0 feeds, 0
catalog items
```
and all nodes get such entries as this:
```
2014-11-06 01:53:24,371 DEBUG
b.m.h.ManagementPlaneSyncRecordPersisterToObjectStore
[brooklyn-execmanager-IEUSHnfF-3]: Checkpointed delta of manager-memento in
1ms: brooklyn.management.ha.ManagementPlaneSyncRecordDeltaImpl[nodes:
[BasicManagementNodeSyncRecord{nodeId=IEUSHnfF, status=HOT_STANDBY}]]
```
each appears once per second, meaning in happy path neither standby nor
master logs too much.
there is some slight hack to prevent rebind over-logging, but i don't think
it's too bad!
`trace` level restores full logging
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ahgittin/incubator-brooklyn
persistence-better-logging
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-brooklyn/pull/303.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #303
----
commit 28150112ebf66c1caef1c281a3589a32e1299148
Author: Alex Heneveld <[email protected]>
Date: 2014-11-05T22:16:32Z
improve logging for persistence and rebind
a few more detailed messages, including timings;
and lots of suppression of messages during redind-read-only
commit 283d17067f1b5eb5954708a2f9a106dec6ef0bfc
Author: Alex Heneveld <[email protected]>
Date: 2014-11-05T22:39:37Z
set entity RO status much earlier when known
mainly to enable suppression of log messages that are hugely noisy in
read-only mode
commit f6316ea4fad0905362b384ed86a08f17a5f87d7e
Author: Alex Heneveld <[email protected]>
Date: 2014-11-06T02:01:31Z
optimize rebind process
remove extraneous obj store lists+reads from process, and a bunch of
code/redundancy tidy, and a bit more log tidy
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
