GitHub user nickwallen opened a pull request:
https://github.com/apache/incubator-metron/pull/387
METRON-606
[METRON-606](https://issues.apache.org/jira/browse/METRON-606)
The Profiler is repetitively overwriting previously written values. Where I
would expect to see a series of multiple values in Hbase, one written per
flush, there is only a single value. This same value is being repetitively
overwritten with new data.
The following changes are included in the PR.
* Refactored the ProfileBuilderBolt so that the logic for building a
profile no longer lives in the bolt itself, but in a separate ProfileBuilder.
The bolt simply maintains a cache of ProfileBuilder instances.
* Ensured that ProfileMeasurement objects are not reused, which reduces the
likelihood of another defect like this in the future.
* Added a Clock interface that allows me to advance time in the unit tests.
This was necessary for testing this specific defect. This will also help with
METRON-590 in dealing with processing versus event time.
* Added a specific unit test for this defect.
* Added a unit test for the recent 'tickUpdate' addition to the Profiler.
The change was tested on a live AWS cluster along with the 'Quick Dev'
environment.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/incubator-metron METRON-606
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/387.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #387
----
commit 4e9fc6307356e9dcb71ec1d9ed78644b1e7f3eb3
Author: Nick Allen <[email protected]>
Date: 2016-11-25T23:24:22Z
METRON-606 Extracted ProfileBuilder logic from the ProfileBuilderBolt.
commit deaaf4cb5066e1c03f20b45bb62a577724cdf22f
Author: Nick Allen <[email protected]>
Date: 2016-12-02T23:07:43Z
METRON-606 Ensure that a new profile measurement is created on each flush
commit cad4674df2eaf67bcfbc8de3cc8fa74a8f1a2b77
Author: Nick Allen <[email protected]>
Date: 2016-12-02T23:34:46Z
METRON-606 Added a unit test specifically for METRON-606
commit d5bfa09248018ad009ac20ccf4032082f4c06d74
Author: Nick Allen <[email protected]>
Date: 2016-12-02T23:53:04Z
METRON-606 Added missing ASF license headers
commit 8b0d84336503898300cf17ddc8ac78c8c25932d3
Author: Nick Allen <[email protected]>
Date: 2016-12-03T01:15:57Z
METRON-606 Exception message can cause an NPE
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
