GitHub user cestella opened a pull request:
https://github.com/apache/incubator-metron/pull/415
METRON-652: Extract indexing config from enrichment config
Currently, the indexing configuration is bound and coupled with the sensor
enrichment configuration. This was done historically because indexing and
enrichment were part of the same topology. When the topologies were separated,
the configurations were never separated, leaving a confusing section about
indexes in the middle of the enrichment configuration. This effort will
separate out the configuration.
Because of the configuration's simplicity, we are treating the config as
just a `Map<String, Object>` and supporting the two existing configurations:
* `batchSize`
* `index`
Because this change is fairly deep, a few non-obvious things were also
updated for consistency and necessity:
* The Stellar Management functions `ENRICHMENT_SET_BATCH` and
`ENRICHMENT_SET_INDEX` functions have been renamed to `INDEXING_SET_BATCH` and
`INDEXING_SET_INDEX`
* Support in `CONFIG_GET` and `CONFIG_PUT` for retrieving/storing indexing
configurations was added.
* Support for indexing configurations were added to the zookeeper utility
function
* The indexing RPM was updated to include the new configs for the shipped
sensors.
I have *not* updated the management pack to create a new panel for managing
indexing configs. That will be part of a follow-on JIRA (METRON-653).
I have done smoke testing on quickdev, but will include more comprehensive
testing explanation in the comments. I feel that this has the possibility of
causing regressions, so I want to be explicit in testing methodology.
Furthermore, I plan on creating a wiki section about about migrations,
because this will necessitate migrating configs when upgrading from 0.3.0 to
the next version.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cestella/incubator-metron METRON-652
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/415.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #415
----
commit 1d03bc57235af2c1b90c901e7895f3c037a054c2
Author: cstella <[email protected]>
Date: 2017-01-10T14:25:33Z
initial update
commit 4c9227cb618a62bfea63b147c32054bfeadf7d08
Author: cstella <[email protected]>
Date: 2017-01-10T15:48:33Z
Updating testing components.
commit 3681edb32dca2876ecb2c0895c58a392ce558ada
Author: cstella <[email protected]>
Date: 2017-01-10T20:32:47Z
second round of changes.
commit facb6518b27fd19fd5c3f40671b57b84013c4076
Author: cstella <[email protected]>
Date: 2017-01-11T15:03:14Z
Merge branch 'master' into METRON-652
commit ec91c279af078c94c4d762bd8ed166a18fa78da3
Author: cstella <[email protected]>
Date: 2017-01-11T15:03:40Z
Merge branch 'master' into METRON-652
commit ed98341e904575f4d1e89533440cb920ee7a6e3d
Author: cstella <[email protected]>
Date: 2017-01-11T17:54:59Z
updating.
commit 375e0260ce2d12e38e66e8ca05740bf6f71e348c
Author: cstella <[email protected]>
Date: 2017-01-11T21:10:16Z
Updating documentation.
commit 3736294bfe9bb4f3f6c3862af300da49960da81f
Author: cstella <[email protected]>
Date: 2017-01-11T21:15:24Z
Updating RPM.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
