Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/760
### Test Plan
1. Run up full dev and verify you have data flowing through to the indexes
2. Change global config options.
1. Go into the Ambari management UI and modify any of the following
config options. e.g. set the es_clustername property to "ivebeenchanged". These
are values that end up in global.json and will prompt for a component restart.
Restart the components.
- es.clustername
- es.ip
- es.date.format
- parser.error.topic
- update.hbase.table
- update.hbase.cf
- profiler.client.period.duration
- profiler.client.period.duration.units
2. Verify $METRON_HOME/config/zookeeper/global.json now has the new
value. For the example above, you would expect to see
es.clustername=ivebeenchanged.
3. Verify Zookeeper contains the new config.
`$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m DUMP -c
GLOBAL`
3. Test changing configs from the command line. The zk config utils has
been updated with new backwards compatible features, so we'll test that here.
1. Change one of the zookeeper configs locally and then try a vanilla
config PUSH using the original method. Verify the values made it into zookeeper.
`${METRON_HOME}/bin/zk_load_configs.sh -i
${METRON_HOME}/config/zookeeper -m PUSH -z $ZOOKEEPER`
2. Change a config locally and push JUST that config to Zookeeper by
using one of the new config loader options. You can then perform a DUMP in a
similar manner.
```
# global config only
$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PUSH -c GLOBAL
# bro config only
$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PUSH -c PARSER
-n bro
# take a dump of config
$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m DUMP -c GLOBAL
$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m DUMP -c PARSER
-n bro
```
3. Try the new JSON patch mechanism with key/value from the command
line. Below are some examples but please feel free to explore other
combinations. **Note** - commons cli strips the first outermost pair of quotes
from all arguments, so you have to provide escaped quotes twice. Neither
\""bar"\" nor "\"bar\"" will work. You must provide both escaped quotes for
string values. \"\"bar\"\". Complex objects are a bit easier, but you still
have to escape quotes inside the string to get them to parse properly.
```
# global config only - add a new key "foo" with simple string value
"bar"
$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c
GLOBAL -pm ADD -pk foo -pv \"\"bar\"\"
# global config only - add a new key "foo" to the JSON doc root
with complex value "{ \"bar\" : { \"baz\" : [ \"bazval1\", \"bazval2\" ] } }"
$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c
GLOBAL -pm ADD -pk "/foo" -pv "{ \"bar\" : { \"baz\" : [ \"bazval1\",
\"bazval2\" ] } }"
# global config only - remove a key altogether
$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c
GLOBAL -pm REMOVE -pk "/foo"
```
4. Try the new JSON patch mechanism with a patch file. Below are some
examples but please feel free to explore other combinations. Note that a patch
file is a JSON array of individual patches.
1. Try a single patch from file
```
# Create a single patch that adds a new node. Create file
"/tmp/mypatch.txt" and add the following JSON array.
[
{
"op": "add",
"path": "/you",
"value": { "cannot" : { "handle" : [ "the", "flow" ] } }
}
]
# Perform the patch
$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c GLOBAL
-pf /tmp/mypatch.txt
# Should result in adding the following to global.json
"you" : {
"cannot" : {
"handle" : [ "the", "flow" ]
}
}
```
2. Try multiple patches in a patch file
```
Create a single patch file that adds multiple nodes. Create file
"/tmp/mygrimespatch.txt" and add the following JSON array.
# 2 patches in a single patch file
[
{
"op": "add",
"path": "/la",
"value": { "jiggy" : { "jar" : { "jar" : "do" } } }
},
{
"op": "add",
"path": "/carl",
"value": [ "poppa", "rhymes" ]
}
]
# Perform the patch
$METRON_HOME/bin/zk_load_configs.sh -z $ZOOKEEPER -m PATCH -c
GLOBAL -pf /tmp/mygrimespatch.txt
# Should result in adding the following to global.json
"la" : {
"jiggy" : {
"jar" : {
"jar" : "do"
}
}
},
"carl" : [ "poppa", "rhymes" ]
```
4. Enable Kerberos to verify config changes still work on service restarts.
---
