Github user cestella commented on the issue:
https://github.com/apache/metron/pull/958
# Testing Plan:
We presume
* `ZOOKEEPER` is an environment variable set to the zk quorum (e.g.
`node1:2181`)
* `BROKER` is an environment variable set to the broker (e.g. `node1:6667`)
## Unbiased Generation
In order to test this, we will:
1. create a new topic
```
# Create dummy_12_unbiased with 12 partitions
/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper $ZOOKEEPER
--create --topic dummy_12_unbiased --partitions 12 --replication-factor 1
```
2. Create some dummy data to send into kafka
```
# Generate some templates of dummy data that look like CSV in
~/dummy.templates
# With a number, a string, a GUID and the timestamp
for i in $(seq 1 100);do echo "$i,foo,\$METRON_GUID,\$METRON_TS";done >
~/dummy.templates
```
3. Write into that topic from a set of templates at a specified rate
```
# Write out 1000 messages per second to dummy_12_unbiased
# Monitor dummy_12_unbiased
# Each message is drawn from the set of templates in step 2.
# Stop after 60 seconds and report how much you've written
$METRON_HOME/bin/load_tool.sh -p 5 -ot dummy_12_unbiased -mt
dummy_12_unbiased -t ~/dummy.template -eps 1000 -z $ZOOKEEPER -tl 60000
```
* It should indicate that it generated something like 60k messages
4. Measure if the messages generated roughly match the requested rate.
```
# You will have to ctrl-c this in a few seconds
/usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
--bootstrap-server node1:6667 --topic dummy_12_topic --from-beginning | wc -l
```
* It should output something like `Processed a total of 60400 messages`,
which would be about 1000 messages per second.
---
