[
https://issues.apache.org/jira/browse/METRON-233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15336556#comment-15336556
]
ASF GitHub Bot commented on METRON-233:
---------------------------------------
GitHub user cestella opened a pull request:
https://github.com/apache/incubator-metron/pull/160
METRON-233: Expose Kafka producer and consumer configs for parser topologies
We need the ability to tune KafkaSpout and KafkaWriter properties in a more
granular way. Currently we do not expose more than a few parameters when
creating the KafkaProducer in the KafkaWriter and do not expose almost any of
the KafkaSpout configurations.
This PR should allow for the KafkaWriter to have arbitrary KafkaProducer
configs exposed in the zookeeper config and have the full range of KafkaSpout
configs be exposed from the CLI used to start parser topologies.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cestella/incubator-metron kafka_writer_config
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/160.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #160
----
commit 3eebff1f4c016b33d90fd2d0cfbacbe1125dc209
Author: cstella <ceste...@gmail.com>
Date: 2016-06-17T17:47:56Z
METRON-233: Expose Kafka producer and consumer configs for parser topologies
----
> Expose Kafka producer and consumer configs for parser topologies
> ----------------------------------------------------------------
>
> Key: METRON-233
> URL: https://issues.apache.org/jira/browse/METRON-233
> Project: Metron
> Issue Type: Improvement
> Reporter: Ajay Yadav
> Assignee: Casey Stella
>
> To tune parser topologies for different traffic and message size requires
> configuring kafka consumer and producer. Its important to configure
> properties like fetchSizeBytes, bufferSizeBytes for consumer and batch.size,
> buffer.memory for producer. We should expose consumer and producer configs
> via a property file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
