Github user garrettlish commented on the issue: https://github.com/apache/incubator-eagle/pull/556 Thx @haoch @RalphSu. I thought we set async for kafka producer and what we need to change is using callback rather than future wait, it is wrong, thanks for pointing out. I have updated the code to add kafka_client_config (list of name/value map) to specify kafka producer configurations. By default, I only set producer.type=async. For batch.num.messages, queue.buffering.max.ms and queue.buffering.max.messages, I think we can use kafka producer default values. The only difference for default value is batch.num.messages, it is 200 if not specified. Could u please share with us what is your reason to set it to 3000? the kafka producer properties could be defined in publish properties as follows: { "name": "***", "properties" : { "kafka_broker": "***", "topics": "***", "kafka_client_config" : [ { "name" : "request.requrie.acks", "value": 1 }, { "name" : "producer.type", "value": "async" }, ... ] } }
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---