Sachin Pasalkar created STORM-1680:
--------------------------------------
Summary: Provide configuration to set min fetch size in KafkaConfig
Key: STORM-1680
URL: https://issues.apache.org/jira/browse/STORM-1680
Project: Apache Storm
Issue Type: Improvement
Components: storm-kafka, trident
Reporter: Sachin Pasalkar
Kafka consumer has provided the configuration to set minimum fetch size.
However, storms kafka spout is not exposing these functionality. This is
helpful in some case where someone writing data to hdfs & want file size of X.
Below are changes needs to be done
1. In KafkaUtils class update fetchMessages API with below change
FetchRequest fetchRequest = builder.addFetch(topic, partitionId, offset,
config.fetchSizeBytes).clientId(config.clientId).maxWait(config.fetchMaxWait).minBytes(config.minFetchByte).build();
2. Update KafkaConfig class with instance variable as minFetchByte
(Default value is 0 as mentioned in FetchRequestBuilder class)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
