[
https://issues.apache.org/jira/browse/STORM-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224887#comment-15224887
]
Narendra Bidari commented on STORM-1680:
----------------------------------------
Submitted Pull Request, Storm-Kafka Project building on my local
STORM-1680, Added Kafka Spout Config FetchByte to Storm-Kafka #1309
> Provide configuration to set min fetch size in KafkaSpout
> ---------------------------------------------------------
>
> Key: STORM-1680
> URL: https://issues.apache.org/jira/browse/STORM-1680
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-kafka, trident
> Reporter: Sachin Pasalkar
> Assignee: Narendra Bidari
>
> Kafka consumer has provided the configuration to set minimum fetch size.
> However, storms kafka spout is not exposing these functionality. This is
> helpful in some case where someone writing data to hdfs & want file size of
> X.
> Below are changes needs to be done
> 1. In KafkaUtils class update fetchMessages API with below change
> FetchRequest fetchRequest = builder.addFetch(topic, partitionId, offset,
> config.fetchSizeBytes).clientId(config.clientId).maxWait(config.fetchMaxWait).minBytes(config.minFetchByte).build();
> 2. Update KafkaConfig class with instance variable as minFetchByte
> (Default value is 0 as mentioned in FetchRequestBuilder class)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
