Tobias Maier created STORM-2077: ----------------------------------- Summary: KafkaSpout doesn't retry failed tuples Key: STORM-2077 URL: https://issues.apache.org/jira/browse/STORM-2077 Project: Apache Storm Issue Type: Bug Components: storm-kafka Affects Versions: 1.0.2 Reporter: Tobias Maier
KafkaSpout does not retry all failed tuples. We used following Configuration: Map<String, Object> props = new HashMap<>(); props.put(KafkaSpoutConfig.Consumer.GROUP_ID, "c1"); props.put(KafkaSpoutConfig.Consumer.KEY_DESERIALIZER, ByteArrayDeserializer.class.getName()); props.put(KafkaSpoutConfig.Consumer.VALUE_DESERIALIZER, ByteArrayDeserializer.class.getName()); props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, broker.bootstrapServer()); KafkaSpoutStreams kafkaSpoutStreams = new KafkaSpoutStreams.Builder(FIELDS_KAFKA_EVENT, new String[]{"test-topic"}).build(); KafkaSpoutTuplesBuilder<byte[], byte[]> kafkaSpoutTuplesBuilder = new KafkaSpoutTuplesBuilder.Builder<>(new KeyValueKafkaSpoutTupleBuilder("test-topic")).build(); KafkaSpoutRetryService retryService = new KafkaSpoutLoggedRetryExponentialBackoff(KafkaSpoutLoggedRetryExponentialBackoff.TimeInterval.milliSeconds(1), KafkaSpoutLoggedRetryExponentialBackoff.TimeInterval.milliSeconds(1), 3, KafkaSpoutLoggedRetryExponentialBackoff.TimeInterval.seconds(1)); KafkaSpoutConfig<byte[], byte[]> config = new KafkaSpoutConfig.Builder<>(props, kafkaSpoutStreams, kafkaSpoutTuplesBuilder, retryService) .setFirstPollOffsetStrategy(UNCOMMITTED_LATEST) .setMaxUncommittedOffsets(30) .setOffsetCommitPeriodMs(10) .setMaxRetries(3) .build(); kafkaSpout = new org.apache.storm.kafka.spout.KafkaSpout<>(config); The downstream bolt fails every tuple and we expect, that those tuple will all be replayed. But that's not the case for every tuple. -- This message was sent by Atlassian JIRA (v6.3.4#6332)