[jira] [Created] (KAFKA-4305) Possible race condition in log segment truncation and Kafka Request Handler
Hari created KAFKA-4305:
---
Summary: Possible race condition in log segment truncation and
Kafka Request Handler
Key: KAFKA-4305
URL: https://issues.apache.org/jira/browse/KAFKA-4305
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 0.10.0.1
Reporter: Hari
Priority: Minor
We started to observe this trace in our Kafka Server logs after migration to
0.10.0.1
{code}
[2016-10-16 20:55:04,825] ERROR [KafkaApi-303] Error while responding to offset
request (kafka.server.KafkaApis)
java.lang.NullPointerException
at kafka.server.KafkaApis.fetchOffsetsBefore(KafkaApis.scala:585)
at kafka.server.KafkaApis.fetchOffsets(KafkaApis.scala:573)
at kafka.server.KafkaApis$$anonfun$27.apply(KafkaApis.scala:530)
at kafka.server.KafkaApis$$anonfun$27.apply(KafkaApis.scala:521)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at kafka.server.KafkaApis.handleOffsetRequest(KafkaApis.scala:521)
at kafka.server.KafkaApis.handle(KafkaApis.scala:78)
at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
at java.lang.Thread.run(Thread.java:745)
{code}
>From a quick investigation, I suspect this could happen because of race
>condition with log segment truncation and Kafka request handler ?
Any thoughts ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KAFKA-1707) ConsumerOffsetChecker shows none partitions assigned
Hari created KAFKA-1707: --- Summary: ConsumerOffsetChecker shows none partitions assigned Key: KAFKA-1707 URL: https://issues.apache.org/jira/browse/KAFKA-1707 Project: Kafka Issue Type: Bug Components: consumer Affects Versions: 0.8.0 Environment: HP 40 x Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz/1.2e+02GB Reporter: Hari Assignee: Neha Narkhede bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker shows some partitions having none consumers after re-balance triggered due to new consumer joined/disconnected to the group. The lag gets piling up till the partitions are assigned to it after re-balance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-1708) Consumers intermittently stop consuming till restart
Hari created KAFKA-1708:
---
Summary: Consumers intermittently stop consuming till restart
Key: KAFKA-1708
URL: https://issues.apache.org/jira/browse/KAFKA-1708
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 0.8.0
Environment: HP 40 x Intel(R) Xeon(R) CPU E5-2470 v2 @
2.40GHz/1.2e+02GB
Reporter: Hari
Assignee: Neha Narkhede
Using a simple consumer, and reading messages using StreamIterator noticed that
the consumptions suddenly stops and the lag starts building up till the
consumer is restarted. Below is the code snippet
final MapString, ListKafkaStreambyte[], byte[] streamsByName =
consumerConnector.createMessageStreams(topicCountMap);
ConsumerIteratorbyte[], byte[] streamIterator =
streamsByName.get(topicName).get(IDX_FIRST_ITEM).iterator();
if (streamIterator.hasNext()) {
final MessageAndMetadatabyte[], byte[] item =
streamIterator.next();
...
}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1707) ConsumerOffsetChecker shows none partitions assigned
[ https://issues.apache.org/jira/browse/KAFKA-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari updated KAFKA-1707: Description: bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker shows some partitions having none consumers after re-balance triggered due to new consumer joined/disconnected to the group. The lag gets piling up till the partitions are assigned to it usually after another re-balance trigger. (was: bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker shows some partitions having none consumers after re-balance triggered due to new consumer joined/disconnected to the group. The lag gets piling up till the partitions are assigned to it after re-balance.) ConsumerOffsetChecker shows none partitions assigned Key: KAFKA-1707 URL: https://issues.apache.org/jira/browse/KAFKA-1707 Project: Kafka Issue Type: Bug Components: consumer Affects Versions: 0.8.0 Environment: HP 40 x Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz/1.2e+02GB Reporter: Hari Assignee: Neha Narkhede Labels: patch bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker shows some partitions having none consumers after re-balance triggered due to new consumer joined/disconnected to the group. The lag gets piling up till the partitions are assigned to it usually after another re-balance trigger. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
