[jira] [Assigned] (KAFKA-2339) broker becomes unavailable if bad data is passed through the protocol
[
https://issues.apache.org/jira/browse/KAFKA-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Chen reassigned KAFKA-2339:
---
Assignee: Timothy Chen
broker becomes unavailable if bad data is passed through the protocol
-
Key: KAFKA-2339
URL: https://issues.apache.org/jira/browse/KAFKA-2339
Project: Kafka
Issue Type: Bug
Reporter: Joe Stein
Assignee: Timothy Chen
Priority: Critical
Fix For: 0.8.3
I ran into a situation that a non integer value got past for the partition
and the brokers went bonkers.
reproducible
{code}
ah=1..2
echo don't do this in production|kafkacat -b localhost:9092 -p $ah
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-2339) broker becomes unavailable if bad data is passed through the protocol
[
https://issues.apache.org/jira/browse/KAFKA-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630981#comment-14630981
]
Timothy Chen commented on KAFKA-2339:
-
Somehow I can't produce this with kafkacat and latest trunk:
Timothys-MacBook-Pro :: ~/src/kafka ‹trunk*› » kafkacat -P -p 3..5 -t test -b
localhost:9092
1 ↵
abc
% ERROR: Failed to produce message (3 bytes): Local: Unknown partition
Which version of Kafka are you using?
broker becomes unavailable if bad data is passed through the protocol
-
Key: KAFKA-2339
URL: https://issues.apache.org/jira/browse/KAFKA-2339
Project: Kafka
Issue Type: Bug
Reporter: Joe Stein
Assignee: Timothy Chen
Priority: Critical
Fix For: 0.8.3
I ran into a situation that a non integer value got past for the partition
and the brokers went bonkers.
reproducible
{code}
ah=1..2
echo don't do this in production|kafkacat -b localhost:9092 -p $ah
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-2188) JBOD Support
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627708#comment-14627708 ] Timothy Chen commented on KAFKA-2188: - Created reviewboard https://reviews.apache.org/r/36503/diff/ against branch origin/trunk JBOD Support Key: KAFKA-2188 URL: https://issues.apache.org/jira/browse/KAFKA-2188 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2188.patch, KAFKA-2188.patch, KAFKA-2188.patch https://cwiki.apache.org/confluence/display/KAFKA/KIP-18+-+JBOD+Support -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2188) JBOD Support
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-2188: Attachment: KAFKA-2188.patch JBOD Support Key: KAFKA-2188 URL: https://issues.apache.org/jira/browse/KAFKA-2188 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2188.patch, KAFKA-2188.patch, KAFKA-2188.patch https://cwiki.apache.org/confluence/display/KAFKA/KIP-18+-+JBOD+Support -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2188) JBOD Support
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625910#comment-14625910 ] Timothy Chen commented on KAFKA-2188: - Created reviewboard https://reviews.apache.org/r/36474/diff/ against branch origin/trunk JBOD Support Key: KAFKA-2188 URL: https://issues.apache.org/jira/browse/KAFKA-2188 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2188.patch, KAFKA-2188.patch https://cwiki.apache.org/confluence/display/KAFKA/KIP-18+-+JBOD+Support -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2188) JBOD Support
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-2188: Attachment: KAFKA-2188.patch JBOD Support Key: KAFKA-2188 URL: https://issues.apache.org/jira/browse/KAFKA-2188 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2188.patch, KAFKA-2188.patch https://cwiki.apache.org/confluence/display/KAFKA/KIP-18+-+JBOD+Support -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1443) Add delete topic to topic commands and update DeleteTopicCommand
[ https://issues.apache.org/jira/browse/KAFKA-1443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1443: Attachment: KAFKA-1443_2014-06-04_20:51:44.patch Add delete topic to topic commands and update DeleteTopicCommand Key: KAFKA-1443 URL: https://issues.apache.org/jira/browse/KAFKA-1443 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Attachments: KAFKA-1443.patch, KAFKA-1443.patch, KAFKA-1443_2014-06-04_20:51:44.patch Add delete topic option to current topic commands -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1443) Add delete topic to topic commands and update DeleteTopicCommand
[ https://issues.apache.org/jira/browse/KAFKA-1443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993760#comment-13993760 ] Timothy Chen commented on KAFKA-1443: - Created reviewboard https://reviews.apache.org/r/21272/ against branch origin/trunk Add delete topic to topic commands and update DeleteTopicCommand Key: KAFKA-1443 URL: https://issues.apache.org/jira/browse/KAFKA-1443 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Attachments: KAFKA-1443.patch, KAFKA-1443.patch Add delete topic option to current topic commands -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1443) Add delete topic to topic commands and update DeleteTopicCommand
[ https://issues.apache.org/jira/browse/KAFKA-1443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1443: Attachment: KAFKA-1443.patch Add delete topic to topic commands and update DeleteTopicCommand Key: KAFKA-1443 URL: https://issues.apache.org/jira/browse/KAFKA-1443 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Attachments: KAFKA-1443.patch, KAFKA-1443.patch Add delete topic option to current topic commands -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1384: Attachment: KAFKA-1384_2014-05-06_13:21:14.patch Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch, KAFKA-1384_2014-05-01_18:47:03.patch, KAFKA-1384_2014-05-02_12:13:24.patch, KAFKA-1384_2014-05-05_11:05:15.patch, KAFKA-1384_2014-05-05_14:25:25.patch, KAFKA-1384_2014-05-05_17:14:57.patch, KAFKA-1384_2014-05-06_13:21:14.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991080#comment-13991080 ] Timothy Chen commented on KAFKA-1384: - Updated reviewboard https://reviews.apache.org/r/20718/ against branch origin/trunk Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch, KAFKA-1384_2014-05-01_18:47:03.patch, KAFKA-1384_2014-05-02_12:13:24.patch, KAFKA-1384_2014-05-05_11:05:15.patch, KAFKA-1384_2014-05-05_14:25:25.patch, KAFKA-1384_2014-05-05_17:14:57.patch, KAFKA-1384_2014-05-06_13:21:14.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989785#comment-13989785 ] Timothy Chen commented on KAFKA-1384: - Updated reviewboard https://reviews.apache.org/r/20718/ against branch origin/trunk Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch, KAFKA-1384_2014-05-01_18:47:03.patch, KAFKA-1384_2014-05-02_12:13:24.patch, KAFKA-1384_2014-05-05_11:05:15.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989793#comment-13989793 ] Timothy Chen commented on KAFKA-1397: - Updated reviewboard https://reviews.apache.org/r/20745/ against branch origin/trunk delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch, KAFKA-1397_2014-04-30_14:55:28.patch, KAFKA-1397_2014-05-01_15:53:57.patch, KAFKA-1397_2014-05-01_18:12:24.patch, KAFKA-1397_2014-05-02_13:38:02.patch, KAFKA-1397_2014-05-05_11:17:59.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1397: Attachment: KAFKA-1397_2014-05-05_11:17:59.patch delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch, KAFKA-1397_2014-04-30_14:55:28.patch, KAFKA-1397_2014-05-01_15:53:57.patch, KAFKA-1397_2014-05-01_18:12:24.patch, KAFKA-1397_2014-05-02_13:38:02.patch, KAFKA-1397_2014-05-05_11:17:59.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989968#comment-13989968 ] Timothy Chen commented on KAFKA-1397: - Updated reviewboard https://reviews.apache.org/r/20745/ against branch origin/trunk delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch, KAFKA-1397_2014-04-30_14:55:28.patch, KAFKA-1397_2014-05-01_15:53:57.patch, KAFKA-1397_2014-05-01_18:12:24.patch, KAFKA-1397_2014-05-02_13:38:02.patch, KAFKA-1397_2014-05-05_11:17:59.patch, KAFKA-1397_2014-05-05_14:00:29.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989994#comment-13989994 ] Timothy Chen commented on KAFKA-1384: - Updated reviewboard https://reviews.apache.org/r/20718/ against branch origin/trunk Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch, KAFKA-1384_2014-05-01_18:47:03.patch, KAFKA-1384_2014-05-02_12:13:24.patch, KAFKA-1384_2014-05-05_11:05:15.patch, KAFKA-1384_2014-05-05_14:25:25.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990123#comment-13990123 ] Timothy Chen commented on KAFKA-1384: - Updated reviewboard https://reviews.apache.org/r/20718/ against branch origin/trunk Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch, KAFKA-1384_2014-05-01_18:47:03.patch, KAFKA-1384_2014-05-02_12:13:24.patch, KAFKA-1384_2014-05-05_11:05:15.patch, KAFKA-1384_2014-05-05_14:25:25.patch, KAFKA-1384_2014-05-05_17:14:57.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1384: Attachment: KAFKA-1384_2014-05-05_17:14:57.patch Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch, KAFKA-1384_2014-05-01_18:47:03.patch, KAFKA-1384_2014-05-02_12:13:24.patch, KAFKA-1384_2014-05-05_11:05:15.patch, KAFKA-1384_2014-05-05_14:25:25.patch, KAFKA-1384_2014-05-05_17:14:57.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1384: Attachment: KAFKA-1384_2014-05-02_12:13:24.patch Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch, KAFKA-1384_2014-05-01_18:47:03.patch, KAFKA-1384_2014-05-02_12:13:24.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988111#comment-13988111 ] Timothy Chen commented on KAFKA-1384: - Updated reviewboard https://reviews.apache.org/r/20718/ against branch origin/trunk Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch, KAFKA-1384_2014-05-01_18:47:03.patch, KAFKA-1384_2014-05-02_12:13:24.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1397: Attachment: KAFKA-1397_2014-05-02_13:38:02.patch delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch, KAFKA-1397_2014-04-30_14:55:28.patch, KAFKA-1397_2014-05-01_15:53:57.patch, KAFKA-1397_2014-05-01_18:12:24.patch, KAFKA-1397_2014-05-02_13:38:02.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988206#comment-13988206 ] Timothy Chen commented on KAFKA-1397: - Updated reviewboard https://reviews.apache.org/r/20745/ against branch origin/trunk delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch, KAFKA-1397_2014-04-30_14:55:28.patch, KAFKA-1397_2014-05-01_15:53:57.patch, KAFKA-1397_2014-05-01_18:12:24.patch, KAFKA-1397_2014-05-02_13:38:02.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1397: Attachment: KAFKA-1397_2014-05-01_15:53:57.patch delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch, KAFKA-1397_2014-04-30_14:55:28.patch, KAFKA-1397_2014-05-01_15:53:57.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987127#comment-13987127 ] Timothy Chen commented on KAFKA-1397: - Updated reviewboard https://reviews.apache.org/r/20745/ against branch origin/trunk delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch, KAFKA-1397_2014-04-30_14:55:28.patch, KAFKA-1397_2014-05-01_15:53:57.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1397: Attachment: KAFKA-1397_2014-05-01_18:12:24.patch delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch, KAFKA-1397_2014-04-30_14:55:28.patch, KAFKA-1397_2014-05-01_15:53:57.patch, KAFKA-1397_2014-05-01_18:12:24.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987259#comment-13987259 ] Timothy Chen commented on KAFKA-1384: - Updated reviewboard https://reviews.apache.org/r/20718/ against branch origin/trunk Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch, KAFKA-1384_2014-05-01_18:47:03.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1384: Attachment: KAFKA-1384_2014-05-01_18:47:03.patch Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch, KAFKA-1384_2014-05-01_18:47:03.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986142#comment-13986142 ] Timothy Chen commented on KAFKA-1397: - Updated reviewboard https://reviews.apache.org/r/20745/ against branch origin/trunk delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch, KAFKA-1397_2014-04-30_14:55:28.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1397: Attachment: KAFKA-1397_2014-04-30_14:55:28.patch delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch, KAFKA-1397_2014-04-30_14:55:28.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1397: Attachment: KAFKA-1397_2014-04-28_14:48:32.patch delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983619#comment-13983619 ] Timothy Chen commented on KAFKA-1397: - Updated reviewboard https://reviews.apache.org/r/20745/ against branch origin/trunk delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1397: Attachment: KAFKA-1397_2014-04-28_17:08:49.patch delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983759#comment-13983759 ] Timothy Chen commented on KAFKA-1397: - Updated reviewboard https://reviews.apache.org/r/20745/ against branch origin/trunk delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch, KAFKA-1397_2014-04-28_14:48:32.patch, KAFKA-1397_2014-04-28_17:08:49.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1397: Attachment: KAFKA-1397.patch delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1397) delete topic is not working
[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982206#comment-13982206 ] Timothy Chen commented on KAFKA-1397: - Created reviewboard https://reviews.apache.org/r/20745/ against branch origin/trunk delete topic is not working Key: KAFKA-1397 URL: https://issues.apache.org/jira/browse/KAFKA-1397 Project: Kafka Issue Type: Bug Affects Versions: 0.8.2 Reporter: Jun Rao Attachments: KAFKA-1397.patch All unit tests are disabled since they hang transiently (see details in KAFKA-1391). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981909#comment-13981909 ] Timothy Chen commented on KAFKA-1384: - Updated reviewboard https://reviews.apache.org/r/20718/ against branch origin/trunk Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1384: Attachment: KAFKA-1384_2014-04-26_00:09:56.patch Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch, KAFKA-1384_2014-04-26_00:09:56.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981272#comment-13981272 ] Timothy Chen commented on KAFKA-1384: - I'm planning just to expose it via JMX, as we don't have a need to write it to zookeeper as other brokers doesn't have a need to query it. Less contention to ZK as well, you have anything in mind why you thought putting it in ZK would be good? Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981275#comment-13981275 ] Timothy Chen commented on KAFKA-1384: - Created reviewboard https://reviews.apache.org/r/20718/ against branch origin/trunk Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1384: Attachment: KAFKA-1384.patch Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1384.patch Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1356: Attachment: KAFKA-1356.patch Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356.patch, KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch, KAFKA-1356_2014-04-04_17:45:37.patch, KAFKA-1356_2014-04-06_01:45:47.patch, KAFKA-1356_2014-04-08_01:38:23.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13966312#comment-13966312 ] Timothy Chen commented on KAFKA-1356: - Created reviewboard https://reviews.apache.org/r/20252/ against branch origin/0.8.1 Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356.patch, KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch, KAFKA-1356_2014-04-04_17:45:37.patch, KAFKA-1356_2014-04-06_01:45:47.patch, KAFKA-1356_2014-04-08_01:38:23.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1363) testTopicConfigChangesDuringDeleteTopic hangs
[ https://issues.apache.org/jira/browse/KAFKA-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13964526#comment-13964526 ] Timothy Chen commented on KAFKA-1363: - Updated reviewboard https://reviews.apache.org/r/20143/ against branch origin/trunk testTopicConfigChangesDuringDeleteTopic hangs - Key: KAFKA-1363 URL: https://issues.apache.org/jira/browse/KAFKA-1363 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8.1 Reporter: Jun Rao Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1363.patch, KAFKA-1363_2014-04-08_17:52:17.patch, KAFKA-1363_2014-04-09_11:38:09.patch Saw the following deadlock during shutting down the delete topic manager. delete-topics-thread prio=10 tid=0x7fd50c003800 nid=0x7d9 waiting on condition [0x7fd53d16] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b41d6318 (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:363) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Test worker prio=10 tid=0x7fd578928800 nid=0x763d waiting on condition [0x7fd570a87000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b5b6f580 (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) at kafka.utils.ShutdownableThread.shutdown(ShutdownableThread.scala:36) at kafka.controller.TopicDeletionManager.shutdown(TopicDeletionManager.scala:100) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:345) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply$mcV$sp(KafkaController.scala:648) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.shutdown(KafkaController.scala:646) at kafka.server.KafkaServer$$anonfun$shutdown$9.apply$mcV$sp(KafkaServer.scala:242) at kafka.utils.Utils$.swallow(Utils.scala:166) at kafka.utils.Logging$class.swallowWarn(Logging.scala:92) at kafka.utils.Utils$.swallowWarn(Utils.scala:45) at kafka.utils.Logging$class.swallow(Logging.scala:94) at kafka.utils.Utils$.swallow(Utils.scala:45) at kafka.server.KafkaServer.shutdown(KafkaServer.scala:242) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61) at scala.collection.immutable.List.foreach(List.scala:45) at
[jira] [Updated] (KAFKA-1363) testTopicConfigChangesDuringDeleteTopic hangs
[ https://issues.apache.org/jira/browse/KAFKA-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1363: Attachment: KAFKA-1363_2014-04-09_11:38:09.patch testTopicConfigChangesDuringDeleteTopic hangs - Key: KAFKA-1363 URL: https://issues.apache.org/jira/browse/KAFKA-1363 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8.1 Reporter: Jun Rao Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1363.patch, KAFKA-1363_2014-04-08_17:52:17.patch, KAFKA-1363_2014-04-09_11:38:09.patch Saw the following deadlock during shutting down the delete topic manager. delete-topics-thread prio=10 tid=0x7fd50c003800 nid=0x7d9 waiting on condition [0x7fd53d16] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b41d6318 (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:363) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Test worker prio=10 tid=0x7fd578928800 nid=0x763d waiting on condition [0x7fd570a87000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b5b6f580 (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) at kafka.utils.ShutdownableThread.shutdown(ShutdownableThread.scala:36) at kafka.controller.TopicDeletionManager.shutdown(TopicDeletionManager.scala:100) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:345) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply$mcV$sp(KafkaController.scala:648) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.shutdown(KafkaController.scala:646) at kafka.server.KafkaServer$$anonfun$shutdown$9.apply$mcV$sp(KafkaServer.scala:242) at kafka.utils.Utils$.swallow(Utils.scala:166) at kafka.utils.Logging$class.swallowWarn(Logging.scala:92) at kafka.utils.Utils$.swallowWarn(Utils.scala:45) at kafka.utils.Logging$class.swallow(Logging.scala:94) at kafka.utils.Utils$.swallow(Utils.scala:45) at kafka.server.KafkaServer.shutdown(KafkaServer.scala:242) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61) at scala.collection.immutable.List.foreach(List.scala:45) at kafka.admin.DeleteTopicTest.testTopicConfigChangesDuringDeleteTopic(DeleteTopicTest.scala:362) -- This message was
[jira] [Updated] (KAFKA-1363) testTopicConfigChangesDuringDeleteTopic hangs
[ https://issues.apache.org/jira/browse/KAFKA-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1363: Attachment: KAFKA-1363.patch testTopicConfigChangesDuringDeleteTopic hangs - Key: KAFKA-1363 URL: https://issues.apache.org/jira/browse/KAFKA-1363 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8.1 Reporter: Jun Rao Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1363.patch, KAFKA-1363.patch, KAFKA-1363_2014-04-08_17:52:17.patch, KAFKA-1363_2014-04-09_11:38:09.patch Saw the following deadlock during shutting down the delete topic manager. delete-topics-thread prio=10 tid=0x7fd50c003800 nid=0x7d9 waiting on condition [0x7fd53d16] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b41d6318 (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:363) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Test worker prio=10 tid=0x7fd578928800 nid=0x763d waiting on condition [0x7fd570a87000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b5b6f580 (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) at kafka.utils.ShutdownableThread.shutdown(ShutdownableThread.scala:36) at kafka.controller.TopicDeletionManager.shutdown(TopicDeletionManager.scala:100) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:345) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply$mcV$sp(KafkaController.scala:648) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.shutdown(KafkaController.scala:646) at kafka.server.KafkaServer$$anonfun$shutdown$9.apply$mcV$sp(KafkaServer.scala:242) at kafka.utils.Utils$.swallow(Utils.scala:166) at kafka.utils.Logging$class.swallowWarn(Logging.scala:92) at kafka.utils.Utils$.swallowWarn(Utils.scala:45) at kafka.utils.Logging$class.swallow(Logging.scala:94) at kafka.utils.Utils$.swallow(Utils.scala:45) at kafka.server.KafkaServer.shutdown(KafkaServer.scala:242) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61) at scala.collection.immutable.List.foreach(List.scala:45) at kafka.admin.DeleteTopicTest.testTopicConfigChangesDuringDeleteTopic(DeleteTopicTest.scala:362) -- This message was sent
[jira] [Commented] (KAFKA-1363) testTopicConfigChangesDuringDeleteTopic hangs
[ https://issues.apache.org/jira/browse/KAFKA-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13964877#comment-13964877 ] Timothy Chen commented on KAFKA-1363: - Created reviewboard https://reviews.apache.org/r/20187/ against branch origin/0.8.1 testTopicConfigChangesDuringDeleteTopic hangs - Key: KAFKA-1363 URL: https://issues.apache.org/jira/browse/KAFKA-1363 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8.1 Reporter: Jun Rao Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1363.patch, KAFKA-1363.patch, KAFKA-1363_2014-04-08_17:52:17.patch, KAFKA-1363_2014-04-09_11:38:09.patch Saw the following deadlock during shutting down the delete topic manager. delete-topics-thread prio=10 tid=0x7fd50c003800 nid=0x7d9 waiting on condition [0x7fd53d16] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b41d6318 (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:363) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Test worker prio=10 tid=0x7fd578928800 nid=0x763d waiting on condition [0x7fd570a87000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b5b6f580 (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) at kafka.utils.ShutdownableThread.shutdown(ShutdownableThread.scala:36) at kafka.controller.TopicDeletionManager.shutdown(TopicDeletionManager.scala:100) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:345) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply$mcV$sp(KafkaController.scala:648) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.shutdown(KafkaController.scala:646) at kafka.server.KafkaServer$$anonfun$shutdown$9.apply$mcV$sp(KafkaServer.scala:242) at kafka.utils.Utils$.swallow(Utils.scala:166) at kafka.utils.Logging$class.swallowWarn(Logging.scala:92) at kafka.utils.Utils$.swallowWarn(Utils.scala:45) at kafka.utils.Logging$class.swallow(Logging.scala:94) at kafka.utils.Utils$.swallow(Utils.scala:45) at kafka.server.KafkaServer.shutdown(KafkaServer.scala:242) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61) at scala.collection.immutable.List.foreach(List.scala:45) at
[jira] [Updated] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1356: Attachment: KAFKA-1356.patch Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch, KAFKA-1356_2014-04-04_17:45:37.patch, KAFKA-1356_2014-04-06_01:45:47.patch, KAFKA-1356_2014-04-08_01:38:23.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13964888#comment-13964888 ] Timothy Chen commented on KAFKA-1356: - Created reviewboard https://reviews.apache.org/r/20190/ against branch origin/trunk Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch, KAFKA-1356_2014-04-04_17:45:37.patch, KAFKA-1356_2014-04-06_01:45:47.patch, KAFKA-1356_2014-04-08_01:38:23.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (KAFKA-1384) Log Broker state
Timothy Chen created KAFKA-1384: --- Summary: Log Broker state Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (KAFKA-1384) Log Broker state
[ https://issues.apache.org/jira/browse/KAFKA-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen reassigned KAFKA-1384: --- Assignee: Timothy Chen Log Broker state - Key: KAFKA-1384 URL: https://issues.apache.org/jira/browse/KAFKA-1384 Project: Kafka Issue Type: New Feature Reporter: Timothy Chen Assignee: Timothy Chen Currently we don't have visibility into what state the broker is currently in, ie: Startup - Running - Waiting Controlled shutdown - Shutting down So without knowing what state the broker it is it's hard to figure out what the current broker is performing. This ticket is to add a new metric to expose the current broker state. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13962707#comment-13962707 ] Timothy Chen commented on KAFKA-1356: - Updated reviewboard https://reviews.apache.org/r/19957/ against branch origin/trunk Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch, KAFKA-1356_2014-04-04_17:45:37.patch, KAFKA-1356_2014-04-06_01:45:47.patch, KAFKA-1356_2014-04-08_01:38:23.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1356: Attachment: KAFKA-1356_2014-04-08_01:38:23.patch Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch, KAFKA-1356_2014-04-04_17:45:37.patch, KAFKA-1356_2014-04-06_01:45:47.patch, KAFKA-1356_2014-04-08_01:38:23.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1363) testTopicConfigChangesDuringDeleteTopic hangs
[ https://issues.apache.org/jira/browse/KAFKA-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1363: Attachment: KAFKA-1363.patch testTopicConfigChangesDuringDeleteTopic hangs - Key: KAFKA-1363 URL: https://issues.apache.org/jira/browse/KAFKA-1363 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8.1 Reporter: Jun Rao Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1363.patch Saw the following deadlock during shutting down the delete topic manager. delete-topics-thread prio=10 tid=0x7fd50c003800 nid=0x7d9 waiting on condition [0x7fd53d16] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b41d6318 (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:363) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Test worker prio=10 tid=0x7fd578928800 nid=0x763d waiting on condition [0x7fd570a87000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b5b6f580 (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) at kafka.utils.ShutdownableThread.shutdown(ShutdownableThread.scala:36) at kafka.controller.TopicDeletionManager.shutdown(TopicDeletionManager.scala:100) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:345) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply$mcV$sp(KafkaController.scala:648) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.shutdown(KafkaController.scala:646) at kafka.server.KafkaServer$$anonfun$shutdown$9.apply$mcV$sp(KafkaServer.scala:242) at kafka.utils.Utils$.swallow(Utils.scala:166) at kafka.utils.Logging$class.swallowWarn(Logging.scala:92) at kafka.utils.Utils$.swallowWarn(Utils.scala:45) at kafka.utils.Logging$class.swallow(Logging.scala:94) at kafka.utils.Utils$.swallow(Utils.scala:45) at kafka.server.KafkaServer.shutdown(KafkaServer.scala:242) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61) at scala.collection.immutable.List.foreach(List.scala:45) at kafka.admin.DeleteTopicTest.testTopicConfigChangesDuringDeleteTopic(DeleteTopicTest.scala:362) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1363) testTopicConfigChangesDuringDeleteTopic hangs
[ https://issues.apache.org/jira/browse/KAFKA-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13963673#comment-13963673 ] Timothy Chen commented on KAFKA-1363: - Created reviewboard https://reviews.apache.org/r/20143/ against branch origin/trunk testTopicConfigChangesDuringDeleteTopic hangs - Key: KAFKA-1363 URL: https://issues.apache.org/jira/browse/KAFKA-1363 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8.1 Reporter: Jun Rao Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1363.patch Saw the following deadlock during shutting down the delete topic manager. delete-topics-thread prio=10 tid=0x7fd50c003800 nid=0x7d9 waiting on condition [0x7fd53d16] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b41d6318 (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:363) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Test worker prio=10 tid=0x7fd578928800 nid=0x763d waiting on condition [0x7fd570a87000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b5b6f580 (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) at kafka.utils.ShutdownableThread.shutdown(ShutdownableThread.scala:36) at kafka.controller.TopicDeletionManager.shutdown(TopicDeletionManager.scala:100) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:345) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply$mcV$sp(KafkaController.scala:648) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.shutdown(KafkaController.scala:646) at kafka.server.KafkaServer$$anonfun$shutdown$9.apply$mcV$sp(KafkaServer.scala:242) at kafka.utils.Utils$.swallow(Utils.scala:166) at kafka.utils.Logging$class.swallowWarn(Logging.scala:92) at kafka.utils.Utils$.swallowWarn(Utils.scala:45) at kafka.utils.Logging$class.swallow(Logging.scala:94) at kafka.utils.Utils$.swallow(Utils.scala:45) at kafka.server.KafkaServer.shutdown(KafkaServer.scala:242) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61) at scala.collection.immutable.List.foreach(List.scala:45) at kafka.admin.DeleteTopicTest.testTopicConfigChangesDuringDeleteTopic(DeleteTopicTest.scala:362) -- This
[jira] [Updated] (KAFKA-1363) testTopicConfigChangesDuringDeleteTopic hangs
[ https://issues.apache.org/jira/browse/KAFKA-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1363: Attachment: KAFKA-1363_2014-04-08_17:52:17.patch testTopicConfigChangesDuringDeleteTopic hangs - Key: KAFKA-1363 URL: https://issues.apache.org/jira/browse/KAFKA-1363 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8.1 Reporter: Jun Rao Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1363.patch, KAFKA-1363_2014-04-08_17:52:17.patch Saw the following deadlock during shutting down the delete topic manager. delete-topics-thread prio=10 tid=0x7fd50c003800 nid=0x7d9 waiting on condition [0x7fd53d16] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b41d6318 (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:363) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Test worker prio=10 tid=0x7fd578928800 nid=0x763d waiting on condition [0x7fd570a87000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b5b6f580 (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) at kafka.utils.ShutdownableThread.shutdown(ShutdownableThread.scala:36) at kafka.controller.TopicDeletionManager.shutdown(TopicDeletionManager.scala:100) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:345) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply$mcV$sp(KafkaController.scala:648) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.shutdown(KafkaController.scala:646) at kafka.server.KafkaServer$$anonfun$shutdown$9.apply$mcV$sp(KafkaServer.scala:242) at kafka.utils.Utils$.swallow(Utils.scala:166) at kafka.utils.Logging$class.swallowWarn(Logging.scala:92) at kafka.utils.Utils$.swallowWarn(Utils.scala:45) at kafka.utils.Logging$class.swallow(Logging.scala:94) at kafka.utils.Utils$.swallow(Utils.scala:45) at kafka.server.KafkaServer.shutdown(KafkaServer.scala:242) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61) at scala.collection.immutable.List.foreach(List.scala:45) at kafka.admin.DeleteTopicTest.testTopicConfigChangesDuringDeleteTopic(DeleteTopicTest.scala:362) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1363) testTopicConfigChangesDuringDeleteTopic hangs
[ https://issues.apache.org/jira/browse/KAFKA-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13963675#comment-13963675 ] Timothy Chen commented on KAFKA-1363: - Updated reviewboard https://reviews.apache.org/r/20143/ against branch origin/trunk testTopicConfigChangesDuringDeleteTopic hangs - Key: KAFKA-1363 URL: https://issues.apache.org/jira/browse/KAFKA-1363 Project: Kafka Issue Type: Bug Components: core Affects Versions: 0.8.1 Reporter: Jun Rao Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1363.patch, KAFKA-1363_2014-04-08_17:52:17.patch Saw the following deadlock during shutting down the delete topic manager. delete-topics-thread prio=10 tid=0x7fd50c003800 nid=0x7d9 waiting on condition [0x7fd53d16] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b41d6318 (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:363) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Test worker prio=10 tid=0x7fd578928800 nid=0x763d waiting on condition [0x7fd570a87000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x0006b5b6f580 (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) at kafka.utils.ShutdownableThread.shutdown(ShutdownableThread.scala:36) at kafka.controller.TopicDeletionManager.shutdown(TopicDeletionManager.scala:100) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:345) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:341) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:341) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply$mcV$sp(KafkaController.scala:648) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.controller.KafkaController$$anonfun$shutdown$1.apply(KafkaController.scala:646) at kafka.utils.Utils$.inLock(Utils.scala:537) at kafka.controller.KafkaController.shutdown(KafkaController.scala:646) at kafka.server.KafkaServer$$anonfun$shutdown$9.apply$mcV$sp(KafkaServer.scala:242) at kafka.utils.Utils$.swallow(Utils.scala:166) at kafka.utils.Logging$class.swallowWarn(Logging.scala:92) at kafka.utils.Utils$.swallowWarn(Utils.scala:45) at kafka.utils.Logging$class.swallow(Logging.scala:94) at kafka.utils.Utils$.swallow(Utils.scala:45) at kafka.server.KafkaServer.shutdown(KafkaServer.scala:242) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at kafka.admin.DeleteTopicTest$$anonfun$testTopicConfigChangesDuringDeleteTopic$1.apply(DeleteTopicTest.scala:362) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61) at scala.collection.immutable.List.foreach(List.scala:45) at
[jira] [Updated] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1356: Attachment: KAFKA-1356_2014-04-06_01:45:47.patch Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch, KAFKA-1356_2014-04-04_17:45:37.patch, KAFKA-1356_2014-04-06_01:45:47.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13961351#comment-13961351 ] Timothy Chen commented on KAFKA-1356: - Updated reviewboard https://reviews.apache.org/r/19957/ against branch origin/trunk Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch, KAFKA-1356_2014-04-04_17:45:37.patch, KAFKA-1356_2014-04-06_01:45:47.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13960456#comment-13960456 ] Timothy Chen commented on KAFKA-1356: - Updated reviewboard https://reviews.apache.org/r/19957/ against branch origin/trunk Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1356: Attachment: KAFKA-1356_2014-04-04_14:40:18.patch Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1356: Attachment: KAFKA-1356_2014-04-04_17:45:37.patch Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch, KAFKA-1356_2014-04-04_17:45:37.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13960865#comment-13960865 ] Timothy Chen commented on KAFKA-1356: - Updated reviewboard https://reviews.apache.org/r/19957/ against branch origin/trunk Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch, KAFKA-1356_2014-04-04_14:40:18.patch, KAFKA-1356_2014-04-04_17:45:37.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1354) Failed to load class org.slf4j.impl.StaticLoggerBinder
[ https://issues.apache.org/jira/browse/KAFKA-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958574#comment-13958574 ] Timothy Chen commented on KAFKA-1354: - Hi Rakesh, this is actually a very common message you get usually when Kafka doesn't find a log4j.properties file to configure a logger to use. How did you start the broker and do you have log4j.properties in the config folder where you read your server.properties? Failed to load class org.slf4j.impl.StaticLoggerBinder Key: KAFKA-1354 URL: https://issues.apache.org/jira/browse/KAFKA-1354 Project: Kafka Issue Type: Bug Components: log Affects Versions: 0.8.1 Environment: RHEL Reporter: RakeshAcharya Assignee: Jay Kreps Labels: newbie, patch, usability Original Estimate: 672h Remaining Estimate: 672h Getting below errors during Kafka startup SLF4J: Failed to load class org.slf4j.impl.StaticLoggerBinder. SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. [2014-03-31 18:55:36,488] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1323) log.dirs server property no longer supports relative directories
[
https://issues.apache.org/jira/browse/KAFKA-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Chen updated KAFKA-1323:
Attachment: KAFKA-1323_2014-04-02_12:17:57.patch
log.dirs server property no longer supports relative directories
Key: KAFKA-1323
URL: https://issues.apache.org/jira/browse/KAFKA-1323
Project: Kafka
Issue Type: Bug
Reporter: Joel Koshy
Priority: Blocker
Fix For: 0.8.1.1
Attachments: KAFKA-1323.patch, KAFKA-1323_2014-03-27_16:17:26.patch,
KAFKA-1323_2014-03-31_11:57:37.patch, KAFKA-1323_2014-04-02_12:17:57.patch,
kafka-1323-trunk-test-failures.png
This seems to have been caused by KAFKA-1315 - we now don't support relative
directories.
Steps to reproduce:
* Set a relative directory for log.dirs. E.g., {{log.dirs=data/kafka-logs}}
* Bring up the broker and produce some messages:
{{./bin/kafka-producer-perf-test.sh --broker-list localhost:9092 --messages
1000 --topic test}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1323) log.dirs server property no longer supports relative directories
[
https://issues.apache.org/jira/browse/KAFKA-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958061#comment-13958061
]
Timothy Chen commented on KAFKA-1323:
-
Updated reviewboard https://reviews.apache.org/r/19626/
against branch origin/trunk
log.dirs server property no longer supports relative directories
Key: KAFKA-1323
URL: https://issues.apache.org/jira/browse/KAFKA-1323
Project: Kafka
Issue Type: Bug
Reporter: Joel Koshy
Priority: Blocker
Fix For: 0.8.1.1
Attachments: KAFKA-1323.patch, KAFKA-1323_2014-03-27_16:17:26.patch,
KAFKA-1323_2014-03-31_11:57:37.patch, KAFKA-1323_2014-04-02_12:17:57.patch,
kafka-1323-trunk-test-failures.png
This seems to have been caused by KAFKA-1315 - we now don't support relative
directories.
Steps to reproduce:
* Set a relative directory for log.dirs. E.g., {{log.dirs=data/kafka-logs}}
* Bring up the broker and produce some messages:
{{./bin/kafka-producer-perf-test.sh --broker-list localhost:9092 --messages
1000 --topic test}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1356: Attachment: KAFKA-1356.patch Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958153#comment-13958153 ] Timothy Chen commented on KAFKA-1356: - Created reviewboard https://reviews.apache.org/r/19957/ against branch origin/trunk Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (KAFKA-1358) Broker throws exception when reconnecting to zookeeper
Timothy Chen created KAFKA-1358: --- Summary: Broker throws exception when reconnecting to zookeeper Key: KAFKA-1358 URL: https://issues.apache.org/jira/browse/KAFKA-1358 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Assignee: Timothy Chen A non-controller broker currently if zk session expires and re-established calls onControllerResignation even though it may not be the controller. The result is that the broker gets exception like this: java.lang.NullPointerException at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:340) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:337) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:337) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:337) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply$mcZ$sp(KafkaController.scala:1068) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1067) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1067) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.KafkaController$SessionExpirationListener.handleNewSession(KafkaController.scala:1067) at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472) at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1358) Broker throws exception when reconnecting to zookeeper
[ https://issues.apache.org/jira/browse/KAFKA-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958387#comment-13958387 ] Timothy Chen commented on KAFKA-1358: - Created reviewboard https://reviews.apache.org/r/19972/ against branch origin/0.8.1 Broker throws exception when reconnecting to zookeeper -- Key: KAFKA-1358 URL: https://issues.apache.org/jira/browse/KAFKA-1358 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1358.patch A non-controller broker currently if zk session expires and re-established calls onControllerResignation even though it may not be the controller. The result is that the broker gets exception like this: java.lang.NullPointerException at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:340) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:337) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:337) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:337) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply$mcZ$sp(KafkaController.scala:1068) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1067) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1067) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.KafkaController$SessionExpirationListener.handleNewSession(KafkaController.scala:1067) at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472) at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1358) Broker throws exception when reconnecting to zookeeper
[ https://issues.apache.org/jira/browse/KAFKA-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1358: Fix Version/s: 0.8.1.1 Broker throws exception when reconnecting to zookeeper -- Key: KAFKA-1358 URL: https://issues.apache.org/jira/browse/KAFKA-1358 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1358.patch A non-controller broker currently if zk session expires and re-established calls onControllerResignation even though it may not be the controller. The result is that the broker gets exception like this: java.lang.NullPointerException at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:340) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:337) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:337) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:337) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply$mcZ$sp(KafkaController.scala:1068) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1067) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1067) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.KafkaController$SessionExpirationListener.handleNewSession(KafkaController.scala:1067) at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472) at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1358) Broker throws exception when reconnecting to zookeeper
[ https://issues.apache.org/jira/browse/KAFKA-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1358: Status: Patch Available (was: Open) Broker throws exception when reconnecting to zookeeper -- Key: KAFKA-1358 URL: https://issues.apache.org/jira/browse/KAFKA-1358 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1358.patch A non-controller broker currently if zk session expires and re-established calls onControllerResignation even though it may not be the controller. The result is that the broker gets exception like this: java.lang.NullPointerException at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:340) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:337) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:337) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:337) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply$mcZ$sp(KafkaController.scala:1068) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1067) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1067) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.KafkaController$SessionExpirationListener.handleNewSession(KafkaController.scala:1067) at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472) at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1358) Broker throws exception when reconnecting to zookeeper
[ https://issues.apache.org/jira/browse/KAFKA-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958388#comment-13958388 ] Timothy Chen commented on KAFKA-1358: - Looks like we already have this guard in trunk, adding this to the 0.8.1 branch Broker throws exception when reconnecting to zookeeper -- Key: KAFKA-1358 URL: https://issues.apache.org/jira/browse/KAFKA-1358 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Assignee: Timothy Chen Fix For: 0.8.1.1 Attachments: KAFKA-1358.patch A non-controller broker currently if zk session expires and re-established calls onControllerResignation even though it may not be the controller. The result is that the broker gets exception like this: java.lang.NullPointerException at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply$mcV$sp(KafkaController.scala:340) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:337) at kafka.controller.KafkaController$$anonfun$onControllerResignation$1.apply(KafkaController.scala:337) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:337) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply$mcZ$sp(KafkaController.scala:1068) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1067) at kafka.controller.KafkaController$SessionExpirationListener$$anonfun$handleNewSession$1.apply(KafkaController.scala:1067) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.KafkaController$SessionExpirationListener.handleNewSession(KafkaController.scala:1067) at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472) at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1356: Attachment: KAFKA-1356_2014-04-02_18:39:36.patch Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13958428#comment-13958428 ] Timothy Chen commented on KAFKA-1356: - Updated reviewboard https://reviews.apache.org/r/19957/ against branch origin/trunk Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Attachments: KAFKA-1356.patch, KAFKA-1356_2014-04-02_18:39:36.patch Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (KAFKA-1356) Topic metadata requests takes too long to process
Timothy Chen created KAFKA-1356: --- Summary: Topic metadata requests takes too long to process Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Currently we're seeing slow response times in handling both get topic metadata requests and update metadata requests. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1356) Topic metadata requests takes too long to process
[ https://issues.apache.org/jira/browse/KAFKA-1356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1356: Description: Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. was: Currently we're seeing slow response times in handling both get topic metadata requests and update metadata requests. Priority: Major (was: Critical) Fix Version/s: (was: 0.8.1.1) Topic metadata requests takes too long to process - Key: KAFKA-1356 URL: https://issues.apache.org/jira/browse/KAFKA-1356 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Timothy Chen Currently we're seeing slow response times in handling get topic metadata requests. Local testing shows that even locally it takes 300 avg ms to respond, even though it's not doing any IO operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1323) log.dirs server property no longer supports relative directories
[
https://issues.apache.org/jira/browse/KAFKA-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Chen updated KAFKA-1323:
Attachment: KAFKA-1323_2014-03-31_11:57:37.patch
log.dirs server property no longer supports relative directories
Key: KAFKA-1323
URL: https://issues.apache.org/jira/browse/KAFKA-1323
Project: Kafka
Issue Type: Bug
Reporter: Joel Koshy
Priority: Blocker
Fix For: 0.8.1.1
Attachments: KAFKA-1323.patch, KAFKA-1323_2014-03-27_16:17:26.patch,
KAFKA-1323_2014-03-31_11:57:37.patch, kafka-1323-trunk-test-failures.png
This seems to have been caused by KAFKA-1315 - we now don't support relative
directories.
Steps to reproduce:
* Set a relative directory for log.dirs. E.g., {{log.dirs=data/kafka-logs}}
* Bring up the broker and produce some messages:
{{./bin/kafka-producer-perf-test.sh --broker-list localhost:9092 --messages
1000 --topic test}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1323) log.dirs server property no longer supports relative directories
[
https://issues.apache.org/jira/browse/KAFKA-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13955532#comment-13955532
]
Timothy Chen commented on KAFKA-1323:
-
Updated reviewboard https://reviews.apache.org/r/19626/
against branch origin/trunk
log.dirs server property no longer supports relative directories
Key: KAFKA-1323
URL: https://issues.apache.org/jira/browse/KAFKA-1323
Project: Kafka
Issue Type: Bug
Reporter: Joel Koshy
Priority: Blocker
Fix For: 0.8.1.1
Attachments: KAFKA-1323.patch, KAFKA-1323_2014-03-27_16:17:26.patch,
KAFKA-1323_2014-03-31_11:57:37.patch, kafka-1323-trunk-test-failures.png
This seems to have been caused by KAFKA-1315 - we now don't support relative
directories.
Steps to reproduce:
* Set a relative directory for log.dirs. E.g., {{log.dirs=data/kafka-logs}}
* Bring up the broker and produce some messages:
{{./bin/kafka-producer-perf-test.sh --broker-list localhost:9092 --messages
1000 --topic test}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1350) Fix excessive state change logging
[ https://issues.apache.org/jira/browse/KAFKA-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13955694#comment-13955694 ] Timothy Chen commented on KAFKA-1350: - Ah got it, just learned that Scala has a call-by-name / call-by-value difference, and our logging uses call-by-name. You're right the format won't be evaluated with our trait. Fix excessive state change logging -- Key: KAFKA-1350 URL: https://issues.apache.org/jira/browse/KAFKA-1350 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Joel Koshy Assignee: Neha Narkhede Priority: Blocker Fix For: 0.8.1.1 Attachments: KAFKA-1350.patch, KAFKA-1350_2014-03-29_23:28:07.patch I can provide steps to reproduce this issue. The state change logger needs to be guarded (to check if trace logging is turned on or not). The delete topic patch significantly increased the amount of logging that we do both on the controller. This results in higher latencies in state transitions and can slow down the controller (as well as the broker). This slow-down was how we ran into KAFKA-1342. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1349) Fatal error during KafkaServerStable startup
[
https://issues.apache.org/jira/browse/KAFKA-1349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13955747#comment-13955747
]
Timothy Chen commented on KAFKA-1349:
-
Hi Pan, the code links you provided is actually not bugs as the brokerId passed
in was to verify the return value read from Zookeeper, and the type was kept
Any since we want to be able to cast to any object we want to verify.
Back to the issue you're seeing, we don't see this problem from our environment
or from anyone else.
Can you perhaps test with a clean zookeeper and try again? And make sure the
jars you are running kafka with is cleaned and rebuilt just to make sure you're
not running old versions of kafka jars.
Fatal error during KafkaServerStable startup
Key: KAFKA-1349
URL: https://issues.apache.org/jira/browse/KAFKA-1349
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 0.8.1
Environment: Ubuntu + scala 2.10 + Kafka 0.8.1 release version
Reporter: Pan Pan
Fix For: 0.8.1
Kafka broker kept on restarting itself. Exceptions are shown in server.log:
[2014-03-28 15:09:01,624] INFO Will not load MX4J, mx4j-tools.jar is not in
the classpath (kafka.utils.Mx4jLoader$)
[2014-03-28 15:09:01,635] INFO conflict in /controller data: 2108872 stored
data: {version:1,brokerid:2310736,timestamp:1396014892001}
(kafka.utils.ZkUtils$)
[2014-03-28 15:09:01,638] FATAL Fatal error during KafkaServerStable startup.
Prepare to shutdown (kafka.server.KafkaServerStartable)
java.lang.NumberFormatException: For input string:
{version:1,brokerid:2310736,timestamp:1396014892001}
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:481)
at java.lang.Integer.parseInt(Integer.java:514)
at
scala.collection.immutable.StringLike$class.toInt(StringLike.scala:207)
at scala.collection.immutable.StringOps.toInt(StringOps.scala:31)
at
kafka.server.ZookeeperLeaderElector.elect(ZookeeperLeaderElector.scala:60)
at
kafka.server.ZookeeperLeaderElector.startup(ZookeeperLeaderElector.scala:43)
at kafka.controller.KafkaController.startup(KafkaController.scala:396)
at kafka.server.KafkaServer.startup(KafkaServer.scala:96)
at
kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
at kafka.Kafka$.main(Kafka.scala:46)
at kafka.Kafka.main(Kafka.scala)
[2014-03-28 15:09:01,639] INFO [Kafka Server 2108872], Shutting down
(kafka.server.KafkaServer)
[2014-03-28 15:09:01,640] INFO Closing zookeeper client...
(kafka.server.KafkaZooKeeper)
[2014-03-28 15:09:01,644] INFO [Socket Server on Broker 2108872], Shutting
down (kafka.network.SocketServer)
[2014-03-28 15:09:01,648] INFO [Socket Server on Broker 2108872], Shutdown
completed (kafka.network.SocketServer)
I checked the source code, there are bugs in code:
https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=blob;f=core/src/main/scala/kafka/server/ZookeeperLeaderElector.scala;h=33b73609b1178c56e692fb60e35aca04ad1af586;hb=8e554c4d2acf5108805905b9f06198f20398ee3a
https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=blob;f=core/src/main/scala/kafka/utils/ZkUtils.scala;h=d1c4b3d7b3d11c88a1d1474aadbb727704cfb759;hb=8e554c4d2acf5108805905b9f06198f20398ee3a
In ZookeeperLeaderElector.scala line 57, Int brokerId is passed in as fourth
parameter, while in ZkUtils.scala line 307, the declaration for the fourth
paramenter is expectedCallerData: Any.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (KAFKA-1341) Client Selector doesn't check connection id properly
[ https://issues.apache.org/jira/browse/KAFKA-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13955751#comment-13955751 ] Timothy Chen edited comment on KAFKA-1341 at 3/31/14 9:41 PM: -- Sorry that was a mistake, uploading a new patch. was (Author: tnachen): Sorry that was a mistake, uploading a new path. Client Selector doesn't check connection id properly Key: KAFKA-1341 URL: https://issues.apache.org/jira/browse/KAFKA-1341 Project: Kafka Issue Type: Bug Components: producer Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1341.patch Reviewing the new producer code I found that we're not checking connection id properly in the Selector, in result connecting using the same id over and over will result in sockets leaking. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1341) Client Selector doesn't check connection id properly
[ https://issues.apache.org/jira/browse/KAFKA-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1341: Attachment: KAFKA-1341_2014-03-31_14:42:44.patch Client Selector doesn't check connection id properly Key: KAFKA-1341 URL: https://issues.apache.org/jira/browse/KAFKA-1341 Project: Kafka Issue Type: Bug Components: producer Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1341.patch, KAFKA-1341_2014-03-31_14:42:44.patch Reviewing the new producer code I found that we're not checking connection id properly in the Selector, in result connecting using the same id over and over will result in sockets leaking. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1341) Client Selector doesn't check connection id properly
[ https://issues.apache.org/jira/browse/KAFKA-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13955755#comment-13955755 ] Timothy Chen commented on KAFKA-1341: - Updated reviewboard https://reviews.apache.org/r/19690/ against branch origin/trunk Client Selector doesn't check connection id properly Key: KAFKA-1341 URL: https://issues.apache.org/jira/browse/KAFKA-1341 Project: Kafka Issue Type: Bug Components: producer Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1341.patch, KAFKA-1341_2014-03-31_14:42:44.patch Reviewing the new producer code I found that we're not checking connection id properly in the Selector, in result connecting using the same id over and over will result in sockets leaking. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1341) Client Selector doesn't check connection id properly
[ https://issues.apache.org/jira/browse/KAFKA-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1341: Attachment: KAFKA-1341_2014-03-31_14:44:27.patch Client Selector doesn't check connection id properly Key: KAFKA-1341 URL: https://issues.apache.org/jira/browse/KAFKA-1341 Project: Kafka Issue Type: Bug Components: producer Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1341.patch, KAFKA-1341_2014-03-31_14:44:27.patch Reviewing the new producer code I found that we're not checking connection id properly in the Selector, in result connecting using the same id over and over will result in sockets leaking. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1341) Client Selector doesn't check connection id properly
[ https://issues.apache.org/jira/browse/KAFKA-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13955758#comment-13955758 ] Timothy Chen commented on KAFKA-1341: - Updated reviewboard https://reviews.apache.org/r/19690/ against branch origin/trunk Client Selector doesn't check connection id properly Key: KAFKA-1341 URL: https://issues.apache.org/jira/browse/KAFKA-1341 Project: Kafka Issue Type: Bug Components: producer Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1341.patch, KAFKA-1341_2014-03-31_14:44:27.patch Reviewing the new producer code I found that we're not checking connection id properly in the Selector, in result connecting using the same id over and over will result in sockets leaking. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1350) Fix excessive state change logging
[ https://issues.apache.org/jira/browse/KAFKA-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13954854#comment-13954854 ] Timothy Chen commented on KAFKA-1350: - [~guozhang] the state change logger log call itself isn't guarded, and although it might not output once it goes into the method, the most expensive call was actually the string.format that forms the log message overall takes a long time to process. Fix excessive state change logging -- Key: KAFKA-1350 URL: https://issues.apache.org/jira/browse/KAFKA-1350 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Joel Koshy Assignee: Neha Narkhede Priority: Blocker Fix For: 0.8.1.1 Attachments: KAFKA-1350.patch, KAFKA-1350_2014-03-29_23:28:07.patch I can provide steps to reproduce this issue. The state change logger needs to be guarded (to check if trace logging is turned on or not). The delete topic patch significantly increased the amount of logging that we do both on the controller. This results in higher latencies in state transitions and can slow down the controller (as well as the broker). This slow-down was how we ran into KAFKA-1342. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1317) KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state
[ https://issues.apache.org/jira/browse/KAFKA-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1317: Attachment: KAFKA-1317_2014-03-28_09:34:02.patch KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state --- Key: KAFKA-1317 URL: https://issues.apache.org/jira/browse/KAFKA-1317 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Brent Bradbury Assignee: Timothy Chen Priority: Blocker Labels: newbie Fix For: 0.8.1.1 Attachments: KAFKA-1317.patch, KAFKA-1317.patch, KAFKA-1317_2014-03-23_23:48:28.patch, KAFKA-1317_2014-03-24_11:06:15.patch, KAFKA-1317_2014-03-25_15:20:14.patch, KAFKA-1317_2014-03-26_09:48:03.patch, KAFKA-1317_2014-03-26_11:30:57.patch, KAFKA-1317_2014-03-26_15:09:48.patch, KAFKA-1317_2014-03-26_15:18:52.patch, KAFKA-1317_2014-03-27_15:15:05.patch, KAFKA-1317_2014-03-28_09:34:02.patch, threaddump.txt When I run an in-process instance of KafkaServer, send a message through it, then call shutdown(), some threads never exit and the process hangs until the process is killed manually. The same scenario does not result in a hang on 0.8.0. The hang happens when calling both shutdown() by itself as well as shutdown() and awaitShutdown() together. I have seen similar behavior shutting down a deployed kafka server as well, but haven't had time to diagnose whether or not it is the same symptom. I suspect either the metrics-meter-tick-thread-1 2 or delete-topics-thread (waiting in kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) is to blame. Since the TopicDeletionManager is new, it seems more suspicious to me. A complete thread dump is attached; the suspect threads are below. delete-topics-thread prio=5 tid=0x7fb3e31d2800 nid=0x6b03 waiting on condition [0x00013c3b3000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e6e6920 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply$mcV$sp(TopicDeletionManager.scala:334) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:333) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Locked ownable synchronizers: - None metrics-meter-tick-thread-2 daemon prio=5 tid=0x7fb3e31c1000 nid=0x5f03 runnable [0x00013ab8f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None metrics-meter-tick-thread-1 daemon prio=5 tid=0x7fb3e31ef800 nid=0x5e03 waiting on condition [0x00013a98c000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a
[jira] [Commented] (KAFKA-1317) KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state
[ https://issues.apache.org/jira/browse/KAFKA-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13950991#comment-13950991 ] Timothy Chen commented on KAFKA-1317: - Updated reviewboard https://reviews.apache.org/r/19696/ against branch origin/trunk KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state --- Key: KAFKA-1317 URL: https://issues.apache.org/jira/browse/KAFKA-1317 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Brent Bradbury Assignee: Timothy Chen Priority: Blocker Labels: newbie Fix For: 0.8.1.1 Attachments: KAFKA-1317.patch, KAFKA-1317.patch, KAFKA-1317_2014-03-23_23:48:28.patch, KAFKA-1317_2014-03-24_11:06:15.patch, KAFKA-1317_2014-03-25_15:20:14.patch, KAFKA-1317_2014-03-26_09:48:03.patch, KAFKA-1317_2014-03-26_11:30:57.patch, KAFKA-1317_2014-03-26_15:09:48.patch, KAFKA-1317_2014-03-26_15:18:52.patch, KAFKA-1317_2014-03-27_15:15:05.patch, KAFKA-1317_2014-03-28_09:34:02.patch, threaddump.txt When I run an in-process instance of KafkaServer, send a message through it, then call shutdown(), some threads never exit and the process hangs until the process is killed manually. The same scenario does not result in a hang on 0.8.0. The hang happens when calling both shutdown() by itself as well as shutdown() and awaitShutdown() together. I have seen similar behavior shutting down a deployed kafka server as well, but haven't had time to diagnose whether or not it is the same symptom. I suspect either the metrics-meter-tick-thread-1 2 or delete-topics-thread (waiting in kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) is to blame. Since the TopicDeletionManager is new, it seems more suspicious to me. A complete thread dump is attached; the suspect threads are below. delete-topics-thread prio=5 tid=0x7fb3e31d2800 nid=0x6b03 waiting on condition [0x00013c3b3000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e6e6920 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply$mcV$sp(TopicDeletionManager.scala:334) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:333) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Locked ownable synchronizers: - None metrics-meter-tick-thread-2 daemon prio=5 tid=0x7fb3e31c1000 nid=0x5f03 runnable [0x00013ab8f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None metrics-meter-tick-thread-1 daemon prio=5 tid=0x7fb3e31ef800 nid=0x5e03 waiting on condition [0x00013a98c000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for
[jira] [Commented] (KAFKA-1310) Zookeeper timeout causes deadlock in Controller
[ https://issues.apache.org/jira/browse/KAFKA-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13951474#comment-13951474 ] Timothy Chen commented on KAFKA-1310: - I tried out the repro scenario described in the latest 0.8.1 branch, and with latest commit 39a5607 I see that after pausing zookeeper for 10 seconds the broker successfully registers itself afterwards. Zookeeper timeout causes deadlock in Controller --- Key: KAFKA-1310 URL: https://issues.apache.org/jira/browse/KAFKA-1310 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Fedor Korotkiy Assignee: Neha Narkhede Priority: Blocker Fix For: 0.8.1.1 Steps to reproduce: 1. Checkout and build 0.8.1 branch from github: git clone g...@github.com:apache/kafka.git cd kafka git checkout origin/0.8.1 ./gradlew jar 2. Start zookeeper server: ./bin/zookeeper-server-start.sh config/zookeeper.properties 3. Start kafka server: ./bin/kafka-server-start.sh config/server.properties 4. Suspend zookeeper process for 10 seconds (ctrl-Z, then %1). 5. And kafka hasn't been re-registered in zookeeper. ./bin/zookeeper-shell.sh ls /brokers/ids [] Root cause of the problem seems to be the deadlock between DeleteTopicsThread and SessionExpirationListener in KafkaController. 1. DeleteTopicsThread acquires controllerLock and await()-s on deleteTopicsCond in awaitTopicDeletionNotification() 2. SessionExpirationListener fires. It acquires controllerLock and tries to shutdown deleteTopicManager(in onControllerResignation()). This interrupts DeleteTopicsThread. 3. DeleteTopicsThread can't return from deleteTopicsCond.await() because controllerLock is taken. We got a deadlock. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1317) KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state
[ https://issues.apache.org/jira/browse/KAFKA-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13950016#comment-13950016 ] Timothy Chen commented on KAFKA-1317: - Updated reviewboard https://reviews.apache.org/r/19696/ against branch origin/trunk KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state --- Key: KAFKA-1317 URL: https://issues.apache.org/jira/browse/KAFKA-1317 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Brent Bradbury Assignee: Timothy Chen Priority: Blocker Labels: newbie Fix For: 0.8.1.1 Attachments: KAFKA-1317.patch, KAFKA-1317.patch, KAFKA-1317_2014-03-23_23:48:28.patch, KAFKA-1317_2014-03-24_11:06:15.patch, KAFKA-1317_2014-03-25_15:20:14.patch, KAFKA-1317_2014-03-26_09:48:03.patch, KAFKA-1317_2014-03-26_11:30:57.patch, KAFKA-1317_2014-03-26_15:09:48.patch, KAFKA-1317_2014-03-26_15:18:52.patch, KAFKA-1317_2014-03-27_15:15:05.patch, threaddump.txt When I run an in-process instance of KafkaServer, send a message through it, then call shutdown(), some threads never exit and the process hangs until the process is killed manually. The same scenario does not result in a hang on 0.8.0. The hang happens when calling both shutdown() by itself as well as shutdown() and awaitShutdown() together. I have seen similar behavior shutting down a deployed kafka server as well, but haven't had time to diagnose whether or not it is the same symptom. I suspect either the metrics-meter-tick-thread-1 2 or delete-topics-thread (waiting in kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) is to blame. Since the TopicDeletionManager is new, it seems more suspicious to me. A complete thread dump is attached; the suspect threads are below. delete-topics-thread prio=5 tid=0x7fb3e31d2800 nid=0x6b03 waiting on condition [0x00013c3b3000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e6e6920 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply$mcV$sp(TopicDeletionManager.scala:334) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:333) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Locked ownable synchronizers: - None metrics-meter-tick-thread-2 daemon prio=5 tid=0x7fb3e31c1000 nid=0x5f03 runnable [0x00013ab8f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None metrics-meter-tick-thread-1 daemon prio=5 tid=0x7fb3e31ef800 nid=0x5e03 waiting on condition [0x00013a98c000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a
[jira] [Updated] (KAFKA-1317) KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state
[ https://issues.apache.org/jira/browse/KAFKA-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1317: Attachment: KAFKA-1317_2014-03-27_15:15:05.patch KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state --- Key: KAFKA-1317 URL: https://issues.apache.org/jira/browse/KAFKA-1317 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Brent Bradbury Assignee: Timothy Chen Priority: Blocker Labels: newbie Fix For: 0.8.1.1 Attachments: KAFKA-1317.patch, KAFKA-1317.patch, KAFKA-1317_2014-03-23_23:48:28.patch, KAFKA-1317_2014-03-24_11:06:15.patch, KAFKA-1317_2014-03-25_15:20:14.patch, KAFKA-1317_2014-03-26_09:48:03.patch, KAFKA-1317_2014-03-26_11:30:57.patch, KAFKA-1317_2014-03-26_15:09:48.patch, KAFKA-1317_2014-03-26_15:18:52.patch, KAFKA-1317_2014-03-27_15:15:05.patch, threaddump.txt When I run an in-process instance of KafkaServer, send a message through it, then call shutdown(), some threads never exit and the process hangs until the process is killed manually. The same scenario does not result in a hang on 0.8.0. The hang happens when calling both shutdown() by itself as well as shutdown() and awaitShutdown() together. I have seen similar behavior shutting down a deployed kafka server as well, but haven't had time to diagnose whether or not it is the same symptom. I suspect either the metrics-meter-tick-thread-1 2 or delete-topics-thread (waiting in kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) is to blame. Since the TopicDeletionManager is new, it seems more suspicious to me. A complete thread dump is attached; the suspect threads are below. delete-topics-thread prio=5 tid=0x7fb3e31d2800 nid=0x6b03 waiting on condition [0x00013c3b3000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e6e6920 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply$mcV$sp(TopicDeletionManager.scala:334) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:333) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Locked ownable synchronizers: - None metrics-meter-tick-thread-2 daemon prio=5 tid=0x7fb3e31c1000 nid=0x5f03 runnable [0x00013ab8f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None metrics-meter-tick-thread-1 daemon prio=5 tid=0x7fb3e31ef800 nid=0x5e03 waiting on condition [0x00013a98c000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at
[jira] [Updated] (KAFKA-1323) log.dirs server property no longer supports relative directories
[
https://issues.apache.org/jira/browse/KAFKA-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Chen updated KAFKA-1323:
Attachment: KAFKA-1323_2014-03-27_16:17:26.patch
log.dirs server property no longer supports relative directories
Key: KAFKA-1323
URL: https://issues.apache.org/jira/browse/KAFKA-1323
Project: Kafka
Issue Type: Bug
Reporter: Joel Koshy
Priority: Blocker
Fix For: 0.8.1.1
Attachments: KAFKA-1323.patch, KAFKA-1323_2014-03-27_16:17:26.patch
This seems to have been caused by KAFKA-1315 - we now don't support relative
directories.
Steps to reproduce:
* Set a relative directory for log.dirs. E.g., {{log.dirs=data/kafka-logs}}
* Bring up the broker and produce some messages:
{{./bin/kafka-producer-perf-test.sh --broker-list localhost:9092 --messages
1000 --topic test}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1323) log.dirs server property no longer supports relative directories
[
https://issues.apache.org/jira/browse/KAFKA-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13950112#comment-13950112
]
Timothy Chen commented on KAFKA-1323:
-
Updated reviewboard https://reviews.apache.org/r/19626/
against branch origin/trunk
log.dirs server property no longer supports relative directories
Key: KAFKA-1323
URL: https://issues.apache.org/jira/browse/KAFKA-1323
Project: Kafka
Issue Type: Bug
Reporter: Joel Koshy
Priority: Blocker
Fix For: 0.8.1.1
Attachments: KAFKA-1323.patch, KAFKA-1323_2014-03-27_16:17:26.patch
This seems to have been caused by KAFKA-1315 - we now don't support relative
directories.
Steps to reproduce:
* Set a relative directory for log.dirs. E.g., {{log.dirs=data/kafka-logs}}
* Bring up the broker and produce some messages:
{{./bin/kafka-producer-perf-test.sh --broker-list localhost:9092 --messages
1000 --topic test}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1317) KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state
[ https://issues.apache.org/jira/browse/KAFKA-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13948082#comment-13948082 ] Timothy Chen commented on KAFKA-1317: - Updated reviewboard https://reviews.apache.org/r/19577/ against branch origin/0.8.1 KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state --- Key: KAFKA-1317 URL: https://issues.apache.org/jira/browse/KAFKA-1317 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Brent Bradbury Assignee: Timothy Chen Priority: Blocker Labels: newbie Fix For: 0.8.1.1 Attachments: KAFKA-1317.patch, KAFKA-1317_2014-03-23_23:48:28.patch, KAFKA-1317_2014-03-24_11:06:15.patch, KAFKA-1317_2014-03-25_15:20:14.patch, KAFKA-1317_2014-03-26_09:48:03.patch, threaddump.txt When I run an in-process instance of KafkaServer, send a message through it, then call shutdown(), some threads never exit and the process hangs until the process is killed manually. The same scenario does not result in a hang on 0.8.0. The hang happens when calling both shutdown() by itself as well as shutdown() and awaitShutdown() together. I have seen similar behavior shutting down a deployed kafka server as well, but haven't had time to diagnose whether or not it is the same symptom. I suspect either the metrics-meter-tick-thread-1 2 or delete-topics-thread (waiting in kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) is to blame. Since the TopicDeletionManager is new, it seems more suspicious to me. A complete thread dump is attached; the suspect threads are below. delete-topics-thread prio=5 tid=0x7fb3e31d2800 nid=0x6b03 waiting on condition [0x00013c3b3000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e6e6920 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply$mcV$sp(TopicDeletionManager.scala:334) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:333) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Locked ownable synchronizers: - None metrics-meter-tick-thread-2 daemon prio=5 tid=0x7fb3e31c1000 nid=0x5f03 runnable [0x00013ab8f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None metrics-meter-tick-thread-1 daemon prio=5 tid=0x7fb3e31ef800 nid=0x5e03 waiting on condition [0x00013a98c000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at
[jira] [Created] (KAFKA-1341) Client Selector doesn't check connection id properly
Timothy Chen created KAFKA-1341: --- Summary: Client Selector doesn't check connection id properly Key: KAFKA-1341 URL: https://issues.apache.org/jira/browse/KAFKA-1341 Project: Kafka Issue Type: Bug Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1341.patch Reviewing the new producer code I found that we're not checking connection id properly in the Selector, in result connecting using the same id over and over will result in sockets leaking. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (KAFKA-1341) Client Selector doesn't check connection id properly
[ https://issues.apache.org/jira/browse/KAFKA-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13948219#comment-13948219 ] Timothy Chen commented on KAFKA-1341: - Created reviewboard https://reviews.apache.org/r/19690/ against branch origin/trunk Client Selector doesn't check connection id properly Key: KAFKA-1341 URL: https://issues.apache.org/jira/browse/KAFKA-1341 Project: Kafka Issue Type: Bug Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1341.patch Reviewing the new producer code I found that we're not checking connection id properly in the Selector, in result connecting using the same id over and over will result in sockets leaking. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1341) Client Selector doesn't check connection id properly
[ https://issues.apache.org/jira/browse/KAFKA-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1341: Attachment: KAFKA-1341.patch Client Selector doesn't check connection id properly Key: KAFKA-1341 URL: https://issues.apache.org/jira/browse/KAFKA-1341 Project: Kafka Issue Type: Bug Reporter: Timothy Chen Assignee: Timothy Chen Attachments: KAFKA-1341.patch Reviewing the new producer code I found that we're not checking connection id properly in the Selector, in result connecting using the same id over and over will result in sockets leaking. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (KAFKA-1317) KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state
[ https://issues.apache.org/jira/browse/KAFKA-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1317: Attachment: KAFKA-1317_2014-03-26_11:30:57.patch KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state --- Key: KAFKA-1317 URL: https://issues.apache.org/jira/browse/KAFKA-1317 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Brent Bradbury Assignee: Timothy Chen Priority: Blocker Labels: newbie Fix For: 0.8.1.1 Attachments: KAFKA-1317.patch, KAFKA-1317_2014-03-23_23:48:28.patch, KAFKA-1317_2014-03-24_11:06:15.patch, KAFKA-1317_2014-03-25_15:20:14.patch, KAFKA-1317_2014-03-26_09:48:03.patch, KAFKA-1317_2014-03-26_11:30:57.patch, threaddump.txt When I run an in-process instance of KafkaServer, send a message through it, then call shutdown(), some threads never exit and the process hangs until the process is killed manually. The same scenario does not result in a hang on 0.8.0. The hang happens when calling both shutdown() by itself as well as shutdown() and awaitShutdown() together. I have seen similar behavior shutting down a deployed kafka server as well, but haven't had time to diagnose whether or not it is the same symptom. I suspect either the metrics-meter-tick-thread-1 2 or delete-topics-thread (waiting in kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) is to blame. Since the TopicDeletionManager is new, it seems more suspicious to me. A complete thread dump is attached; the suspect threads are below. delete-topics-thread prio=5 tid=0x7fb3e31d2800 nid=0x6b03 waiting on condition [0x00013c3b3000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e6e6920 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply$mcV$sp(TopicDeletionManager.scala:334) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:333) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Locked ownable synchronizers: - None metrics-meter-tick-thread-2 daemon prio=5 tid=0x7fb3e31c1000 nid=0x5f03 runnable [0x00013ab8f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None metrics-meter-tick-thread-1 daemon prio=5 tid=0x7fb3e31ef800 nid=0x5e03 waiting on condition [0x00013a98c000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at
[jira] [Commented] (KAFKA-1317) KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state
[ https://issues.apache.org/jira/browse/KAFKA-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13948273#comment-13948273 ] Timothy Chen commented on KAFKA-1317: - Updated reviewboard https://reviews.apache.org/r/19577/ against branch origin/0.8.1 KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state --- Key: KAFKA-1317 URL: https://issues.apache.org/jira/browse/KAFKA-1317 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Brent Bradbury Assignee: Timothy Chen Priority: Blocker Labels: newbie Fix For: 0.8.1.1 Attachments: KAFKA-1317.patch, KAFKA-1317_2014-03-23_23:48:28.patch, KAFKA-1317_2014-03-24_11:06:15.patch, KAFKA-1317_2014-03-25_15:20:14.patch, KAFKA-1317_2014-03-26_09:48:03.patch, KAFKA-1317_2014-03-26_11:30:57.patch, threaddump.txt When I run an in-process instance of KafkaServer, send a message through it, then call shutdown(), some threads never exit and the process hangs until the process is killed manually. The same scenario does not result in a hang on 0.8.0. The hang happens when calling both shutdown() by itself as well as shutdown() and awaitShutdown() together. I have seen similar behavior shutting down a deployed kafka server as well, but haven't had time to diagnose whether or not it is the same symptom. I suspect either the metrics-meter-tick-thread-1 2 or delete-topics-thread (waiting in kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) is to blame. Since the TopicDeletionManager is new, it seems more suspicious to me. A complete thread dump is attached; the suspect threads are below. delete-topics-thread prio=5 tid=0x7fb3e31d2800 nid=0x6b03 waiting on condition [0x00013c3b3000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e6e6920 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply$mcV$sp(TopicDeletionManager.scala:334) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:333) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Locked ownable synchronizers: - None metrics-meter-tick-thread-2 daemon prio=5 tid=0x7fb3e31c1000 nid=0x5f03 runnable [0x00013ab8f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None metrics-meter-tick-thread-1 daemon prio=5 tid=0x7fb3e31ef800 nid=0x5e03 waiting on condition [0x00013a98c000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[jira] [Updated] (KAFKA-1317) KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state
[ https://issues.apache.org/jira/browse/KAFKA-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated KAFKA-1317: Attachment: KAFKA-1317.patch KafkaServer 0.8.1 not responding to .shutdown() cleanly, possibly related to TopicDeletionManager or MetricsMeter state --- Key: KAFKA-1317 URL: https://issues.apache.org/jira/browse/KAFKA-1317 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1 Reporter: Brent Bradbury Assignee: Timothy Chen Priority: Blocker Labels: newbie Fix For: 0.8.1.1 Attachments: KAFKA-1317.patch, KAFKA-1317.patch, KAFKA-1317_2014-03-23_23:48:28.patch, KAFKA-1317_2014-03-24_11:06:15.patch, KAFKA-1317_2014-03-25_15:20:14.patch, KAFKA-1317_2014-03-26_09:48:03.patch, KAFKA-1317_2014-03-26_11:30:57.patch, threaddump.txt When I run an in-process instance of KafkaServer, send a message through it, then call shutdown(), some threads never exit and the process hangs until the process is killed manually. The same scenario does not result in a hang on 0.8.0. The hang happens when calling both shutdown() by itself as well as shutdown() and awaitShutdown() together. I have seen similar behavior shutting down a deployed kafka server as well, but haven't had time to diagnose whether or not it is the same symptom. I suspect either the metrics-meter-tick-thread-1 2 or delete-topics-thread (waiting in kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) is to blame. Since the TopicDeletionManager is new, it seems more suspicious to me. A complete thread dump is attached; the suspect threads are below. delete-topics-thread prio=5 tid=0x7fb3e31d2800 nid=0x6b03 waiting on condition [0x00013c3b3000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e6e6920 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$awaitTopicDeletionNotification(TopicDeletionManager.scala:178) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply$mcV$sp(TopicDeletionManager.scala:334) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:333) at kafka.utils.Utils$.inLock(Utils.scala:538) at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:333) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) Locked ownable synchronizers: - None metrics-meter-tick-thread-2 daemon prio=5 tid=0x7fb3e31c1000 nid=0x5f03 runnable [0x00013ab8f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None metrics-meter-tick-thread-1 daemon prio=5 tid=0x7fb3e31ef800 nid=0x5e03 waiting on condition [0x00013a98c000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for 0x00012e7d05d8 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at
