Re: [DISCUSS] KIP-31 - Message format change proposal

Ewen and Jay, They way I see the LogAppendTime is another format of "offset". It serves the following purpose: 1. Locate messages not only by position, but also by time. The difference from offset is timestamp is not unique for all messags. 2. Allow broker to manage messages based on time, e.g.

[jira] [Commented] (KAFKA-1929) Convert core kafka module to use the errors in org.apache.kafka.common.errors

[ https://issues.apache.org/jira/browse/KAFKA-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741299#comment-14741299 ] Grant Henke commented on KAFKA-1929: [~hachikuji], feel free to pick this up > Convert core kafka

[jira] [Commented] (KAFKA-2477) Replicas spuriously deleting all segments in partition

[ https://issues.apache.org/jira/browse/KAFKA-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741275#comment-14741275 ] ASF GitHub Bot commented on KAFKA-2477: --- GitHub user becketqin reopened a pull request:

[jira] [Commented] (KAFKA-2477) Replicas spuriously deleting all segments in partition

[ https://issues.apache.org/jira/browse/KAFKA-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741274#comment-14741274 ] ASF GitHub Bot commented on KAFKA-2477: --- Github user becketqin closed the pull request at:

[jira] [Resolved] (KAFKA-2529) Brokers should write current version to log when they first start

[ https://issues.apache.org/jira/browse/KAFKA-2529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira resolved KAFKA-2529. - Resolution: Fixed Sorry, missed the jira that fixed this. > Brokers should write current version

[jira] [Commented] (KAFKA-2276) Initial patch for KIP-25

[ https://issues.apache.org/jira/browse/KAFKA-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741211#comment-14741211 ] Geoff Anderson commented on KAFKA-2276: --- Hey [~jkreps] - if we're ok with adding Kafka core as a

Re: Review Request 36858: Patch for KAFKA-2120

> On Sept. 11, 2015, 1:44 a.m., Jason Gustafson wrote: > > clients/src/main/java/org/apache/kafka/clients/InFlightRequests.java, line > > 140 > > > > > > This line is puzzling me a little bit. Wouldn't we want to

[jira] [Commented] (KAFKA-2276) Initial patch for KIP-25

[ https://issues.apache.org/jira/browse/KAFKA-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741225#comment-14741225 ] Jay Kreps commented on KAFKA-2276: -- [~granders] This is common code so it should go under

[jira] [Created] (KAFKA-2536) topics tool should allow users to alter topic configuration

Gwen Shapira created KAFKA-2536: --- Summary: topics tool should allow users to alter topic configuration Key: KAFKA-2536 URL: https://issues.apache.org/jira/browse/KAFKA-2536 Project: Kafka

[jira] [Updated] (KAFKA-2477) Replicas spuriously deleting all segments in partition

[ https://issues.apache.org/jira/browse/KAFKA-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-2477: Status: Patch Available (was: Open) > Replicas spuriously deleting all segments in partition >

[jira] [Created] (KAFKA-2535) When removing a partition from an out of sync broker, it should not need to join the ISR first

Sean Fellows created KAFKA-2535: --- Summary: When removing a partition from an out of sync broker, it should not need to join the ISR first Key: KAFKA-2535 URL: https://issues.apache.org/jira/browse/KAFKA-2535

[jira] [Commented] (KAFKA-2229) Phase 1: Requests and KafkaApis

[ https://issues.apache.org/jira/browse/KAFKA-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741307#comment-14741307 ] Grant Henke commented on KAFKA-2229: [~abiletskyi] Are you still working on this? If not, perhaps I

Re: Review Request 36858: Patch for KAFKA-2120

> On Sept. 10, 2015, 6:19 p.m., Ismael Juma wrote: > > clients/src/main/java/org/apache/kafka/clients/InFlightRequests.java, line > > 137 > > > > > > Out of curiosity, why are we using `LinkedList` here? Generally

[jira] [Created] (KAFKA-2534) SSLTransportLayer does not handle buffer overflow correctly

Rajini Sivaram created KAFKA-2534: - Summary: SSLTransportLayer does not handle buffer overflow correctly Key: KAFKA-2534 URL: https://issues.apache.org/jira/browse/KAFKA-2534 Project: Kafka

[jira] [Created] (KAFKA-2537) Mirrormaker defaults to localhost with no sanity checks

Evan Huus created KAFKA-2537: Summary: Mirrormaker defaults to localhost with no sanity checks Key: KAFKA-2537 URL: https://issues.apache.org/jira/browse/KAFKA-2537 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-31 - Message format change proposal

Jay had mentioned the scenario of mirror-maker bootstrap which would effectively reset the logAppendTimestamps for the bootstrapped data. If we don't include logAppendTimestamps in each message there is a similar scenario when rebuilding indexes during recovery. So it seems it may be worth adding

Re: Review Request 36858: Patch for KAFKA-2120

> On Sept. 11, 2015, 1:44 a.m., Jason Gustafson wrote: > > clients/src/main/java/org/apache/kafka/clients/InFlightRequests.java, line > > 140 > > > > > > This line is puzzling me a little bit. Wouldn't we want to

[jira] [Updated] (KAFKA-2278) JmxTool should support querying all objects when object-name is omitted

[ https://issues.apache.org/jira/browse/KAFKA-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2278: --- Fix Version/s: (was: 0.9.0) 0.8.3 > JmxTool should support querying all

[jira] [Commented] (KAFKA-2278) JmxTool should support querying all objects when object-name is omitted

[ https://issues.apache.org/jira/browse/KAFKA-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741675#comment-14741675 ] ASF GitHub Bot commented on KAFKA-2278: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-2278; JmxTool should support querying al...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/181 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-2440) Use `NetworkClient` instead of `SimpleConsumer` to fetch data from replica

[ https://issues.apache.org/jira/browse/KAFKA-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2440: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 194

[jira] [Commented] (KAFKA-2440) Use `NetworkClient` instead of `SimpleConsumer` to fetch data from replica

[ https://issues.apache.org/jira/browse/KAFKA-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741699#comment-14741699 ] ASF GitHub Bot commented on KAFKA-2440: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-2440; Use `NetworkClient` instead of `Si...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/194 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Build failed in Jenkins: Kafka-trunk #620

See Changes: [wangguoz] KAFKA-2389: remove commit type from new consumer. [wangguoz] KAFKA-2278: JmxTool should support querying all objects when object name is omitted [junrao] KAFKA-2440; Use `NetworkClient` instead of `SimpleConsumer`

[jira] [Commented] (KAFKA-2389) CommitType seems not necessary in commit().

[ https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741654#comment-14741654 ] ASF GitHub Bot commented on KAFKA-2389: --- Github user asfgit closed the pull request at:

[jira] [Updated] (KAFKA-2389) CommitType seems not necessary in commit().

[ https://issues.apache.org/jira/browse/KAFKA-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2389: - Resolution: Fixed Fix Version/s: 0.8.3 Status: Resolved (was: Patch Available)

[GitHub] kafka pull request: KAFKA-2389: remove commit type from new consum...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/134 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-2373) Copycat distributed offset storage

[ https://issues.apache.org/jira/browse/KAFKA-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2373: - Status: Patch Available (was: Open) > Copycat distributed offset storage >

Re: Review Request 36858: Patch for KAFKA-2120

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36858/ --- (Updated Sept. 11, 2015, 9:54 p.m.) Review request for kafka. Bugs:

[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient

[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741594#comment-14741594 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard

[jira] [Updated] (KAFKA-2120) Add a request timeout to NetworkClient

[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat updated KAFKA-2120: --- Attachment: KAFKA-2120_2015-09-11_14:54:15.patch > Add a request timeout to NetworkClient >

[jira] [Updated] (KAFKA-2278) JmxTool should support querying all objects when object-name is omitted

[ https://issues.apache.org/jira/browse/KAFKA-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2278: - Resolution: Fixed Fix Version/s: 0.9.0 Status: Resolved (was: Patch Available)

Kafka KIP meeting Sep 14 at 9:30am PST

Hi, Everyone, Since most people from Confluent will be out Tuesday, we will have a Kafka KIP meeting on Sep. 14 (Monday) at 9:30am PST. If you plan to attend but haven't received an invite, please let me know. The following is the agenda. Agenda: 1. KIP-28: Add a processor client 2. KIP-31: Move

[GitHub] kafka pull request: support commit offset after consumed

GitHub user wyzssw opened a pull request: https://github.com/apache/kafka/pull/207 support commit offset after consumed Kafka Of Original Version do not support "commit offset after consuming",when kafka.consumer.ConsumerIterator[K, V].next() return a MessageAndMetadata,which

[jira] [Reopened] (KAFKA-2300) Error in controller log when broker tries to rejoin cluster

[ https://issues.apache.org/jira/browse/KAFKA-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira reopened KAFKA-2300: - I'm observing sometimes that the same controller gets elected, but it is not restarting with

[GitHub] kafka pull request: KAFKA-2477: Fix a race condition between log a...

Github user becketqin closed the pull request at: https://github.com/apache/kafka/pull/204 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request: KAFKA-2477: Fix a race condition between log a...

GitHub user becketqin reopened a pull request: https://github.com/apache/kafka/pull/204 KAFKA-2477: Fix a race condition between log append and fetch that causes OffsetOutOfRangeException. Tried two fixes. I prefer the second approach because it saves an additional offset search.

[jira] [Updated] (KAFKA-1929) Convert core kafka module to use the errors in org.apache.kafka.common.errors

[ https://issues.apache.org/jira/browse/KAFKA-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-1929: --- Assignee: (was: Grant Henke) > Convert core kafka module to use the errors in

[GitHub] kafka pull request: KAFKA-2534: Fixes and unit tests for SSLTransp...

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/205 KAFKA-2534: Fixes and unit tests for SSLTransportLayer buffer overflow Unit tests which mock buffer overflow and underflow in the SSL transport layer and fixes for the couple of issues in

[jira] [Updated] (KAFKA-2534) SSLTransportLayer does not handle buffer overflow correctly

[ https://issues.apache.org/jira/browse/KAFKA-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram updated KAFKA-2534: -- Reviewer: Sriharsha Chintalapani Status: Patch Available (was: Open) > SSLTransportLayer

[jira] [Commented] (KAFKA-2534) SSLTransportLayer does not handle buffer overflow correctly

[ https://issues.apache.org/jira/browse/KAFKA-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740540#comment-14740540 ] ASF GitHub Bot commented on KAFKA-2534: --- GitHub user rajinisivaram opened a pull request:

[GitHub] kafka pull request: let kafka support "commit offset after consumi...

GitHub user wyzssw opened a pull request: https://github.com/apache/kafka/pull/206 let kafka support "commit offset after consuming" Kafka Of Original Version do not support "commit offset after consuming",when kafka.consumer.ConsumerIterator[K, V].next() return a

Re: Kafka cluster cannot start anymore after unexpected shutdown

And I tried to clean up the whole kafka-logs folder, and then starts the kafka server again. It will the following errors: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in

[jira] [Assigned] (KAFKA-2538) Compilation in trunk is failing due to https://github.com/apache/kafka/commit/845514d62329be8382e6d02b8041fc858718d534

[ https://issues.apache.org/jira/browse/KAFKA-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt reassigned KAFKA-2538: --- Assignee: Parth Brahmbhatt > Compilation in trunk is failing due to >

[jira] [Created] (KAFKA-2538) Compilation in trunk is failing due to https://github.com/apache/kafka/commit/845514d62329be8382e6d02b8041fc858718d534

Parth Brahmbhatt created KAFKA-2538: --- Summary: Compilation in trunk is failing due to https://github.com/apache/kafka/commit/845514d62329be8382e6d02b8041fc858718d534 Key: KAFKA-2538 URL:

[jira] [Commented] (KAFKA-2530) metrics for old replica fetcher thread need to be deregistered

[ https://issues.apache.org/jira/browse/KAFKA-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741840#comment-14741840 ] Otis Gospodnetic commented on KAFKA-2530: - Is that, by any chance, related to

[jira] [Commented] (KAFKA-2538) Compilation in trunk is failing due to https://github.com/apache/kafka/commit/845514d62329be8382e6d02b8041fc858718d534

[ https://issues.apache.org/jira/browse/KAFKA-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14741860#comment-14741860 ] ASF GitHub Bot commented on KAFKA-2538: --- GitHub user Parth-Brahmbhatt opened a pull request:

[jira] [Resolved] (KAFKA-1679) JmxTool outputs nothing if any mbean attributes can't be retrieved

[ https://issues.apache.org/jira/browse/KAFKA-1679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy resolved KAFKA-1679. Resolution: Fixed Fix Version/s: 0.8.3 This got fixed in KAFKA-2278 > JmxTool

Kafka cluster cannot start anymore after unexpected shutdown

Hi, We're running the Trunk version of Kafka (for its SSL feature) and recently I'm trying to enable the kafka manager with it. After enabling that, I find out some machine's Kafka Server is dead. Looking at the server.log, it has the following logs. java.lang.OutOfMemoryError: Java heap space

[GitHub] kafka pull request: KAFKA-2538: Fixing a compilation error in trun...

GitHub user Parth-Brahmbhatt opened a pull request: https://github.com/apache/kafka/pull/208 KAFKA-2538: Fixing a compilation error in trunk. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Parth-Brahmbhatt/kafka KAFKA-2538