Re: [DISCUSS] KIP-185: Make exactly once in order delivery per partition the default producer setting

Hi Apurva, Thanks for the clarification of the definition. The definitions are clear and helpful. It seems the scope of this KIP is just about the producer side configuration change, but not attempting to achieve the exactly once semantic with all default settings out of the box. The broker

[GitHub] kafka pull request #3672: KAFKA-5731 Corrected how the sink task worker upda...

Github user rhauch closed the pull request at: https://github.com/apache/kafka/pull/3672 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-5731) Connect WorkerSinkTask out of order offset commit can lead to inconsistent state

[ https://issues.apache.org/jira/browse/KAFKA-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-5731. Resolution: Fixed Fix Version/s: (was: 1.0.0) Issue resolved by pull request

[GitHub] kafka pull request #3674: KAFKA-5737. KafkaAdminClient thread should be daem...

GitHub user cmccabe opened a pull request: https://github.com/apache/kafka/pull/3674 KAFKA-5737. KafkaAdminClient thread should be daemon You can merge this pull request into a Git repository by running: $ git pull https://github.com/cmccabe/kafka KAFKA-5737 Alternatively

[jira] [Created] (KAFKA-5737) KafkaAdminClient thread should be daemon

Colin P. McCabe created KAFKA-5737: -- Summary: KafkaAdminClient thread should be daemon Key: KAFKA-5737 URL: https://issues.apache.org/jira/browse/KAFKA-5737 Project: Kafka Issue Type: Bug

Build failed in Jenkins: kafka-trunk-jdk8 #1911

See Changes: [jason] KAFKA-5731; Corrected how the sink task worker updates the last -- [...truncated 4.28 MB...] kafka.producer.AsyncProducerTest > testRandomPartitioner

[GitHub] kafka pull request #3673: MINOR: Consolidate broker request/response handlin...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3673 MINOR: Consolidate broker request/response handling This patch contains a few small improvements to make request/response handling more consistent. Primarily it consolidates request/response

Re: [DISCUSS] KIP-131 : Add access to OffsetStorageReader from SourceConnector

Sorry it's taken me so long to come back to this. Have you considered creating a `SourceConnectorContext` interface that extends `ConnectorContext` and that adds the method to access the offset storage? This would very closely match the existing `SourceTaskContext`. `SourceConnector`

Re: [DISCUSS] KIP-91 Provide Intuitive User Timeouts in The Producer

Question about "the closing of a batch can be delayed longer than linger.ms": Is it possible to cause an indefinite delay? At some point bytes limit might kick in. Also, why is closing of a batch coupled with availability of its destination? In this approach a batch chosen for eviction due to

[jira] [Created] (KAFKA-5736) Improve error message in Connect when all kafka brokers are down

Konstantine Karantasis created KAFKA-5736: - Summary: Improve error message in Connect when all kafka brokers are down Key: KAFKA-5736 URL: https://issues.apache.org/jira/browse/KAFKA-5736

Jenkins build is back to normal : kafka-trunk-jdk7 #2642

See

[GitHub] kafka pull request #3672: KAFKA-5731 Corrected how the sink task worker upda...

GitHub user rhauch opened a pull request: https://github.com/apache/kafka/pull/3672 KAFKA-5731 Corrected how the sink task worker updates the last committed offsets (0.11.0) Prior to this change, it was possible for the synchronous consumer commit request to be handled before

Re: [KAFKA-5138] MirrorMaker doesn't exit on send failure occasionally

Hey, I think I have found something. My guess is that when the AbstactCoordinator.maybeLeaveGroup(...) function's pollNoWakeUp part throws an exception, then it can not call resetGeneration() and the HearBeatThread stays in STABLE state. The broker won't be notified about the consumer's leave

[GitHub] kafka pull request #3662: KAFKA-5731 Corrected how the sink task worker upda...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3662 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-185: Make exactly once in order delivery per partition the default producer setting

Hi Jay, I chatted with Apurva offline, and we think the key of the discussion is that, as summarized in the updated KIP wiki, whether we should consider replication as a necessary condition of at-least-once, and of course also exactly-once. Originally I think replication is not a necessary

Re: [DISCUSS] KIP-152 - Improve diagnostics for SASL authentication failures

I have updated the KIP based on the discussions so far. It will be good if we can get some more feedback so that this can be implemented for 1.0.0. Thanks, Rajini On Thu, May 4, 2017 at 10:22 PM, Ismael Juma wrote: > Hi Rajini, > > I think we were talking about slightly

Re: [DISCUSS] KIP-185: Make exactly once in order delivery per partition the default producer setting

Hey Guozhang, I think the argument is that with acks=1 the message could be lost and hence you aren't guaranteeing exactly once delivery. -Jay On Mon, Aug 14, 2017 at 1:36 PM, Guozhang Wang wrote: > Just want to clarify that regarding 1), I'm fine with changing it to `all`

[jira] [Resolved] (KAFKA-2283) scheduler exception on non-controller node when shutdown

[ https://issues.apache.org/jira/browse/KAFKA-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-2283. -- Resolution: Fixed > scheduler exception on non-controller node when shutdown >

[jira] [Resolved] (KAFKA-2220) Improvement: Could we support rewind by time ?

[ https://issues.apache.org/jira/browse/KAFKA-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-2220. -- Resolution: Fixed This got fixed in KAFKA-4743 / KIP-122. > Improvement: Could we support rewind by

[jira] [Resolved] (KAFKA-1832) Async Producer will cause 'java.net.SocketException: Too many open files' when broker host does not exist

[ https://issues.apache.org/jira/browse/KAFKA-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-1832. -- Resolution: Fixed Fixed in KAFKA-1041 > Async Producer will cause 'java.net.SocketException: Too many

[GitHub] kafka pull request #3622: HOTFIX: state transition cherry picking

Github user enothereska closed the pull request at: https://github.com/apache/kafka/pull/3622 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Jenkins build is back to normal : kafka-0.11.0-jdk7 #271

See

[GitHub] kafka pull request #3671: KAFKA-5723[WIP]: Refactor BrokerApiVersionsCommand...

GitHub user adyach opened a pull request: https://github.com/apache/kafka/pull/3671 KAFKA-5723[WIP]: Refactor BrokerApiVersionsCommand to use the new AdminClient This PR brings refactoring to new AdminClient java class for BrokerApiVersionsCommand. The code was not tested, because

[KAFKA-5138] MirrorMaker doesn't exit on send failure occasionally

Hi Guys, I have just started to work on this ticket a little more than a week ago: https://issues.apache.org/jira/browse/KAFKA-5138 I could not reproduce it sadly, but from the logs Dustin gave and from the code it seems like this might not be just a MirrorMaker issue but a consumer one. My

[jira] [Created] (KAFKA-5735) Client-ids are not handled consistently by clients and broker

Rajini Sivaram created KAFKA-5735: - Summary: Client-ids are not handled consistently by clients and broker Key: KAFKA-5735 URL: https://issues.apache.org/jira/browse/KAFKA-5735 Project: Kafka

[GitHub] kafka pull request #3670: MINOR: fixed StickyAssignor javadoc

GitHub user mimaison opened a pull request: https://github.com/apache/kafka/pull/3670 MINOR: fixed StickyAssignor javadoc StickyAssignor javadoc has a bunch of formatting issues which make it pretty hard to read:

[jira] [Created] (KAFKA-5734) Heap (Old generation space) gradually increase

jang created KAFKA-5734: --- Summary: Heap (Old generation space) gradually increase Key: KAFKA-5734 URL: https://issues.apache.org/jira/browse/KAFKA-5734 Project: Kafka Issue Type: Bug

[GitHub] kafka pull request #3669: KAFKA-5726: KafkaConsumer.subscribe() overload tha...

GitHub user attilakreiner opened a pull request: https://github.com/apache/kafka/pull/3669 KAFKA-5726: KafkaConsumer.subscribe() overload that takes just Pattern - changed the interface & implementations - updated tests to use the new method where applicable You can merge this

[GitHub] kafka pull request #3668: KAFKA-5679: Add logging for broker termination due...

GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/3668 KAFKA-5679: Add logging for broker termination due to SIGTERM or SIGINT You can merge this pull request into a Git repository by running: $ git pull