Re: [DISCUSS] KIP-204 : adding records deletion operation to the new Admin Client API

Hi, I have just updated the KIP-204 description with the new TopicDeletionPolicy suggested by the KIP-201. Paolo Patierno Senior Software Engineer (IoT) @ Red Hat Microsoft MVP on Azure & IoT Microsoft Azure Advisor Twitter : @ppatierno Linkedin :

[jira] [Created] (KAFKA-5984) API changes review for Kafka

Andrey Ponomarenko created KAFKA-5984: - Summary: API changes review for Kafka Key: KAFKA-5984 URL: https://issues.apache.org/jira/browse/KAFKA-5984 Project: Kafka Issue Type: Test

[GitHub] kafka pull request #3932: KAFKA-5867: Log Kafka Connect worker info during s...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3932 ---

[GitHub] kafka pull request #3980: MINOR: Push JMX metric name mangling int the JmxRe...

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/3980 MINOR: Push JMX metric name mangling int the JmxReporter (KIP-190 follow up) You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka

[GitHub] kafka pull request #3979: KAFKA-5979: Use single AtomicCounter to generate i...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/3979 KAFKA-5979: Use single AtomicCounter to generate internal names You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka

[GitHub] kafka-site pull request #87: Update main bullet verbiage on the homepage

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka-site/pull/87 Update main bullet verbiage on the homepage Updates to the 3 main bullets at the top of the homepage

[GitHub] kafka pull request #3977: MINOR: Mark log cleaner integration tests as integ...

Github user ewencp closed the pull request at: https://github.com/apache/kafka/pull/3977 ---

[GitHub] kafka pull request #3117: KAFKA-5225:StreamsResetter doesn't allow custom Co...

Github user bharatviswa504 closed the pull request at: https://github.com/apache/kafka/pull/3117 ---

[GitHub] kafka pull request #3978: KAFKA-5932: Avoid call to fetchPrevious in FlushLi...

GitHub user bbejeck opened a pull request: https://github.com/apache/kafka/pull/3978 KAFKA-5932: Avoid call to fetchPrevious in FlushListeners You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbejeck/kafka

[GitHub] kafka pull request #3977: MINOR: Mark log cleaner integration tests as integ...

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/3977 MINOR: Mark log cleaner integration tests as integration tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka

Jenkins build is back to normal : kafka-trunk-jdk8 #2076

See

[GitHub] kafka pull request #3976: MINOR: Follow-up cleanup of KAFKA-5960

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3976 MINOR: Follow-up cleanup of KAFKA-5960 You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka version-fix-followup Alternatively

[jira] [Resolved] (KAFKA-5369) Producer hangs on metadata fetch from broker in security authorization related failures and no meaningful exception is thrown

[ https://issues.apache.org/jira/browse/KAFKA-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-5369. -- Resolution: Duplicate Resolving this as duplicate of KAFKA-4764, KAFKA-5920 > Producer hangs on

Build failed in Jenkins: kafka-trunk-jdk7 #2821

See Changes: [ismael] KAFKA-5750; Elevate log messages for denials to INFO in [ismael] KAFKA-5597; Improve Metrics docs generation -- [...truncated 357.82 KB...]

[GitHub] kafka pull request #3962: KAFKA-5973 Exit when ShutdownableThread encounters...

Github user tedyu closed the pull request at: https://github.com/apache/kafka/pull/3962 ---

[GitHub] kafka pull request #3975: KAFKA-5902 Added sink task metrics

GitHub user rhauch opened a pull request: https://github.com/apache/kafka/pull/3975 KAFKA-5902 Added sink task metrics Added Connect metrics specific to source tasks, and builds upon #3864 and #3911 that have already been merged into `trunk`, and #3959 that has yet to be merged.

[GitHub] kafka pull request #3974: Kafka 5932[WIP] DON'T Merge avoid fetch previous i...

Github user bbejeck closed the pull request at: https://github.com/apache/kafka/pull/3974 ---

[GitHub] kafka pull request #3974: Kafka 5932[WIP] DON'T Merge avoid fetch previous i...

GitHub user bbejeck opened a pull request: https://github.com/apache/kafka/pull/3974 Kafka 5932[WIP] DON'T Merge avoid fetch previous in flush listeners You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbejeck/kafka

[GitHub] kafka pull request #3799: KAFKA-5597 Alternate way to do Metrics docs genera...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3799 ---

[GitHub] kafka pull request #3698: KAFKA-5750: Elevate log messages for denials to IN...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3698 ---

Build failed in Jenkins: kafka-trunk-jdk8 #2075

See Changes: [damian.guy] MINOR:Updated Rabobank description -- [...truncated 1.89 MB...] org.apache.kafka.connect.runtime.distributed.DistributedHerderTest >

[VOTE] KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule

Hi All, I'd like to start the vote on KIP-203. Details are here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-203%3A+Add+toLowerCase+support+to+sasl.kerberos.principal.to.local+rule Thanks,

Re: [VOTE] KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule

+1 (nonbinding) On 27 September 2017 at 16:10, Manikumar wrote: > Hi All, > > I'd like to start the vote on KIP-203. Details are here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 203%3A+Add+toLowerCase+support+to+sasl.kerberos.principal.to.local+rule

[GitHub] kafka pull request #3973: Kafka-5958: Global stores access state restore lis...

GitHub user bbejeck opened a pull request: https://github.com/apache/kafka/pull/3973 Kafka-5958: Global stores access state restore listener You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbejeck/kafka

[jira] [Created] (KAFKA-5983) Cannot mirror Avro-encoded data using the Apache Kafka MirrorMaker

Giulio Vito de Musso created KAFKA-5983: --- Summary: Cannot mirror Avro-encoded data using the Apache Kafka MirrorMaker Key: KAFKA-5983 URL: https://issues.apache.org/jira/browse/KAFKA-5983

[GitHub] kafka pull request #3972: *WIP* MINOR: Add documentation about each of the m...

GitHub user tombentley opened a pull request: https://github.com/apache/kafka/pull/3972 *WIP* MINOR: Add documentation about each of the messages in the protocol Following recent refactoring of `Protocol` I suggested to add documentation for the messages to explain the purpose of

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

Hi Ismael, 1. I don't have a real requirement now but "deleting" is an operation that could be really dangerous so it's always better having a way for having more control on that. I know that we have the authorizer used for that (delete on topic) but fine grained control could be better

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

Hi Ismael, Thanks for looking at the KIP and explaining the thinking behind the original API. Looking at the updated KIP, I notice that we actually have a > TopicDeletionPolicy with a separate config. That seems to be a halfway > house. Not sure about that. > I can certainly see that point of

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

A couple of questions: 1. Is this a concrete requirement from a user or is it hypothetical? 2. You sure you would want to do this in the broker instead of the clients? It's worth remembering that updating broker policies involves a rolling restart of the cluster, so it's not the right place for

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

Hi Tom, I guess that at "On topic deletion will be applied on topic and message deletion." you meant something like "The TopicDeletionPolicy will be applied on topic and messages deletion". Paolo Patierno Senior Software Engineer (IoT) @ Red Hat Microsoft MVP on Azure & IoT Microsoft

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

Hi Ismael, regarding motivations for delete records, as I said during the discussion on KIP-204, it gives the possibility to avoid deleting messages for specific partitions (inside the topic) and starting from a specific offset. I could think on some users solutions where they know exactly

[GitHub] kafka pull request #3971: MINOR: additional kip-182 doc updates

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/3971 MINOR: additional kip-182 doc updates You can merge this pull request into a Git repository by running: $ git pull https://github.com/dguy/kafka kip-182-docs Alternatively you can review and

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

Another quick comment: the KIP states that having multiple interfaces imply that the logic must be in 2 places. That is not true because the same class can implement multiple interfaces (this aspect was considered when we decided to introduce policies incrementally). The main reason why I think

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

A couple more comments: 1. "If this KIP is accepted for Kafka 1.1.0 this removal could happen in Kafka 1.2.0 or a later release." -> we only remove code in major releases. So, if it's deprecated in 1.1.0, it would be removed in 2.0.0. 2. Deleting all messages in a topic is not really the same as

Re: [DISCUSS] KIP-201: Rationalising Policy interfaces

I have updated the KIP to add a common policy interface for topic and message deletion. This included pulling ClusterState and TopicState interfaces up to the top level so that they can be shared between the two policies. Cheers, Tom On 26 September 2017 at 18:09, Edoardo Comar

[jira] [Created] (KAFKA-5982) Kafka 0.9.0.1 doesn't start after delete storage dir

Marc Cortinas created KAFKA-5982: Summary: Kafka 0.9.0.1 doesn't start after delete storage dir Key: KAFKA-5982 URL: https://issues.apache.org/jira/browse/KAFKA-5982 Project: Kafka Issue

Re: [DISCUSS] KIP-190: Handle client-ids consistently between clients and brokers

I don't know the history either, I quickly scanned the KIP-55 threads and couldn't see it being discussed. Anyway, your suggestion sounds good to me, are you planning to do that as part of KIP-196 or should I create a new JIRA ? On Wed, Sep 27, 2017 at 3:20 AM, Ewen Cheslack-Postava

[jira] [Created] (KAFKA-5981) Update the affected versions on bug KAFKA-1194

Amund Gjersøe created KAFKA-5981: Summary: Update the affected versions on bug KAFKA-1194 Key: KAFKA-5981 URL: https://issues.apache.org/jira/browse/KAFKA-5981 Project: Kafka Issue Type:

[GitHub] kafka pull request #3963: MINOR:Updated Rabobank description

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3963 ---

[GitHub] kafka pull request #3970: KAFKA-5225: StreamsResetter doesn't allow custom C...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/3970 KAFKA-5225: StreamsResetter doesn't allow custom Consumer properties You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka