Build failed in Jenkins: kafka-2.0-jdk8 #112

See Changes: [matthias] MINOR: Fixed log in Topology Builder. (#5477) -- [...truncated 432.54 KB...] kafka.controller.PartitionStateMachineTest >

Build failed in Jenkins: kafka-trunk-jdk8 #2880

See Changes: [wangguoz] MINOR: Follow up for KAFKA-6761 graph should add stores for consistency [jason] KAFKA-4950; Fix ConcurrentModificationException on assigned-partitions

[jira] [Created] (KAFKA-7264) Support Java 11

Ismael Juma created KAFKA-7264: -- Summary: Support Java 11 Key: KAFKA-7264 URL: https://issues.apache.org/jira/browse/KAFKA-7264 Project: Kafka Issue Type: Improvement Reporter:

[jira] [Created] (KAFKA-7263) Container exception java.lang.IllegalStateException: Coordinator selected invalid assignment protocol: null

laomei created KAFKA-7263: - Summary: Container exception java.lang.IllegalStateException: Coordinator selected invalid assignment protocol: null Key: KAFKA-7263 URL: https://issues.apache.org/jira/browse/KAFKA-7263

[jira] [Created] (KAFKA-7262) Ability to support heterogeneous storage in Kafka

fangbin sun created KAFKA-7262: -- Summary: Ability to support heterogeneous storage in Kafka Key: KAFKA-7262 URL: https://issues.apache.org/jira/browse/KAFKA-7262 Project: Kafka Issue Type:

Re: [VOTE] KIP-349 Priorities for Source Topics

Can you guys spell it out for me? I just don't really see when I want to subscribe to two topics but not get events from both at the same time. Is this a work-queue type pattern? On Wed, Aug 8, 2018 at 6:10 PM, Matt Farmer wrote: > Oh, almost forgot, thanks for the KIP - I can see this being a

Build failed in Jenkins: kafka-trunk-jdk10 #388

See Changes: [wangguoz] MINOR: Follow up for KAFKA-6761 graph should add stores for consistency [jason] KAFKA-4950; Fix ConcurrentModificationException on assigned-partitions

Build failed in Jenkins: kafka-trunk-jdk8 #2879

See Changes: [github] KAFKA-7158: Add unit test for window store range queries (#5466) -- [...truncated 2.48 MB...]

Re: [VOTE] KIP-349 Priorities for Source Topics

Oh, almost forgot, thanks for the KIP - I can see this being a very useful addition. :) On Wed, Aug 8, 2018 at 9:09 PM Matt Farmer wrote: > Is it worth spelling out explicitly what the behavior is when two topics > have the same priority? I'm a bit fuzzy on how we choose what topics to >

Re: [VOTE] KIP-349 Priorities for Source Topics

Is it worth spelling out explicitly what the behavior is when two topics have the same priority? I'm a bit fuzzy on how we choose what topics to consume from right now, if I'm being honest, so it could be useful to outline the current behavior in the background and to spell out how that would

[VOTE] KIP-349 Priorities for Source Topics

Hi All, Calling for a vote on KIP-349 https://cwiki.apache.org/confluence/display/KAFKA/KIP-349%3A+Priorities+for+Source+Topics Cheers, -- Nick

Jenkins build is back to normal : kafka-1.0-jdk7 #230

See

Jenkins build is back to normal : kafka-trunk-jdk10 #387

See

Re: [DISCUSS] KIP-291: Have separate queues for control requests and data requests

Hi Lucas, Yes, a separate Jira is OK. Since the proposal has significantly changed since the initial vote started. We probably should let the others who have already voted know and ensure they are happy with the updated proposal. Also, it seems the motivation part of the KIP wiki is still just

[DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

Hi all, I would like to start a discussion on: KIP-347: Enable batching in FindCoordinatorRequest https://cwiki.apache.org/confluence/x/CgZPBQ Thanks @Guozhang Wang for his help and patience! Thanks, Yishun

Jenkins build is back to normal : kafka-2.0-jdk8 #109

See

[jira] [Created] (KAFKA-7261) Request and response total metrics record bytes instead of request count

Rajini Sivaram created KAFKA-7261: - Summary: Request and response total metrics record bytes instead of request count Key: KAFKA-7261 URL: https://issues.apache.org/jira/browse/KAFKA-7261 Project:

[jira] [Resolved] (KAFKA-7158) Duplicates when searching kafka stream state store with caching

[ https://issues.apache.org/jira/browse/KAFKA-7158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7158. -- Resolution: Fixed Assignee: John Roesler Fix Version/s: 2.1.0

Build failed in Jenkins: kafka-trunk-jdk8 #2878

See Changes: [github] KAFKA-7255: Fix timing issue with create/update in SimpleAclAuthorizer [harsha] KAFKA-7247: Update a link to Apache BookKeeper project (#5460) --

Jenkins build is back to normal : kafka-1.1-jdk7 #175

See

Re: Kafka stream - Internal topic name and schema avro compatibility

Hi Cédric, The suffix is generated when we build the topology in such a way to guarantee each node/interna-topic/state-store gets a unique name. Generally speaking, it is unsafe to modify the topology and restart it. We recommend using the app reset tool whenever you update your topology. That

Build failed in Jenkins: kafka-1.0-jdk7 #229

See Changes: [rajinisivaram] KAFKA-7255: Fix timing issue with create/update in SimpleAclAuthorizer -- [...truncated 380.21 KB...]

Re: [VOTE] KIP-320: Allow fetchers to detect and handle log truncation

Thanks for the KIP Jason! +1 (binding) On Wed, Aug 8, 2018 at 1:04 PM, Jason Gustafson wrote: > Hi All, > > I'd like to start a vote for KIP-320: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-320% > 3A+Allow+fetchers+to+detect+and+handle+log+truncation. > Thanks to everyone who

[VOTE] KIP-320: Allow fetchers to detect and handle log truncation

Hi All, I'd like to start a vote for KIP-320: https://cwiki.apache.org/confluence/display/KAFKA/KIP-320%3A+Allow+fetchers+to+detect+and+handle+log+truncation. Thanks to everyone who reviewed the proposal. Please feel free to send additional questions to the discussion thread if you have any. +1

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Rajini. I think we are considering the following two options. Let me try to describe them along with their relative advantages/disadvantages. Option #1: Send two callbacks in a single array to the callback handler: ch.handle(new Callback[] {tokenCallback, extensionsCallback}); Option

[jira] [Created] (KAFKA-7260) Failures while attempting to move partitions with kafka-reassign-partitions.sh

Kristopher Browne created KAFKA-7260: Summary: Failures while attempting to move partitions with kafka-reassign-partitions.sh Key: KAFKA-7260 URL: https://issues.apache.org/jira/browse/KAFKA-7260

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Ron, Yes, I was thinking of a SaslExtensionsValidatorCallback with additional context as well initially, but I didn't like the idea of name-value pairs and I didn't want generic objects passed around through the callback So providing context through other callbacks felt like a neater fit.

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Ron, I was also thinking along the lines of send the callbacks separately. Initially I thought the `CallbackHandler` can simply maintain state but that's not a good idea since it'll be dependent on the implementation of the `SaslServer` rather than the interface. Having the

Build failed in Jenkins: kafka-trunk-jdk10 #386

See Changes: [github] KAFKA-7255: Fix timing issue with create/update in SimpleAclAuthorizer [harsha] KAFKA-7247: Update a link to Apache BookKeeper project (#5460) --

Re: [EXTERNAL] [VOTE] KIP-310: Add a Kafka Source Connector to Kafka Connect

Hi Could we get a couple of votes on this KIP - voting closes in 24 hours. Thanks, Rhys > On Aug 6, 2018, at 11:51 AM, McCaig, Rhys wrote: > > Hi All, > > I’m starting a vote on KIP-310: Add a Kafka Source Connector to Kafka Connect > > KIP: >

[jira] [Resolved] (KAFKA-7247) A link to Apache BookKeeper project is broken

[ https://issues.apache.org/jira/browse/KAFKA-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-7247. -- Resolution: Fixed Fix Version/s: 2.1.0 > A link to Apache BookKeeper project is broken >

Re: Kafka stream - Internal topic name and schema avro compatibility

Hi Cédric I do not know how the topology names are chosen, but provided that you didn't change any of the topology then new topics will not be created or require alteration. If you modify the topology then the naming can indeed change, but it would then create a new internal topic and there

[jira] [Created] (KAFKA-7259) Remove deprecated ZKUtils usage from ZkSecurityMigrator

Manikumar created KAFKA-7259: Summary: Remove deprecated ZKUtils usage from ZkSecurityMigrator Key: KAFKA-7259 URL: https://issues.apache.org/jira/browse/KAFKA-7259 Project: Kafka Issue Type:

Re: [DISCUSS] Applying scalafmt to core code

By doing piecemeal formatting, I don't think we can do a "hard" enforcement on using scalafmt with every PR, but by allowing the tool to run on already modified files in a patch, we can slowly migrate towards getting the entire code base clean.  The trade offs are pretty standard (giant patch

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Rajini. I also like that idea, but I think it might rely on one or possibly two implicit assumptions that I'm not sure are guaranteed to be true. First, I'm not sure if the CallbackHandler interface guarantees that implementations must process callbacks in order. Second (and more plausibly

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Rajini, That approach makes more sense to me. I like it On Wed, Aug 8, 2018 at 5:35 PM Rajini Sivaram wrote: > Hi Ron/Stanislav, > > Do you think it makes sense to separate out OAuthBearerValidatorCallback > and SaslExtensionsValidatorCallback so that it is clearer that these are > two

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Ron/Stanislav, Do you think it makes sense to separate out OAuthBearerValidatorCallback and SaslExtensionsValidatorCallback so that it is clearer that these are two separate entities that need validation? When we add support for customisable extensions in other mechanisms, we could reuse

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Ron, Yes, I agree we should document it thoroughly On Wed, Aug 8, 2018 at 5:02 PM Ron Dagostino wrote: > Hi Stanislav. If the community agrees we should add it then we should at a > minimum add explicit warnings in the Javadoc making it very clear how this > should not be used. > > Ron > >

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Stanislav. If the community agrees we should add it then we should at a minimum add explicit warnings in the Javadoc making it very clear how this should not be used. Ron On Wed, Aug 8, 2018 at 11:54 AM Stanislav Kozlovski wrote: > Hey Ron, > > I fully agree that token validation is a

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hey Ron, I fully agree that token validation is a serious security operation. Although, I believe allowing the user to do more validation with the extensions does not hurt - the user is fully responsible for his security once he starts implementing custom code for token validation. You would

Kafka stream - Internal topic name and schema avro compatibility

Within the Kafka Stream topology, internal topic are created. For this internal topics, schema avro for key and value are registered into schema registry. For the topic internal-MYAPPS-KSTREAM-KEY-SELECT-99-repartition, I have 2 subjects into schema registry : -

[jira] [Created] (KAFKA-7258) Different offset numbers in Windows and Linux

Ansel Zandegran created KAFKA-7258: -- Summary: Different offset numbers in Windows and Linux Key: KAFKA-7258 URL: https://issues.apache.org/jira/browse/KAFKA-7258 Project: Kafka Issue Type:

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

+1 (non-binding). Thanks for the KIP. On Wed, Aug 8, 2018 at 7:36 PM Ted Yu wrote: > +1 > > On Wed, Aug 8, 2018 at 4:09 AM Mickael Maison > wrote: > > > +1 (non-binding) > > Thanks Vahid > > On Wed, Aug 8, 2018 at 11:26 AM Kamal Chandraprakash > > wrote: > > > > > > +1 (non-binding) > > > >

[jira] [Created] (KAFKA-7257) --property (silently) fails when configuring SSL parameters

Andrew Pennebaker created KAFKA-7257: Summary: --property (silently) fails when configuring SSL parameters Key: KAFKA-7257 URL: https://issues.apache.org/jira/browse/KAFKA-7257 Project: Kafka

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Stanislav. If you wanted to do this a good way might be to add a constructor to the org.apache.kafka.common.security.oauthbearer. OAuthBearerValidatorCallback class that accepts a SaslExtensions instance in addition to a token value. Then it would give the callback handler the option to

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

+1 On Wed, Aug 8, 2018 at 4:09 AM Mickael Maison wrote: > +1 (non-binding) > Thanks Vahid > On Wed, Aug 8, 2018 at 11:26 AM Kamal Chandraprakash > wrote: > > > > +1 (non-binding) > > > > Thanks for the KIP. > > > > On Wed, Aug 8, 2018 at 3:11 PM Stanislav Kozlovski < > stanis...@confluent.io>

Re: [DISCUSS] add connect-related packages to "What is considered a "major change" that needs a KIP?"

Oops, ignore that messy shell command -- it misses the streams packages. If anyone knows how to use gradle to get the actual list of packages I would be interested in knowing the command. Ron On Wed, Aug 8, 2018 at 9:59 AM Ron Dagostino wrote: > Yes, I have recently been wondering what exactly

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hey everybody, Sorry for reviving this, but I neglected something the first time around. I believe it would be useful to provide the `OAuthBearerUnsecuredValidatorCallbackHandler` with the OAuth extensions too. This would enable use cases where validators want to reconcile information from the

Re: [DISCUSS] add connect-related packages to "What is considered a "major change" that needs a KIP?"

Yes, I have recently been wondering what exactly the set of public-facing APIs really is. If it is fact defined by the Javadoc then the KIP info page should state this. In case anyone finds it helpful, see below for one (admittedly imperfect) way to see the list of packages. There is probably a

[jira] [Created] (KAFKA-7256) First started Kafka server & leader failure blocks consumers

Petre Adrian created KAFKA-7256: --- Summary: First started Kafka server & leader failure blocks consumers Key: KAFKA-7256 URL: https://issues.apache.org/jira/browse/KAFKA-7256 Project: Kafka

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

More followup, and +dev as Guozhang replied to me directly previously. I am currently porting the code over to trunk. One of the major changes since 1.0 is the usage of GraphNodes. I have a question about this: For a foreignKey joiner, should it have its own dedicated node type? Or would it be

Re: [VOTE] KIP-346 - Improve LogCleaner behavior on error

+1 (non-binding) Thanks for the KIP! On Tue, Aug 7, 2018 at 6:53 PM Gwen Shapira wrote: > > +1 (binding) > > On Tue, Aug 7, 2018 at 4:19 AM, Stanislav Kozlovski > wrote: > > > Hey everybody, > > I'm starting a vote on KIP-346 > > >

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

+1 (non-binding) Thanks Vahid On Wed, Aug 8, 2018 at 11:26 AM Kamal Chandraprakash wrote: > > +1 (non-binding) > > Thanks for the KIP. > > On Wed, Aug 8, 2018 at 3:11 PM Stanislav Kozlovski > wrote: > > > +1 (non-binding) > > Thanks! > > > > On Tue, Aug 7, 2018 at 11:47 PM Jason Gustafson > >

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

+1 (non-binding) Thanks for the KIP. On Wed, Aug 8, 2018 at 3:11 PM Stanislav Kozlovski wrote: > +1 (non-binding) > Thanks! > > On Tue, Aug 7, 2018 at 11:47 PM Jason Gustafson > wrote: > > > +1 Thanks Vahid. > > > > On Tue, Aug 7, 2018 at 11:14 AM, Vahid S Hashemian < > >

Re: [DISCUSS] add connect-related packages to "What is considered a "major change" that needs a KIP?"

hi Ewen > However, note that we also document the exact set of public APIs, or at > least try to, in the user-facing javadocs > http://kafka.apache.org/20/javadoc/index.html?overview-summary.html If its > not in there, it likely is not public API. Thanks for the information! The user-facing

Re: [VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

> If you are inheriting from SerializationException, your derived class should also be a kind of serialization exception. Not something more general. Yeah, the reason for inheriting it would be for backwards-compatibility. > Hmm. Can you think of any new scenarios that would make Kafka force

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

+1 (non-binding) Thanks! On Tue, Aug 7, 2018 at 11:47 PM Jason Gustafson wrote: > +1 Thanks Vahid. > > On Tue, Aug 7, 2018 at 11:14 AM, Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > > > Hi all, > > > > I'd like to start a vote on KIP-289 to modify the default group id of > >

[jira] [Created] (KAFKA-7255) Timing issue in SimpleAclAuthorizer with concurrent create/update

Rajini Sivaram created KAFKA-7255: - Summary: Timing issue in SimpleAclAuthorizer with concurrent create/update Key: KAFKA-7255 URL: https://issues.apache.org/jira/browse/KAFKA-7255 Project: Kafka