Jenkins build is back to normal : kafka-trunk-jdk8 #3344

See

Build failed in Jenkins: kafka-2.1-jdk8 #114

See Changes: [cmccabe] MINOR: update copyright year in the NOTICE file. (#6196) -- [...truncated 693.88 KB...] kafka.api.PlaintextProducerSendTest >

Build failed in Jenkins: kafka-trunk-jdk11 #241

See Changes: [jason] KAFKA-7692; Fix ProducerStateManager SequenceNumber overflow (#5990) [jason] KAFKA-7693; Fix SequenceNumber overflow in producer (#5989) --

Build failed in Jenkins: kafka-1.1-jdk7 #240

See Changes: [jason] KAFKA-7692; Fix ProducerStateManager SequenceNumber overflow (#5990) [jason] KAFKA-7693; Fix SequenceNumber overflow in producer (#5989) --

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

Thank you for the questions Cyrus! Replies inline: On Thu, Jan 24, 2019 at 6:03 PM Cyrus Vafadari wrote: > "scheduled.rebalance.max.delay.ms" -- "...the actual delay used by the > leader to hold off redistribution of connectors and tasks and maintain > imbalance may be less or equal to this

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

"scheduled.rebalance.max.delay.ms" -- "...the actual delay used by the leader to hold off redistribution of connectors and tasks and maintain imbalance may be less or equal to this value." How is the actual delay determined and specified in the AssignmentFlatBuffer? I see this defaults to a max

Re: [VOTE] - KIP-213 (new vote) - Simplified and revised.

Bumping this vote because I don't want it to languish. It is very unlikely to go into 2.2 at this point, but I would like to avoid resurrecting a dead thread in 30 days time. On Tue, Jan 15, 2019 at 5:07 PM Adam Bellemare wrote: > All good Matthias. If it doesn’t get in for 2.2 I’ll just do it

Build failed in Jenkins: kafka-trunk-jdk11 #240

See Changes: [github] MINOR: update copyright year in the NOTICE file. (#6196) -- [...truncated 2.27 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

Hi Jason! Thank you for the comments. Replying below: On Thu, Jan 24, 2019 at 9:10 AM Jason Gustafson wrote: > Hi Konstantine, > > Thanks for the clear, well-written proposal. > > I think my only doubt is about the `connect.protocol` configuration. A > couple questions: > > 1. Do we need the

Build failed in Jenkins: kafka-trunk-jdk8 #3343

See Changes: [github] MINOR: update copyright year in the NOTICE file. (#6196) -- [...truncated 2.27 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: [VOTE] KIP-307: Allow to define custom processor names with KStreams DSL

Hello folks, I'd like to call out for another time on voting for this KIP, given the deadline is approaching now. Guozhang On Thu, Jan 17, 2019 at 3:11 PM John Roesler wrote: > FWIW, I'm also +1 on the consistency changes to Joined. > > -John > > On Thu, Jan 17, 2019 at 5:04 PM Guozhang Wang

Re: [VOTE] KIP-349 Priorities for Source Topics

Hi Colin, > On Jan 24, 2019, at 12:14 PM, Colin McCabe wrote: > > Users almost always like the idea of new features, whatever they are. But > that doesn't mean that the feature would necessarily work well or be > necessary. Yes, though we should certainly consider the responses on the user

Re: [DISCUSS] KIP-382: MirrorMaker 2.0

Pippin, thanks for your interest. I will publish a PR soon (several days?) which you'll be able to build and play with. Watch this space :) Ryanne On Thu, Jan 24, 2019 at 5:19 PM Pippin Wallace wrote: > > I see that the Current state of KIP-382 recently changed from Voting to > Accepted on

Re: [VOTE] KIP-421: Support resolving externalized secrets in AbstractConfig

> It allows _all_ existing clients of the class, e.g. those in Apache Kafka or in applications written by other people that use the class, to get this functionality for free, i.e. without any code changes. (I realize this is probably where the 'unexpected effects' comes from). > Personally, I

[DISCUSS] KIP-382: MirrorMaker 2.0

I see that the Current state of KIP-382 recently changed from Voting to Accepted on Confluence page https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0 I am just looking for a best guess as to when this might make it into an alpha, beta, or GA release? Regards,

[jira] [Resolved] (KAFKA-7741) Bad dependency via SBT

[ https://issues.apache.org/jira/browse/KAFKA-7741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe resolved KAFKA-7741. Resolution: Fixed > Bad dependency via SBT > -- > >

Re: [VOTE] KIP-420: Add Single Value Fetch in Session Stores

+1 from myself as well. Folks, thanks for all the votes. I'm closing this ticket now with three biding votes (Damian, Matthas, Guozhang) and one non-binding votes (Bill). Guozhang On Thu, Jan 24, 2019 at 2:53 PM Matthias J. Sax wrote: > +1 (binding) > > -Matthias > > On 1/24/19 8:23 AM,

Re: [VOTE] KIP-420: Add Single Value Fetch in Session Stores

+1 (binding) -Matthias On 1/24/19 8:23 AM, Damian Guy wrote: > +1 > > On Wed, 23 Jan 2019 at 23:23, Guozhang Wang wrote: > >> Hello Matthias, >> >> Cool. I'd add it to the wiki page as well. >> >> >> Guozhang >> >> On Sat, Jan 19, 2019 at 10:59 AM Matthias J. Sax >> wrote: >> >>> Thanks for

[jira] [Resolved] (KAFKA-7856) Cryptographic Issues by Insufficient Entropy

[ https://issues.apache.org/jira/browse/KAFKA-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin P. McCabe resolved KAFKA-7856. Resolution: Invalid > Cryptographic Issues by Insufficient Entropy >

[jira] [Resolved] (KAFKA-7796) structured streaming fetched wrong current offset from kafka

[ https://issues.apache.org/jira/browse/KAFKA-7796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryne Yang resolved KAFKA-7796. -- Resolution: Invalid Not related to kafka, it's actually spark structured streaming integer overflow: 

Re: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL Factory

Hi Rajini, Since you helped review the KIP if you don't mind can you vote on this KIP. Thanks, Harsha On Wed, Jan 9, 2019, at 8:05 AM, Harsha wrote: > HI All, > We are looking forward to this KIP. Appreciate if others can > take a look at the kip and > vote on this thread. >

Jenkins build is back to normal : kafka-trunk-jdk11 #239

See

Getting added to the contributor list

Hi! I’m using Kafka in my everyday work, and want to start start contributing. If I understand it correctly, I need to be added to the contributors list in order to assign a Jira issue to myself? If this is correct, can you please add me to this list? My Jira username is: aurlien Thanks!

Build failed in Jenkins: kafka-trunk-jdk8 #3342

See Changes: [github] KAFKA-7793: Improve the Trogdor command line. (#6133) -- [...truncated 2.48 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Re: [VOTE] KIP-421: Support resolving externalized secrets in AbstractConfig

On Thu, Jan 24, 2019, at 11:25, TEJAL ADSUL wrote: > > > On 2019/01/24 17:26:02, Andy Coates wrote: > > I'm wondering why we're rejected changing AbstractConfig to automatically > > resolve the variables? > > > > > 1. Change AbstractConfig to *automatically* resolve variables of the form > >

Re: [VOTE] KIP-421: Support resolving externalized secrets in AbstractConfig

On 2019/01/24 18:54:13, Rajini Sivaram wrote: > Hi Tejal, > > Thanks for the KIP. I have a couple of comments/questions. > > It sounds like we are addressing password protection for clients, Connect > etc., but not for brokers, even though the changes proposed are in classes > common to

Re: [VOTE] KIP-421: Support resolving externalized secrets in AbstractConfig

On 2019/01/24 17:26:02, Andy Coates wrote: > I'm wondering why we're rejected changing AbstractConfig to automatically > resolve the variables? > > > 1. Change AbstractConfig to *automatically* resolve variables of the form > specified in KIP-297. This was rejected because it would change

[jira] [Created] (KAFKA-7869) Refactor RocksDBConfigSetter API to separate DBOptions and CFOptions

Guozhang Wang created KAFKA-7869: Summary: Refactor RocksDBConfigSetter API to separate DBOptions and CFOptions Key: KAFKA-7869 URL: https://issues.apache.org/jira/browse/KAFKA-7869 Project: Kafka

Re: [VOTE] KIP-421: Support resolving externalized secrets in AbstractConfig

Hi Tejal, Thanks for the KIP. I have a couple of comments/questions. It sounds like we are addressing password protection for clients, Connect etc., but not for brokers, even though the changes proposed are in classes common to brokers and clients. It is ok for the scope to be limited, but we

Re: [VOTE] KIP-421: Support resolving externalized secrets in AbstractConfig

I'm wondering why we're rejected changing AbstractConfig to automatically resolve the variables? > 1. Change AbstractConfig to *automatically* resolve variables of the form specified in KIP-297. This was rejected because it would change the behavior of existing code and might cause unexpected

Re: [DISCUSS] KIP-419 Safely notify Kafka Connect SourceTask is stopped

Ah, I'm sorta wrong -- in the current implementation, restartTask() stops the task and starts a *new* task instance with the same task ID. (I'm not certain that is clear from the documentation or interfaces, or if that may change in the future.) Ryanne On Thu, Jan 24, 2019 at 10:25 AM Ryanne

Re: [VOTE] KIP-421: Support resolving externalized secrets in AbstractConfig

On Wed, Jan 23, 2019, at 17:22, TEJAL ADSUL wrote: > > Hi Colin, > > If the configProvider is not configured we will return the unresolved > value as is, so from the example the value returned will be > ${file:/path/to/variables.properties:foo.bar} instead of the resolved > value. OK. This

Re: [VOTE] KIP-349 Priorities for Source Topics

On Thu, Jan 24, 2019, at 05:31, n...@afshartous.com wrote: > > > On Jan 17, 2019, at 8:49 PM, n...@afshartous.com wrote: > > > >> On Jan 15, 2019, at 2:26 PM, Colin McCabe >> > wrote: > >> > >> I think it makes sense to go back to use-cases again. So far, all of the

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

Hi Konstantine, Thanks for the clear, well-written proposal. I think my only doubt is about the `connect.protocol` configuration. A couple questions: 1. Do we need the `eager` option? It sounds like we're planning to deprecate it anyway. 2. Can you explain how `compatible` works? The basic idea

Re: [VOTE] KIP-349 Priorities for Source Topics

On 24.01.2019 15:51, Thomas Becker wrote: > Yes, I think this type of strategy interface would be valuable. > Thank you for leaving this here!

Re: [DISCUSS] KIP-419 Safely notify Kafka Connect SourceTask is stopped

I've now added a diagram to illustrate the states of a SourceTask. The KIP is essentially trying to give a clear signal to SourceTask when all work has stopped. In particular, if a SourceTask has a session to the source system that it uses in poll() and commit(), it now has a safe way to

Re: [DISCUSS] KIP-390: Add producer option to adjust compression level

Hi Becket, Thank you for your opinion. Frankly, I have no strong opinion on configuration name. In this problem, I will follow the community's choice. (I like your idea in that it has a notion of 'scope' per compression codec. However, it should be implemented on top of new config type like Map;

Re: [DISCUSS] KIP-419 Safely notify Kafka Connect SourceTask is stopped

Andrew, I believe the task can be started again with start() during the stopping and stopped states in your diagram. Ryanne On Thu, Jan 24, 2019, 10:20 AM Andrew Schofield I've now added a diagram to illustrate the states of a SourceTask. The KIP > is essentially trying to give a clear signal

Re: [VOTE] KIP-420: Add Single Value Fetch in Session Stores

+1 On Wed, 23 Jan 2019 at 23:23, Guozhang Wang wrote: > Hello Matthias, > > Cool. I'd add it to the wiki page as well. > > > Guozhang > > On Sat, Jan 19, 2019 at 10:59 AM Matthias J. Sax > wrote: > > > Thanks for the KIP Guozhang! > > > > Would it make sense to add a default implementation for

Re: [VOTE] KIP-349 Priorities for Source Topics

Yes, I think this type of strategy interface would be valuable. On Wed, 2019-01-16 at 15:41 +, Jan Filipiak wrote: On 16.01.2019 14:05, Thomas Becker wrote: I'm going to bow out of this discussion since it's been made clear that the feature is not targeted at streams. But for the record,

[jira] [Resolved] (KAFKA-7223) KIP-328: Add in-memory Suppression

[ https://issues.apache.org/jira/browse/KAFKA-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-7223. - Resolution: Fixed > KIP-328: Add in-memory Suppression > -- > >

Re: [VOTE] KIP-349 Priorities for Source Topics

> On Jan 17, 2019, at 8:49 PM, n...@afshartous.com wrote: > >> On Jan 15, 2019, at 2:26 PM, Colin McCabe > > wrote: >> >> I think it makes sense to go back to use-cases again. So far, all of the >> use-cases we discussed could be handled by pause and resume. So it

[jira] [Created] (KAFKA-7867) Broker fails after corrupted page table

Maurits Hartman created KAFKA-7867: -- Summary: Broker fails after corrupted page table Key: KAFKA-7867 URL: https://issues.apache.org/jira/browse/KAFKA-7867 Project: Kafka Issue Type: Bug