[jira] [Created] (KAFKA-8932) Add tag for TopicConfigErrorCode in CreateTopics response when KIP-482 is merged

[Rajini Sivaram (Jira)]

Rajini Sivaram created KAFKA-8932:
-

 Summary: Add tag for TopicConfigErrorCode in CreateTopics response 
when KIP-482 is merged
 Key: KAFKA-8932
 URL: https://issues.apache.org/jira/browse/KAFKA-8932
 Project: Kafka
  Issue Type: Sub-task
  Components: clients
Reporter: Rajini Sivaram
Assignee: Rajini Sivaram
 Fix For: 2.4.0


See 
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response]
 for protocol changes. Splitting this out to get the main PR with AdminClient 
changes merged first under KAFKA-8907.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Build failed in Jenkins: kafka-trunk-jdk8 #3919

[Apache Jenkins Server]

See 

--
[...truncated 2.64 MB...]
org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessConsumerRecordList[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessConsumerRecordList[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUseSinkSpecificSerializers[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUseSinkSpecificSerializers[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldFlushStoreForFirstInput[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldFlushStoreForFirstInput[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessFromSourceThatMatchPattern[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessFromSourceThatMatchPattern[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUpdateStoreForNewKey[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUpdateStoreForNewKey[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateOnWallClockTime[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateOnWallClockTime[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > shouldSetRecordMetadata[Eos 
enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > shouldSetRecordMetadata[Eos 
enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForLargerValue[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForLargerValue[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnCorrectInMemoryStoreTypeOnly[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnCorrectInMemoryStoreTypeOnly[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessRecordForTopic[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessRecordForTopic[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldForwardRecordsFromSubtopologyToSubtopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldForwardRecordsFromSubtopologyToSubtopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForSmallerValue[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForSmallerValue[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] PASSED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis STARTED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis PASSED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime STARTED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime PASSED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep PASSED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnIsOpen 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnIsOpen 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldDeleteAndReturnPlainValue STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldDeleteAndReturnPlainValue PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnName 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnName 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutAllWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutAllWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 

Re: [DISCUSS] KIP-504 - Add new Java Authorizer Interface

[Rajini Sivaram]

Hi all,

I have submitted a PR to address a few review comments from Ismael. There
are a couple of interface changes in the PR, I have updated the KIP as well.

   1. The `listener()` method in AuthorizableRequestContext and Endpoint
   has been renamed to listenerName()
   2. Endpoint.listenerName() now returns an Optional. Since this
   is in the common package, we may use it in future in clients where we
   don't have listener names

Let me know if there are any concerns.

Regards,

Rajini

On Mon, Sep 9, 2019 at 4:53 PM Rajini Sivaram 
wrote:

> Hi Jun,
>
> Thanks for the response. If we use the existing purgatory implementation,
> we should get additional purgatory metrics for ACL updates with the new
> purgatory name as tag, consistent with what we have for other delayed
> operations. I will add these to the KIP. We also have request metrics which
> indicate local vs remote time with the API key as tag, so that should
> indicate the portion of time waiting for async operations.
>
> Regards,
>
> Rajini
>
> On Mon, Sep 9, 2019 at 4:31 PM Jun Rao  wrote:
>
>> Hi, Rajini, Ismael,
>>
>> Yes, I can see the argument for making CreateAcls/DeleteAcls async. I am
>> ok
>> with that if you feel the implementation is not too complicated. Should we
>> consider adding some additional metric to reflect the portion of the time
>> spent in waiting for the async operation to complete?
>>
>> Thanks,
>>
>> Jun
>>
>> On Mon, Sep 9, 2019 at 5:20 AM Ismael Juma  wrote:
>>
>> > Hi Jun,
>> >
>> > As you say, even though the average number of operations may be low, the
>> > request rate can be high in bursts. This can overwhelm the request queue
>> > and cause an outage for no good reason. Or the backing storage system
>> may
>> > be slow for a period of time, causing a similar issue.
>> >
>> > I've seen several such issues in production affecting Kafka's
>> > create/alter/delete paths (e.g people creating/deleting thousands/tens
>> of
>> > thousands of topics in a hot loop). As you say, it's not just ACLs that
>> are
>> > vulnerable. In my opinion, we should learn from these experiences when
>> > designing new interfaces and we should adjust the existing ones to be
>> more
>> > resilient. It's also worth mentioning that create/delete topics already
>> > relies on the purgatory if the request timeout is greater than 0.
>> >
>> > Can you elaborate on how CPU throttling would help? It seems to me that
>> it
>> > would not trigger fast enough if there was a slow back-end, for example
>> > (the number of request threads is often lower than 10, so it doesn't
>> take
>> > many blocked requests to cause major problems).
>> >
>> > Ismael
>> >
>> > On Sun, Sep 8, 2019, 9:5k2 PM Jun Rao  wrote:
>> >
>> > > Hi, Rajini,
>> > >
>> > > Thanks for the reply. The 4-step approach that you outlined seems to
>> > work.
>> > > Overall, I can see that the async authorize() api could lead to an
>> > overall
>> > > more efficient implementation. The tradeoff is that we have to code
>> every
>> > > request with an extra stage. To me, this optimization seems too early.
>> > With
>> > > 100K topics, each with 1KB worth of users, the required ACL space is
>> > about
>> > > 100MB and we should be able to cache everything. Beyond that, we still
>> > have
>> > > the option of dividing up the topic and ACL metadata such that every
>> > broker
>> > > is only required to cache a subset of all metadata.
>> > >
>> > > Regarding making create/delete api async, I still have mixed feelings
>> on
>> > > this. I am wondering if the benefit is worth the added complexity. To
>> me,
>> > > both operations are rare. If one request thread is blocked
>> occasionally
>> > for
>> > > a few millisecs, it's probably ok since it mostly just affects one
>> > client.
>> > > In the case that a particular user issues many those operations in a
>> > short
>> > > window. Could we just use CPU throttling to prevent too much resource
>> > being
>> > > used? There are a few other similar types of requests such as
>> > create/alter
>> > > configs, create/alter topic, etc. Do we plan to add an extra
>> processing
>> > > stage for each of them too in the future?
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> >
>>
>

Re: [VOTE] KIP-525 - Return topic metadata and configs in CreateTopics response

[Rajini Sivaram]

Hi Satish, I have updated the KIP to use TopicMetadataAndConfig for the
inner class instead of CreateResult.

I also fixed the version in the protocol to use 5 instead of 6 since the
update to version 5 in KIP-482 will be going out in the same release as
this KIP.

Since both are minor changes that don't affect the externals, I won't
restart the vote.

Regards,

Rajini

On Mon, Sep 23, 2019 at 1:01 PM Kamal Chandraprakash <
kamal.chandraprak...@gmail.com> wrote:

> +1 (non-binding), Thanks for the KIP!
>
> On Mon, Sep 23, 2019 at 10:45 AM Satish Duggana 
> wrote:
>
> > Thanks for the KIP, +1 (non binding)
> >
> > This looks to be a useful API for admin tools. When we worked on an
> > admin tool for Kafka, we had to explicitly call the config API for
> > showing the configs for the newly created topic.
> > I have a minor-nit comment on `CreateResult` which seems to be a
> > slightly confusing name as it is not really about the CreateResult of
> > a topic but it is about topic metadata and config(maybe
> > TopicMetadataWithConfig). This does not need to be addressed in the
> > KIP now as it is not exposed to clients. But you may want to address
> > this in the PR later.
> >
> >
> > On Sun, Sep 22, 2019 at 9:02 PM Harsha Chintalapani 
> > wrote:
> > >
> > > +1 (binding).
> > > -Harsha
> > >
> > >
> > > On Sat, Sep 21, 2019 at 12:13 AM, Manikumar  >
> > > wrote:
> > >
> > > > +1 (binding), Thanks for the KIP.
> > > >
> > > > Thanks,
> > > >
> > > > On Sat, Sep 21, 2019 at 2:49 AM Colin McCabe 
> > wrote:
> > > >
> > > > +1 (binding). Thanks, Rajini.
> > > >
> > > > best,
> > > > Colin
> > > >
> > > > On Fri, Sep 20, 2019, at 00:43, Rajini Sivaram wrote:
> > > >
> > > > Hi all,
> > > >
> > > > I would like to start vote on KIP-525 to return configs in
> CreateTopics
> > > > response. This is a minor KIP that returns additional data in the
> > > >
> > > > response
> > > >
> > > > without breaking compatibility.
> > > >
> > > > -
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/
> > > > KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response
> > > >
> > > > Thank you,
> > > >
> > > > Rajini
> > > >
> > > >
> >
>

Build failed in Jenkins: kafka-trunk-jdk8 #3920

[Apache Jenkins Server]

See 


Changes:

[github] KAFKA-8848; Update system tests to use new AclAuthorizer (#7374)

--
Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on H41 (ubuntu xenial) in workspace 

[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
[WS-CLEANUP] Done
No credentials specified
Cloning the remote Git repository
Cloning repository https://github.com/apache/kafka.git
 > git init  # timeout=10
Fetching upstream changes from https://github.com/apache/kafka.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/kafka.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # 
 > timeout=10
 > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10
Fetching upstream changes from https://github.com/apache/kafka.git
 > git fetch --tags --progress https://github.com/apache/kafka.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/trunk^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/trunk^{commit} # timeout=10
Checking out Revision 0d31272b350adf40bda09709304721545553d7f8 
(refs/remotes/origin/trunk)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 0d31272b350adf40bda09709304721545553d7f8
Commit message: "KAFKA-8848; Update system tests to use new AclAuthorizer 
(#7374)"
 > git rev-list --no-walk bc4fd676e5ea0e3cf44a8684dca0db46f912c828 # timeout=10
Setting GRADLE_4_10_3_HOME=/home/jenkins/tools/gradle/4.10.3
[kafka-trunk-jdk8] $ /bin/bash -xe /tmp/jenkins7043756291710883065.sh
+ rm -rf 
+ /home/jenkins/tools/gradle/4.10.3/bin/gradle
To honour the JVM settings for this build a new JVM will be forked. Please 
consider using the daemon: 
https://docs.gradle.org/4.10.3/userguide/gradle_daemon.html.

FAILURE: Build failed with an exception.

* What went wrong:
Unable to start the daemon process.
This problem might be caused by incorrect configuration of the daemon.
For example, an unrecognized jvm option is used.
Please refer to the user guide chapter on the daemon at 
https://docs.gradle.org/4.10.3/userguide/gradle_daemon.html
Please read the following process output to find out more:
---
Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread


* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org
Build step 'Execute shell' marked build as failure
[FINDBUGS] Collecting findbugs analysis files...
Setting GRADLE_4_10_3_HOME=/home/jenkins/tools/gradle/4.10.3
[FINDBUGS] Searching for all files in 
 that match the pattern 
**/build/reports/*bugs/*.xml
[FINDBUGS] No files found. Configuration error?
Setting GRADLE_4_10_3_HOME=/home/jenkins/tools/gradle/4.10.3
No credentials specified
Setting GRADLE_4_10_3_HOME=/home/jenkins/tools/gradle/4.10.3
 Using GitBlamer to create author and commit information for all 
warnings.
 GIT_COMMIT=0d31272b350adf40bda09709304721545553d7f8, 
workspace=
[FINDBUGS] Computing warning deltas based on reference build #3914
Recording test results
Setting GRADLE_4_10_3_HOME=/home/jenkins/tools/gradle/4.10.3
ERROR: Step ?Publish JUnit test result report? failed: No test report files 
were found. Configuration error?
Setting GRADLE_4_10_3_HOME=/home/jenkins/tools/gradle/4.10.3
Not sending mail to unregistered user nore...@github.com

[jira] [Resolved] (KAFKA-8848) Update system test to use new authorizer

[Rajini Sivaram (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-8848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajini Sivaram resolved KAFKA-8848.
---
  Reviewer: Manikumar
Resolution: Fixed

> Update system test to use new authorizer
> 
>
> Key: KAFKA-8848
> URL: https://issues.apache.org/jira/browse/KAFKA-8848
> Project: Kafka
>  Issue Type: Sub-task
>  Components: security
>Reporter: Rajini Sivaram
>Assignee: Rajini Sivaram
>Priority: Major
> Fix For: 2.4.0
>
>
> We should run system tests with the new authorizer.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Apache Kafka 2.4.0 release

[Rajini Sivaram]

Hi Manikumar,

Can we add KIP-525 (
https://cwiki.apache.org/confluence/display/KAFKA/KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response)
to 2.4.0 release? It has been accepted and is a minor KIP.

Thank you,

Rajini

On Fri, Sep 20, 2019 at 6:14 AM Manikumar  wrote:

> Hi,
>
>  Added KIP-401 to the wiki page for tracking.
>
> Thanks,
>
> On Fri, Sep 20, 2019 at 7:55 AM Paul Whalen  wrote:
>
> > Manikumar,
> >
> > KIP-401 was accepted a few weeks ago and there is a PR pending review,
> can
> > it be included in the release as well?
> >
> > Thanks,
> > Paul
> >
> > On Mon, Sep 16, 2019 at 6:14 AM Manikumar 
> > wrote:
> >
> > > Hi All,
> > >
> > > Just a reminder that any new/pending KIP must pass vote by next
> Wednesday
> > > (Sep 25, 2019) to be included
> > > in Apache Kafka 2.4.0 release.
> > >
> > > Also keep in mind that deadline for feature freeze is Oct 2, 2019.
> > > In order to be included in the release, major features/KIPs need to be
> > > merged and minor features need to be
> > > have PR ready.  Any feature/KIP not in this state will be automatically
> > > moved to the next release after Oct 2.
> > >
> > > There are total 84 open
> > > <
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%202.4.0%20AND%20status%20not%20in%20(resolved%2C%20closed)%20ORDER%20BY%20priority%20DESC%2C%20status%20DESC%2C%20updated%20DESC%20%20%20%20%20%20%20%20
> > > >
> > > JIRAs. Please update your assigned JIRAs, if you know they cannot make
> it
> > > to 2.4.0.
> > > There are also quite a few JIRAs related to flaky tests. We really
> > > appreciate any help on fixing these failing tests.
> > >
> > > Thanks,
> > > Manikumar
> > >
> > > On Mon, Sep 16, 2019 at 4:08 PM Manikumar 
> > > wrote:
> > >
> > > > Hi Mickael,
> > > >
> > > > Yes, we can include. Added KIP-396 to the wiki page for tracking.
> > > >
> > > >
> > > > Thanks,
> > > > Manikumar
> > > >
> > > > On Mon, Sep 16, 2019 at 3:36 PM Mickael Maison <
> > mickael.mai...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > >> Hi Manikumar,
> > > >>
> > > >> Can we also include KIP-396?
> > > >> (
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97551484
> > > >> )
> > > >> It has been accepted and the PR is ready for review:
> > > >> https://github.com/apache/kafka/pull/7296
> > > >>
> > > >> Thanks
> > > >>
> > > >> On Mon, Sep 16, 2019 at 10:56 AM Manikumar <
> manikumar.re...@gmail.com
> > >
> > > >> wrote:
> > > >> >
> > > >> > Hi Viktor,
> > > >> >
> > > >> > Yes, we can include KIP-434.
> > > >> >
> > > >> > Thanks,
> > > >> >
> > > >> > On Mon, Sep 16, 2019 at 3:09 PM Viktor Somogyi-Vass <
> > > >> viktorsomo...@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > Hi Manikumar,
> > > >> > >
> > > >> > > Can we please also include KIP-434?
> > > >> > >
> > > >> > >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-434%3A+Add+Replica+Fetcher+and+Log+Cleaner+Count+Metrics
> > > >> > > It has been accepted and there is already a pull request under
> > > review.
> > > >> > >
> > > >> > > Thanks,
> > > >> > > Viktor
> > > >> > >
> > > >> > > On Fri, Sep 6, 2019 at 9:59 AM Manikumar <
> > manikumar.re...@gmail.com
> > > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi David,
> > > >> > > >
> > > >> > > > Yes, we can include KIP-511.  KIP must be accepted by KIP
> Freeze
> > > >> date
> > > >> > > (Sep
> > > >> > > > 25, 2019 )
> > > >> > > >
> > > >> > > > Thanks,
> > > >> > > >
> > > >> > > >
> > > >> > > > On Fri, Sep 6, 2019 at 12:53 PM David Jacot <
> > dja...@confluent.io>
> > > >> wrote:
> > > >> > > >
> > > >> > > > > Hi Manikumar,
> > > >> > > > >
> > > >> > > > > Could we add KIP-511 to the plan? I think it will make it.
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > > David
> > > >> > > > >
> > > >> > > > > On Tue, Aug 27, 2019 at 5:32 PM Manikumar <
> > > >> manikumar.re...@gmail.com>
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Hi all,
> > > >> > > > > >
> > > >> > > > > > I put together a draft release plan with Oct 2019 as the
> > > release
> > > >> > > month
> > > >> > > > > and
> > > >> > > > > > a list of KIPs that have already been voted:
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=125307901
> > > >> > > > > >
> > > >> > > > > > Here are the dates:
> > > >> > > > > >
> > > >> > > > > > 1) KIP Freeze:  Sep 25, 2019 (A KIP must be accepted by
> this
> > > >> date in
> > > >> > > > > order
> > > >> > > > > > to be considered for this release)
> > > >> > > > > >
> > > >> > > > > > 2) Feature Freeze:  Oct 2, 2019 (Major features merged &
> > > >> working on
> > > >> > > > > > stabilization, minor features have PR,
> > > >> > > > > >  release branch cut; anything not in this state will be
> > > >> automatically
> > > >> > > > > moved
> > > >> > > > > > to the next 

Re: [VOTE] KIP-525 - Return topic metadata and configs in CreateTopics response

[Rajini Sivaram]

The vote has passed with 4 binding votes (Colin, Manikumar, Harsha, Rajini)
and 2 non-binding votes (Satish, Kamal). I will update the KIP page and
submit a PR.

Many thanks for the discussion and votes.

Regards,

Rajini



On Tue, Sep 24, 2019 at 9:22 AM Rajini Sivaram 
wrote:

>
> Hi Satish, I have updated the KIP to use TopicMetadataAndConfig for the
> inner class instead of CreateResult.
>
> I also fixed the version in the protocol to use 5 instead of 6 since the
> update to version 5 in KIP-482 will be going out in the same release as
> this KIP.
>
> Since both are minor changes that don't affect the externals, I won't
> restart the vote.
>
> Regards,
>
> Rajini
>
> On Mon, Sep 23, 2019 at 1:01 PM Kamal Chandraprakash <
> kamal.chandraprak...@gmail.com> wrote:
>
>> +1 (non-binding), Thanks for the KIP!
>>
>> On Mon, Sep 23, 2019 at 10:45 AM Satish Duggana > >
>> wrote:
>>
>> > Thanks for the KIP, +1 (non binding)
>> >
>> > This looks to be a useful API for admin tools. When we worked on an
>> > admin tool for Kafka, we had to explicitly call the config API for
>> > showing the configs for the newly created topic.
>> > I have a minor-nit comment on `CreateResult` which seems to be a
>> > slightly confusing name as it is not really about the CreateResult of
>> > a topic but it is about topic metadata and config(maybe
>> > TopicMetadataWithConfig). This does not need to be addressed in the
>> > KIP now as it is not exposed to clients. But you may want to address
>> > this in the PR later.
>> >
>> >
>> > On Sun, Sep 22, 2019 at 9:02 PM Harsha Chintalapani 
>> > wrote:
>> > >
>> > > +1 (binding).
>> > > -Harsha
>> > >
>> > >
>> > > On Sat, Sep 21, 2019 at 12:13 AM, Manikumar <
>> manikumar.re...@gmail.com>
>> > > wrote:
>> > >
>> > > > +1 (binding), Thanks for the KIP.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > On Sat, Sep 21, 2019 at 2:49 AM Colin McCabe 
>> > wrote:
>> > > >
>> > > > +1 (binding). Thanks, Rajini.
>> > > >
>> > > > best,
>> > > > Colin
>> > > >
>> > > > On Fri, Sep 20, 2019, at 00:43, Rajini Sivaram wrote:
>> > > >
>> > > > Hi all,
>> > > >
>> > > > I would like to start vote on KIP-525 to return configs in
>> CreateTopics
>> > > > response. This is a minor KIP that returns additional data in the
>> > > >
>> > > > response
>> > > >
>> > > > without breaking compatibility.
>> > > >
>> > > > -
>> > > >
>> > > > https://cwiki.apache.org/confluence/display/KAFKA/
>> > > > KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response
>> > > >
>> > > > Thank you,
>> > > >
>> > > > Rajini
>> > > >
>> > > >
>> >
>>
>

[jira] [Created] (KAFKA-8933) An unhandled SSL handshake exception in polling event - needed a retry logic

[Remigius (Jira)]

Remigius created KAFKA-8933:
---

 Summary: An unhandled SSL handshake exception in polling event - 
needed a retry logic
 Key: KAFKA-8933
 URL: https://issues.apache.org/jira/browse/KAFKA-8933
 Project: Kafka
  Issue Type: Wish
  Components: clients
Affects Versions: 2.2.1
 Environment: software platform
Reporter: Remigius


Already client is connected and during polling event, SSL handshake failure 
happened. it led to leaving the co-ordinator. Even on SSL handshake failure 
which was actually intermittent issue, polling should have some resilient and 
retry the polling. Leaving group caused all instances of clients to drop and 
left the messages in Kafka for long time until re-subscribe the kafka topic 
manually.

 

 
{noformat}
2019-09-06 04:03:09,016 ERROR [reactive-kafka-] 
org.apache.kafka.clients.NetworkClient [Consumer clientId=aaa, groupId=bbb] 
Connection to node 150 (host:port) failed authentication due to: SSL handshake 
failed
2019-09-06 04:03:09,021 ERROR [reactive-kafka-]  
reactor.kafka.receiver.internals.DefaultKafkaReceiver Unexpected exception
java.lang.NullPointerException: null
 at 
org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.handleCompletedMetadataResponse(NetworkClient.java:1012)
 ~[kafka-clients-2.2.1.jar!/:?]
 at 
org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:822)
 ~[kafka-clients-2.2.1.jar!/:?]
 at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:544) 
~[kafka-clients-2.2.1.jar!/:?]
 at 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
 ~[kafka-clients-2.2.1.jar!/:?]
 at 
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
 ~[kafka-clients-2.2.1.jar!/:?]
 at 
org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1256)
 ~[kafka-clients-2.2.1.jar!/:?]
 at 
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1200) 
~[kafka-clients-2.2.1.jar!/:?]
 at 
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1176) 
~[kafka-clients-2.2.1.jar!/:?]
 at 
reactor.kafka.receiver.internals.DefaultKafkaReceiver$PollEvent.run(DefaultKafkaReceiver.java:470)
 ~[reactor-kafka-1.1.1.RELEASE.jar!/:1.1.1.RELEASE]
 at 
reactor.kafka.receiver.internals.DefaultKafkaReceiver.doEvent(DefaultKafkaReceiver.java:401)
 ~[reactor-kafka-1.1.1.RELEASE.jar!/:1.1.1.RELEASE]
 at 
reactor.kafka.receiver.internals.DefaultKafkaReceiver.lambda$start$14(DefaultKafkaReceiver.java:335)
 ~[reactor-kafka-1.1.1.RELEASE.jar!/:1.1.1.RELEASE]
 at reactor.core.publisher.LambdaSubscriber.onNext(LambdaSubscriber.java:130) 
~[reactor-core-3.2.10.RELEASE.jar!/:3.2.10.RELEASE]
 at 
reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:398)
 ~[reactor-core-3.2.10.RELEASE.jar!/:3.2.10.RELEASE]
 at 
reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:484)
 ~[reactor-core-3.2.10.RELEASE.jar!/:3.2.10.RELEASE]
 at 
reactor.kafka.receiver.internals.KafkaSchedulers$EventScheduler.lambda$decorate$1(KafkaSchedulers.java:100)
 ~[reactor-kafka-1.1.1.RELEASE.jar!/:1.1.1.RELEASE]
 at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) 
~[reactor-core-3.2.10.RELEASE.jar!/:3.2.10.RELEASE]
 at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) 
~[reactor-core-3.2.10.RELEASE.jar!/:3.2.10.RELEASE]
 at 
org.springframework.cloud.sleuth.instrument.async.TraceCallable.call(TraceCallable.java:70)
 ~[spring-cloud-sleuth-core-2.1.1.RELEASE.jar!/:2.1.1.RELEASE]
 at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
 ~[?:?]
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 
~[?:?]
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 
~[?:?]
 at java.lang.Thread.run(Thread.java:834) [?:?]

2019-09-06 04:03:09,023 INFO  [reactive-kafka-] 
org.apache.kafka.clients.consumer.internals.AbstractCoordinator [Consumer 
clientId=aaa, groupId=bbb] Member x_13-081e61ec-1509-4e0e-819e-58063d1ce8f6 
sending LeaveGroup request to coordinator{noformat}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Apache Kafka 2.4.0 release

[Manikumar]

Hi All,

Just a reminder that tomorrow (Sep 25, 2019) is the last date for KIP
acceptance for 2.4.0 release.

Thanks,

On Tue, Sep 24, 2019 at 7:05 PM Manikumar  wrote:

> Hi Rajini,
>
> Added KIP-525 to the release plan for tracking.
>
> Thanks,
>
>
> On Tue, Sep 24, 2019 at 4:36 PM Rajini Sivaram 
> wrote:
>
>> Hi Manikumar,
>>
>> Can we add KIP-525 (
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response
>> )
>> to 2.4.0 release? It has been accepted and is a minor KIP.
>>
>> Thank you,
>>
>> Rajini
>>
>> On Fri, Sep 20, 2019 at 6:14 AM Manikumar 
>> wrote:
>>
>> > Hi,
>> >
>> >  Added KIP-401 to the wiki page for tracking.
>> >
>> > Thanks,
>> >
>> > On Fri, Sep 20, 2019 at 7:55 AM Paul Whalen  wrote:
>> >
>> > > Manikumar,
>> > >
>> > > KIP-401 was accepted a few weeks ago and there is a PR pending review,
>> > can
>> > > it be included in the release as well?
>> > >
>> > > Thanks,
>> > > Paul
>> > >
>> > > On Mon, Sep 16, 2019 at 6:14 AM Manikumar 
>> > > wrote:
>> > >
>> > > > Hi All,
>> > > >
>> > > > Just a reminder that any new/pending KIP must pass vote by next
>> > Wednesday
>> > > > (Sep 25, 2019) to be included
>> > > > in Apache Kafka 2.4.0 release.
>> > > >
>> > > > Also keep in mind that deadline for feature freeze is Oct 2, 2019.
>> > > > In order to be included in the release, major features/KIPs need to
>> be
>> > > > merged and minor features need to be
>> > > > have PR ready.  Any feature/KIP not in this state will be
>> automatically
>> > > > moved to the next release after Oct 2.
>> > > >
>> > > > There are total 84 open
>> > > > <
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%202.4.0%20AND%20status%20not%20in%20(resolved%2C%20closed)%20ORDER%20BY%20priority%20DESC%2C%20status%20DESC%2C%20updated%20DESC%20%20%20%20%20%20%20%20
>> > > > >
>> > > > JIRAs. Please update your assigned JIRAs, if you know they cannot
>> make
>> > it
>> > > > to 2.4.0.
>> > > > There are also quite a few JIRAs related to flaky tests. We really
>> > > > appreciate any help on fixing these failing tests.
>> > > >
>> > > > Thanks,
>> > > > Manikumar
>> > > >
>> > > > On Mon, Sep 16, 2019 at 4:08 PM Manikumar <
>> manikumar.re...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi Mickael,
>> > > > >
>> > > > > Yes, we can include. Added KIP-396 to the wiki page for tracking.
>> > > > >
>> > > > >
>> > > > > Thanks,
>> > > > > Manikumar
>> > > > >
>> > > > > On Mon, Sep 16, 2019 at 3:36 PM Mickael Maison <
>> > > mickael.mai...@gmail.com
>> > > > >
>> > > > > wrote:
>> > > > >
>> > > > >> Hi Manikumar,
>> > > > >>
>> > > > >> Can we also include KIP-396?
>> > > > >> (
>> > > > >>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97551484
>> > > > >> )
>> > > > >> It has been accepted and the PR is ready for review:
>> > > > >> https://github.com/apache/kafka/pull/7296
>> > > > >>
>> > > > >> Thanks
>> > > > >>
>> > > > >> On Mon, Sep 16, 2019 at 10:56 AM Manikumar <
>> > manikumar.re...@gmail.com
>> > > >
>> > > > >> wrote:
>> > > > >> >
>> > > > >> > Hi Viktor,
>> > > > >> >
>> > > > >> > Yes, we can include KIP-434.
>> > > > >> >
>> > > > >> > Thanks,
>> > > > >> >
>> > > > >> > On Mon, Sep 16, 2019 at 3:09 PM Viktor Somogyi-Vass <
>> > > > >> viktorsomo...@gmail.com>
>> > > > >> > wrote:
>> > > > >> >
>> > > > >> > > Hi Manikumar,
>> > > > >> > >
>> > > > >> > > Can we please also include KIP-434?
>> > > > >> > >
>> > > > >> > >
>> > > > >>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-434%3A+Add+Replica+Fetcher+and+Log+Cleaner+Count+Metrics
>> > > > >> > > It has been accepted and there is already a pull request
>> under
>> > > > review.
>> > > > >> > >
>> > > > >> > > Thanks,
>> > > > >> > > Viktor
>> > > > >> > >
>> > > > >> > > On Fri, Sep 6, 2019 at 9:59 AM Manikumar <
>> > > manikumar.re...@gmail.com
>> > > > >
>> > > > >> > > wrote:
>> > > > >> > >
>> > > > >> > > > Hi David,
>> > > > >> > > >
>> > > > >> > > > Yes, we can include KIP-511.  KIP must be accepted by KIP
>> > Freeze
>> > > > >> date
>> > > > >> > > (Sep
>> > > > >> > > > 25, 2019 )
>> > > > >> > > >
>> > > > >> > > > Thanks,
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > > On Fri, Sep 6, 2019 at 12:53 PM David Jacot <
>> > > dja...@confluent.io>
>> > > > >> wrote:
>> > > > >> > > >
>> > > > >> > > > > Hi Manikumar,
>> > > > >> > > > >
>> > > > >> > > > > Could we add KIP-511 to the plan? I think it will make
>> it.
>> > > > >> > > > >
>> > > > >> > > > > Thanks,
>> > > > >> > > > > David
>> > > > >> > > > >
>> > > > >> > > > > On Tue, Aug 27, 2019 at 5:32 PM Manikumar <
>> > > > >> manikumar.re...@gmail.com>
>> > > > >> > > > > wrote:
>> > > > >> > > > >
>> > > > >> > > > > > Hi all,
>> > > > >> > > > > >
>> > > > >> > > > > > I put together a draft release plan with Oct 2019 as
>> the
>> > > > release
>> > > > >> > > month
>> 

[jira] [Resolved] (KAFKA-6958) Allow to define custom processor names with KStreams DSL

[Bill Bejeck (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bill Bejeck resolved KAFKA-6958.

Resolution: Fixed

Thanks again [~fhussonnois] for your hard work and persistence in seeing this 
valuable contribution through to completion!

> Allow to define custom processor names with KStreams DSL
> 
>
> Key: KAFKA-6958
> URL: https://issues.apache.org/jira/browse/KAFKA-6958
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 1.1.0
>Reporter: Florian Hussonnois
>Assignee: Florian Hussonnois
>Priority: Minor
>  Labels: kip
> Fix For: 2.4.0
>
>
> Currently, while building a new Topology through the KStreams DSL the 
> processors are automatically named.
> The genarated names are prefixed depending of the operation (i.e 
> KSTREAM-SOURCE, KSTREAM-FILTER, KSTREAM-MAP, etc).
> To debug/understand a topology it is possible to display the processor 
> lineage with the method Topology#describe(). However, a complex topology with 
> dozens of operations can be hard to understand if the processor names are not 
> relevant.
> It would be useful to be able to set more meaningful names. For example, a 
> processor name could describe the business rule performed by a map() 
> operation.
> [KIP-307|https://cwiki.apache.org/confluence/display/KAFKA/KIP-307%3A+Allow+to+define+custom+processor+names+with+KStreams+DSL]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (KAFKA-8934) Introduce Instance-level Metrics

[Bruno Cadonna (Jira)]

Bruno Cadonna created KAFKA-8934:


 Summary: Introduce Instance-level Metrics
 Key: KAFKA-8934
 URL: https://issues.apache.org/jira/browse/KAFKA-8934
 Project: Kafka
  Issue Type: Improvement
  Components: streams
Reporter: Bruno Cadonna


Introduce instance-level metrics as proposed in KIP-444.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] KIP-507: Securing Internal Connect REST Endpoints

[Randall Hauch]

Thanks, Chris!

+1 (binding)

On Thu, Sep 19, 2019 at 7:43 PM Chris Egerton  wrote:

> Hi all,
>
> I'd like to begin voting on KIP-507:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-507%3A+Securing+Internal+Connect+REST+Endpoints
>
> Thanks to Ryanne, Magesh, Konstantine, Greg, and Randall for the fruitful
> discussion!
>
> Cheers,
>
> Chris
>

Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

[aishwarya kumar]

Thank you for the suggestion Matthais, i've made the necessary changes in
the KIP.

Keeping this thread open for further input.
KIP link:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL

Best,
Aishwarya

On Thu, Sep 19, 2019 at 10:50 AM aishwarya kumar  wrote:

> Thanks Matthias,
>
> That does make sense, let me update the KIP to reflect the Materialization
> scenario.
>
> Best,
> Aishwarya
>
> On Tue, Sep 17, 2019, 2:49 PM Matthias J. Sax 
> wrote:
>
>> Aishwarya,
>>
>> thanks for the KIP. Overall, I think it makes sense to allow converting
>> a KStream into a KTable.
>>
>> From the KIP:
>>
>> > materializing these KTables should only be allowed if the overloaded
>> function with Materialized is used (and if optimization is turned on it may
>> still be only logically materialized if the queryable name is not set).
>>
>> Can you elaborate? I think the behavior we want should align with the
>> behavior of `StreamsBuilder#table()`.
>>
>> From my understanding (correct me if I am wrong) it should be:
>>
>> (1) If optimization is turned off, the KTable will always be
>> materialized, independent which method is used. The KTable will not be
>> queryable though.
>>
>> (2) If optimization is turned on and if `toTable()` is used, the KTable
>> may or may not be materialized. For this case, even if the KTable is
>> materialized, the store would not be queryable.
>>
>> (3) If `toTable(Materialized)` is use and a `storeName` or
>> `StoreSupplier` is specified, the store will always be materialized and
>> also be queryable. Otherwise, case (1) or (2) applies.
>>
>>
>>
>> -Matthias
>>
>>
>> On 9/17/19 6:42 AM, aishwarya kumar wrote:
>> > Hi All,
>> >
>> > Keeping this thread alive!!
>> >
>> > The aim is to add two methods Kstream.toTable() &
>> > Kstream.toTable(Materialized), so users can choose to convert their
>> > event stream into a changelog stream at any stage.
>> > wiki link :
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
>> > jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
>> >
>> > Best,
>> > Aishwarya
>> >
>> > On Fri, Sep 13, 2019 at 10:49 AM aishwarya kumar 
>> wrote:
>> >
>> >> Hello,
>> >>
>> >> Starting this thread to discuss KIP-532:
>> >> wiki link :
>> >>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
>> >> jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
>> >>
>> >> There has been some discussion around the use-case of this KIP in the
>> Jira
>> >> ticket.
>> >>
>> >> Regards,
>> >> Aishwarya
>> >>
>> >
>>
>>

Re: [DISCUSS] KIP-527: Add NothingSerde to Serdes

[Nikolay Izhikov]

Hello,

KIP [1] updated to VoidSerde.

[1] 
https://cwiki.apache.org/confluence/display/KAFKA/KIP-527%3A+Add+VoidSerde+to+Serdes


В Вт, 24/09/2019 в 09:11 -0400, Andrew Otto пишет:
> Ah it is!  +1 to VoidSerde
> 
> On Mon, Sep 23, 2019 at 11:25 PM Matthias J. Sax 
> wrote:
> 
> > Because the actually data type is `Void`, I am wondering if `VoidSerde`
> > might be a more descriptive name?
> > 
> > -Matthias
> > 
> > On 9/23/19 12:25 PM, Nikolay Izhikov wrote:
> > > Hello, guys
> > > 
> > > Any additional feeback on this KIP?
> > > Should I start a vote?
> > > 
> > > В Пт, 20/09/2019 в 08:52 +0300, Nikolay Izhikov пишет:
> > > > Hello, Andrew.
> > > > 
> > > > OK, if nobody mind, let's change it to Null.
> > > > 
> > > > В Чт, 19/09/2019 в 13:54 -0400, Andrew Otto пишет:
> > > > > NullSerdes seems more descriptive, but up to you!  :)
> > > > > 
> > > > > On Thu, Sep 19, 2019 at 1:37 PM Nikolay Izhikov 
> > 
> > wrote:
> > > > > 
> > > > > > Hello, Andrew.
> > > > > > 
> > > > > > Seems, usage null or nothing is matter of taste. I dont mind if we
> > 
> > call it
> > > > > > NullSerde
> > > > > > 
> > > > > > чт, 19 сент. 2019 г., 20:28 Andrew Otto :
> > > > > > 
> > > > > > > Why 'NothingSerdes' instead of 'NullSerdes'?
> > > > > > > 
> > > > > > > On Thu, Sep 19, 2019 at 1:10 PM Nikolay Izhikov 
> > > > > > >  > > > > > > wrote:
> > > > > > > 
> > > > > > > > All,
> > > > > > > > 
> > > > > > > > I'd like to start a discussion for adding a NothingSerde to 
> > > > > > > > Serdes.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > 
> > > > > > 
> > > > > > 
> > 
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-527%3A+Add+NothingSerde+to+Serdes
> > > > > > > > 
> > > > > > > > Your comments and suggestions are welcome.
> > > > > > > > 
> > 
> > 


signature.asc
Description: This is a digitally signed message part

Re: [DISCUSS] Apache Kafka 2.4.0 release

[Manikumar]

Hi Rajini,

Added KIP-525 to the release plan for tracking.

Thanks,


On Tue, Sep 24, 2019 at 4:36 PM Rajini Sivaram 
wrote:

> Hi Manikumar,
>
> Can we add KIP-525 (
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response
> )
> to 2.4.0 release? It has been accepted and is a minor KIP.
>
> Thank you,
>
> Rajini
>
> On Fri, Sep 20, 2019 at 6:14 AM Manikumar 
> wrote:
>
> > Hi,
> >
> >  Added KIP-401 to the wiki page for tracking.
> >
> > Thanks,
> >
> > On Fri, Sep 20, 2019 at 7:55 AM Paul Whalen  wrote:
> >
> > > Manikumar,
> > >
> > > KIP-401 was accepted a few weeks ago and there is a PR pending review,
> > can
> > > it be included in the release as well?
> > >
> > > Thanks,
> > > Paul
> > >
> > > On Mon, Sep 16, 2019 at 6:14 AM Manikumar 
> > > wrote:
> > >
> > > > Hi All,
> > > >
> > > > Just a reminder that any new/pending KIP must pass vote by next
> > Wednesday
> > > > (Sep 25, 2019) to be included
> > > > in Apache Kafka 2.4.0 release.
> > > >
> > > > Also keep in mind that deadline for feature freeze is Oct 2, 2019.
> > > > In order to be included in the release, major features/KIPs need to
> be
> > > > merged and minor features need to be
> > > > have PR ready.  Any feature/KIP not in this state will be
> automatically
> > > > moved to the next release after Oct 2.
> > > >
> > > > There are total 84 open
> > > > <
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%202.4.0%20AND%20status%20not%20in%20(resolved%2C%20closed)%20ORDER%20BY%20priority%20DESC%2C%20status%20DESC%2C%20updated%20DESC%20%20%20%20%20%20%20%20
> > > > >
> > > > JIRAs. Please update your assigned JIRAs, if you know they cannot
> make
> > it
> > > > to 2.4.0.
> > > > There are also quite a few JIRAs related to flaky tests. We really
> > > > appreciate any help on fixing these failing tests.
> > > >
> > > > Thanks,
> > > > Manikumar
> > > >
> > > > On Mon, Sep 16, 2019 at 4:08 PM Manikumar  >
> > > > wrote:
> > > >
> > > > > Hi Mickael,
> > > > >
> > > > > Yes, we can include. Added KIP-396 to the wiki page for tracking.
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Manikumar
> > > > >
> > > > > On Mon, Sep 16, 2019 at 3:36 PM Mickael Maison <
> > > mickael.mai...@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Manikumar,
> > > > >>
> > > > >> Can we also include KIP-396?
> > > > >> (
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97551484
> > > > >> )
> > > > >> It has been accepted and the PR is ready for review:
> > > > >> https://github.com/apache/kafka/pull/7296
> > > > >>
> > > > >> Thanks
> > > > >>
> > > > >> On Mon, Sep 16, 2019 at 10:56 AM Manikumar <
> > manikumar.re...@gmail.com
> > > >
> > > > >> wrote:
> > > > >> >
> > > > >> > Hi Viktor,
> > > > >> >
> > > > >> > Yes, we can include KIP-434.
> > > > >> >
> > > > >> > Thanks,
> > > > >> >
> > > > >> > On Mon, Sep 16, 2019 at 3:09 PM Viktor Somogyi-Vass <
> > > > >> viktorsomo...@gmail.com>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Hi Manikumar,
> > > > >> > >
> > > > >> > > Can we please also include KIP-434?
> > > > >> > >
> > > > >> > >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-434%3A+Add+Replica+Fetcher+and+Log+Cleaner+Count+Metrics
> > > > >> > > It has been accepted and there is already a pull request under
> > > > review.
> > > > >> > >
> > > > >> > > Thanks,
> > > > >> > > Viktor
> > > > >> > >
> > > > >> > > On Fri, Sep 6, 2019 at 9:59 AM Manikumar <
> > > manikumar.re...@gmail.com
> > > > >
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hi David,
> > > > >> > > >
> > > > >> > > > Yes, we can include KIP-511.  KIP must be accepted by KIP
> > Freeze
> > > > >> date
> > > > >> > > (Sep
> > > > >> > > > 25, 2019 )
> > > > >> > > >
> > > > >> > > > Thanks,
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Fri, Sep 6, 2019 at 12:53 PM David Jacot <
> > > dja...@confluent.io>
> > > > >> wrote:
> > > > >> > > >
> > > > >> > > > > Hi Manikumar,
> > > > >> > > > >
> > > > >> > > > > Could we add KIP-511 to the plan? I think it will make it.
> > > > >> > > > >
> > > > >> > > > > Thanks,
> > > > >> > > > > David
> > > > >> > > > >
> > > > >> > > > > On Tue, Aug 27, 2019 at 5:32 PM Manikumar <
> > > > >> manikumar.re...@gmail.com>
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Hi all,
> > > > >> > > > > >
> > > > >> > > > > > I put together a draft release plan with Oct 2019 as the
> > > > release
> > > > >> > > month
> > > > >> > > > > and
> > > > >> > > > > > a list of KIPs that have already been voted:
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=125307901
> > > > >> > > > > >
> > > > >> > > > > > Here are the dates:
> > > > >> > > > > >
> > > > >> > > > > > 

Build failed in Jenkins: kafka-trunk-jdk11 #830

[Apache Jenkins Server]

See 

--
[...truncated 2.69 MB...]

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullTableOnJoinWithGlobalTable STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullTableOnJoinWithGlobalTable PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullOtherStreamOnJoin STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullOtherStreamOnJoin PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullTopicChooserOnTo STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullTopicChooserOnTo PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullTransformerSupplierOnTransform STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullTransformerSupplierOnTransform PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldThrowNullPointerOnPrintIfPrintedIsNull STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldThrowNullPointerOnPrintIfPrintedIsNull PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullTableOnJLeftJoinWithGlobalTable STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullTableOnJLeftJoinWithGlobalTable PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldMergeTwoStreams STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldMergeTwoStreams PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldSendDataToDynamicTopics STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldSendDataToDynamicTopics PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldSendDataToTopicUsingProduced STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldSendDataToTopicUsingProduced PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullMapperOnFlatMap STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullMapperOnFlatMap PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldProcessFromSourceThatMatchPattern STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldProcessFromSourceThatMatchPattern PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldCantHaveNullPredicate STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldCantHaveNullPredicate PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldPreserveSerdesForOperators STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldPreserveSerdesForOperators PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullMapperOnJoinWithGlobalTable STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullMapperOnJoinWithGlobalTable PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullActionOnForEach STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullActionOnForEach PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullValueMapperOnTableJoin STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullValueMapperOnTableJoin PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullMapperOnLeftJoinWithGlobalTable STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullMapperOnLeftJoinWithGlobalTable PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldUseRecordMetadataTimestampExtractorWithThrough STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldUseRecordMetadataTimestampExtractorWithThrough PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullTransformerSupplierOnFlatTransform STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullTransformerSupplierOnFlatTransform PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullPredicateOnFilterNot STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldNotAllowNullPredicateOnFilterNot PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldHaveAtLeastOnPredicateWhenBranching STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldHaveAtLeastOnPredicateWhenBranching PASSED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 
shouldThrowNullPointerOnJoinWithStreamWhenJoinedIsNull STARTED

org.apache.kafka.streams.kstream.internals.KStreamImplTest > 

[jira] [Created] (KAFKA-8936) Connect metrics have a chance to disappear on rebalance

[Steven Blumenthal (Jira)]

Steven Blumenthal created KAFKA-8936:


 Summary: Connect metrics have a chance to disappear on rebalance
 Key: KAFKA-8936
 URL: https://issues.apache.org/jira/browse/KAFKA-8936
 Project: Kafka
  Issue Type: Bug
  Components: KafkaConnect, metrics
Affects Versions: 2.0.0
Reporter: Steven Blumenthal


We encountered an interesting problem with our connect cluster. At times, 
seemingly randomly, some connect sink task metrics would randomly disappear 
from Datadog (which is where we are sending these metrics to). After some 
investigation, I noticed that the metrics in question weren't being reported by 
the connect servers themselves.

After some more investigation, I noticed that the metrics stopped reporting 
after a rebalance was triggered. Our logs were filled with "Graceful stop of 
task ... failed". So, further digging to understand what was happening in the 
code when this happens, it appears that this error means that the stopping of 
tasks timed out for whatever reason, and the connect cluster will no longer 
wait for them to stop. They will still stop eventually, but in the meantime new 
tasks can be spun up. 
([Worker.java|[https://github.com/apache/kafka/blob/2.0.0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L587]],
 which calls 
[WorkerTask.java:cancel()|[https://github.com/apache/kafka/blob/2.0.0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerTask.java#L120]])

So, new tasks are being spun up, and begin consuming records and doing work. 
Then, at some point, the old task is removed, and the very last thing that 
happens when the old task is removed is that the metric group associated with 
that task is removed. 
([WorkerTask.java|[https://github.com/apache/kafka/blob/2.0.0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerTask.java#L232]]
 which, in this case, calls 
[WorkerSinkTask.java|[https://github.com/apache/kafka/blob/2.0.0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L179]])

The issue with this is that task based metrics are registered based on a set of 
tags that one would expect to not change during runtime. Meaning that, when the 
old task IS EVENTUALLY REMOVED, it is removing the metric group that the new 
task is using (if the new task came up on the same connect node that the old 
task was running on). 
([WorkerSinkTask.java|[https://github.com/apache/kafka/blob/2.0.0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L721]])

I tried increasing the "task.shutdown.graceful.timeout.ms" config by 3 times 
what it had previously been set to, however that did not completely remove the 
problem. Also, even if it did, it doesn't change the fact that a minor network 
blip on my connect cluster could result in us needing to redeploy the code 
simply because metrics went missing due to task shut downs taking longer than 
intended.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] KIP-527: Add NothingSerde to Serdes

[Andrew Otto]

Ah it is!  +1 to VoidSerde

On Mon, Sep 23, 2019 at 11:25 PM Matthias J. Sax 
wrote:

> Because the actually data type is `Void`, I am wondering if `VoidSerde`
> might be a more descriptive name?
>
> -Matthias
>
> On 9/23/19 12:25 PM, Nikolay Izhikov wrote:
> > Hello, guys
> >
> > Any additional feeback on this KIP?
> > Should I start a vote?
> >
> > В Пт, 20/09/2019 в 08:52 +0300, Nikolay Izhikov пишет:
> >> Hello, Andrew.
> >>
> >> OK, if nobody mind, let's change it to Null.
> >>
> >> В Чт, 19/09/2019 в 13:54 -0400, Andrew Otto пишет:
> >>> NullSerdes seems more descriptive, but up to you!  :)
> >>>
> >>> On Thu, Sep 19, 2019 at 1:37 PM Nikolay Izhikov 
> wrote:
> >>>
>  Hello, Andrew.
> 
>  Seems, usage null or nothing is matter of taste. I dont mind if we
> call it
>  NullSerde
> 
>  чт, 19 сент. 2019 г., 20:28 Andrew Otto :
> 
> > Why 'NothingSerdes' instead of 'NullSerdes'?
> >
> > On Thu, Sep 19, 2019 at 1:10 PM Nikolay Izhikov  >
> > wrote:
> >
> >> All,
> >>
> >> I'd like to start a discussion for adding a NothingSerde to Serdes.
> >>
> >>
> >>
> 
> 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-527%3A+Add+NothingSerde+to+Serdes
> >>
> >> Your comments and suggestions are welcome.
> >>
>
>

Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

[John Roesler]

Hey Aishwarya,

Thanks for the KIP! It looks good to me, although in a post-KIP-307
world, we also need a "Named" parameter (to give the processor node a
name, as opposed to the store itself).

This would result in a total of four overloads:
1. no args
2. Named
3. Materialized
4. Materialized, Named

I'd like to propose a re-design of the DSL in the future to clean this
up, but for now, this is the pattern we have to follow.

Thoughts?

Thanks,
-John

On Tue, Sep 24, 2019 at 9:54 AM aishwarya kumar  wrote:
>
> Thank you for the suggestion Matthais, i've made the necessary changes in
> the KIP.
>
> Keeping this thread open for further input.
> KIP link:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL
>
> Best,
> Aishwarya
>
> On Thu, Sep 19, 2019 at 10:50 AM aishwarya kumar  wrote:
>
> > Thanks Matthias,
> >
> > That does make sense, let me update the KIP to reflect the Materialization
> > scenario.
> >
> > Best,
> > Aishwarya
> >
> > On Tue, Sep 17, 2019, 2:49 PM Matthias J. Sax 
> > wrote:
> >
> >> Aishwarya,
> >>
> >> thanks for the KIP. Overall, I think it makes sense to allow converting
> >> a KStream into a KTable.
> >>
> >> From the KIP:
> >>
> >> > materializing these KTables should only be allowed if the overloaded
> >> function with Materialized is used (and if optimization is turned on it may
> >> still be only logically materialized if the queryable name is not set).
> >>
> >> Can you elaborate? I think the behavior we want should align with the
> >> behavior of `StreamsBuilder#table()`.
> >>
> >> From my understanding (correct me if I am wrong) it should be:
> >>
> >> (1) If optimization is turned off, the KTable will always be
> >> materialized, independent which method is used. The KTable will not be
> >> queryable though.
> >>
> >> (2) If optimization is turned on and if `toTable()` is used, the KTable
> >> may or may not be materialized. For this case, even if the KTable is
> >> materialized, the store would not be queryable.
> >>
> >> (3) If `toTable(Materialized)` is use and a `storeName` or
> >> `StoreSupplier` is specified, the store will always be materialized and
> >> also be queryable. Otherwise, case (1) or (2) applies.
> >>
> >>
> >>
> >> -Matthias
> >>
> >>
> >> On 9/17/19 6:42 AM, aishwarya kumar wrote:
> >> > Hi All,
> >> >
> >> > Keeping this thread alive!!
> >> >
> >> > The aim is to add two methods Kstream.toTable() &
> >> > Kstream.toTable(Materialized), so users can choose to convert their
> >> > event stream into a changelog stream at any stage.
> >> > wiki link :
> >> >
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
> >> > jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
> >> >
> >> > Best,
> >> > Aishwarya
> >> >
> >> > On Fri, Sep 13, 2019 at 10:49 AM aishwarya kumar 
> >> wrote:
> >> >
> >> >> Hello,
> >> >>
> >> >> Starting this thread to discuss KIP-532:
> >> >> wiki link :
> >> >>
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL
> >> >> jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658
> >> >>
> >> >> There has been some discussion around the use-case of this KIP in the
> >> Jira
> >> >> ticket.
> >> >>
> >> >> Regards,
> >> >> Aishwarya
> >> >>
> >> >
> >>
> >>

Re: [VOTE] KIP-517: Add consumer metrics to observe user poll behavior

[Kevin Lu]

Hi All,

It has been a few days so I am closing the vote now.

This KIP has passed with +4 binding votes (Harsha, Manikumar, Jason, Bill)
and +3 non-binding votes (Mickael, Maulin, Tom).

Regards,
Kevin

On Fri, Sep 20, 2019 at 1:37 PM Kevin Lu  wrote:

> Hi All,
>
> Thanks for the votes!
>
> I have to point out that Jason brought up a good point about making the "
> *poll-idle-ratio*" measurement consistent with existing broker idle
> metrics so we made a small change.
>
> Previously it was calculated as:
>
>> poll-idle-ratio = time-between-poll / time-inside-poll
>
>
> We have changed it to:
>
>> poll-idle-ratio = time-inside-poll / total-time
>
>
> I will leave the vote open for a few more days in case people have
> concerns over this change. If there are no issues then, I will close the
> vote as we have enough votes.
>
> Regards,
> Kevin
>
> On Fri, Sep 20, 2019 at 1:33 PM Bill Bejeck  wrote:
>
>> +1 (binding)
>>
>> Thanks for the KIP!
>>
>> -Bill
>>
>> On Fri, Sep 20, 2019 at 1:28 PM Jason Gustafson 
>> wrote:
>>
>> > +1 Thanks!
>> >
>> > On Thu, Sep 19, 2019 at 11:22 PM Tom Bentley 
>> wrote:
>> >
>> > > +1 (non-binding).
>> > >
>> > > On Fri, Sep 20, 2019 at 7:00 AM Maulin Vasavada <
>> > maulin.vasav...@gmail.com
>> > > >
>> > > wrote:
>> > >
>> > > > +1 (non-binding). Thanks for the KIP.
>> > > >
>> > > > On Thu, Sep 19, 2019 at 10:38 PM Manikumar <
>> manikumar.re...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > +1 (binding), Thanks for the KIP.
>> > > > >
>> > > > > On Fri, Sep 20, 2019 at 12:41 AM Harsha Chintalapani <
>> > ka...@harsha.io>
>> > > > > wrote:
>> > > > >
>> > > > > > +1 (binding). Thanks for the KIP.
>> > > > > >
>> > > > > > -Harsha
>> > > > > >
>> > > > > > On Wed, Sep 18, 2019 at 9:07 AM Mickael Maison <
>> > > > mickael.mai...@gmail.com
>> > > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > +1 (non binding)
>> > > > > > > Thanks for the KIP, I can see this being really useful!
>> > > > > > >
>> > > > > > > On Wed, Sep 18, 2019 at 4:40 PM Kevin Lu <
>> lu.ke...@berkeley.edu>
>> > > > > wrote:
>> > > > > > > >
>> > > > > > > > Hi All,
>> > > > > > > >
>> > > > > > > > Since we have a bit of support, I'd like to start the vote
>> on
>> > > > > KIP-517:
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-517%3A+Add+consumer+metrics+to+observe+user+poll+behavior
>> > > > > > > >
>> > > > > > > > Thanks!
>> > > > > > > >
>> > > > > > > > Regards,
>> > > > > > > > Kevin
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [VOTE] KIP-507: Securing Internal Connect REST Endpoints

[Rajini Sivaram]

Thanks for the KIP, Chris!

+1 (binding)

Regards,

Rajini

On Tue, Sep 24, 2019 at 3:12 PM Randall Hauch  wrote:

> Thanks, Chris!
>
> +1 (binding)
>
> On Thu, Sep 19, 2019 at 7:43 PM Chris Egerton  wrote:
>
> > Hi all,
> >
> > I'd like to begin voting on KIP-507:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-507%3A+Securing+Internal+Connect+REST+Endpoints
> >
> > Thanks to Ryanne, Magesh, Konstantine, Greg, and Randall for the fruitful
> > discussion!
> >
> > Cheers,
> >
> > Chris
> >
>

Re: [DISCUSS] Apache Kafka 2.4.0 release

[Kevin Lu]

Hi Manikumar,

Please add KIP-517 as it is now accepted too
https://cwiki.apache.org/confluence/display/KAFKA/KIP-517%3A+Add+consumer+metrics+to+observe+user+poll+behavior

Thanks.

Regards,
Kevin

On Tue, Sep 24, 2019 at 7:25 AM Manikumar  wrote:

> Hi All,
>
> Just a reminder that tomorrow (Sep 25, 2019) is the last date for KIP
> acceptance for 2.4.0 release.
>
> Thanks,
>
> On Tue, Sep 24, 2019 at 7:05 PM Manikumar 
> wrote:
>
> > Hi Rajini,
> >
> > Added KIP-525 to the release plan for tracking.
> >
> > Thanks,
> >
> >
> > On Tue, Sep 24, 2019 at 4:36 PM Rajini Sivaram 
> > wrote:
> >
> >> Hi Manikumar,
> >>
> >> Can we add KIP-525 (
> >>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response
> >> )
> >> to 2.4.0 release? It has been accepted and is a minor KIP.
> >>
> >> Thank you,
> >>
> >> Rajini
> >>
> >> On Fri, Sep 20, 2019 at 6:14 AM Manikumar 
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> >  Added KIP-401 to the wiki page for tracking.
> >> >
> >> > Thanks,
> >> >
> >> > On Fri, Sep 20, 2019 at 7:55 AM Paul Whalen 
> wrote:
> >> >
> >> > > Manikumar,
> >> > >
> >> > > KIP-401 was accepted a few weeks ago and there is a PR pending
> review,
> >> > can
> >> > > it be included in the release as well?
> >> > >
> >> > > Thanks,
> >> > > Paul
> >> > >
> >> > > On Mon, Sep 16, 2019 at 6:14 AM Manikumar <
> manikumar.re...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Hi All,
> >> > > >
> >> > > > Just a reminder that any new/pending KIP must pass vote by next
> >> > Wednesday
> >> > > > (Sep 25, 2019) to be included
> >> > > > in Apache Kafka 2.4.0 release.
> >> > > >
> >> > > > Also keep in mind that deadline for feature freeze is Oct 2, 2019.
> >> > > > In order to be included in the release, major features/KIPs need
> to
> >> be
> >> > > > merged and minor features need to be
> >> > > > have PR ready.  Any feature/KIP not in this state will be
> >> automatically
> >> > > > moved to the next release after Oct 2.
> >> > > >
> >> > > > There are total 84 open
> >> > > > <
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%202.4.0%20AND%20status%20not%20in%20(resolved%2C%20closed)%20ORDER%20BY%20priority%20DESC%2C%20status%20DESC%2C%20updated%20DESC%20%20%20%20%20%20%20%20
> >> > > > >
> >> > > > JIRAs. Please update your assigned JIRAs, if you know they cannot
> >> make
> >> > it
> >> > > > to 2.4.0.
> >> > > > There are also quite a few JIRAs related to flaky tests. We really
> >> > > > appreciate any help on fixing these failing tests.
> >> > > >
> >> > > > Thanks,
> >> > > > Manikumar
> >> > > >
> >> > > > On Mon, Sep 16, 2019 at 4:08 PM Manikumar <
> >> manikumar.re...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Mickael,
> >> > > > >
> >> > > > > Yes, we can include. Added KIP-396 to the wiki page for
> tracking.
> >> > > > >
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Manikumar
> >> > > > >
> >> > > > > On Mon, Sep 16, 2019 at 3:36 PM Mickael Maison <
> >> > > mickael.mai...@gmail.com
> >> > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > >> Hi Manikumar,
> >> > > > >>
> >> > > > >> Can we also include KIP-396?
> >> > > > >> (
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97551484
> >> > > > >> )
> >> > > > >> It has been accepted and the PR is ready for review:
> >> > > > >> https://github.com/apache/kafka/pull/7296
> >> > > > >>
> >> > > > >> Thanks
> >> > > > >>
> >> > > > >> On Mon, Sep 16, 2019 at 10:56 AM Manikumar <
> >> > manikumar.re...@gmail.com
> >> > > >
> >> > > > >> wrote:
> >> > > > >> >
> >> > > > >> > Hi Viktor,
> >> > > > >> >
> >> > > > >> > Yes, we can include KIP-434.
> >> > > > >> >
> >> > > > >> > Thanks,
> >> > > > >> >
> >> > > > >> > On Mon, Sep 16, 2019 at 3:09 PM Viktor Somogyi-Vass <
> >> > > > >> viktorsomo...@gmail.com>
> >> > > > >> > wrote:
> >> > > > >> >
> >> > > > >> > > Hi Manikumar,
> >> > > > >> > >
> >> > > > >> > > Can we please also include KIP-434?
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-434%3A+Add+Replica+Fetcher+and+Log+Cleaner+Count+Metrics
> >> > > > >> > > It has been accepted and there is already a pull request
> >> under
> >> > > > review.
> >> > > > >> > >
> >> > > > >> > > Thanks,
> >> > > > >> > > Viktor
> >> > > > >> > >
> >> > > > >> > > On Fri, Sep 6, 2019 at 9:59 AM Manikumar <
> >> > > manikumar.re...@gmail.com
> >> > > > >
> >> > > > >> > > wrote:
> >> > > > >> > >
> >> > > > >> > > > Hi David,
> >> > > > >> > > >
> >> > > > >> > > > Yes, we can include KIP-511.  KIP must be accepted by KIP
> >> > Freeze
> >> > > > >> date
> >> > > > >> > > (Sep
> >> > > > >> > > > 25, 2019 )
> >> > > > >> > > >
> >> > > > >> > > > Thanks,
> >> > > > >> > > >
> >> > > > >> > > >
> >> > > > >> > > > On Fri, Sep 6, 2019 at 12:53 PM David 

Re: [VOTE] KIP-528: Deprecate PartitionGrouper configuration and interface

[Matthias J. Sax]

Thanks for voting. +1 (binding) for myself.

I am closing this vote. The KIP is accepted with

 4 binding (Gouzhang, Bill, Gwen, Matthias) and
 2 non-binding (John, Bruno) votes.


-Matthias


On 9/20/19 10:41 AM, Gwen Shapira wrote:
> +1 (binding)
> 
> On Thu, Sep 19, 2019 at 4:28 PM Matthias J. Sax  wrote:
>>
>> Hi,
>>
>> I would like to propose a small KIP to deprecate some public APIs that
>> are considered "dangerous" to use, with the goal to remove them in the
>> next major release.
>>
>> Because the KIP is straight forward and to make it into 2.4 release, I
>> call directly for a vote.
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-528%3A+Deprecate+PartitionGrouper+configuration+and+interface
>>
>>
>> -Matthias
>>



signature.asc
Description: OpenPGP digital signature

[VOTE] KIP-524: Allow users to choose config source when describing configs

[Jason Gustafson]

Hi All,

I'm starting a vote for KIP-524, which is a small change to the config
tool:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-524%3A+Allow+users+to+choose+config+source+when+describing+configs
.

+1 from me.

Thanks,
Jason

Build failed in Jenkins: kafka-2.2-jdk8 #167

[Apache Jenkins Server]

See 


Changes:

[github] HOTFIX: fix Kafka Streams upgrade note for broker backward 
compatibility

--
Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on H40 (ubuntu xenial) in workspace 

[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
No credentials specified
Cloning the remote Git repository
Cloning repository https://github.com/apache/kafka.git
 > git init  # timeout=10
Fetching upstream changes from https://github.com/apache/kafka.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/kafka.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # 
 > timeout=10
 > git config remote.origin.url https://github.com/apache/kafka.git # timeout=10
Fetching upstream changes from https://github.com/apache/kafka.git
 > git fetch --tags --progress https://github.com/apache/kafka.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/2.2^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/2.2^{commit} # timeout=10
Checking out Revision 67dfa1d8225f2468a5f69218531703a72a63b1df 
(refs/remotes/origin/2.2)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 67dfa1d8225f2468a5f69218531703a72a63b1df
Commit message: "HOTFIX: fix Kafka Streams upgrade note for broker backward 
compatibility (#7364)"
 > git rev-list --no-walk bf0e10e5a79b4652bf630c48d488065837170f01 # timeout=10
Setting GRADLE_4_8_1_HOME=/home/jenkins/tools/gradle/4.8.1
Setting GRADLE_4_8_1_HOME=/home/jenkins/tools/gradle/4.8.1
[kafka-2.2-jdk8] $ /bin/bash -xe /tmp/jenkins2830300533167398190.sh
+ rm -rf 
+ /home/jenkins/tools/gradle/4.8.1/bin/gradle
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# An error report file with more information is saved as:
# 
Build step 'Execute shell' marked build as failure
[FINDBUGS] Collecting findbugs analysis files...
Setting GRADLE_4_8_1_HOME=/home/jenkins/tools/gradle/4.8.1
[FINDBUGS] Searching for all files in 
 that match the pattern 
**/build/reports/findbugs/*.xml
[FINDBUGS] No files found. Configuration error?
Setting GRADLE_4_8_1_HOME=/home/jenkins/tools/gradle/4.8.1
No credentials specified
Setting GRADLE_4_8_1_HOME=/home/jenkins/tools/gradle/4.8.1
 Using GitBlamer to create author and commit information for all 
warnings.
 GIT_COMMIT=67dfa1d8225f2468a5f69218531703a72a63b1df, 
workspace=
[FINDBUGS] Computing warning deltas based on reference build #163
Recording test results
Setting GRADLE_4_8_1_HOME=/home/jenkins/tools/gradle/4.8.1
ERROR: Step ?Publish JUnit test result report? failed: No test report files 
were found. Configuration error?
Setting GRADLE_4_8_1_HOME=/home/jenkins/tools/gradle/4.8.1
Not sending mail to unregistered user wangg...@gmail.com
Not sending mail to unregistered user nore...@github.com

Re: [VOTE] KIP-507: Securing Internal Connect REST Endpoints

[Konstantine Karantasis]

Nicely done!

+1 (non-binding)

Konstantine

On Tue, Sep 24, 2019 at 9:10 AM Rajini Sivaram 
wrote:

> Thanks for the KIP, Chris!
>
> +1 (binding)
>
> Regards,
>
> Rajini
>
> On Tue, Sep 24, 2019 at 3:12 PM Randall Hauch  wrote:
>
> > Thanks, Chris!
> >
> > +1 (binding)
> >
> > On Thu, Sep 19, 2019 at 7:43 PM Chris Egerton 
> wrote:
> >
> > > Hi all,
> > >
> > > I'd like to begin voting on KIP-507:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-507%3A+Securing+Internal+Connect+REST+Endpoints
> > >
> > > Thanks to Ryanne, Magesh, Konstantine, Greg, and Randall for the
> fruitful
> > > discussion!
> > >
> > > Cheers,
> > >
> > > Chris
> > >
> >
>

[jira] [Created] (KAFKA-8940) Flaky SmokeTestDriverIntegrationTest.shouldWorkWithRebalance

[Guozhang Wang (Jira)]

Guozhang Wang created KAFKA-8940:


 Summary: Flaky 
SmokeTestDriverIntegrationTest.shouldWorkWithRebalance
 Key: KAFKA-8940
 URL: https://issues.apache.org/jira/browse/KAFKA-8940
 Project: Kafka
  Issue Type: Bug
  Components: streams, unit tests
Reporter: Guozhang Wang


I lost the screen shot unfortunately... it reports the set of expected records 
does not match the received records.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] KIP-521: Redirect Connect log4j messages also to a file by default

[Konstantine Karantasis]

Thanks everyone!

After 3+ business days since this thread started, I'm concluding the vote
on KIP-521.

The KIP has passed with:

3 binding votes from Gwen, Randall and Bill.
1 non-binding vote from Mitchell.

Thank you all for voting!

Best,
Konstantine


On Fri, Sep 20, 2019 at 5:50 PM Bill Bejeck  wrote:

> Nice job on the KIP Konstantine. This is a great improvement.
>
> +1 (binding)
>
> -Bill
>
> On Fri, Sep 20, 2019 at 5:07 PM Mitchell  wrote:
>
> > +1 (non-binding).
> >
> > Love this change.
> >
> > On Fri, Sep 20, 2019, 4:37 PM Randall Hauch  wrote:
> >
> > > +1 (binding)
> > >
> > > Great work, Konstantine.
> > >
> > > On Thu, Sep 19, 2019 at 11:50 AM Gwen Shapira 
> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > On Thu, Sep 19, 2019 at 8:53 AM Konstantine Karantasis
> > > >  wrote:
> > > > >
> > > > > I'd like to start the vote on KIP-521.
> > > > >
> > > > > The proposal seems straightforward and no major concerns came up
> > during
> > > > its
> > > > > recent brief discussions. I'd appreciate your votes and, of course,
> > > your
> > > > > comments are still welcome.
> > > > >
> > > > > Hopefully we could meet the forthcoming KIP deadline for this
> simple
> > > yet
> > > > > useful operational feature in Kafka Connect.
> > > > >
> > > > > Cheers,
> > > > > Konstantine
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-515: Reorganize checkpoint system in log cleaner to be per partition

[Jason Gustafson]

Hi Richard,

It would be unsatisfying to make a big change to the checkpointing logic in
order to handle only one case of this problem, right?

I did have one idea about how to do this. It's a bit of a hack, but keep an
open mind ;). The basic problem is having somewhere to embed the delete
horizon for each batch. In the v2 format, each batch header contains two
timestamps: the base timestamp and the max timestamp. Each record in the
batch contains a timestamp delta which is relative to the base timestamp.
In other words, to get the record timestamp, you add the record delta to
the base timestamp.

Typically there is no reason for the base timestamp to be different from
the timestamp of the first message, but this is not a strict requirement.
As long as you can get to the record timestamp by adding the base timestamp
and delta, then we are good. So the idea is to set the base timestamp to
the delete horizon and adjust the deltas accordingly. We could then use one
bit from the batch attributes to indicate when the base timestamp had been
set to the delete horizon. There would be no change to the batch max
timestamp, so indexing would not be affected by this change.

So the logic would look something like this when cleaning the log.

Case 1: Normal batch

a. If delete horizon flag is set, then retain tombstones as long as the
current time is before the horizon.
b. If no delete horizon is set, then retain tombstones and set the delete
horizon in the cleaned batch to current time +
log.cleaner.delete.retention.ms.

Case 2: Control batch

a. If delete horizon flag is set, then retain the batch and the marker
as long as the current time is before the horizon.
b. If no delete horizon is set and there are no records remaining from the
transaction, then retain the marker and set the delete horizon in the
cleaned batch to current time + log.cleaner.delete.retention.ms.

What do you think?

-Jason



On Thu, Sep 19, 2019 at 3:21 PM Richard Yu 
wrote:

> Hi Jason,
>
> That hadn't occurred to me.
>
> I think I missed your comment in the discussion, so I created this KIP only
> with resolving the problem regarding tombstones.
> Whats your thoughts? If the problem regarding transaction markers is a
> little too complex, then we can we just leave it out of the KIP and fix the
> tombstones issue.
>
> Cheers,
> Richard
>
> On Thu, Sep 19, 2019 at 8:47 AM Jason Gustafson 
> wrote:
>
> > Hi Richard,
> >
> > Just reposting my comment from the JIRA:
> >
> > The underlying problem here also impacts the cleaning of transaction
> > markers. We use the same delete horizon in order to tell when it is safe
> to
> > remove the marker. If all the data from a transaction has been cleaned
> and
> > the delete horizon has passed enough time, then the marker is eligible
> for
> > deletion.
> >
> > However, I don't think the same approach that we're proposing to fix the
> > problem for tombstones will work transaction markers. What we need to
> track
> > is the timestamp when all the records from a transaction have been
> removed.
> > That is when we start the timer for deletion. But this would be different
> > for every transaction and there is no guarantee that earlier transactions
> > will be eligible for deletion before later ones. It all depends on the
> keys
> > written in the transaction. I don't see an obvious way to solve this
> > problem without some record-level bookkeeping, but I might be missing
> > something.
> >
> > Thanks,
> > Jason
> >
> > On Mon, Sep 9, 2019 at 7:21 PM Richard Yu 
> > wrote:
> >
> > > Hi Jun,
> > >
> > > Thanks for chipping in. :)
> > >
> > > The description you provided is pretty apt in describing the motivation
> > of
> > > the KIP, so I will add it. I've made some changes to the KIP and
> outlined
> > > the basic approaches of what we have so far (basically changing the
> > > checkpoint file organization or incorporating an extra internal header
> > > field for a record). I will expand on them shortly.
> > >
> > > Any comments are appreciated!
> > >
> > > Cheers,
> > > Richard
> > >
> > > On Mon, Sep 9, 2019 at 3:10 PM Jun Rao  wrote:
> > >
> > > > Hi, Richard,
> > > >
> > > > Thanks for drafting the KIP. A few comments below.
> > > >
> > > > 1. We need to provide a better motivation for the KIP. The goal of
> the
> > > KIP
> > > > is not to reorganize the checkpoint for log cleaning. It's just an
> > > > implementation detail. I was thinking that we could add sth like the
> > > > following in the Motivation/Problem section.
> > > >
> > > > "The idea of the configuration delete.retention.ms for compacted
> > topics
> > > is
> > > > to prevent an application that has read a key to not see a subsequent
> > > > deletion of the key because it's physically removed too early. To
> solve
> > > > this problem, from the latest possible time (deleteHorizonMs) that an
> > > > application could have read a non tombstone key before a tombstone,
> we
> > > > preserve that tombstone for at least delete.retention.ms 

Re: [VOTE] KIP-479 Add Materialized to Join

[Bill Bejeck]

Adding a +1 (binding) from myself.

Thanks to everyone for voting.

I am closing this vote. The KIP is accepted with

 3 binding (Gouzhang, Bill, Matthias) and
 1 non-binding (John) votes.

Thanks,
Bill

On Thu, Sep 19, 2019 at 9:23 PM John Roesler  wrote:

> FWIW, I'm also +1 (non-binding).
>
> Thanks for tackling this, Bill.
> -John
>
> On Thu, Sep 19, 2019 at 3:09 PM Guozhang Wang  wrote:
> >
> > Thanks Bill for the update, I'm +1 as well (binding).
> >
> > On Thu, Sep 19, 2019 at 11:25 AM Bill Bejeck  wrote:
> >
> > > Thanks for the comments, Matthias.
> > >
> > > I don't have a strong preference, so given that Matthias is ok with
> > > "StreamJoined" and Guozhang seems to prefer "StreamJoined" I'll update
> the
> > > KIP accordingly.
> > >
> > > Thanks,
> > > Bill
> > >
> > >
> > >
> > > On Thu, Sep 19, 2019 at 11:04 AM Matthias J. Sax <
> matth...@confluent.io>
> > > wrote:
> > >
> > > > As I mentioned on the DISCUSS thread, it think either `StreamsJoined`
> > > > (plural) or `StreamJoin` are good names.
> > > >
> > > > But I am also ok with `StreamJoined` if anyone insist on it. I leave
> it
> > > > up to Bill to pick any of the three variant.
> > > >
> > > > +1 (binding)
> > > >
> > > > -Matthias
> > > >
> > > > On 9/19/19 9:40 AM, John Roesler wrote:
> > > > > I'm +1 either way :)
> > > > > -John
> > > > >
> > > > > On Wed, Sep 18, 2019 at 5:37 PM Bill Bejeck 
> wrote:
> > > > >>
> > > > >> Good catch!  I meant to propose the name to be "StreamJoin". I
> have
> > > > updated
> > > > >> the KIP accordingly.
> > > > >>
> > > > >> As for the name, I originally had "StreamJoined" and updated it
> after
> > > > some
> > > > >> comments on the KIP.
> > > > >> I do feel that the name "StreamJoin" is better in this case since
> it
> > > is
> > > > >> used to represent a stream join configuration vs. "StreamJoined"
> which
> > > > >> feels more like it's being used as a verb (past tense).
> > > > >>
> > > > >> WDYT?
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Wed, Sep 18, 2019 at 4:48 PM Guozhang Wang  >
> > > > wrote:
> > > > >>
> > > > >>> Hello Bill,
> > > > >>>
> > > > >>> The KIP's proposal has the code snippet name as "StreamJoined"
> but
> > > the
> > > > >>> class name defined is StreamJoin.Which one did you propose?
> > > Personally
> > > > I
> > > > >>> think StreamJoined with better aligned with other control
> objects,
> > > but
> > > > if
> > > > >>> you think otherwise is better I can be convinced too :)
> > > > >>>
> > > > >>>
> > > > >>> Guozhang
> > > > >>>
> > > > >>> On Wed, Sep 18, 2019 at 4:38 PM Bill Bejeck 
> > > wrote:
> > > > >>>
> > > >  All, since we have updated KIP-479
> > > > 
> > > > 
> > > > >>>
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-479%3A+Add+StreamJoin+config+object+to+Join
> > > >  and
> > > >  seem to have completed the discussion for the updates, I'd like
> to
> > > > call
> > > > >>> for
> > > >  everyone to vote again.
> > > > 
> > > >  Thanks,
> > > >  Bill
> > > > 
> > > >  On Fri, Aug 2, 2019 at 10:46 AM Bill Bejeck 
> > > > wrote:
> > > > 
> > > > > +1 (binding) from myself.
> > > > >
> > > > >
> > > > > This vote has been open for 7 days now. so I'm closing this
> vote
> > > > >>> thread.
> > > > >
> > > > > KIP-479 had the following votes:
> > > > >
> > > > > binding +1s: 3 (Guozhang, Matthias, and Bill)
> > > > > -1 votes: none
> > > > >
> > > > > Thanks to everyone who voted and participated in the
> discussion for
> > > > >>> this
> > > > > KIP!
> > > > >
> > > > > -Bill
> > > > >
> > > > > On Mon, Jul 29, 2019 at 6:03 PM Guozhang Wang <
> wangg...@gmail.com>
> > > >  wrote:
> > > > >
> > > > >> +1 (binding)
> > > > >>
> > > > >> On Thu, Jul 25, 2019 at 7:39 PM Matthias J. Sax <
> > > > >>> matth...@confluent.io>
> > > > >> wrote:
> > > > >>
> > > > >>> +1 (binding)
> > > > >>>
> > > > >>> On 7/25/19 1:05 PM, Bill Bejeck wrote:
> > > >  All,
> > > > 
> > > >  After a great discussion on KIP-479 (
> > > > 
> > > > >>>
> > > > >>
> > > > 
> > > > >>>
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-479%3A+Add+Materialized+to+Join
> > > > >>> )
> > > >  I'd
> > > >  like to start a vote.
> > > > 
> > > >  Thanks,
> > > >  Bill
> > > > 
> > > > >>>
> > > > >>>
> > > > >>
> > > > >> --
> > > > >> -- Guozhang
> > > > >>
> > > > >
> > > > 
> > > > >>>
> > > > >>>
> > > > >>> --
> > > > >>> -- Guozhang
> > > > >>>
> > > >
> > > >
> > >
> >
> >
> > --
> > -- Guozhang
>

[jira] [Created] (KAFKA-8939) Flaky test ReassignPartitionsClusterTest#shouldTriggerReassignmentWithZnodePrecedenceOnControllerStartup

[Sophie Blee-Goldman (Jira)]

Sophie Blee-Goldman created KAFKA-8939:
--

 Summary: Flaky test 
ReassignPartitionsClusterTest#shouldTriggerReassignmentWithZnodePrecedenceOnControllerStartup
 Key: KAFKA-8939
 URL: https://issues.apache.org/jira/browse/KAFKA-8939
 Project: Kafka
  Issue Type: Bug
  Components: admin, core
Affects Versions: 2.4.0
Reporter: Sophie Blee-Goldman


h3. Stacktrace

java.lang.AssertionError: expected: but was: at 
org.junit.Assert.fail(Assert.java:89) at 
org.junit.Assert.failNotEquals(Assert.java:835) at 
org.junit.Assert.assertEquals(Assert.java:120) at 
org.junit.Assert.assertEquals(Assert.java:146) at 
kafka.admin.ReassignPartitionsClusterTest.shouldTriggerReassignmentWithZnodePrecedenceOnControllerStartup(ReassignPartitionsClusterTest.scala:687)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method) at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.base/java.lang.reflect.Method.invoke(Method.java:566) at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
 at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
 at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305) at 
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:365) at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
 at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:330) at 
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:78) at 
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:328) at 
org.junit.runners.ParentRunner.access$100(ParentRunner.java:65) at 
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305) at 
org.junit.runners.ParentRunner.run(ParentRunner.java:412) at 
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
 at 
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
 at 
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
 at 
org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
 at 
org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
 at jdk.internal.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.base/java.lang.reflect.Method.invoke(Method.java:566) at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
 at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
 at 
org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
 at 
org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
 at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at 
org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
 at jdk.internal.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.base/java.lang.reflect.Method.invoke(Method.java:566) at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
 at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
 at 
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
 at 
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
 at 

[jira] [Created] (KAFKA-8937) Flaky Test SaslPlainSslEndToEndAuthorizationTest#testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl

[Sophie Blee-Goldman (Jira)]

Sophie Blee-Goldman created KAFKA-8937:
--

 Summary: Flaky Test 
SaslPlainSslEndToEndAuthorizationTest#testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl
 Key: KAFKA-8937
 URL: https://issues.apache.org/jira/browse/KAFKA-8937
 Project: Kafka
  Issue Type: Bug
  Components: core
Reporter: Sophie Blee-Goldman


h3. Error Message

org.scalatest.exceptions.TestFailedException: Consumed 0 records before timeout 
instead of the expected 1 records
h3. Stacktrace

org.scalatest.exceptions.TestFailedException: Consumed 0 records before timeout 
instead of the expected 1 records at 
org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:530)
 at 
org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1389) at 
org.scalatest.Assertions$class.fail(Assertions.scala:1091) at 
org.scalatest.Assertions$.fail(Assertions.scala:1389) at 
kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:830) at 
kafka.utils.TestUtils$.pollRecordsUntilTrue(TestUtils.scala:792) at 
kafka.utils.TestUtils$.pollUntilAtLeastNumRecords(TestUtils.scala:1324) at 
kafka.utils.TestUtils$.consumeRecords(TestUtils.scala:1333) at 
kafka.api.EndToEndAuthorizationTest.consumeRecords(EndToEndAuthorizationTest.scala:529)
 at 
kafka.api.EndToEndAuthorizationTest.testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl(EndToEndAuthorizationTest.scala:368)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498) at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
 at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
 at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305) at 
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:365) at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
 at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:330) at 
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:78) at 
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:328) at 
org.junit.runners.ParentRunner.access$100(ParentRunner.java:65) at 
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305) at 
org.junit.runners.ParentRunner.run(ParentRunner.java:412) at 
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
 at 
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
 at 
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
 at 
org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
 at 
org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
 at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498) at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
 at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
 at 
org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
 at 
org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
 at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at 
org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
 at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498) at 
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
 at 

[jira] [Reopened] (KAFKA-7965) Flaky Test ConsumerBounceTest#testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup

[Sophie Blee-Goldman (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-7965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sophie Blee-Goldman reopened KAFKA-7965:

  Assignee: (was: Jason Gustafson)

Saw another failure in PR builder (2.4)

> Flaky Test 
> ConsumerBounceTest#testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup
> 
>
> Key: KAFKA-7965
> URL: https://issues.apache.org/jira/browse/KAFKA-7965
> Project: Kafka
>  Issue Type: Bug
>  Components: clients, consumer, unit tests
>Affects Versions: 1.1.1, 2.2.0, 2.3.0
>Reporter: Matthias J. Sax
>Priority: Critical
>  Labels: flaky-test
> Fix For: 2.3.0
>
>
> To get stable nightly builds for `2.2` release, I create tickets for all 
> observed test failures.
> [https://jenkins.confluent.io/job/apache-kafka-test/job/2.2/21/]
> {quote}java.lang.AssertionError: Received 0, expected at least 68 at 
> org.junit.Assert.fail(Assert.java:88) at 
> org.junit.Assert.assertTrue(Assert.java:41) at 
> kafka.api.ConsumerBounceTest.receiveAndCommit(ConsumerBounceTest.scala:557) 
> at 
> kafka.api.ConsumerBounceTest.$anonfun$testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup$1(ConsumerBounceTest.scala:320)
>  at 
> kafka.api.ConsumerBounceTest.$anonfun$testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup$1$adapted(ConsumerBounceTest.scala:319)
>  at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) 
> at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) 
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) at 
> kafka.api.ConsumerBounceTest.testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup(ConsumerBounceTest.scala:319){quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (KAFKA-8938) Connect - Improve Memory Allocations During Struct Validation

[Auston McReynolds (Jira)]

Auston McReynolds created KAFKA-8938:


 Summary: Connect - Improve Memory Allocations During Struct 
Validation
 Key: KAFKA-8938
 URL: https://issues.apache.org/jira/browse/KAFKA-8938
 Project: Kafka
  Issue Type: Improvement
  Components: KafkaConnect
Reporter: Auston McReynolds


Summary: Struct value validation in Kafka Connect can be optimized to avoid 
creating an Iterator when the expectedClasses list is of size 1. This is a 
meaningful enhancement for high throughput connectors.

Stack Trace from the Couchbase Kafka Connector:
 * java.util.Collections.singletonIterator(Object)
 * java.util.Collections$SingletonList.iterator()
 * org.apache.kafka.connect.data.ConnectSchema.validateValue(String, Schema, 
Object)
 * org.apache.kafka.connect.data.Struct.put(Field, Object)
 * org.apache.kafka.connect.data.Struct.put(String, Object)
 * 
com.couchbase.connect.kafka.handler.source.DefaultSchemaSourceHandler.buildValue(SourceHandlerParams,
 CouchbaseSourceRecord$Builder)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] KIP-502: Connect SinkTask.put(...) to specify ArrayList in Signature

[Cyrus Vafadari]

Almog, I think that's a great point -- I will update the KIP to reflect
your suggestion!

On Tue, Sep 24, 2019 at 12:46 PM Almog Gavra  wrote:

> Thanks Cyrus! I think this change is a good step in hardening the API. I do
> believe that APIs should be defined by functionality and not performance
> characteristics, so I'd prefer using List<> over ArrayList<> (the
> alternative you mention in rejected). That also gives us leeway in the
> future to swap implementations without breaking compatibility (e.g. we want
> to pass an ImmutableList or Collections.unmodifiableList instead of
> ArrayList or discover some unknown more efficient implementation than
> ArrayList).
>
> Almog
>
> On Sat, Aug 3, 2019 at 7:01 PM Cyrus Vafadari  wrote:
>
> > Hi all,
> >
> > I've written a KIP to update the SinkTask abstract class to specify that
> > the `put` method will take ArrayList. I think this will greatly simplify
> > connector development, so you aren't limited to the simplest iterations.
> It
> > will also harden the ordering guarantees of the API.
> >
> > Looking forward to the feedback!
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-502%3A+Connect+SinkTask.put%28...%29+to+specify+ArrayList%3CSinkRecord%3E+in+Signature
> >
> > Cyrus
> >
>

[jira] [Resolved] (KAFKA-8880) Augment Consumer.committed(partition) to allow multiple partitions

[Guozhang Wang (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guozhang Wang resolved KAFKA-8880.
--
Fix Version/s: 2.4.0
 Assignee: Guozhang Wang
   Resolution: Fixed

> Augment Consumer.committed(partition) to allow multiple partitions
> --
>
> Key: KAFKA-8880
> URL: https://issues.apache.org/jira/browse/KAFKA-8880
> Project: Kafka
>  Issue Type: Improvement
>  Components: consumer
>Reporter: Guozhang Wang
>Assignee: Guozhang Wang
>Priority: Major
>  Labels: needs-kip, newbie++
> Fix For: 2.4.0
>
>
> We've observed that many usage of the consumer.committed calls are made for 
> not only one partition, but for a batch of partitions. On the other hand, the 
> OffsetFetchRequest protocol actually allows for multiple partitions within 
> one request.
> I'd propose we add an overloaded function of KafkaConsumer that takes 
> {code}
> Map committed(Collection 
> partitions, final Duration timeout)
> {code}
> And then deprecate the existing function that only takes on partition.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] KIP-507: Securing Internal Connect REST Endpoints

[Ewen Cheslack-Postava]

+1 (binding)

A couple of comments on the details:

- I think this might be the first time we're adding a new message type to
the config topic. Might be worth noting that (somewhat unfortunately) we
currently log unknown keys as an error. Kinda unclear whether it should be
an error or warn (since it could legitimately indicate bad data in the
config topic). During upgrade/downgrade this means we could log errors
(given all nodes will have had to upgrade first, I think probably only in
case of reverting).

- I think it would be worthwhile considering a more flexible, extensible
solution to the connect subprotocol. In particular, we're not even changing
the protocol, but are just using the version number to indicate feature
support. Another way of doing this which would still require a protocol
version bump but would allow us more flexibility moving forward *without*
bumping versions would be to introduce a field that does something like
KIP-35, where the field is a flexible list of "features" supported by the
client. We lose a bit of automatic resolution of the best protocol to use,
but it avoids having to bump versions unnecessarily. In this case, we'd
just include something like "internal_security" as a supported feature, the
leader would determine if everyone supports it, and if so make use of it.
The main reason I bring this up is that bumping versions for every small
semantic change gets expensive given the defaults we want to support. We
can eventually retire some of the older protocols on a major version bump,
but given this new proposal we'll be sending 3x the subprotocol data and
any further changes will only make that cost worse.

- There is a seemingly arbitrary 1 minute timeout for picking up renewed
session ids. It seems like this should probably be tied to some
corresponding group timeout, e.g. maybe the session.timeout.

-Ewen

On Tue, Sep 24, 2019 at 11:37 AM Konstantine Karantasis <
konstant...@confluent.io> wrote:

> Nicely done!
>
> +1 (non-binding)
>
> Konstantine
>
> On Tue, Sep 24, 2019 at 9:10 AM Rajini Sivaram 
> wrote:
>
> > Thanks for the KIP, Chris!
> >
> > +1 (binding)
> >
> > Regards,
> >
> > Rajini
> >
> > On Tue, Sep 24, 2019 at 3:12 PM Randall Hauch  wrote:
> >
> > > Thanks, Chris!
> > >
> > > +1 (binding)
> > >
> > > On Thu, Sep 19, 2019 at 7:43 PM Chris Egerton 
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'd like to begin voting on KIP-507:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-507%3A+Securing+Internal+Connect+REST+Endpoints
> > > >
> > > > Thanks to Ryanne, Magesh, Konstantine, Greg, and Randall for the
> > fruitful
> > > > discussion!
> > > >
> > > > Cheers,
> > > >
> > > > Chris
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-502: Connect SinkTask.put(...) to specify ArrayList in Signature

[Almog Gavra]

Thanks Cyrus! I think this change is a good step in hardening the API. I do
believe that APIs should be defined by functionality and not performance
characteristics, so I'd prefer using List<> over ArrayList<> (the
alternative you mention in rejected). That also gives us leeway in the
future to swap implementations without breaking compatibility (e.g. we want
to pass an ImmutableList or Collections.unmodifiableList instead of
ArrayList or discover some unknown more efficient implementation than
ArrayList).

Almog

On Sat, Aug 3, 2019 at 7:01 PM Cyrus Vafadari  wrote:

> Hi all,
>
> I've written a KIP to update the SinkTask abstract class to specify that
> the `put` method will take ArrayList. I think this will greatly simplify
> connector development, so you aren't limited to the simplest iterations. It
> will also harden the ordering guarantees of the API.
>
> Looking forward to the feedback!
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-502%3A+Connect+SinkTask.put%28...%29+to+specify+ArrayList%3CSinkRecord%3E+in+Signature
>
> Cyrus
>

Re: [VOTE] 2.3.1 RC0

[Jason Gustafson]

Hi David,

Thanks for running the release. I think we should consider getting this bug
fixed: https://issues.apache.org/jira/browse/KAFKA-8896. The impact of this
bug is that consumer groups cannot commit offsets or rebalance. The patch
should be ready shortly.

Thanks,
Jason



On Fri, Sep 13, 2019 at 3:53 PM David Arthur  wrote:

> Hello Kafka users, developers and client-developers,
>
>
> This is the first candidate for release of Apache Kafka 2.3.1 which
> includes many bug fixes for Apache Kafka 2.3.
>
>
> Release notes for the 2.3.1 release:
>
> https://home.apache.org/~davidarthur/kafka-2.3.1-rc0/RELEASE_NOTES.html
>
>
> *** Please download, test and vote by Wednesday, September 18, 9am PT
>
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
>
> https://kafka.apache.org/KEYS
>
>
> * Release artifacts to be voted upon (source and binary):
>
> https://home.apache.org/~davidarthur/kafka-2.3.1-rc0/
>
>
> * Maven artifacts to be voted upon:
>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
>
> * Javadoc:
>
> https://home.apache.org/~davidarthur/kafka-2.3.1-rc0/javadoc/
>
>
> * Tag to be voted upon (off 2.3 branch) is the 2.3.1 tag:
>
> https://github.com/apache/kafka/releases/tag/2.3.1-rc0
>
>
> * Documentation:
>
> https://kafka.apache.org/23/documentation.html
>
>
> * Protocol:
>
> https://kafka.apache.org/23/protocol.html
>
>
> * Successful Jenkins builds for the 2.3 branch:
>
> Unit/integration tests: https://builds.apache.org/job/kafka-2.3-jdk8/
>
> System tests:
> https://jenkins.confluent.io/job/system-test-kafka/job/2.3/119
>
>
>
> We have yet to get a successful unit/integration job run due to some flaky
> failures. I will send out a follow-up email once we have a passing build.
>
>
> Thanks!
>
> David
>

Build failed in Jenkins: kafka-2.3-jdk8 #111

[Apache Jenkins Server]

See 


Changes:

[matthias] HOTFIX: fix Kafka Streams upgrade note for broker backward 
compatibility

--
[...truncated 2.74 MB...]
kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testReassignPartitionLeaderElection PASSED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testOfflinePartitionLeaderElection STARTED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testOfflinePartitionLeaderElection PASSED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testPreferredReplicaPartitionLeaderElection STARTED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testPreferredReplicaPartitionLeaderElection PASSED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testReassignPartitionLeaderElectionWithEmptyIsr STARTED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testReassignPartitionLeaderElectionWithEmptyIsr PASSED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testControlledShutdownPartitionLeaderElectionAllIsrSimultaneouslyShutdown 
STARTED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testControlledShutdownPartitionLeaderElectionAllIsrSimultaneouslyShutdown PASSED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testOfflinePartitionLeaderElectionLastIsrOfflineUncleanLeaderElectionEnabled 
STARTED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testOfflinePartitionLeaderElectionLastIsrOfflineUncleanLeaderElectionEnabled 
PASSED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testPreferredReplicaPartitionLeaderElectionPreferredReplicaNotInIsrNotLive 
STARTED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testPreferredReplicaPartitionLeaderElectionPreferredReplicaNotInIsrNotLive 
PASSED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testOfflinePartitionLeaderElectionLastIsrOfflineUncleanLeaderElectionDisabled 
STARTED

kafka.controller.PartitionLeaderElectionAlgorithmsTest > 
testOfflinePartitionLeaderElectionLastIsrOfflineUncleanLeaderElectionDisabled 
PASSED

kafka.controller.ControllerChannelManagerTest > 
testUpdateMetadataInterBrokerProtocolVersion STARTED

kafka.controller.ControllerChannelManagerTest > 
testUpdateMetadataInterBrokerProtocolVersion PASSED

kafka.controller.ControllerChannelManagerTest > testLeaderAndIsrRequestIsNew 
STARTED

kafka.controller.ControllerChannelManagerTest > testLeaderAndIsrRequestIsNew 
PASSED

kafka.controller.ControllerChannelManagerTest > 
testStopReplicaRequestsWhileTopicQueuedForDeletion STARTED

kafka.controller.ControllerChannelManagerTest > 
testStopReplicaRequestsWhileTopicQueuedForDeletion PASSED

kafka.controller.ControllerChannelManagerTest > 
testLeaderAndIsrRequestSentToLiveOrShuttingDownBrokers STARTED

kafka.controller.ControllerChannelManagerTest > 
testLeaderAndIsrRequestSentToLiveOrShuttingDownBrokers PASSED

kafka.controller.ControllerChannelManagerTest > 
testStopReplicaInterBrokerProtocolVersion STARTED

kafka.controller.ControllerChannelManagerTest > 
testStopReplicaInterBrokerProtocolVersion PASSED

kafka.controller.ControllerChannelManagerTest > 
testStopReplicaSentOnlyToLiveAndShuttingDownBrokers STARTED

kafka.controller.ControllerChannelManagerTest > 
testStopReplicaSentOnlyToLiveAndShuttingDownBrokers PASSED

kafka.controller.ControllerChannelManagerTest > testStopReplicaGroupsByBroker 
STARTED

kafka.controller.ControllerChannelManagerTest > testStopReplicaGroupsByBroker 
PASSED

kafka.controller.ControllerChannelManagerTest > 
testUpdateMetadataDoesNotIncludePartitionsWithoutLeaderAndIsr STARTED

kafka.controller.ControllerChannelManagerTest > 
testUpdateMetadataDoesNotIncludePartitionsWithoutLeaderAndIsr PASSED

kafka.controller.ControllerChannelManagerTest > 
testMixedDeleteAndNotDeleteStopReplicaRequests STARTED

kafka.controller.ControllerChannelManagerTest > 
testMixedDeleteAndNotDeleteStopReplicaRequests PASSED

kafka.controller.ControllerChannelManagerTest > 
testLeaderAndIsrInterBrokerProtocolVersion STARTED

kafka.controller.ControllerChannelManagerTest > 
testLeaderAndIsrInterBrokerProtocolVersion PASSED

kafka.controller.ControllerChannelManagerTest > testUpdateMetadataRequestSent 
STARTED

kafka.controller.ControllerChannelManagerTest > testUpdateMetadataRequestSent 
PASSED

kafka.controller.ControllerChannelManagerTest > 
testUpdateMetadataRequestDuringTopicDeletion STARTED

kafka.controller.ControllerChannelManagerTest > 
testUpdateMetadataRequestDuringTopicDeletion PASSED

kafka.controller.ControllerChannelManagerTest > 
testUpdateMetadataIncludesLiveOrShuttingDownBrokers STARTED

kafka.controller.ControllerChannelManagerTest > 
testUpdateMetadataIncludesLiveOrShuttingDownBrokers PASSED

kafka.controller.ControllerChannelManagerTest > testStopReplicaRequestSent 
STARTED

kafka.controller.ControllerChannelManagerTest > testStopReplicaRequestSent 
PASSED

Build failed in Jenkins: kafka-trunk-jdk11 #831

[Apache Jenkins Server]

See 


Changes:

[github] KAFKA-8848; Update system tests to use new AclAuthorizer (#7374)

--
[...truncated 2.80 MB...]

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertEmptyString STARTED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertEmptyString PASSED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertMapWithStringKeysAndMixedValuesToMapWithoutSchema STARTED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertMapWithStringKeysAndMixedValuesToMapWithoutSchema PASSED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldParseStringOfMapWithIntValuesWithoutWhitespaceAsMap STARTED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldParseStringOfMapWithIntValuesWithoutWhitespaceAsMap PASSED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldParseStringOfMapWithShortValuesWithWhitespaceAsMap STARTED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldParseStringOfMapWithShortValuesWithWhitespaceAsMap PASSED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertMapWithStringKeysAndIntegerValues STARTED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertMapWithStringKeysAndIntegerValues PASSED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertNullValue STARTED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertNullValue PASSED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldParseStringOfMapWithShortValuesWithoutWhitespaceAsMap STARTED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldParseStringOfMapWithShortValuesWithoutWhitespaceAsMap PASSED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertListWithStringValues STARTED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertListWithStringValues PASSED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldParseStringOfMapWithIntValuesWithWhitespaceAsMap STARTED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldParseStringOfMapWithIntValuesWithWhitespaceAsMap PASSED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertListWithIntegerValues STARTED

org.apache.kafka.connect.storage.SimpleHeaderConverterTest > 
shouldConvertListWithIntegerValues PASSED

org.apache.kafka.connect.storage.StringConverterTest > testBytesNullToString 
STARTED

org.apache.kafka.connect.storage.StringConverterTest > testBytesNullToString 
PASSED

org.apache.kafka.connect.storage.StringConverterTest > 
testToBytesNonUtf8Encoding STARTED

org.apache.kafka.connect.storage.StringConverterTest > 
testToBytesNonUtf8Encoding PASSED

org.apache.kafka.connect.storage.StringConverterTest > testNonStringToBytes 
STARTED

org.apache.kafka.connect.storage.StringConverterTest > testNonStringToBytes 
PASSED

org.apache.kafka.connect.storage.StringConverterTest > 
testBytesToStringNonUtf8Encoding STARTED

org.apache.kafka.connect.storage.StringConverterTest > 
testBytesToStringNonUtf8Encoding PASSED

org.apache.kafka.connect.storage.StringConverterTest > testNullToBytes STARTED

org.apache.kafka.connect.storage.StringConverterTest > testNullToBytes PASSED

org.apache.kafka.connect.storage.StringConverterTest > 
testNullHeaderValueToBytes STARTED

org.apache.kafka.connect.storage.StringConverterTest > 
testNullHeaderValueToBytes PASSED

org.apache.kafka.connect.storage.StringConverterTest > 
testStringHeaderValueToBytes STARTED

org.apache.kafka.connect.storage.StringConverterTest > 
testStringHeaderValueToBytes PASSED

org.apache.kafka.connect.storage.StringConverterTest > testToBytesIgnoresSchema 
STARTED

org.apache.kafka.connect.storage.StringConverterTest > testToBytesIgnoresSchema 
PASSED

org.apache.kafka.connect.storage.StringConverterTest > 
testNonStringHeaderValueToBytes STARTED

org.apache.kafka.connect.storage.StringConverterTest > 
testNonStringHeaderValueToBytes PASSED

org.apache.kafka.connect.storage.StringConverterTest > testStringToBytes STARTED

org.apache.kafka.connect.storage.StringConverterTest > testStringToBytes PASSED

org.apache.kafka.connect.storage.StringConverterTest > testBytesToString STARTED

org.apache.kafka.connect.storage.StringConverterTest > testBytesToString PASSED

org.apache.kafka.connect.data.SchemaBuilderTest > testTypeNotNull STARTED

org.apache.kafka.connect.data.SchemaBuilderTest > testTypeNotNull PASSED

org.apache.kafka.connect.data.SchemaBuilderTest > testStringBuilder STARTED

org.apache.kafka.connect.data.SchemaBuilderTest > testStringBuilder PASSED

org.apache.kafka.connect.data.SchemaBuilderTest > 
testDefaultFieldsDifferentValueOverwriting STARTED

org.apache.kafka.connect.data.SchemaBuilderTest > 

Build failed in Jenkins: kafka-trunk-jdk8 #3921

[Apache Jenkins Server]

See 


Changes:

[github] HOTFIX: fix Kafka Streams upgrade note for broker backward 
compatibility

--
[...truncated 2.99 MB...]

org.apache.kafka.streams.MockProcessorContextTest > shouldCapturePunctuator 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnIsOpen 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnIsOpen 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldDeleteAndReturnPlainValue STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldDeleteAndReturnPlainValue PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnName 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnName 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutAllWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutAllWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldReturnIsPersistent STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldReturnIsPersistent PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutIfAbsentWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutIfAbsentWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardClose 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardClose 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardFlush 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardFlush 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardInit 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardInit 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnIsOpen 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnIsOpen 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnName 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnName 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWindowStartTimestampWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWindowStartTimestampWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldReturnIsPersistent STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldReturnIsPersistent PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardClose 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardClose 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardFlush 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardFlush 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
PASSED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis STARTED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis PASSED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime STARTED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime PASSED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep PASSED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep PASSED

> Task :streams:upgrade-system-tests-0100:processTestResources NO-SOURCE
> Task :streams:upgrade-system-tests-0100:testClasses
> Task :streams:upgrade-system-tests-0100:checkstyleTest
> Task :streams:upgrade-system-tests-0100:spotbugsMain NO-SOURCE
> Task :streams:upgrade-system-tests-0100:test
> Task :streams:upgrade-system-tests-0101:compileJava NO-SOURCE
> Task :streams:upgrade-system-tests-0101:processResources NO-SOURCE
> Task :streams:upgrade-system-tests-0101:classes UP-TO-DATE
> Task :streams:upgrade-system-tests-0101:checkstyleMain NO-SOURCE
> Task :streams:upgrade-system-tests-0101:compileTestJava
> Task :streams:upgrade-system-tests-0101:processTestResources 

Build failed in Jenkins: kafka-trunk-jdk8 #3922

[Apache Jenkins Server]

See 


Changes:

[github] KAFKA-8880: Add overloaded function of Consumer.committed (#7304)

[wangguoz] KAFKA-8580: Compute RocksDB metrics (#7263)

[wangguoz] KAFKA-8179: do not suspend standby tasks during rebalance (#7321)

--
[...truncated 2.64 MB...]
org.apache.kafka.streams.TopologyTestDriverTest > 
shouldSendRecordViaCorrectSourceTopic[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldSendRecordViaCorrectSourceTopic[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotCreateStateDirectoryForStatelessTopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotCreateStateDirectoryForStatelessTopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnAllStoresNames[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnAllStoresNames[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessConsumerRecordList[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessConsumerRecordList[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUseSinkSpecificSerializers[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUseSinkSpecificSerializers[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldFlushStoreForFirstInput[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldFlushStoreForFirstInput[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessFromSourceThatMatchPattern[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessFromSourceThatMatchPattern[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUpdateStoreForNewKey[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldUpdateStoreForNewKey[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateOnWallClockTime[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateOnWallClockTime[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > shouldSetRecordMetadata[Eos 
enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > shouldSetRecordMetadata[Eos 
enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForLargerValue[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForLargerValue[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnCorrectInMemoryStoreTypeOnly[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldReturnCorrectInMemoryStoreTypeOnly[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessRecordForTopic[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldProcessRecordForTopic[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldForwardRecordsFromSubtopologyToSubtopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldForwardRecordsFromSubtopologyToSubtopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForSmallerValue[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForSmallerValue[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] PASSED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis STARTED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis PASSED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime STARTED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime PASSED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep PASSED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnIsOpen 
STARTED

Build failed in Jenkins: kafka-trunk-jdk11 #832

[Apache Jenkins Server]

See 


Changes:

[github] HOTFIX: fix Kafka Streams upgrade note for broker backward 
compatibility

[github] KAFKA-8880: Add overloaded function of Consumer.committed (#7304)

[wangguoz] KAFKA-8580: Compute RocksDB metrics (#7263)

--
[...truncated 2.73 MB...]

org.apache.kafka.streams.kstream.internals.UnlimitedWindowTest > 
cannotCompareUnlimitedWindowWithDifferentWindowType PASSED

org.apache.kafka.streams.kstream.internals.KTableReduceTest > 
shouldAddAndSubtract STARTED

org.apache.kafka.streams.kstream.internals.KTableReduceTest > 
shouldAddAndSubtract PASSED

org.apache.kafka.streams.kstream.internals.SessionTupleForwarderTest > 
shouldForwardRecordsIfWrappedStateStoreDoesNotCache STARTED

org.apache.kafka.streams.kstream.internals.SessionTupleForwarderTest > 
shouldForwardRecordsIfWrappedStateStoreDoesNotCache PASSED

org.apache.kafka.streams.kstream.internals.SessionTupleForwarderTest > 
shouldNotForwardRecordsIfWrappedStateStoreDoesCache STARTED

org.apache.kafka.streams.kstream.internals.SessionTupleForwarderTest > 
shouldNotForwardRecordsIfWrappedStateStoreDoesCache PASSED

org.apache.kafka.streams.kstream.internals.SessionTupleForwarderTest > 
shouldSetFlushListenerOnWrappedStateStore STARTED

org.apache.kafka.streams.kstream.internals.SessionTupleForwarderTest > 
shouldSetFlushListenerOnWrappedStateStore PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullMapperOnMapValues STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullMapperOnMapValues PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullPredicateOnFilter STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullPredicateOnFilter PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldThrowNullPointerOnOuterJoinWhenMaterializedIsNull STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldThrowNullPointerOnOuterJoinWhenMaterializedIsNull PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullOtherTableOnJoin STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullOtherTableOnJoin PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldThrowNullPointerOnFilterNotWhenMaterializedIsNull STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldThrowNullPointerOnFilterNotWhenMaterializedIsNull PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullSelectorOnGroupBy STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullSelectorOnGroupBy PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullJoinerOnLeftJoin STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullJoinerOnLeftJoin PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldThrowNullPointerOnJoinWhenMaterializedIsNull STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldThrowNullPointerOnJoinWhenMaterializedIsNull PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > testStateStore 
STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > testStateStore 
PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullMapperOnMapValueWithKey STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullMapperOnMapValueWithKey PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldThrowNullPointerOnTransformValuesWithKeyWhenStoreNamesNull STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldThrowNullPointerOnTransformValuesWithKeyWhenStoreNamesNull PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullOtherTableOnLeftJoin STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullOtherTableOnLeftJoin PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullSelectorOnToStream STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullSelectorOnToStream PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullOtherTableOnOuterJoin STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullOtherTableOnOuterJoin PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullJoinerOnOuterJoin STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldNotAllowNullJoinerOnOuterJoin PASSED

org.apache.kafka.streams.kstream.internals.KTableImplTest > 
shouldCreateSourceAndSinkNodesForRepartitioningTopic STARTED

org.apache.kafka.streams.kstream.internals.KTableImplTest >