[DISCUSS] KIP-491: Preferred Leader Deprioritized List (Temporary Blacklist)

[George Li]

 Hi,

I have created KIP-491 
(https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=120736982) 
for putting a broker to the preferred leader blacklist or deprioritized list so 
when determining leadership,  it's moved to the lowest priority for some of the 
listed use-cases. 

Please provide your comments/feedbacks. 

Thanks,
George



   - Forwarded Message - From: Jose Armando Garcia Sancio (JIRA) 
To: "sql_consult...@yahoo.com" Sent: 
Tuesday, July 9, 2019, 01:06:05 PM PDTSubject: [jira] [Commented] (KAFKA-8638) 
Preferred Leader Blacklist (deprioritized list)
 
    [ 
https://issues.apache.org/jira/browse/KAFKA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881511#comment-16881511
 ] 

Jose Armando Garcia Sancio commented on KAFKA-8638:
---

Thanks for feedback and clear use cases [~sql_consulting].

> Preferred Leader Blacklist (deprioritized list)
> ---
>
>                Key: KAFKA-8638
>                URL: https://issues.apache.org/jira/browse/KAFKA-8638
>            Project: Kafka
>          Issue Type: Improvement
>          Components: config, controller, core
>    Affects Versions: 1.1.1, 2.3.0, 2.2.1
>            Reporter: GEORGE LI
>            Assignee: GEORGE LI
>            Priority: Major
>
> Currently, the kafka preferred leader election will pick the broker_id in the 
> topic/partition replica assignments in a priority order when the broker is in 
> ISR. The preferred leader is the broker id in the first position of replica. 
> There are use-cases that, even the first broker in the replica assignment is 
> in ISR, there is a need for it to be moved to the end of ordering (lowest 
> priority) when deciding leadership during  preferred leader election. 
> Let’s use topic/partition replica (1,2,3) as an example. 1 is the preferred 
> leader.  When preferred leadership is run, it will pick 1 as the leader if 
> it's ISR, if 1 is not online and in ISR, then pick 2, if 2 is not in ISR, 
> then pick 3 as the leader. There are use cases that, even 1 is in ISR, we 
> would like it to be moved to the end of ordering (lowest priority) when 
> deciding leadership during preferred leader election.  Below is a list of use 
> cases:
> * (If broker_id 1 is a swapped failed host and brought up with last segments 
> or latest offset without historical data (There is another effort on this), 
> it's better for it to not serve leadership till it's caught-up.
> * The cross-data center cluster has AWS instances which have less computing 
> power than the on-prem bare metal machines.  We could put the AWS broker_ids 
> in Preferred Leader Blacklist, so on-prem brokers can be elected leaders, 
> without changing the reassignments ordering of the replicas. 
> * If the broker_id 1 is constantly losing leadership after some time: 
> "Flapping". we would want to exclude 1 to be a leader unless all other 
> brokers of this topic/partition are offline.  The “Flapping” effect was seen 
> in the past when 2 or more brokers were bad, when they lost leadership 
> constantly/quickly, the sets of partition replicas they belong to will see 
> leadership constantly changing.  The ultimate solution is to swap these bad 
> hosts.  But for quick mitigation, we can also put the bad hosts in the 
> Preferred Leader Blacklist to move the priority of its being elected as 
> leaders to the lowest. 
> *  If the controller is busy serving an extra load of metadata requests and 
> other tasks. we would like to put the controller's leaders to other brokers 
> to lower its CPU load. currently bouncing to lose leadership would not work 
> for Controller, because after the bounce, the controller fails over to 
> another broker.
> * Avoid bouncing broker in order to lose its leadership: it would be good if 
> we have a way to specify which broker should be excluded from serving 
> traffic/leadership (without changing the replica assignment ordering by 
> reassignments, even though that's quick), and run preferred leader election.  
> A bouncing broker will cause temporary URP, and sometimes other issues.  Also 
> a bouncing of broker (e.g. broker_id 1) can temporarily lose all its 
> leadership, but if another broker (e.g. broker_id 2) fails or gets bounced, 
> some of its leaderships will likely failover to broker_id 1 on a replica with 
> 3 brokers.  If broker_id 1 is in the blacklist, then in such a scenario even 
> broker_id 2 offline,  the 3rd broker can take leadership. 
> The current work-around of the above is to change the topic/partition's 
> replica reassignments to move the broker_id 1 from the first position to the 
> last position and run preferred leader election. e.g. (1, 2, 3) => (2, 3, 1). 
> This changes the replica reassignments, and we need to keep track of the 
> original one and restore if things change (e.g. controller fails over to 
> another broker, the swapped empty 

Re: [DISCUSS] KIP-455 Create an Admin API for Replica Reassignments

[George Li]

 Hi Stanislav,

sorry for the late reply. comments below: 

> Thanks for the reminder. A lot of your suggestions are outlined in the
> "Future Work" section of KIP-455. The pointer towards different
> ReplicaFetcher thread pools is interesting -- do you think there's
> much value in that? My intuition is that having appropriate quotas for
> the reassignment traffic is the better way to separate the two,
> whereas a separate thread pool might provide less of a benefit.


I think separating the Reassignment replication from the normal follower 
traffic of the ReplicaFetcher Threads will give us some benefits: 

1.  The throttling on Reassignment traffic will be much cleaner to the 
Reassignment replication threads, Right now, it's complicated using the topic 
config (+broker level) for throttle. e.g. 


Topic:georgeli_test PartitionCount:8 ReplicationFactor:4 
Configs:leader.replication.throttled.replicas=0:1026,0:1028,0:1025,follower.replication.throttled.replicas=0:1027
 Topic: georgeli_test Partition: 0 Leader: 1026 Replicas: 1027,1025,1028,1026 
Isr: 1026,1028,1025


When trying to do reassignment (1026,1028,1025) ==> (1027,1025,1028), it puts 
the partition#, brokers in the Topic config, if 1027 caught up in ISR, and 
reassignment completed,  1027 could still be throttled. unless we remove the 
throttle.  If it's dedicated Reassignment replication thread, the completed 
reassignment replica will move back to the normal ReplicaFetcher threads. 
Making managing throttle much easier. 


2.  One of the major issues in reassignments is the performance impact on 
"lossless" type of cluster/topics, the producer latency spikes 
(min.insync.replicas>1) even with throttling, the major contribution of latency 
spike from RemoteTimeMs (replication).  In theory, since Reassignment traffic 
is sharing the same ReplicaFetcher threads, for large partition, it will affect 
the existing Leader => Follower replication. Also after separating, the 
reassignment replication can have different settings to increase network 
throughput, etc. because it's pulling large amount of historical data? The 
normal replication follower traffic is doing less incremental data. Two 
different type of use cases. 


> With regards to keeping the original replicas info before
> reassignments are kicked off - this KIP proposes that we store the
> `targetReplicas` in a different collection and thus preserve the
> original replicas info until the reassignment is fully complete. It
> should allow you to implement rollback functionality. Please take a
> look at the KIP and confirm if that is the case. It would be good to
> synergize both KIPs.

I think the "targetReplicas" in ZK node /topics/ is fine.  But the code 
(makeFollower()?) to determine where ReplicaFetcher should pull needs a change, 
right now, it's using the partition current assignment replicas. Current 
implementation is first updating  OAR + RAR. I just submit 
https://issues.apache.org/jira/browse/KAFKA-8663 for a minor improvement 
request. 

I just take a look at the updated KIP-455 again. I noticed this 
"targetReplicas" is removed and instead, put "addingReplicas" & 
"removingReplicas".  So where does the new reassigned "targetReplicas" is 
stored? When all replicas in ISR, and reassignment completes, it needs to 
update the partition assignment with "targetReplicas" which has the exact 
replica ordering, e.g. Preferred Leader, 2nd preferred leader, ., not just 
by removing the "removingReplicas" from the existing partition assignments? 

Also I still hope this KIP-455 can provide a feature/solution of clean 
cancel/rollback as much as possible. It would be easier for the user. e.g. (1, 
2, 3) => (4, 5, 6),  If ISR goes to (1,2,3,4), and cancel, since 1, 2, 3 all in 
ISR, why not cleanly rollback to (1,2,3),  but leave it as (1,2,3,4)?  having 4 
replicas there will increase storage of the cluster, and replicationFactor 
discrepancies among partitions of the same topic (e.g. future expand partition 
operation of this topic will fail).  With "originalReplicas" & 
"targetReplicas", I think we can derive "addingReplicas" & "removingReplicas".  
  


Can KIP-455 elaborate a bit more detail on how to remove the reassignment 
throttle ?  Current implementation is for the user to submit the list of 
current reassignments json with a "--verify" option. It will check whether the 
current partition assignments matches the one of the "targetReplicas" and 
remove the throttles at the Topic Level and Broker Level 
(leader.replication.throttled.rate/follower.replication.throttled.rate).  
KIP-455 will remove Broker Level throttle only when no more reassignments in 
the cluster?  race conditions with another newly submitted reassignment with 
throttle?  actually general handling of race conditions of concurrent 
submissions of reassignments which was guarded against before by the 
/admin/reassign_partitions znode. 

Overall, I think KIP-455 is a great improvement and direction to go. 

[jira] [Created] (KAFKA-8663) partition assignment would be better original_assignment + new_reassignment during reassignments

[GEORGE LI (JIRA)]

GEORGE LI created KAFKA-8663:


 Summary: partition assignment would be better original_assignment 
+ new_reassignment during reassignments
 Key: KAFKA-8663
 URL: https://issues.apache.org/jira/browse/KAFKA-8663
 Project: Kafka
  Issue Type: Improvement
  Components: controller, core
Affects Versions: 2.3.0, 1.1.1
Reporter: GEORGE LI


>From my observation/experience during reassignment,  the partition assignment 
>replica ordering gets changed.   because it's  OAR + RAR  (original replicas + 
>reassignment replicas)  set union. 

However, it seems like the preferred leaders changed during the reassignments.  
Normally if there is no cluster preferred leader election,  the leader is still 
the old leader.  But if during the reassignments, there is a leader election,  
the leadership changes.  This caused some side effects.  Let's look at this 
example.

{code}
Topic:georgeli_test PartitionCount:8ReplicationFactor:3 Configs:
Topic: georgeli_testPartition: 0Leader: 1026Replicas: 
1026,1028,1025Isr: 1026,1028,1025
{code}

reassignment  (1026,1028,1025) => (1027,1025,1028)

{code}
Topic:georgeli_test PartitionCount:8ReplicationFactor:4 
Configs:leader.replication.throttled.replicas=0:1026,0:1028,0:1025,follower.replication.throttled.replicas=0:1027
Topic: georgeli_testPartition: 0Leader: 1026Replicas: 
1027,1025,1028,1026   Isr: 1026,1028,1025
{code}

Notice the above:   Leader remains 1026.   but Replicas: 1027,1025,1028,1026.   
If we run preferred leader election,  it will try 1027 first, then 1025.
After  1027 is in ISR,  then the final assignment will be  (1027,1025,1028).

My proposal for a minor improvement is to keep the original ordering replicas 
during the reassignment (could be long for big topic/partitions).  and after 
all replicas in ISR, then finally set the partition assignment to New 
reassignment.  

{code}
  val newAndOldReplicas = (reassignedPartitionContext.newReplicas ++ 
controllerContext.partitionReplicaAssignment(topicPartition)).toSet
  //1. Update AR in ZK with OAR + RAR.
  updateAssignedReplicasForPartition(topicPartition, 
newAndOldReplicas.toSeq)
{code} 

above code changed to below to keep the original ordering during reassignment: 

{code}
  val newAndOldReplicas = 
(controllerContext.partitionReplicaAssignment(topicPartition) ++ 
reassignedPartitionContext.newReplicas).toSet
{ code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[DISCUSS] KIP-490: log when consumer groups lose a message because offset has been deleted

[Jose M]

Hello,

I would like to know what do you think on KIP-490:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-490%3A+log+when+consumer+groups+lose+a+message+because+offset+has+been+deleted



Thanks a lot !
-- 
Jose M

Re: [DISCUSS] KIP-484: Expose metrics for group and transaction metadata loading duration

[Anastasia Vela]

Hey! Thanks for looking over this KIP. These are all partitions per
manager. I'll clear this up in the KIP.

Anastasia

On Sat, Jul 13, 2019, 2:15 PM Stanislav Kozlovski 
wrote:

> Hey there,
>
> Thanks for the KIP! I always smile when I see new metrics
> I have just one quick question - are these metrics per partition or a total
> of all the partitions per manager? The KIP has conflicting sentences so I'm
> not sure what it is.
>
> Thanks,
> Stanislav
>
>
> On Wed, Jul 3, 2019 at 9:13 PM Gwen Shapira  wrote:
>
> > It looks great! If there are no more concerns, lets start a vote.
> >
> > On Tue, Jul 2, 2019 at 2:59 PM Anastasia Vela 
> wrote:
> > >
> > > After further discussion with Anna, we decided the following:
> > > - add the average metric, but noted in the KIP that the average value
> may
> > > look low at times when there are many empty partitions that have a 0ms
> > load
> > > time
> > > - set the window to 30sec, because there is no significance difference
> if
> > > we set the window time to 3hrs, so I will keep the default value
> instead.
> > >
> > > Thanks. Let me know any more concerns.
> > > Anastasia
> > >
> > > On Mon, Jul 1, 2019 at 9:13 AM Anastasia Vela 
> > wrote:
> > >
> > > > Hey Gwen!
> > > >
> > > > Thanks for reviewing my KIP!
> > > >
> > > > 1. I did consider adding an Avg metric as well. Anna and I decided
> > that a
> > > > max would provide the crucial information. We just need to know if
> > there
> > > > was a long load time, and expose what that duration was so we
> > understand
> > > > there's downtime for such a long time. However I do agree that it may
> > be
> > > > necessary to compute averages if we want to give the max a reference
> > point.
> > > > I can easily add this if we believe it is necessary.
> > > > 2. The default refers to the metric configuration set when you
> > initialize
> > > > KafkaServer. When I was running tests, the max value was computed
> over
> > a
> > > > window of 30 seconds, unless I changed the metrics config. So I noted
> > that
> > > > unless we change the config for this specific metric, it will be
> > computed
> > > > over the default window.
> > > > 3. I proposed a 3 hour window because we have (very rarely) seen
> > > > partitions take hours to load. 3 hours was an upper bound for how
> long
> > a
> > > > load could take. The way max works is that it computes the running
> max
> > > > until the window has lapsed. Then it starts a new window and forgets
> > the
> > > > max value of the last window. So if a partition takes more than the
> > window
> > > > time to load, there will be one value in that window and the next
> load
> > will
> > > > be part of a new window. I guess it just depends on how we want it to
> > be
> > > > displayed on the graph. If it's ok for this behavior to happen, the
> > window
> > > > can be shrunk. Regarding the rate metric, I was actually thinking
> about
> > > > doing this, but I was told that loads don't happen very often. But it
> > is
> > > > true that if the reload happens very often then that may be a
> problem.
> > > >
> > > > Thanks,
> > > > Anastasia
> > > >
> > > > On Fri, Jun 28, 2019 at 4:27 PM Gwen Shapira 
> > wrote:
> > > >
> > > >> Hey,
> > > >>
> > > >> Thank you for proposing this! Sounds really useful - we have
> > > >> definitely seem some difficult to explain pauses in consumer
> activity
> > > >> and this metric will let us correlate those.
> > > >>
> > > >> Few questions:
> > > >> 1. Did you consider adding both Max and Avg metrics? Many of our
> > > >> metrics have both (batch-size and message-size for example) and it
> > > >> helps put the max value in context.
> > > >> 2. You wrote: "Lengthening or shortening the 3 hour time window is
> up
> > > >> for discussion (default is 30sec)."  and I'm not sure what default
> you
> > > >> are referring to?
> > > >> 3. Can you also give some background on why you are proposing 3h?
> I'm
> > > >> guessing it is because loading the state from the topic happens
> rarely
> > > >> enough that in 3h it will probably only happen once or not at all?
> > > >> Perhaps we need a rate metric to see how often it actually happens
> (if
> > > >> we have to reload offsets very often it is a different problem).
> > > >>
> > > >> Gwen
> > > >>
> > > >> On Tue, Jun 25, 2019 at 4:43 PM Anastasia Vela 
> > > >> wrote:
> > > >> >
> > > >> > Hi all,
> > > >> >
> > > >> > I'd like to discuss KIP-484:
> > > >> >
> > > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-484%3A+Expose+metrics+for+group+and+transaction+metadata+loading+duration
> > > >> >
> > > >> > Let me know what you think!
> > > >> >
> > > >> > Thanks,
> > > >> > Anastasia
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Gwen Shapira
> > > >> Product Manager | Confluent
> > > >> 650.450.2760 | @gwenshap
> > > >> Follow us: Twitter | blog
> > > >>
> > > >
> >
> >
> >
> > --
> > Gwen Shapira
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter | blog
> >
>
>

[jira] [Resolved] (KAFKA-8183) Trogdor - ProduceBench should retry on UnknownTopicOrPartitionException during topic creation

[Stanislav Kozlovski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stanislav Kozlovski resolved KAFKA-8183.

Resolution: Fixed

> Trogdor - ProduceBench should retry on UnknownTopicOrPartitionException 
> during topic creation
> -
>
> Key: KAFKA-8183
> URL: https://issues.apache.org/jira/browse/KAFKA-8183
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Stanislav Kozlovski
>Assignee: Stanislav Kozlovski
>Priority: Minor
>
> There exists a race condition in the Trogdor produce bench worker code where 
> `WorkerUtils#createTopics()` [notices the topic 
> exists|https://github.com/apache/kafka/blob/4824dc994d7fc56b7540b643a78aadb4bdd0f14d/tools/src/main/java/org/apache/kafka/trogdor/common/WorkerUtils.java#L159]
>  yet when it goes on to verify the topics, the DescribeTopics call throws an 
> `UnknownTopicOrPartitionException`.
> We should add sufficient retries such that this does not fail the task.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Re: [VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

[Stanislav Kozlovski]

Hello,

Could we restart the discussion here again?

My last message was sent on the 3rd of June but I haven't received replies
since then.

I'd like to get this KIP to a finished state, regardless of whether that is
merged-in or discarded. It has been almost one year since the publication
of the KIP.

Thanks,
Stanislav

On Mon, Jun 3, 2019 at 11:19 AM Stanislav Kozlovski 
wrote:

> Do people agree with the approach I outlined in my last reply?
>
> On Mon, May 6, 2019 at 2:12 PM Stanislav Kozlovski 
> wrote:
>
>> Hey there Kamal,
>>
>> I'm sincerely sorry for missing your earlier message. As I open this
>> thread up, I see I have an unsent draft message about resuming discussion
>> from some time ago.
>>
>> In retrospect, I think I may have been too pedantic with the exception
>> naming and hierarchy.
>> I now believe a single exception type of `RecordDeserializationException`
>> is enough. Let's go with that.
>>
>> On Mon, May 6, 2019 at 6:40 AM Kamal Chandraprakash <
>> kamal.chandraprak...@gmail.com> wrote:
>>
>>> Matthias,
>>>
>>> We already have CorruptRecordException which doesn't extend the
>>> SerializationException. So, we need an alternate
>>> name suggestion for the corrupted record error if we decide to extend the
>>> FaultyRecordException class.
>>>
>>> Stanislav,
>>>
>>> Our users are also facing this error. Could we bump up this discussion?
>>>
>>> I think we can have a single exception type
>>> FaultyRecordException/RecordDeserialization exception to capture both
>>> the errors. We can add an additional enum field to differentiate the
>>> errors
>>> if required.
>>>
>>> Thanks,
>>> Kamal Chandraprakash
>>>
>>> On Wed, Apr 24, 2019 at 1:49 PM Kamal Chandraprakash <
>>> kamal.chandraprak...@gmail.com> wrote:
>>>
>>> > Stanislav,
>>> >
>>> > Any updates on this KIP? We have internal users who want to skip the
>>> > corrupted message while consuming the records.
>>> >
>>> >
>>> > On Fri, Oct 19, 2018 at 11:34 PM Matthias J. Sax <
>>> matth...@confluent.io>
>>> > wrote:
>>> >
>>> >> I am not 100% familiar with the details of the consumer code, however
>>> I
>>> >> tend to disagree with:
>>> >>
>>> >> > There's no difference between the two cases -- if (and only if) the
>>> >> message is corrupt, it can't be deserialized.  If (and only if) it
>>> can't be
>>> >> deserialized, it is corrupt.
>>> >>
>>> >> Assume that a user configures a JSON deserializer but a faulty
>>> upstream
>>> >> producer writes an Avro message. For this case, the message is not
>>> >> corrupted, but still can't be deserialized. And I can imaging that
>>> users
>>> >> want to handle both cases differently.
>>> >>
>>> >> Thus, I think it makes sense to have two different exceptions
>>> >> `RecordDeserializationException` and `CorruptedRecordException` that
>>> can
>>> >> both extend `FaultyRecordException` (don't like this name too much
>>> >> honestly, but don't have a better idea for it anyway).
>>> >>
>>> >> Side remark. If we introduce class `RecordDeserializationException`
>>> and
>>> >> `CorruptedRecordException`, we can also add an interface that both
>>> >> implement to return partition/offset information and let both extend
>>> >> `SerializationException` directly without an intermediate class in the
>>> >> exception hierarchy.
>>> >>
>>> >>
>>> >> -Matthias
>>> >>
>>> >> On 8/8/18 2:57 AM, Stanislav Kozlovski wrote:
>>> >> >> If you are inheriting from SerializationException, your derived
>>> class
>>> >> > should also be a kind of serialization exception.  Not something
>>> more
>>> >> > general.
>>> >> > Yeah, the reason for inheriting it would be for
>>> backwards-compatibility.
>>> >> >
>>> >> >> Hmm.  Can you think of any new scenarios that would make Kafka
>>> force
>>> >> the
>>> >> > user need to skip a specific record?  Perhaps one scenario is if
>>> records
>>> >> > are lost but we don't know how many.
>>> >> > Not on the spot, but I do wonder how likely a new scenario is to
>>> >> surface in
>>> >> > the future and how we'd handle the exceptions' class hierarchy then.
>>> >> >
>>> >> >> Which offset were we planning to use in the
>>> >> > exception?
>>> >> > The offset of the record which caused the exception. In the case of
>>> >> > batches, we use the last offset of the batch. In both cases, users
>>> >> should
>>> >> > have to seek +1 from the given offset. You can review the PR to
>>> ensure
>>> >> its
>>> >> > accurate
>>> >> >
>>> >> >
>>> >> > If both of you prefer `RecordDeserializationException`, we can go
>>> with
>>> >> > that. Please do confirm that is okay
>>> >> >
>>> >> > On Tue, Aug 7, 2018 at 11:35 PM Jason Gustafson >> >
>>> >> wrote:
>>> >> >
>>> >> >> One difference between the two cases is that we can't generally
>>> trust
>>> >> the
>>> >> >> offset of a corrupt message. Which offset were we planning to use
>>> in
>>> >> the
>>> >> >> exception? Maybe it should be either the fetch offset or one plus
>>> the
>>> >> last
>>> >> >> consumed offset? I think I'm with Colin in preferring

Re: [VOTE] KIP-480 : Sticky Partitioner

[Stanislav Kozlovski]

+1 (non-binding)

Thanks!

On Fri, Jul 12, 2019 at 6:02 PM Gwen Shapira  wrote:

> +1 (binding)
>
> Thank you for the KIP. This was long awaited.
>
> On Tue, Jul 9, 2019 at 5:15 PM Justine Olshan 
> wrote:
> >
> > Hello all,
> >
> > I'd like to start the vote for KIP-480 : Sticky Partitioner.
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-480%3A+Sticky+Partitioner
> >
> > Thank you,
> > Justine Olshan
>
>
>
> --
> Gwen Shapira
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter | blog
>


-- 
Best,
Stanislav

Re: [DISCUSS] KIP-484: Expose metrics for group and transaction metadata loading duration

[Stanislav Kozlovski]

Hey there,

Thanks for the KIP! I always smile when I see new metrics
I have just one quick question - are these metrics per partition or a total
of all the partitions per manager? The KIP has conflicting sentences so I'm
not sure what it is.

Thanks,
Stanislav


On Wed, Jul 3, 2019 at 9:13 PM Gwen Shapira  wrote:

> It looks great! If there are no more concerns, lets start a vote.
>
> On Tue, Jul 2, 2019 at 2:59 PM Anastasia Vela  wrote:
> >
> > After further discussion with Anna, we decided the following:
> > - add the average metric, but noted in the KIP that the average value may
> > look low at times when there are many empty partitions that have a 0ms
> load
> > time
> > - set the window to 30sec, because there is no significance difference if
> > we set the window time to 3hrs, so I will keep the default value instead.
> >
> > Thanks. Let me know any more concerns.
> > Anastasia
> >
> > On Mon, Jul 1, 2019 at 9:13 AM Anastasia Vela 
> wrote:
> >
> > > Hey Gwen!
> > >
> > > Thanks for reviewing my KIP!
> > >
> > > 1. I did consider adding an Avg metric as well. Anna and I decided
> that a
> > > max would provide the crucial information. We just need to know if
> there
> > > was a long load time, and expose what that duration was so we
> understand
> > > there's downtime for such a long time. However I do agree that it may
> be
> > > necessary to compute averages if we want to give the max a reference
> point.
> > > I can easily add this if we believe it is necessary.
> > > 2. The default refers to the metric configuration set when you
> initialize
> > > KafkaServer. When I was running tests, the max value was computed over
> a
> > > window of 30 seconds, unless I changed the metrics config. So I noted
> that
> > > unless we change the config for this specific metric, it will be
> computed
> > > over the default window.
> > > 3. I proposed a 3 hour window because we have (very rarely) seen
> > > partitions take hours to load. 3 hours was an upper bound for how long
> a
> > > load could take. The way max works is that it computes the running max
> > > until the window has lapsed. Then it starts a new window and forgets
> the
> > > max value of the last window. So if a partition takes more than the
> window
> > > time to load, there will be one value in that window and the next load
> will
> > > be part of a new window. I guess it just depends on how we want it to
> be
> > > displayed on the graph. If it's ok for this behavior to happen, the
> window
> > > can be shrunk. Regarding the rate metric, I was actually thinking about
> > > doing this, but I was told that loads don't happen very often. But it
> is
> > > true that if the reload happens very often then that may be a problem.
> > >
> > > Thanks,
> > > Anastasia
> > >
> > > On Fri, Jun 28, 2019 at 4:27 PM Gwen Shapira 
> wrote:
> > >
> > >> Hey,
> > >>
> > >> Thank you for proposing this! Sounds really useful - we have
> > >> definitely seem some difficult to explain pauses in consumer activity
> > >> and this metric will let us correlate those.
> > >>
> > >> Few questions:
> > >> 1. Did you consider adding both Max and Avg metrics? Many of our
> > >> metrics have both (batch-size and message-size for example) and it
> > >> helps put the max value in context.
> > >> 2. You wrote: "Lengthening or shortening the 3 hour time window is up
> > >> for discussion (default is 30sec)."  and I'm not sure what default you
> > >> are referring to?
> > >> 3. Can you also give some background on why you are proposing 3h? I'm
> > >> guessing it is because loading the state from the topic happens rarely
> > >> enough that in 3h it will probably only happen once or not at all?
> > >> Perhaps we need a rate metric to see how often it actually happens (if
> > >> we have to reload offsets very often it is a different problem).
> > >>
> > >> Gwen
> > >>
> > >> On Tue, Jun 25, 2019 at 4:43 PM Anastasia Vela 
> > >> wrote:
> > >> >
> > >> > Hi all,
> > >> >
> > >> > I'd like to discuss KIP-484:
> > >> >
> > >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-484%3A+Expose+metrics+for+group+and+transaction+metadata+loading+duration
> > >> >
> > >> > Let me know what you think!
> > >> >
> > >> > Thanks,
> > >> > Anastasia
> > >>
> > >>
> > >>
> > >> --
> > >> Gwen Shapira
> > >> Product Manager | Confluent
> > >> 650.450.2760 | @gwenshap
> > >> Follow us: Twitter | blog
> > >>
> > >
>
>
>
> --
> Gwen Shapira
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter | blog
>


-- 
Best,
Stanislav

Re: [DISCUSS] KIP-488: Clean up Sum,Count,Total Metrics

[Stanislav Kozlovski]

Hello,

Thanks for the KIP, I'm happy we're converging on implementations.

I was wondering whether we need to deprecate the old metrics - are those
classes public interfaces? I see we don't build a JavaDoc for them (
https://kafka.apache.org/22/javadoc/allclasses-noframe.html) and, as far as
I know, that means we don't necessarily need to go with the deprecation
route.

I think we also have the Measurable and Gauge classes which seem to do the
same thing. From the names, I intuitively thought that Measurable indicates
something that might require some amount of processing to get the value of
and Gauge to be something that is instantly read. Reading their usages,
though, I see that is not the case. Measurable is inherited by
MeasurableStat which itself is inherited by most of the classes you
mentioned in the KIP. Is it worth it converging on those as well (perhaps
deprecating/removing Measurable and opting for Gauge) or do we prefer to
keep the scope of this KIP small?

Thanks,
Stanislav

On Fri, Jul 12, 2019 at 7:42 PM John Roesler  wrote:

> Hey, thanks Matthias and Bruno,
>
> I agree, "Cumulative" is a mouthful. "TotalX" sounds fine to me.
>
> Also, yes, I would have liked to not have any modifier for
> "non-sampled", but there is a name conflict with Sum.
>
> I'll update the KIP to reflect "TotalX" and then start the vote thread.
>
> Thanks again,
> -John
>
> On Fri, Jul 12, 2019 at 11:27 AM Bruno Cadonna  wrote:
> >
> > OK, makes sense. Then, I am in favour of TotalCount and TotalSum.
> >
> > Best,
> > Bruno
> >
> > On Fri, Jul 12, 2019 at 12:57 AM Matthias J. Sax 
> wrote:
> > >
> > > `Sum` is an existing name, for the "sampled sum" metric, that gets
> > > deprecated. Hence, we cannot use it.
> > >
> > > If we cannot use `Sum` and use `TotalSum`, we should also not use
> > > `Count` but `TotalCount` for consistency.
> > >
> > >
> > > -Matthias
> > >
> > >
> > >
> > > On 7/11/19 12:58 PM, Bruno Cadonna wrote:
> > > > Hi John,
> > > >
> > > > Thank you for the KIP.
> > > >
> > > > LGTM
> > > >
> > > > I also do not like CumulativeSum/Count so much. I propose to just
> call
> > > > it Sum and Count.
> > > >
> > > > I understand that you want to unequivocally distinguish the two
> metric
> > > > functions by their names, but I have the feeling the names become
> > > > artificially complex. The exact semantics can also be documented in
> > > > the javadocs, which btw could also be improved in those classes.
> > > >
> > > > Best,
> > > > Bruno
> > > >
> > > >
> > > >
> > > > On Thu, Jul 11, 2019 at 8:25 PM Matthias J. Sax <
> matth...@confluent.io> wrote:
> > > >>
> > > >> Thanks for the KIP. Overall LGTM.
> > > >>
> > > >> The only though I have is, if we may want to use `TotalSum` and
> > > >> `TotalCount` instead of `CumulativeSum/Count` as names?
> > > >>
> > > >>
> > > >> -Matthias
> > > >>
> > > >>
> > > >> On 7/11/19 9:31 AM, John Roesler wrote:
> > > >>> Hi Kafka devs,
> > > >>>
> > > >>> I'd like to propose KIP-488 as a minor cleanup of some of our
> metric
> > > >>> implementations.
> > > >>>
> > > >>> KIP-488: https://cwiki.apache.org/confluence/x/kkAyBw
> > > >>>
> > > >>> Over time, iterative updates to these metrics has resulted in a
> pretty
> > > >>> confusing little collection of classes, and I've personally been
> > > >>> involved in three separate moderately time-consuming iterations of
> me
> > > >>> or someone else trying to work out which metrics are available, and
> > > >>> which ones are desired for a given use case. One of these was
> actually
> > > >>> a long-running bug in Kafka Streams' metrics, so not only has this
> > > >>> confusion been a time sink, but it has also led to bugs.
> > > >>>
> > > >>> I'm hoping this change won't be too controversial.
> > > >>>
> > > >>> Thanks,
> > > >>> -John
> > > >>>
> > > >>
> > >
>

Re: [DISCUSS] KIP-480 : Sticky Partitioner

[Jun Rao]

Hi, Justine,

Thanks for the explanation. Your PR made the RecordAccumulator more
cooperative with the new partitioning scheme. So, what I mentioned won't be
an issue. The KIP looks good to me then.

Jun

On Fri, Jul 12, 2019 at 9:02 AM Justine Olshan  wrote:

> Hello all,
>
> Jun, thanks for taking a look at my KIP! We were also concerned about
> batches containing a single record so we kept this in mind for the
> implementation. The decision to switch the sticky partition actually
> involves returning from the record accumulator and assigning the new
> partition before the new batch is created. That way all of the records will
> go to this new partition's batch. If you would like to get a better look at
> how this works, please check out the PR:
> https://github.com/apache/kafka/pull/6997/files. The most important lines
> are in the append method of the RecordAccumulator and doSend in
> KafkaProducer.
>
> Colin, I think this makes sense to me except for the name
> StickyRoundRobinPartitioner seems to not really explain the behavior of
> what would be implemented. Perhaps a name indicating the sticky behavior is
> always used, or that it will be used on keys is more descriptive. Calling
> it "RoundRobin" seems a bit misleading to me.
>
> Thanks again for reviewing,
> Justine
>
> On Thu, Jul 11, 2019 at 6:07 PM Jun Rao  wrote:
>
> > Hi, Justine,
> >
> > Thanks for the KIP. Nice writeup and great results. Just one comment.
> >
> > 100. To add a record to the accumulator, the producer needs to know the
> > partition id. The decision of whether the record can be added to the
> > current batch is only made after the accumulator.append() call. So, when
> a
> > batch is full, it seems that the KIP will try to append the next record
> to
> > the same partition, which will trigger the creation of a new batch with a
> > single record. After that, new records will be routed to a new partition.
> > If the producer doesn't come back to the first partition in time, the
> > producer will send a single record batch. In the worse case, it can be
> that
> > every other batch has only a single record. Is this correct? If so, could
> > we avoid that?
> >
> > Jun
> >
> > On Thu, Jul 11, 2019 at 5:23 PM Colin McCabe  wrote:
> >
> > > Hi Justine,
> > >
> > > I agree that we shouldn't change RoundRobinPartitioner, since its
> > behavior
> > > is already specified.
> > >
> > > However, we could add a new, separate StickyRoundRobinPartitioner class
> > to
> > > KIP-480 which just implemented the sticky behavior regardless of
> whether
> > > the key was null.  That seems pretty easy to add (and it wouldn't have
> to
> > > implemented right away in the first PR, of course.)  It would be an
> > option
> > > for people who wanted to configure this behavior.
> > >
> > > best,
> > > Colin
> > >
> > >
> > > On Wed, Jul 10, 2019, at 08:48, Justine Olshan wrote:
> > > > Hi M,
> > > >
> > > > I'm a little confused by what you mean by extending the behavior on
> to
> > > the
> > > > RoundRobinPartitioner.
> > > > The sticky partitioner plans to remove the round-robin behavior from
> > > > records with no keys. Instead of sending them to each partition in
> > order,
> > > > it sends them all to the same partition until the batch is sent.
> > > > I don't think you can have both round-robin and sticky partition
> > > behavior.
> > > >
> > > > Thank you,
> > > > Justine Olshan
> > > >
> > > > On Wed, Jul 10, 2019 at 1:54 AM M. Manna  wrote:
> > > >
> > > > > Thanks for the comments Colin.
> > > > >
> > > > > My only concern is that this KIP is addressing a good feature and
> > > having
> > > > > that extended to RoundRobinPartitioner means 1 less KIP in the
> > future.
> > > > >
> > > > > Would it be appropriate to extend the support to
> > RoundRobinPartitioner
> > > too?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > On Tue, 9 Jul 2019 at 17:24, Colin McCabe 
> > wrote:
> > > > >
> > > > > > Hi M,
> > > > > >
> > > > > > The RoundRobinPartitioner added by KIP-369 doesn't interact with
> > this
> > > > > > KIP.  If you configure your producer to use
> RoundRobinPartitioner,
> > > then
> > > > > the
> > > > > > DefaultPartitioner will not be used.  And the "sticky" behavior
> is
> > > > > > implemented only in the DefaultPartitioner.
> > > > > >
> > > > > > regards,
> > > > > > Colin
> > > > > >
> > > > > >
> > > > > > On Tue, Jul 9, 2019, at 05:12, M. Manna wrote:
> > > > > > > Hello Justine,
> > > > > > >
> > > > > > > I have one item I wanted to discuss.
> > > > > > >
> > > > > > > We are currently in review stage for KAFKA- where we can
> > choose
> > > > > > always
> > > > > > > RoundRobin regardless of null/usable key.
> > > > > > >
> > > > > > > If I understood this KIP motivation correctly, you are still
> > > honouring
> > > > > > how
> > > > > > > the hashing of key works for DefaultPartitioner. Would you say
> > that
> > > > > > having
> > > > > > > an always "Round-Robin" partitioning with "Sticky" assignment
> > > > > 

[jira] [Created] (KAFKA-8662) Produce fails if a previous produce was to an unauthorized topic

[Rajini Sivaram (JIRA)]

Rajini Sivaram created KAFKA-8662:
-

 Summary: Produce fails if a previous produce was to an 
unauthorized topic
 Key: KAFKA-8662
 URL: https://issues.apache.org/jira/browse/KAFKA-8662
 Project: Kafka
  Issue Type: Bug
  Components: producer 
Affects Versions: 2.3.0
Reporter: Rajini Sivaram
Assignee: Rajini Sivaram
 Fix For: 2.4.0, 2.3.1


This is a regression introduced by the commit 
[https://github.com/apache/kafka/commit/460e46c3bb76a361d0706b263c03696005e12566|https://github.com/apache/kafka/commit/460e46c3bb76a361d0706b263c03696005e12566.].

When we produce to a topic, was add the topic to the producer's Metadata 
instance. If metadata authorization fails for the topic, we fail the send and 
propagate the authorization exception to the caller. The topic remains in the 
Metadata instance. We expire the topic and remove from Metadata after a fixed 
interval of 5 minutes. This has been the case for a while.

 

If a subsequent send is to a different authorized topic, we may still get 
metadata authorization failures for the previous unauthorized topic that is 
still in Metadata. Prior to that commit in 2.3.0, send to authorized topics 
completed successfully even if there were other unauthorized or invalid topics 
in the Metadata. Now, we propagate the exceptions without checking topic. This 
is a regression and not the expected behaviour since producer becomes unusable 
for 5 minutes unless authorization is granted to the first topic.

 



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Jenkins build is back to normal : kafka-trunk-jdk11 #695

[Apache Jenkins Server]

See 

Build failed in Jenkins: kafka-trunk-jdk8 #3786

[Apache Jenkins Server]

See 


Changes:

[jason] MINOR: Refactor admin client helpers for checking leader and ISR (#7074)

--
[...truncated 2.55 MB...]
org.apache.kafka.connect.converters.IntegerConverterTest > 
testBytesNullToNumber PASSED

org.apache.kafka.connect.converters.IntegerConverterTest > 
testSerializingIncorrectType STARTED

org.apache.kafka.connect.converters.IntegerConverterTest > 
testSerializingIncorrectType PASSED

org.apache.kafka.connect.converters.IntegerConverterTest > 
testDeserializingHeaderWithTooManyBytes STARTED

org.apache.kafka.connect.converters.IntegerConverterTest > 
testDeserializingHeaderWithTooManyBytes PASSED

org.apache.kafka.connect.converters.IntegerConverterTest > testNullToBytes 
STARTED

org.apache.kafka.connect.converters.IntegerConverterTest > testNullToBytes 
PASSED

org.apache.kafka.connect.converters.IntegerConverterTest > 
testSerializingIncorrectHeader STARTED

org.apache.kafka.connect.converters.IntegerConverterTest > 
testSerializingIncorrectHeader PASSED

org.apache.kafka.connect.converters.IntegerConverterTest > 
testDeserializingDataWithTooManyBytes STARTED

org.apache.kafka.connect.converters.IntegerConverterTest > 
testDeserializingDataWithTooManyBytes PASSED

org.apache.kafka.connect.converters.IntegerConverterTest > 
testConvertingSamplesToAndFromBytes STARTED

org.apache.kafka.connect.converters.IntegerConverterTest > 
testConvertingSamplesToAndFromBytes PASSED

org.apache.kafka.connect.converters.ByteArrayConverterTest > 
testFromConnectSchemaless STARTED

org.apache.kafka.connect.converters.ByteArrayConverterTest > 
testFromConnectSchemaless PASSED

org.apache.kafka.connect.converters.ByteArrayConverterTest > testToConnectNull 
STARTED

org.apache.kafka.connect.converters.ByteArrayConverterTest > testToConnectNull 
PASSED

org.apache.kafka.connect.converters.ByteArrayConverterTest > 
testFromConnectBadSchema STARTED

org.apache.kafka.connect.converters.ByteArrayConverterTest > 
testFromConnectBadSchema PASSED

org.apache.kafka.connect.converters.ByteArrayConverterTest > 
testFromConnectNull STARTED

org.apache.kafka.connect.converters.ByteArrayConverterTest > 
testFromConnectNull PASSED

org.apache.kafka.connect.converters.ByteArrayConverterTest > testToConnect 
STARTED

org.apache.kafka.connect.converters.ByteArrayConverterTest > testToConnect 
PASSED

org.apache.kafka.connect.converters.ByteArrayConverterTest > testFromConnect 
STARTED

org.apache.kafka.connect.converters.ByteArrayConverterTest > testFromConnect 
PASSED

org.apache.kafka.connect.converters.ByteArrayConverterTest > 
testFromConnectInvalidValue STARTED

org.apache.kafka.connect.converters.ByteArrayConverterTest > 
testFromConnectInvalidValue PASSED

org.apache.kafka.connect.converters.DoubleConverterTest > testBytesNullToNumber 
STARTED

org.apache.kafka.connect.converters.DoubleConverterTest > testBytesNullToNumber 
PASSED

org.apache.kafka.connect.converters.DoubleConverterTest > 
testSerializingIncorrectType STARTED

org.apache.kafka.connect.converters.DoubleConverterTest > 
testSerializingIncorrectType PASSED

org.apache.kafka.connect.converters.DoubleConverterTest > 
testDeserializingHeaderWithTooManyBytes STARTED

org.apache.kafka.connect.converters.DoubleConverterTest > 
testDeserializingHeaderWithTooManyBytes PASSED

org.apache.kafka.connect.converters.DoubleConverterTest > testNullToBytes 
STARTED

org.apache.kafka.connect.converters.DoubleConverterTest > testNullToBytes PASSED

org.apache.kafka.connect.converters.DoubleConverterTest > 
testSerializingIncorrectHeader STARTED

org.apache.kafka.connect.converters.DoubleConverterTest > 
testSerializingIncorrectHeader PASSED

org.apache.kafka.connect.converters.DoubleConverterTest > 
testDeserializingDataWithTooManyBytes STARTED

org.apache.kafka.connect.converters.DoubleConverterTest > 
testDeserializingDataWithTooManyBytes PASSED

org.apache.kafka.connect.converters.DoubleConverterTest > 
testConvertingSamplesToAndFromBytes STARTED

org.apache.kafka.connect.converters.DoubleConverterTest > 
testConvertingSamplesToAndFromBytes PASSED

org.apache.kafka.connect.converters.LongConverterTest > testBytesNullToNumber 
STARTED

org.apache.kafka.connect.converters.LongConverterTest > testBytesNullToNumber 
PASSED

org.apache.kafka.connect.converters.LongConverterTest > 
testSerializingIncorrectType STARTED

org.apache.kafka.connect.converters.LongConverterTest > 
testSerializingIncorrectType PASSED

org.apache.kafka.connect.converters.LongConverterTest > 
testDeserializingHeaderWithTooManyBytes STARTED

org.apache.kafka.connect.converters.LongConverterTest > 
testDeserializingHeaderWithTooManyBytes PASSED

org.apache.kafka.connect.converters.LongConverterTest > testNullToBytes STARTED

org.apache.kafka.connect.converters.LongConverterTest > testNullToBytes PASSED

org.apache.kafka.connect.converters.LongConverterTest > 

Build failed in Jenkins: kafka-trunk-jdk11 #694

[Apache Jenkins Server]

See 


Changes:

[github] MINOR: Use dynamic port in `RestServerTest` (#7079)

[jason] KAFKA-8614; Consistent naming for IncrementalAlterConfig and AlterConfig

--
[...truncated 2.55 MB...]
org.apache.kafka.streams.TopologyTest > 
tableNamedMaterializedCountShouldPreserveTopologyStructure STARTED

org.apache.kafka.streams.TopologyTest > 
tableNamedMaterializedCountShouldPreserveTopologyStructure PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullStoreNameWhenConnectingProcessorAndStateStores STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullStoreNameWhenConnectingProcessorAndStateStores PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullNameWhenAddingSourceWithTopic STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullNameWhenAddingSourceWithTopic PASSED

org.apache.kafka.streams.TopologyTest > 
sourceAndProcessorWithStateShouldHaveSingleSubtopology STARTED

org.apache.kafka.streams.TopologyTest > 
sourceAndProcessorWithStateShouldHaveSingleSubtopology PASSED

org.apache.kafka.streams.TopologyTest > shouldNotAllowNullTopicWhenAddingSink 
STARTED

org.apache.kafka.streams.TopologyTest > shouldNotAllowNullTopicWhenAddingSink 
PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowToAddGlobalStoreWithSourceNameEqualsProcessorName STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowToAddGlobalStoreWithSourceNameEqualsProcessorName PASSED

org.apache.kafka.streams.TopologyTest > 
singleSourceWithListOfTopicsShouldHaveSingleSubtopology STARTED

org.apache.kafka.streams.TopologyTest > 
singleSourceWithListOfTopicsShouldHaveSingleSubtopology PASSED

org.apache.kafka.streams.TopologyTest > 
sourceWithMultipleProcessorsShouldHaveSingleSubtopology STARTED

org.apache.kafka.streams.TopologyTest > 
sourceWithMultipleProcessorsShouldHaveSingleSubtopology PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowZeroStoreNameWhenConnectingProcessorAndStateStores STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowZeroStoreNameWhenConnectingProcessorAndStateStores PASSED

org.apache.kafka.streams.TopologyTest > 
kTableNamedMaterializedMapValuesShouldPreserveTopologyStructure STARTED

org.apache.kafka.streams.TopologyTest > 
kTableNamedMaterializedMapValuesShouldPreserveTopologyStructure PASSED

org.apache.kafka.streams.TopologyTest > 
timeWindowZeroArgCountShouldPreserveTopologyStructure STARTED

org.apache.kafka.streams.TopologyTest > 
timeWindowZeroArgCountShouldPreserveTopologyStructure PASSED

org.apache.kafka.streams.TopologyTest > 
multipleSourcesWithProcessorsShouldHaveDistinctSubtopologies STARTED

org.apache.kafka.streams.TopologyTest > 
multipleSourcesWithProcessorsShouldHaveDistinctSubtopologies PASSED

org.apache.kafka.streams.TopologyTest > shouldThrowOnUnassignedStateStoreAccess 
STARTED

org.apache.kafka.streams.TopologyTest > shouldThrowOnUnassignedStateStoreAccess 
PASSED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullProcessorNameWhenConnectingProcessorAndStateStores STARTED

org.apache.kafka.streams.TopologyTest > 
shouldNotAllowNullProcessorNameWhenConnectingProcessorAndStateStores PASSED

org.apache.kafka.streams.TopologyTest > 
kTableAnonymousMaterializedMapValuesShouldPreserveTopologyStructure STARTED

org.apache.kafka.streams.TopologyTest > 
kTableAnonymousMaterializedMapValuesShouldPreserveTopologyStructure PASSED

org.apache.kafka.streams.TopologyTest > 
sessionWindowZeroArgCountShouldPreserveTopologyStructure STARTED

org.apache.kafka.streams.TopologyTest > 
sessionWindowZeroArgCountShouldPreserveTopologyStructure PASSED

org.apache.kafka.streams.TopologyTest > 
tableAnonymousMaterializedCountShouldPreserveTopologyStructure STARTED

org.apache.kafka.streams.TopologyTest > 
tableAnonymousMaterializedCountShouldPreserveTopologyStructure PASSED

org.apache.kafka.streams.TopologyTest > shouldDescribeGlobalStoreTopology 
STARTED

org.apache.kafka.streams.TopologyTest > shouldDescribeGlobalStoreTopology PASSED

org.apache.kafka.streams.TopologyTest > 
kTableNonMaterializedMapValuesShouldPreserveTopologyStructure STARTED

org.apache.kafka.streams.TopologyTest > 
kTableNonMaterializedMapValuesShouldPreserveTopologyStructure PASSED

org.apache.kafka.streams.TopologyTest > 
kGroupedStreamAnonymousMaterializedCountShouldPreserveTopologyStructure STARTED

org.apache.kafka.streams.TopologyTest > 
kGroupedStreamAnonymousMaterializedCountShouldPreserveTopologyStructure PASSED

org.apache.kafka.streams.TopologyTest > 
shouldDescribeMultipleGlobalStoreTopology STARTED

org.apache.kafka.streams.TopologyTest > 
shouldDescribeMultipleGlobalStoreTopology PASSED

org.apache.kafka.streams.TopologyTest > shouldNotAllowToAddSinkWithSameName 
STARTED

org.apache.kafka.streams.TopologyTest > shouldNotAllowToAddSinkWithSameName 
PASSED

org.apache.kafka.streams.TopologyTest > 

Build failed in Jenkins: kafka-trunk-jdk8 #3785

[Apache Jenkins Server]

See 


Changes:

[github] MINOR: Create a new topic for each test for flaky

[matthias] HOT FIX: close RocksDB objects in correct order (#7076)

[github] MINOR: Use dynamic port in `RestServerTest` (#7079)

[jason] KAFKA-8614; Consistent naming for IncrementalAlterConfig and AlterConfig

--
[...truncated 6.51 MB...]
org.apache.kafka.tools.PushHttpMetricsReporterTest > testClientError STARTED

org.apache.kafka.tools.PushHttpMetricsReporterTest > testClientError PASSED

org.apache.kafka.tools.PushHttpMetricsReporterTest > testServerError STARTED

org.apache.kafka.tools.PushHttpMetricsReporterTest > testServerError PASSED

org.apache.kafka.tools.PushHttpMetricsReporterTest > testMetricValues STARTED

org.apache.kafka.tools.PushHttpMetricsReporterTest > testMetricValues PASSED

> Task :streams:streams-scala:test

org.apache.kafka.streams.scala.kstream.ProducedTest > Create a Produced should 
create a Produced with Serdes STARTED

org.apache.kafka.streams.scala.kstream.ProducedTest > Create a Produced should 
create a Produced with Serdes PASSED

org.apache.kafka.streams.scala.kstream.ProducedTest > Create a Produced with 
timestampExtractor and resetPolicy should create a Consumed with Serdes, 
timestampExtractor and resetPolicy STARTED

org.apache.kafka.streams.scala.kstream.ProducedTest > Create a Produced with 
timestampExtractor and resetPolicy should create a Consumed with Serdes, 
timestampExtractor and resetPolicy PASSED

org.apache.kafka.streams.scala.kstream.SuppressedTest > 
Suppressed.untilWindowCloses should produce the correct suppression STARTED

org.apache.kafka.streams.scala.kstream.SuppressedTest > 
Suppressed.untilWindowCloses should produce the correct suppression PASSED

org.apache.kafka.streams.scala.kstream.SuppressedTest > 
Suppressed.untilTimeLimit should produce the correct suppression STARTED

org.apache.kafka.streams.scala.kstream.SuppressedTest > 
Suppressed.untilTimeLimit should produce the correct suppression PASSED

org.apache.kafka.streams.scala.kstream.SuppressedTest > BufferConfig.maxRecords 
should produce the correct buffer config STARTED

org.apache.kafka.streams.scala.kstream.SuppressedTest > BufferConfig.maxRecords 
should produce the correct buffer config PASSED

org.apache.kafka.streams.scala.kstream.SuppressedTest > BufferConfig.maxBytes 
should produce the correct buffer config STARTED

org.apache.kafka.streams.scala.kstream.SuppressedTest > BufferConfig.maxBytes 
should produce the correct buffer config PASSED

org.apache.kafka.streams.scala.kstream.SuppressedTest > BufferConfig.unbounded 
should produce the correct buffer config STARTED

org.apache.kafka.streams.scala.kstream.SuppressedTest > BufferConfig.unbounded 
should produce the correct buffer config PASSED

org.apache.kafka.streams.scala.kstream.SuppressedTest > BufferConfig should 
support very long chains of factory methods STARTED

org.apache.kafka.streams.scala.kstream.SuppressedTest > BufferConfig should 
support very long chains of factory methods PASSED

org.apache.kafka.streams.scala.kstream.MaterializedTest > Create a Materialized 
should create a Materialized with Serdes STARTED

org.apache.kafka.streams.scala.kstream.MaterializedTest > Create a Materialized 
should create a Materialized with Serdes PASSED

org.apache.kafka.streams.scala.kstream.MaterializedTest > Create a Materialize 
with a store name should create a Materialized with Serdes and a store name 
STARTED

org.apache.kafka.streams.scala.kstream.MaterializedTest > Create a Materialize 
with a store name should create a Materialized with Serdes and a store name 
PASSED

org.apache.kafka.streams.scala.kstream.MaterializedTest > Create a Materialize 
with a window store supplier should create a Materialized with Serdes and a 
store supplier STARTED

org.apache.kafka.streams.scala.kstream.MaterializedTest > Create a Materialize 
with a window store supplier should create a Materialized with Serdes and a 
store supplier PASSED

org.apache.kafka.streams.scala.kstream.MaterializedTest > Create a Materialize 
with a key value store supplier should create a Materialized with Serdes and a 
store supplier STARTED

org.apache.kafka.streams.scala.kstream.MaterializedTest > Create a Materialize 
with a key value store supplier should create a Materialized with Serdes and a 
store supplier PASSED

org.apache.kafka.streams.scala.kstream.MaterializedTest > Create a Materialize 
with a session store supplier should create a Materialized with Serdes and a 
store supplier STARTED

org.apache.kafka.streams.scala.kstream.MaterializedTest > Create a Materialize 
with a session store supplier should create a Materialized with Serdes and a 
store supplier PASSED

org.apache.kafka.streams.scala.kstream.KStreamTest > filter a KStream should 
filter records satisfying the predicate STARTED

org.apache.kafka.streams.scala.kstream.KStreamTest >