Re: Hi / Requesting permission to create KIP

[Kowshik Prakasam]

Hi Guozhang,

Yes, it was granted earlier today. Thank you!


Cheers,
Kowshik


On Fri, Mar 20, 2020 at 8:59 PM Guozhang Wang  wrote:

> Hello Kowshik,
>
> I saw your username is granted the permission already on wiki.
>
> Cheers,
> Guozhang
>
>
> On Fri, Mar 20, 2020 at 5:13 PM Kowshik Prakasam 
> wrote:
>
> > Hi,
> >
> > I'm looking for permission to create a KIP under
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > .
> > My username is 'kprakasam'. Could you please grant me access?
> >
> >
> > Cheers,
> > Kowshik
> >
>
>
> --
> -- Guozhang
>

[jira] [Resolved] (KAFKA-9701) Consumer could catch InconsistentGroupProtocolException during rebalance

[Guozhang Wang (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guozhang Wang resolved KAFKA-9701.
--
Fix Version/s: 2.5.0
   Resolution: Fixed

> Consumer could catch InconsistentGroupProtocolException during rebalance
> 
>
> Key: KAFKA-9701
> URL: https://issues.apache.org/jira/browse/KAFKA-9701
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 2.5.0
>Reporter: Boyang Chen
>Assignee: Boyang Chen
>Priority: Blocker
> Fix For: 2.5.0
>
> Attachments: cluster.log
>
>
> The bug was due to an out-of-order handling of the SyncGroupRequest after the 
> LeaveGroupRequest.
> The sequence of events are:
>  # The stream thread tries to rejoin the group during runOnce#poll
>  # The join group call was successful and group was waiting for sync group 
> result
>  # Outside the poll, task producer hits FencedException, triggering a 
> partition lost
>  # Stream thread unsubscribes and sends out an leave group, and gets the 
> local generation wipe out 
>  # The sync group response was processed. Although it is legitimate, the 
> local protocol type becomes null in this case
>  # The sync group response hits the protocol type mismatch fatal exception
>  
> [2020-03-20T*10:40:08-07:00*] 
> (streams-soak-trunk-eos_soak_i-01629239fa39901b4_streamslog) [2020-03-20 
> 17:40:08,754] INFO 
> [stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1] 
> [Consumer 
> clientId=stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1-consumer,
>  groupId=stream-soak-test] (Re-)joining group 
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
> [2020-03-20T*10:40:11-07:00*] 
> (streams-soak-trunk-eos_soak_i-01629239fa39901b4_streamslog) [2020-03-20 
> 17:40:11,152] ERROR [kafka-producer-network-thread | 
> stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1-0_1-producer]
>  stream-thread 
> [stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1] task 
> [0_1] Error encountered sending record to topic network-id-repartition for 
> task 0_1 due to:
> [2020-03-20T10:40:11-07:00] 
> (streams-soak-trunk-eos_soak_i-01629239fa39901b4_streamslog) 
> org.apache.kafka.common.errors.ProducerFencedException: Producer attempted an 
> operation with an old epoch. Either there is a newer producer with the same 
> transactionalId, or the producer's transaction has been expired by the broker.
> [2020-03-20T10:40:12-07:00] 
> (streams-soak-trunk-eos_soak_i-01629239fa39901b4_streamslog) [2020-03-20 
> 17:40:12,048] INFO 
> [stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1] 
> stream-thread 
> [stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1] at 
> state RUNNING: partitions [logs.json.kafka-1, node-name-repartition-1, 
> logs.json.zookeeper-1, logs.kubernetes-1, windowed-node-counts-1, 
> logs.operator-1, logs.syslog-1] lost due to missed rebalance.
>         lost active tasks: []
>         lost assigned standby tasks: []
>  (org.apache.kafka.streams.processor.internals.StreamThread)
>  
> [2020-03-20T*10:40:12-07:00*] 
> (streams-soak-trunk-eos_soak_i-01629239fa39901b4_streamslog) [2020-03-20 
> 17:40:12,048] INFO 
> [stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1] 
> [Consumer 
> clientId=stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1-consumer,
>  groupId=stream-soak-test] Member 
> stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1-consumer-34c2198b-5bdd-470b-ae50-30a39873edab
>  sending LeaveGroup request to coordinator 
> ip-172-31-18-29.us-west-2.compute.internal:9092 (id: 2147482644 rack: null) 
> due to the consumer *unsubscribed from all topics* 
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
> [2020-03-20T10:40:12-07:00] 
> (streams-soak-trunk-eos_soak_i-01629239fa39901b4_streamslog) [2020-03-20 
> 17:40:12,048] INFO 
> [stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1] 
> [Consumer 
> clientId=stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1-consumer,
>  groupId=stream-soak-test] Unsubscribed all topics or patterns and assigned 
> partitions (org.apache.kafka.clients.consumer.KafkaConsumer)
> [2020-03-20T10:40:17-07:00] 
> (streams-soak-trunk-eos_soak_i-01629239fa39901b4_streamslog) [2020-03-20 
> 17:40:16,972] ERROR 
> [stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1] 
> [Consumer 
> clientId=stream-soak-test-f7392d33-55d7-484f-8b72-578e22fead96-StreamThread-1-consumer,
>  groupId=stream-soak-test] SyncGroup failed due to inconsistent Protocol 
> Name, received stream but expected null 
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
> [2020-03-20T10:40:17-07:00] 
> 

Re: Hi / Requesting permission to create KIP

[Guozhang Wang]

Hello Kowshik,

I saw your username is granted the permission already on wiki.

Cheers,
Guozhang


On Fri, Mar 20, 2020 at 5:13 PM Kowshik Prakasam 
wrote:

> Hi,
>
> I'm looking for permission to create a KIP under
>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> .
> My username is 'kprakasam'. Could you please grant me access?
>
>
> Cheers,
> Kowshik
>


-- 
-- Guozhang

Jenkins build is back to normal : kafka-trunk-jdk11 #1268

[Apache Jenkins Server]

See 

Build failed in Jenkins: kafka-2.5-jdk8 #69

[Apache Jenkins Server]

See 


Changes:

[konstantine] MINOR: Use Exit.exit instead of System.exit in MM2 (#8321)


--
[...truncated 2.91 MB...]

org.apache.kafka.streams.MockProcessorContextTest > shouldCapturePunctuator 
PASSED

org.apache.kafka.streams.TestTopicsTest > testNonUsedOutputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonUsedOutputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testEmptyTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testEmptyTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testStartTimestamp STARTED

org.apache.kafka.streams.TestTopicsTest > testStartTimestamp PASSED

org.apache.kafka.streams.TestTopicsTest > testNegativeAdvance STARTED

org.apache.kafka.streams.TestTopicsTest > testNegativeAdvance PASSED

org.apache.kafka.streams.TestTopicsTest > shouldNotAllowToCreateWithNullDriver 
STARTED

org.apache.kafka.streams.TestTopicsTest > shouldNotAllowToCreateWithNullDriver 
PASSED

org.apache.kafka.streams.TestTopicsTest > testDuration STARTED

org.apache.kafka.streams.TestTopicsTest > testDuration PASSED

org.apache.kafka.streams.TestTopicsTest > testOutputToString STARTED

org.apache.kafka.streams.TestTopicsTest > testOutputToString PASSED

org.apache.kafka.streams.TestTopicsTest > testValue STARTED

org.apache.kafka.streams.TestTopicsTest > testValue PASSED

org.apache.kafka.streams.TestTopicsTest > testTimestampAutoAdvance STARTED

org.apache.kafka.streams.TestTopicsTest > testTimestampAutoAdvance PASSED

org.apache.kafka.streams.TestTopicsTest > testOutputWrongSerde STARTED

org.apache.kafka.streams.TestTopicsTest > testOutputWrongSerde PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputTopicWithNullTopicName STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputTopicWithNullTopicName PASSED

org.apache.kafka.streams.TestTopicsTest > testWrongSerde STARTED

org.apache.kafka.streams.TestTopicsTest > testWrongSerde PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMapWithNull STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMapWithNull PASSED

org.apache.kafka.streams.TestTopicsTest > testNonExistingOutputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonExistingOutputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testMultipleTopics STARTED

org.apache.kafka.streams.TestTopicsTest > testMultipleTopics PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValueList STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValueList PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputWithNullDriver STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputWithNullDriver PASSED

org.apache.kafka.streams.TestTopicsTest > testValueList STARTED

org.apache.kafka.streams.TestTopicsTest > testValueList PASSED

org.apache.kafka.streams.TestTopicsTest > testRecordList STARTED

org.apache.kafka.streams.TestTopicsTest > testRecordList PASSED

org.apache.kafka.streams.TestTopicsTest > testNonExistingInputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonExistingInputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMap STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMap PASSED

org.apache.kafka.streams.TestTopicsTest > testRecordsToList STARTED

org.apache.kafka.streams.TestTopicsTest > testRecordsToList PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValueListDuration STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValueListDuration PASSED

org.apache.kafka.streams.TestTopicsTest > testInputToString STARTED

org.apache.kafka.streams.TestTopicsTest > testInputToString PASSED

org.apache.kafka.streams.TestTopicsTest > testTimestamp STARTED

org.apache.kafka.streams.TestTopicsTest > testTimestamp PASSED

org.apache.kafka.streams.TestTopicsTest > testWithHeaders STARTED

org.apache.kafka.streams.TestTopicsTest > testWithHeaders PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValue STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValue PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateTopicWithNullTopicName STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateTopicWithNullTopicName PASSED

> Task :streams:upgrade-system-tests-0100:compileJava NO-SOURCE
> Task :streams:upgrade-system-tests-0100:processResources NO-SOURCE
> Task :streams:upgrade-system-tests-0100:classes UP-TO-DATE
> Task :streams:upgrade-system-tests-0100:checkstyleMain NO-SOURCE
> Task :streams:upgrade-system-tests-0100:compileTestJava
> Task :streams:upgrade-system-tests-0100:processTestResources NO-SOURCE
> Task :streams:upgrade-system-tests-0100:testClasses
> Task :streams:upgrade-system-tests-0100:checkstyleTest
> Task :streams:upgrade-system-tests-0100:spotbugsMain NO-SOURCE
> Task 

Build failed in Jenkins: kafka-trunk-jdk8 #4347

[Apache Jenkins Server]

See 


Changes:

[github] KAFKA-9727: cleanup the state store for standby task dirty close and


--
[...truncated 5.91 MB...]

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForSmallerValue[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] PASSED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis STARTED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis PASSED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime STARTED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime PASSED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep PASSED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnIsOpen 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnIsOpen 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnName 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnName 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWindowStartTimestampWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWindowStartTimestampWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldReturnIsPersistent STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldReturnIsPersistent PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardClose 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardClose 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardFlush 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardFlush 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
PASSED

org.apache.kafka.streams.test.TestRecordTest > testConsumerRecord STARTED

org.apache.kafka.streams.test.TestRecordTest > testConsumerRecord PASSED

org.apache.kafka.streams.test.TestRecordTest > testToString STARTED

org.apache.kafka.streams.test.TestRecordTest > testToString PASSED

org.apache.kafka.streams.test.TestRecordTest > testInvalidRecords STARTED

org.apache.kafka.streams.test.TestRecordTest > testInvalidRecords PASSED

org.apache.kafka.streams.test.TestRecordTest > testPartialConstructorEquals 
STARTED

org.apache.kafka.streams.test.TestRecordTest > testPartialConstructorEquals 
PASSED

org.apache.kafka.streams.test.TestRecordTest > testMultiFieldMatcher STARTED

org.apache.kafka.streams.test.TestRecordTest > testMultiFieldMatcher PASSED

org.apache.kafka.streams.test.TestRecordTest > testFields STARTED

org.apache.kafka.streams.test.TestRecordTest > testFields PASSED

org.apache.kafka.streams.test.TestRecordTest > testProducerRecord STARTED

org.apache.kafka.streams.test.TestRecordTest > testProducerRecord PASSED

org.apache.kafka.streams.test.TestRecordTest > testEqualsAndHashCode STARTED

org.apache.kafka.streams.test.TestRecordTest > testEqualsAndHashCode PASSED

> Task :streams:upgrade-system-tests-0100:compileJava NO-SOURCE
> Task :streams:upgrade-system-tests-0100:processResources NO-SOURCE
> Task :streams:upgrade-system-tests-0100:classes UP-TO-DATE
> Task :streams:upgrade-system-tests-0100:checkstyleMain NO-SOURCE
> Task :streams:upgrade-system-tests-0100:compileTestJava
> Task :streams:upgrade-system-tests-0100:processTestResources NO-SOURCE
> Task :streams:upgrade-system-tests-0100:testClasses
> Task :streams:upgrade-system-tests-0100:checkstyleTest
> Task :streams:upgrade-system-tests-0100:spotbugsMain NO-SOURCE
> Task :streams:upgrade-system-tests-0100:test
> Task :streams:upgrade-system-tests-0101:compileJava NO-SOURCE
> Task :streams:upgrade-system-tests-0101:processResources NO-SOURCE
> Task :streams:upgrade-system-tests-0101:classes UP-TO-DATE
> Task :streams:upgrade-system-tests-0101:checkstyleMain 

Re: [VOTE] 2.5.0 RC2

[Ismael Juma]

Hi Boyang,

Is this a regression?

Ismael

On Fri, Mar 20, 2020, 5:43 PM Boyang Chen 
wrote:

> Hey David,
>
> I would like to raise https://issues.apache.org/jira/browse/KAFKA-9701 as
> a
> 2.5 blocker. The impact of this bug is that it could throw fatal exception
> and kill a stream thread on Kafka Streams level. It could also create a
> crashing scenario for plain Kafka Consumer users as well as the exception
> will be thrown all the way up.
>
> Let me know your thoughts.
>
> Boyang
>
> On Tue, Mar 17, 2020 at 8:10 AM David Arthur  wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the third candidate for release of Apache Kafka 2.5.0.
> >
> > * TLS 1.3 support (1.2 is now the default)
> > * Co-groups for Kafka Streams
> > * Incremental rebalance for Kafka Consumer
> > * New metrics for better operational insight
> > * Upgrade Zookeeper to 3.5.7
> > * Deprecate support for Scala 2.11
> >
> >
> >  Release notes for the 2.5.0 release:
> > https://home.apache.org/~davidarthur/kafka-2.5.0-rc2/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by Tuesday March 24, 2020 by 5pm PT.
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://home.apache.org/~davidarthur/kafka-2.5.0-rc2/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://home.apache.org/~davidarthur/kafka-2.5.0-rc2/javadoc/
> >
> > * Tag to be voted upon (off 2.5 branch) is the 2.5.0 tag:
> > https://github.com/apache/kafka/releases/tag/2.5.0-rc2
> >
> > * Documentation:
> > https://kafka.apache.org/25/documentation.html
> >
> > * Protocol:
> > https://kafka.apache.org/25/protocol.html
> >
> >
> > I'm thrilled to be able to include links to both build jobs with
> successful
> > builds! Thanks to everyone who has helped reduce our flaky test exposure
> > these past few weeks :)
> >
> > * Successful Jenkins builds for the 2.5 branch:
> > Unit/integration tests: https://builds.apache.org/job/kafka-2.5-jdk8/64/
> > System tests:
> > https://jenkins.confluent.io/job/system-test-kafka/job/2.5/42/
> >
> > --
> > David Arthur
> >
>

Consumer Rebalancing Issues

[Ravi Kanth]

Hi All,

I have a Kafka Consumer that polls the data and gets *paused* for 15-20
mins for the post-processing of the polled records. However, during the
time of pause, the broker assumes that the consumer group is dead(check
below log) and rebalances the consumer group.

*Application Log:*
k8s-worker403: 2020-03-20 21:41:14.147 [kafka-coordinator-heartbeat-thread
| ] INFO  o.a.k.c.c.i.AbstractCoordinator - [Consumer
clientId=consumer-1, groupId=] Marking the coordinator
:9093 (id: 2147483480 rack: null) dead

*Broker Log:*
Member consumer-1-4eb939a6-c37d-4853-96dc-4255ec8663ff in group  has failed, removing it from the group

When rebalancing happens, consumer is automatically *resumed* and starts
polling for more records though I am expecting it to be paused. This is
causing unexpected behavior.

When reading through Kafka docs, it looks like tuning up
request.timeout.ms/session.timeout.ms will help avoid running into this
problem. Please correct if my understanding is incorrect. However, I am not
willing to tweak the configurations, instead handle the problem at the code
level.

*Question:*
1. Is this a bug in Kafka Java API letting the consumer be resumed after
rebalancing even when it is initially paused? If so, details of the bug
would be helpful.
2. Is there anyother way to avoid running into this problem without
tweaking configurations?

*Kafka Version Using:*
*1.0.1*

Any suggestions would be of great help.

Thanks,
Ravi

Build failed in Jenkins: kafka-2.4-jdk8 #170

[Apache Jenkins Server]

See 


Changes:

[jason] KAFKA-9654; Update epoch in `ReplicaAlterLogDirsThread` after new


--
[...truncated 2.76 MB...]

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairsAndCustomTimestamps
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairsAndCustomTimestamps
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairs 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicNameAndTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicNameAndTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairsAndCustomTimestamps
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairsAndCustomTimestamps
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairsWithCustomTimestampAndIncrementsAndNotAdvanceTime
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairsWithCustomTimestampAndIncrementsAndNotAdvanceTime
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithTimestampWithTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithTimestampWithTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKeyAndDefaultTimestamp
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKeyAndDefaultTimestamp
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairsWithTimestampAndIncrements STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairsWithTimestampAndIncrements PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnIsOpen 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnIsOpen 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldDeleteAndReturnPlainValue STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldDeleteAndReturnPlainValue PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnName 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldReturnName 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutAllWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutAllWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldReturnIsPersistent STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldReturnIsPersistent PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutIfAbsentWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > 
shouldPutIfAbsentWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardClose 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardClose 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardFlush 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardFlush 
PASSED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardInit 
STARTED

org.apache.kafka.streams.internals.KeyValueStoreFacadeTest > shouldForwardInit 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnIsOpen 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnIsOpen 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnName 
STARTED

[jira] [Created] (KAFKA-9741) ConsumerCoordinator must update ConsumerGroupMetadata before calling onPartitionsRevoked()

[Matthias J. Sax (Jira)]

Matthias J. Sax created KAFKA-9741:
--

 Summary: ConsumerCoordinator must update ConsumerGroupMetadata 
before calling onPartitionsRevoked()
 Key: KAFKA-9741
 URL: https://issues.apache.org/jira/browse/KAFKA-9741
 Project: Kafka
  Issue Type: Bug
  Components: consumer
Affects Versions: 2.5.0
Reporter: Matthias J. Sax


If partitions are revoked, an application may want to commit the current 
offsets.

Using transactions, committing offsets would be done via the producer passing 
in the current `ConsumerGroupMetadata`. If the metadata is not updates before 
the callback, the call to `commitTransaction(...)` fails as and old 
generationId would be used.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] 2.5.0 RC2

[Boyang Chen]

Hey David,

I would like to raise https://issues.apache.org/jira/browse/KAFKA-9701 as a
2.5 blocker. The impact of this bug is that it could throw fatal exception
and kill a stream thread on Kafka Streams level. It could also create a
crashing scenario for plain Kafka Consumer users as well as the exception
will be thrown all the way up.

Let me know your thoughts.

Boyang

On Tue, Mar 17, 2020 at 8:10 AM David Arthur  wrote:

> Hello Kafka users, developers and client-developers,
>
> This is the third candidate for release of Apache Kafka 2.5.0.
>
> * TLS 1.3 support (1.2 is now the default)
> * Co-groups for Kafka Streams
> * Incremental rebalance for Kafka Consumer
> * New metrics for better operational insight
> * Upgrade Zookeeper to 3.5.7
> * Deprecate support for Scala 2.11
>
>
>  Release notes for the 2.5.0 release:
> https://home.apache.org/~davidarthur/kafka-2.5.0-rc2/RELEASE_NOTES.html
>
> *** Please download, test and vote by Tuesday March 24, 2020 by 5pm PT.
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~davidarthur/kafka-2.5.0-rc2/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~davidarthur/kafka-2.5.0-rc2/javadoc/
>
> * Tag to be voted upon (off 2.5 branch) is the 2.5.0 tag:
> https://github.com/apache/kafka/releases/tag/2.5.0-rc2
>
> * Documentation:
> https://kafka.apache.org/25/documentation.html
>
> * Protocol:
> https://kafka.apache.org/25/protocol.html
>
>
> I'm thrilled to be able to include links to both build jobs with successful
> builds! Thanks to everyone who has helped reduce our flaky test exposure
> these past few weeks :)
>
> * Successful Jenkins builds for the 2.5 branch:
> Unit/integration tests: https://builds.apache.org/job/kafka-2.5-jdk8/64/
> System tests:
> https://jenkins.confluent.io/job/system-test-kafka/job/2.5/42/
>
> --
> David Arthur
>

Hi / Requesting permission to create KIP

[Kowshik Prakasam]

Hi,

I'm looking for permission to create a KIP under
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals .
My username is 'kprakasam'. Could you please grant me access?


Cheers,
Kowshik

Build failed in Jenkins: kafka-trunk-jdk11 #1267

[Apache Jenkins Server]

See 


Changes:

[github] KAFKA-6145: Add constrained balanced assignment algorithm (#8262)


--
[...truncated 5.95 MB...]
org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardFlush 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
PASSED

org.apache.kafka.streams.TestTopicsTest > testNonUsedOutputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonUsedOutputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testEmptyTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testEmptyTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testStartTimestamp STARTED

org.apache.kafka.streams.TestTopicsTest > testStartTimestamp PASSED

org.apache.kafka.streams.TestTopicsTest > testNegativeAdvance STARTED

org.apache.kafka.streams.TestTopicsTest > testNegativeAdvance PASSED

org.apache.kafka.streams.TestTopicsTest > shouldNotAllowToCreateWithNullDriver 
STARTED

org.apache.kafka.streams.TestTopicsTest > shouldNotAllowToCreateWithNullDriver 
PASSED

org.apache.kafka.streams.TestTopicsTest > testDuration STARTED

org.apache.kafka.streams.TestTopicsTest > testDuration PASSED

org.apache.kafka.streams.TestTopicsTest > testOutputToString STARTED

org.apache.kafka.streams.TestTopicsTest > testOutputToString PASSED

org.apache.kafka.streams.TestTopicsTest > testValue STARTED

org.apache.kafka.streams.TestTopicsTest > testValue PASSED

org.apache.kafka.streams.TestTopicsTest > testTimestampAutoAdvance STARTED

org.apache.kafka.streams.TestTopicsTest > testTimestampAutoAdvance PASSED

org.apache.kafka.streams.TestTopicsTest > testOutputWrongSerde STARTED

org.apache.kafka.streams.TestTopicsTest > testOutputWrongSerde PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputTopicWithNullTopicName STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputTopicWithNullTopicName PASSED

org.apache.kafka.streams.TestTopicsTest > testWrongSerde STARTED

org.apache.kafka.streams.TestTopicsTest > testWrongSerde PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMapWithNull STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMapWithNull PASSED

org.apache.kafka.streams.TestTopicsTest > testNonExistingOutputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonExistingOutputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testMultipleTopics STARTED

org.apache.kafka.streams.TestTopicsTest > testMultipleTopics PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValueList STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValueList PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputWithNullDriver STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputWithNullDriver PASSED

org.apache.kafka.streams.TestTopicsTest > testValueList STARTED

org.apache.kafka.streams.TestTopicsTest > testValueList PASSED

org.apache.kafka.streams.TestTopicsTest > testRecordList STARTED

org.apache.kafka.streams.TestTopicsTest > testRecordList PASSED

org.apache.kafka.streams.TestTopicsTest > testNonExistingInputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonExistingInputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMap STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMap PASSED

org.apache.kafka.streams.TestTopicsTest > testRecordsToList STARTED

org.apache.kafka.streams.TestTopicsTest > testRecordsToList PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValueListDuration STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValueListDuration PASSED

org.apache.kafka.streams.TestTopicsTest > testInputToString STARTED

org.apache.kafka.streams.TestTopicsTest > testInputToString PASSED

org.apache.kafka.streams.TestTopicsTest > testTimestamp STARTED

org.apache.kafka.streams.TestTopicsTest > testTimestamp PASSED

org.apache.kafka.streams.TestTopicsTest > testWithHeaders STARTED

org.apache.kafka.streams.TestTopicsTest > testWithHeaders PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValue STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValue PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateTopicWithNullTopicName STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateTopicWithNullTopicName PASSED

> Task :streams:upgrade-system-tests-0100:compileJava NO-SOURCE
> Task :streams:upgrade-system-tests-0100:processResources NO-SOURCE
> Task :streams:upgrade-system-tests-0100:classes UP-TO-DATE
> Task :streams:upgrade-system-tests-0100:checkstyleMain NO-SOURCE
> Task :streams:upgrade-system-tests-0100:compileTestJava
> Task :streams:upgrade-system-tests-0100:processTestResources NO-SOURCE
> Task 

[jira] [Reopened] (KAFKA-7509) Kafka Connect logs unnecessary warnings about unused configurations

[Randall Hauch (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Randall Hauch reopened KAFKA-7509:
--

> Kafka Connect logs unnecessary warnings about unused configurations
> ---
>
> Key: KAFKA-7509
> URL: https://issues.apache.org/jira/browse/KAFKA-7509
> Project: Kafka
>  Issue Type: Improvement
>  Components: clients, KafkaConnect
>Affects Versions: 0.10.2.0
>Reporter: Randall Hauch
>Assignee: Randall Hauch
>Priority: Major
>
> When running Connect, the logs contain quite a few warnings about "The 
> configuration '{}' was supplied but isn't a known config." This occurs when 
> Connect creates producers, consumers, and admin clients, because the 
> AbstractConfig is logging unused configuration properties upon construction. 
> It's complicated by the fact that the Producer, Consumer, and AdminClient all 
> create their own AbstractConfig instances within the constructor, so we can't 
> even call its {{ignore(String key)}} method.
> See also KAFKA-6793 for a similar issue with Streams.
> There are no arguments in the Producer, Consumer, or AdminClient constructors 
> to control  whether the configs log these warnings, so a simpler workaround 
> is to only pass those configuration properties to the Producer, Consumer, and 
> AdminClient that the ProducerConfig, ConsumerConfig, and AdminClientConfig 
> configdefs know about.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-9727) Flaky system test StreamsEOSTest.test_failure_and_recovery

[Boyang Chen (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-9727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Boyang Chen resolved KAFKA-9727.

Resolution: Fixed

> Flaky system test StreamsEOSTest.test_failure_and_recovery
> --
>
> Key: KAFKA-9727
> URL: https://issues.apache.org/jira/browse/KAFKA-9727
> Project: Kafka
>  Issue Type: Bug
>  Components: streams, system tests
>Reporter: Boyang Chen
>Assignee: Boyang Chen
>Priority: Major
>
> Hits no lock available exceptions sometime after task revive:
>  
> [2020-03-13 05:40:50,224] ERROR stream-thread 
> [EosTest-46de8ee5-82a1-4bd4-af23-f4acd8515f0f-StreamThread-1] Encountered the 
> following exception during processing and the thread is going to shut down:  
> (org.apache.kafka.streams.processor.internals.StreamThread)
> org.apache.kafka.streams.errors.ProcessorStateException: Error opening store 
> KSTREAM-AGGREGATE-STATE-STORE-03 at location 
> /mnt/streams/EosTest/0_0/rocksdb/KSTREAM-AGGREGATE-STATE-STORE-03
>         at 
> org.apache.kafka.streams.state.internals.RocksDBTimestampedStore.openRocksDB(RocksDBTimestampedStore.java:87)
>         at 
> org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:191)
>         at 
> org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:230)
>         at 
> org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:48)
>         at 
> org.apache.kafka.streams.state.internals.ChangeLoggingKeyValueBytesStore.init(ChangeLoggingKeyValueBytesStore.java:44)
>         at 
> org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:48)
>         at 
> org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$init$0(MeteredKeyValueStore.java:101)
>         at 
> org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:806)
>         at 
> org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:101)
>         at 
> org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:81)
>         at 
> org.apache.kafka.streams.processor.internals.StandbyTask.initializeIfNeeded(StandbyTask.java:86)
>         at 
> org.apache.kafka.streams.processor.internals.TaskManager.tryToCompleteRestoration(TaskManager.java:275)
>  
>         at 
> org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:583)
>         at 
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:498)
>         at 
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:472)
> Caused by: org.rocksdb.RocksDBException: lock : 
> /mnt/streams/EosTest/0_0/rocksdb/KSTREAM-AGGREGATE-STATE-STORE-03/LOCK:
>  No locks available
>         at org.rocksdb.RocksDB.open(Native Method)
>         at org.rocksdb.RocksDB.open(RocksDB.java:286)
>         at 
> org.apache.kafka.streams.state.internals.RocksDBTimestampedStore.openRocksDB(RocksDBTimestampedStore.java:75)
>         ... 14 more



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Hi / Requesting permission to create KIP

[Kowshik Prakasam]

Thanks a lot, Matthias!


Cheers,
Kowshik


On Fri, Mar 20, 2020 at 3:55 PM Matthias J. Sax  wrote:

> Done.
>
> On 3/20/20 3:52 PM, Kowshik Prakasam wrote:
> > Hey everyone,
> >
> > I'm looking for permission to create a KIP under
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> .
> > My username is 'kprakasam'. If you are an admin, and you are reading this
> > email, could you please grant me access? Thank you.
> >
> >
> > Cheers,
> > Kowshik
> >
>
>

Re: Hi / Requesting permission to create KIP

[Matthias J. Sax]

Done.

On 3/20/20 3:52 PM, Kowshik Prakasam wrote:
> Hey everyone,
> 
> I'm looking for permission to create a KIP under
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals 
> .
> My username is 'kprakasam'. If you are an admin, and you are reading this
> email, could you please grant me access? Thank you.
> 
> 
> Cheers,
> Kowshik
> 



signature.asc
Description: OpenPGP digital signature

Hi / Requesting permission to create KIP

[Kowshik Prakasam]

Hey everyone,

I'm looking for permission to create a KIP under
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals .
My username is 'kprakasam'. If you are an admin, and you are reading this
email, could you please grant me access? Thank you.


Cheers,
Kowshik

Jenkins build is back to normal : kafka-trunk-jdk8 #4346

[Apache Jenkins Server]

See 

[jira] [Created] (KAFKA-9740) Add a "continue" option for Kafka Connect error handling

[Zihan Li (Jira)]

Zihan Li created KAFKA-9740:
---

 Summary: Add a "continue" option for Kafka Connect error handling
 Key: KAFKA-9740
 URL: https://issues.apache.org/jira/browse/KAFKA-9740
 Project: Kafka
  Issue Type: Improvement
Reporter: Zihan Li


Currently there are two error handling options in Kafka Connect, "none" and 
"all". Option "none" will config the connector to fail fast, and option "all" 
will ignore broken records.

If users want to store their broken records, they have to config a broken 
record queue, which is too much work for them in some cases. 

Some sink connectors have the ability to deal with broken records, for example, 
a JDBC sink connector can store the broken raw bytes into a separate table, a 
S3 connector can store that in a zipped file.

Therefore, it would be idea if Kafka Connect provides an additional option that 
sends the broken raw bytes to SinkTask directly. 

SinkTask is then responsible for handling the unparsed bytes input.

The benefits of having this additional option are:
 * Being user friendly. Connectors can handle broken record and hide that from 
clients.
 * Providing more flexibility to SinkTask in terms of broken record handling.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (KAFKA-9654) ReplicaAlterLogDirsThread can't be created again if the previous ReplicaAlterLogDirsThreadmeet encounters leader epoch error

[Jason Gustafson (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-9654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Gustafson resolved KAFKA-9654.

Fix Version/s: 2.5.1
   2.4.2
   Resolution: Fixed

> ReplicaAlterLogDirsThread can't be created again if the previous 
> ReplicaAlterLogDirsThreadmeet encounters leader epoch error
> 
>
> Key: KAFKA-9654
> URL: https://issues.apache.org/jira/browse/KAFKA-9654
> Project: Kafka
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.4.2, 2.5.1
>
>
> ReplicaManager does create ReplicaAlterLogDirsThread only if an new future 
> log is created. If the previous ReplicaAlterLogDirsThread encounters error 
> when moving data, the target partition is moved to "failedPartitions" and 
> ReplicaAlterLogDirsThread get idle due to empty partitions. The future log is 
> still existent so we CAN'T either create another ReplicaAlterLogDirsThread to 
> handle the parition or update the paritions of the idler 
> ReplicaAlterLogDirsThread.
> ReplicaManager should call ReplicaAlterLogDirsManager#addFetcherForPartitions 
> even if there is already a future log since we can create an new 
> ReplicaAlterLogDirsThread to handle the new partitions or update the 
> partitions of existent ReplicaAlterLogDirsThread to make it busy again.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Jenkins build is back to normal : kafka-trunk-jdk11 #1266

[Apache Jenkins Server]

See 

Build failed in Jenkins: kafka-trunk-jdk8 #4345

[Apache Jenkins Server]

See 


Changes:

[github] KAFKA-9734: Fix IllegalState in Streams transit to standby (#8319)


--
[...truncated 2.91 MB...]

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldNotUpdateStoreForSmallerValue[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldCreateStateDirectoryForStatefulTopology[Eos enabled = false] PASSED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] STARTED

org.apache.kafka.streams.TopologyTestDriverTest > 
shouldPunctuateIfWallClockTimeAdvances[Eos enabled = false] PASSED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis STARTED

org.apache.kafka.streams.MockTimeTest > shouldGetNanosAsMillis PASSED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime STARTED

org.apache.kafka.streams.MockTimeTest > shouldSetStartTime PASSED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldNotAllowNegativeSleep PASSED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep STARTED

org.apache.kafka.streams.MockTimeTest > shouldAdvanceTimeOnSleep PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnIsOpen 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnIsOpen 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnName 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldReturnName 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWindowStartTimestampWithUnknownTimestamp STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldPutWindowStartTimestampWithUnknownTimestamp PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldReturnIsPersistent STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > 
shouldReturnIsPersistent PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardClose 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardClose 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardFlush 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardFlush 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
PASSED

org.apache.kafka.streams.test.TestRecordTest > testConsumerRecord STARTED

org.apache.kafka.streams.test.TestRecordTest > testConsumerRecord PASSED

org.apache.kafka.streams.test.TestRecordTest > testToString STARTED

org.apache.kafka.streams.test.TestRecordTest > testToString PASSED

org.apache.kafka.streams.test.TestRecordTest > testInvalidRecords STARTED

org.apache.kafka.streams.test.TestRecordTest > testInvalidRecords PASSED

org.apache.kafka.streams.test.TestRecordTest > testPartialConstructorEquals 
STARTED

org.apache.kafka.streams.test.TestRecordTest > testPartialConstructorEquals 
PASSED

org.apache.kafka.streams.test.TestRecordTest > testMultiFieldMatcher STARTED

org.apache.kafka.streams.test.TestRecordTest > testMultiFieldMatcher PASSED

org.apache.kafka.streams.test.TestRecordTest > testFields STARTED

org.apache.kafka.streams.test.TestRecordTest > testFields PASSED

org.apache.kafka.streams.test.TestRecordTest > testProducerRecord STARTED

org.apache.kafka.streams.test.TestRecordTest > testProducerRecord PASSED

org.apache.kafka.streams.test.TestRecordTest > testEqualsAndHashCode STARTED

org.apache.kafka.streams.test.TestRecordTest > testEqualsAndHashCode PASSED

> Task :streams:upgrade-system-tests-0100:compileJava NO-SOURCE
> Task :streams:upgrade-system-tests-0100:processResources NO-SOURCE
> Task :streams:upgrade-system-tests-0100:classes UP-TO-DATE
> Task :streams:upgrade-system-tests-0100:checkstyleMain NO-SOURCE
> Task :streams:upgrade-system-tests-0100:compileTestJava
> Task :streams:upgrade-system-tests-0100:processTestResources NO-SOURCE
> Task :streams:upgrade-system-tests-0100:testClasses
> Task :streams:upgrade-system-tests-0100:checkstyleTest
> Task :streams:upgrade-system-tests-0100:spotbugsMain NO-SOURCE
> Task :streams:upgrade-system-tests-0100:test
> Task :streams:upgrade-system-tests-0101:compileJava NO-SOURCE
> Task :streams:upgrade-system-tests-0101:processResources NO-SOURCE
> Task :streams:upgrade-system-tests-0101:classes UP-TO-DATE
> Task :streams:upgrade-system-tests-0101:checkstyleMain NO-SOURCE

Re: JMX expired topic metircs

[Shailesh Panwar]

Believe this is already fixed
https://issues.apache.org/jira/browse/KAFKA-3572

On Thu, Mar 19, 2020 at 9:27 PM 张祥  wrote:

> Hi,
>
> I notice that there are jmx metrics for deleted topics when using java code
> and jmxterm. Has anyone also run into this ? If yes, what is the reason
> behind this and how can I filter expired metrics ? Thanks.
>

[jira] [Resolved] (KAFKA-7958) Transactions are broken with kubernetes hosted brokers

[Alexandre Dupriez (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-7958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Dupriez resolved KAFKA-7958.
--
Fix Version/s: 2.1.1
   Resolution: Fixed

> Transactions are broken with kubernetes hosted brokers
> --
>
> Key: KAFKA-7958
> URL: https://issues.apache.org/jira/browse/KAFKA-7958
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 2.1.0
> Environment: cp-kakfka 2.1.1-1, kafka-streams 2.1.1
>Reporter: Thomas Dickinson
>Priority: Major
> Fix For: 2.1.1
>
>
> After a rolling re-start in a kubernetes-like environment, brokers may change 
> IP address.  From our logs it seems that the transaction manager in the 
> brokers never re-resolves the DNS name of other brokers, keeping stale pod 
> IPs.  Thus transactions stop working.  
> ??[2019-02-20 02:20:20,085] WARN [TransactionCoordinator id=1001] Connection 
> to node 0 
> (khaki-joey-kafka-0.khaki-joey-kafka-headless.hyperspace-dev/[10.233.124.181:9092|http://10.233.124.181:9092/])
>  could not be established. Broker may not be available. 
> (org.apache.kafka.clients.NetworkClient)??
> ??[2019-02-20 02:20:57,205] WARN [TransactionCoordinator id=1001] Connection 
> to node 1 
> (khaki-joey-kafka-1.khaki-joey-kafka-headless.hyperspace-dev/[10.233.122.67:9092|http://10.233.122.67:9092/])
>  could not be established. Broker may not be available. 
> (org.apache.kafka.clients.NetworkClient)??
> This is from the log from broker 1001 which was restarted first, followed by 
> 1 and then 0.  The log entries are from the day after the rolling restart.
> I note a similar issue was fixed for clients 2.1.1  
> https://issues.apache.org/jira/browse/KAFKA-7755.  We are using streams lib 
> 2.1.1
> *Update* We are now testing with Kafka 2.1.1 (docker cp-kafka 5.1.2-1) and it 
> looks like this may be resolved.  Would be good to get confirmation though.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [Vote] KIP-569: Update DescribeConfigsResponse to include additional metadata information

[Shailesh Panwar]

I have 3 +1s and 1 +1(non-binding) vote for this Kip. Thank you all for the
feedback. I'll start working on the PR.

Thanks
Shailesh


On Fri, Mar 20, 2020 at 8:35 AM Brian Byrne  wrote:

> +1 (non-binding) - thanks!
>
> My only suggestion would be to make the enum-to-int conversion explicit for
> the new ConfigType, with a surrounding comment, to ensure that no
> accidental reordering and for easier readability should the response
> message message be read.
>
> Brian
>
> On Fri, Mar 20, 2020 at 8:13 AM David Arthur  wrote:
>
> > +1 binding. Thanks for the KIP 
> >
> > -David
> >
> > On Tue, Mar 17, 2020 at 4:44 AM Rajini Sivaram 
> > wrote:
> >
> > > Hi Shailesh,
> > >
> > > +1 (binding)
> > >
> > > Thanks for the KIP!
> > >
> > > Regards,
> > >
> > > Rajini
> > >
> > >
> > > On Tue, Mar 10, 2020 at 2:37 AM Gwen Shapira 
> wrote:
> > >
> > > > +1
> > > > Looks great. Thanks for the proposal, Shailesh.
> > > >
> > > > Gwen Shapira
> > > > Engineering Manager | Confluent
> > > > 650.450.2760 | @gwenshap
> > > > Follow us: Twitter | blog
> > > >
> > > > On Mon, Mar 09, 2020 at 6:00 AM, Shailesh Panwar <
> > span...@confluent.io
> > > >
> > > > wrote:
> > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi All,
> > > > > I would like to start a vote on KIP-569: Update
> > > > > DescribeConfigsResponse to include additional metadata information
> > > > >
> > > > >
> > > > >
> > > > > The KIP is here:
> > > > > https:/ / cwiki. apache. org/ confluence/ display/ KAFKA/
> > > >
> > >
> >
> KIP-569%3A+DescribeConfigsResponse+-+Update+the+schema+to+include+additional+metadata+information+of+the+field
> > > > > (
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-569%3A+DescribeConfigsResponse+-+Update+the+schema+to+include+additional+metadata+information+of+the+field
> > > > > )
> > > > >
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Shailesh
> > > > >
> > > > >
> > > > >
> > >
> >
> >
> > --
> > David Arthur
> >
>

Re: [Vote] KIP-569: Update DescribeConfigsResponse to include additional metadata information

[Brian Byrne]

+1 (non-binding) - thanks!

My only suggestion would be to make the enum-to-int conversion explicit for
the new ConfigType, with a surrounding comment, to ensure that no
accidental reordering and for easier readability should the response
message message be read.

Brian

On Fri, Mar 20, 2020 at 8:13 AM David Arthur  wrote:

> +1 binding. Thanks for the KIP 
>
> -David
>
> On Tue, Mar 17, 2020 at 4:44 AM Rajini Sivaram 
> wrote:
>
> > Hi Shailesh,
> >
> > +1 (binding)
> >
> > Thanks for the KIP!
> >
> > Regards,
> >
> > Rajini
> >
> >
> > On Tue, Mar 10, 2020 at 2:37 AM Gwen Shapira  wrote:
> >
> > > +1
> > > Looks great. Thanks for the proposal, Shailesh.
> > >
> > > Gwen Shapira
> > > Engineering Manager | Confluent
> > > 650.450.2760 | @gwenshap
> > > Follow us: Twitter | blog
> > >
> > > On Mon, Mar 09, 2020 at 6:00 AM, Shailesh Panwar <
> span...@confluent.io
> > >
> > > wrote:
> > >
> > > >
> > > >
> > > >
> > > > Hi All,
> > > > I would like to start a vote on KIP-569: Update
> > > > DescribeConfigsResponse to include additional metadata information
> > > >
> > > >
> > > >
> > > > The KIP is here:
> > > > https:/ / cwiki. apache. org/ confluence/ display/ KAFKA/
> > >
> >
> KIP-569%3A+DescribeConfigsResponse+-+Update+the+schema+to+include+additional+metadata+information+of+the+field
> > > > (
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-569%3A+DescribeConfigsResponse+-+Update+the+schema+to+include+additional+metadata+information+of+the+field
> > > > )
> > > >
> > > >
> > > >
> > > > Thanks,
> > > > Shailesh
> > > >
> > > >
> > > >
> >
>
>
> --
> David Arthur
>

[jira] [Resolved] (KAFKA-9734) Streams IllegalStateException in trunk during rebalance

[John Roesler (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-9734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Roesler resolved KAFKA-9734.
-
Resolution: Fixed

> Streams IllegalStateException in trunk during rebalance
> ---
>
> Key: KAFKA-9734
> URL: https://issues.apache.org/jira/browse/KAFKA-9734
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: John Roesler
>Assignee: John Roesler
>Priority: Major
>
> I have observed the following exception to kill a thread in the current trunk 
> of Streams:
> {noformat}
> [2020-03-19 07:04:35,206] ERROR 
> [stream-soak-test-e60443b4-aa2d-4107-abf7-ce90407cb70e-StreamThread-1] 
> stream-thread [stream-soak-test-e60443b4-aa
> 2d-4107-abf7-ce90407cb70e-StreamThread-1] Encountered the following exception 
> during processing and the thread is going to shut down:  
> (org.apache.kafka.streams.processor.internals.StreamThread)
> java.lang.IllegalStateException: The changelog reader is not restoring active 
> tasks while trying to transit to update standby tasks: {}
> at 
> org.apache.kafka.streams.processor.internals.StoreChangelogReader.transitToUpdateStandby(StoreChangelogReader.java:303)
>   
>
> at 
> org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:582)
> at 
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:498)
> at 
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:472){noformat}
> Judging from the fact that the standby tasks are reported as an empty set, I 
> think this is just a missed case in the task manager. PR to follow shortly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [Vote] KIP-569: Update DescribeConfigsResponse to include additional metadata information

[David Arthur]

+1 binding. Thanks for the KIP 

-David

On Tue, Mar 17, 2020 at 4:44 AM Rajini Sivaram 
wrote:

> Hi Shailesh,
>
> +1 (binding)
>
> Thanks for the KIP!
>
> Regards,
>
> Rajini
>
>
> On Tue, Mar 10, 2020 at 2:37 AM Gwen Shapira  wrote:
>
> > +1
> > Looks great. Thanks for the proposal, Shailesh.
> >
> > Gwen Shapira
> > Engineering Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter | blog
> >
> > On Mon, Mar 09, 2020 at 6:00 AM, Shailesh Panwar < span...@confluent.io
> >
> > wrote:
> >
> > >
> > >
> > >
> > > Hi All,
> > > I would like to start a vote on KIP-569: Update
> > > DescribeConfigsResponse to include additional metadata information
> > >
> > >
> > >
> > > The KIP is here:
> > > https:/ / cwiki. apache. org/ confluence/ display/ KAFKA/
> >
> KIP-569%3A+DescribeConfigsResponse+-+Update+the+schema+to+include+additional+metadata+information+of+the+field
> > > (
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-569%3A+DescribeConfigsResponse+-+Update+the+schema+to+include+additional+metadata+information+of+the+field
> > > )
> > >
> > >
> > >
> > > Thanks,
> > > Shailesh
> > >
> > >
> > >
>


-- 
David Arthur

[jira] [Created] (KAFKA-9739) StreamsBuilder.build fails with StreamsException "Found a null keyChangingChild node for OptimizableRepartitionNode"

[Artur (Jira)]

Artur created KAFKA-9739:


 Summary: StreamsBuilder.build fails with StreamsException "Found a 
null keyChangingChild node for OptimizableRepartitionNode"
 Key: KAFKA-9739
 URL: https://issues.apache.org/jira/browse/KAFKA-9739
 Project: Kafka
  Issue Type: Bug
  Components: streams
Affects Versions: 2.4.1
Reporter: Artur
 Attachments: streams-exception-log.txt, topology-description.txt

We created a topology using Streams DSL (topology description is available in 
the attached topology-description.txt, no optimization).

Topology works fine with {{topology.optimization=none}}, however it fails to 
build with StreamsException "Found a null keyChangingChild node for 
OptimizableRepartitionNode" if we set {{topology.optimization=all}} (exception 
stack trace is attached streams-exception-log.txt).

We used [https://zz85.github.io/kafka-streams-viz/] to visualize topology and 
try to guess what might be upsetting optimizer, yet did not manage to figure it 
out ourselves.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (KAFKA-9738) Add Generics Type Parameters to forwarded() in MockProcessorContext

[Bruno Cadonna (Jira)]

Bruno Cadonna created KAFKA-9738:


 Summary: Add Generics Type Parameters to forwarded() in 
MockProcessorContext 
 Key: KAFKA-9738
 URL: https://issues.apache.org/jira/browse/KAFKA-9738
 Project: Kafka
  Issue Type: Improvement
  Components: streams
Affects Versions: 2.5.0
Reporter: Bruno Cadonna


The method {{forwarded()}} to capture the forwarded records in 
{{MockProcessorContext}} does not have any type parameters although the 
corresponding {{forward()}} does have them. To enable type checking at compile 
time in tests, generics parameters shall be added to the {{forwarded()}} method.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Jenkins build is back to normal : kafka-trunk-jdk8 #4344

[Apache Jenkins Server]

See 

Build failed in Jenkins: kafka-trunk-jdk11 #1265

[Apache Jenkins Server]

See 


Changes:

[github] KAFKA-8820: kafka-reassign-partitions.sh should support the KIP-455 API

[github] KAFKA-9553; Improve measurement for loading groups and transactions


--
[...truncated 5.93 MB...]
org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardFlush 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
PASSED

org.apache.kafka.streams.TestTopicsTest > testNonUsedOutputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonUsedOutputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testEmptyTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testEmptyTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testStartTimestamp STARTED

org.apache.kafka.streams.TestTopicsTest > testStartTimestamp PASSED

org.apache.kafka.streams.TestTopicsTest > testNegativeAdvance STARTED

org.apache.kafka.streams.TestTopicsTest > testNegativeAdvance PASSED

org.apache.kafka.streams.TestTopicsTest > shouldNotAllowToCreateWithNullDriver 
STARTED

org.apache.kafka.streams.TestTopicsTest > shouldNotAllowToCreateWithNullDriver 
PASSED

org.apache.kafka.streams.TestTopicsTest > testDuration STARTED

org.apache.kafka.streams.TestTopicsTest > testDuration PASSED

org.apache.kafka.streams.TestTopicsTest > testOutputToString STARTED

org.apache.kafka.streams.TestTopicsTest > testOutputToString PASSED

org.apache.kafka.streams.TestTopicsTest > testValue STARTED

org.apache.kafka.streams.TestTopicsTest > testValue PASSED

org.apache.kafka.streams.TestTopicsTest > testTimestampAutoAdvance STARTED

org.apache.kafka.streams.TestTopicsTest > testTimestampAutoAdvance PASSED

org.apache.kafka.streams.TestTopicsTest > testOutputWrongSerde STARTED

org.apache.kafka.streams.TestTopicsTest > testOutputWrongSerde PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputTopicWithNullTopicName STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputTopicWithNullTopicName PASSED

org.apache.kafka.streams.TestTopicsTest > testWrongSerde STARTED

org.apache.kafka.streams.TestTopicsTest > testWrongSerde PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMapWithNull STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMapWithNull PASSED

org.apache.kafka.streams.TestTopicsTest > testNonExistingOutputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonExistingOutputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testMultipleTopics STARTED

org.apache.kafka.streams.TestTopicsTest > testMultipleTopics PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValueList STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValueList PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputWithNullDriver STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputWithNullDriver PASSED

org.apache.kafka.streams.TestTopicsTest > testValueList STARTED

org.apache.kafka.streams.TestTopicsTest > testValueList PASSED

org.apache.kafka.streams.TestTopicsTest > testRecordList STARTED

org.apache.kafka.streams.TestTopicsTest > testRecordList PASSED

org.apache.kafka.streams.TestTopicsTest > testNonExistingInputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonExistingInputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMap STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMap PASSED

org.apache.kafka.streams.TestTopicsTest > testRecordsToList STARTED

org.apache.kafka.streams.TestTopicsTest > testRecordsToList PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValueListDuration STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValueListDuration PASSED

org.apache.kafka.streams.TestTopicsTest > testInputToString STARTED

org.apache.kafka.streams.TestTopicsTest > testInputToString PASSED

org.apache.kafka.streams.TestTopicsTest > testTimestamp STARTED

org.apache.kafka.streams.TestTopicsTest > testTimestamp PASSED

org.apache.kafka.streams.TestTopicsTest > testWithHeaders STARTED

org.apache.kafka.streams.TestTopicsTest > testWithHeaders PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValue STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValue PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateTopicWithNullTopicName STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateTopicWithNullTopicName PASSED

> Task :streams:upgrade-system-tests-0100:compileJava NO-SOURCE
> Task :streams:upgrade-system-tests-0100:processResources NO-SOURCE
> Task :streams:upgrade-system-tests-0100:classes UP-TO-DATE
> Task :streams:upgrade-system-tests-0100:checkstyleMain NO-SOURCE
> Task :streams:upgrade-system-tests-0100:compileTestJava
> 

[jira] [Resolved] (KAFKA-9692) Flaky test - kafka.admin.ReassignPartitionsClusterTest#znodeReassignmentShouldOverrideApiTriggeredReassignment

[Tom Bentley (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tom Bentley resolved KAFKA-9692.

Resolution: Done

This test got deleted as part of KAFKA-8820, when much of the testing changed 
to unit tests.

> Flaky test - 
> kafka.admin.ReassignPartitionsClusterTest#znodeReassignmentShouldOverrideApiTriggeredReassignment
> --
>
> Key: KAFKA-9692
> URL: https://issues.apache.org/jira/browse/KAFKA-9692
> Project: Kafka
>  Issue Type: Bug
>  Components: unit tests
>Affects Versions: 2.5.0
>Reporter: Tom Bentley
>Priority: Major
>  Labels: flaky-test
>
> {noformat}
> java.lang.AssertionError: expected: but was: 101)>
>   at org.junit.Assert.fail(Assert.java:89)
>   at org.junit.Assert.failNotEquals(Assert.java:835)
>   at org.junit.Assert.assertEquals(Assert.java:120)
>   at org.junit.Assert.assertEquals(Assert.java:146)
>   at 
> kafka.admin.ReassignPartitionsClusterTest.assertReplicas(ReassignPartitionsClusterTest.scala:1220)
>   at 
> kafka.admin.ReassignPartitionsClusterTest.assertIsReassigning(ReassignPartitionsClusterTest.scala:1191)
>   at 
> kafka.admin.ReassignPartitionsClusterTest.znodeReassignmentShouldOverrideApiTriggeredReassignment(ReassignPartitionsClusterTest.scala:897)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
>   at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
>   at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
>   at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
>   at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
>   at 
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
>   at 
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
>   at 
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
>   at jdk.internal.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>   at 
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
>   at 
> org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
>   at 
> org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
>   at 
> 

Jenkins build is back to normal : kafka-2.5-jdk8 #68

[Apache Jenkins Server]

See 

Build failed in Jenkins: kafka-trunk-jdk11 #1264

[Apache Jenkins Server]

See 


Changes:

[github] MINOR: Bump RocksDB version from 5.18.3 to 5.18.4 (#8284)

[github] KAFKA-9654; Update epoch in `ReplicaAlterLogDirsThread` after new


--
[...truncated 5.94 MB...]
org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardFlush 
PASSED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
STARTED

org.apache.kafka.streams.internals.WindowStoreFacadeTest > shouldForwardInit 
PASSED

org.apache.kafka.streams.TestTopicsTest > testNonUsedOutputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonUsedOutputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testEmptyTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testEmptyTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testStartTimestamp STARTED

org.apache.kafka.streams.TestTopicsTest > testStartTimestamp PASSED

org.apache.kafka.streams.TestTopicsTest > testNegativeAdvance STARTED

org.apache.kafka.streams.TestTopicsTest > testNegativeAdvance PASSED

org.apache.kafka.streams.TestTopicsTest > shouldNotAllowToCreateWithNullDriver 
STARTED

org.apache.kafka.streams.TestTopicsTest > shouldNotAllowToCreateWithNullDriver 
PASSED

org.apache.kafka.streams.TestTopicsTest > testDuration STARTED

org.apache.kafka.streams.TestTopicsTest > testDuration PASSED

org.apache.kafka.streams.TestTopicsTest > testOutputToString STARTED

org.apache.kafka.streams.TestTopicsTest > testOutputToString PASSED

org.apache.kafka.streams.TestTopicsTest > testValue STARTED

org.apache.kafka.streams.TestTopicsTest > testValue PASSED

org.apache.kafka.streams.TestTopicsTest > testTimestampAutoAdvance STARTED

org.apache.kafka.streams.TestTopicsTest > testTimestampAutoAdvance PASSED

org.apache.kafka.streams.TestTopicsTest > testOutputWrongSerde STARTED

org.apache.kafka.streams.TestTopicsTest > testOutputWrongSerde PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputTopicWithNullTopicName STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputTopicWithNullTopicName PASSED

org.apache.kafka.streams.TestTopicsTest > testWrongSerde STARTED

org.apache.kafka.streams.TestTopicsTest > testWrongSerde PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMapWithNull STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMapWithNull PASSED

org.apache.kafka.streams.TestTopicsTest > testNonExistingOutputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonExistingOutputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testMultipleTopics STARTED

org.apache.kafka.streams.TestTopicsTest > testMultipleTopics PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValueList STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValueList PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputWithNullDriver STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateOutputWithNullDriver PASSED

org.apache.kafka.streams.TestTopicsTest > testValueList STARTED

org.apache.kafka.streams.TestTopicsTest > testValueList PASSED

org.apache.kafka.streams.TestTopicsTest > testRecordList STARTED

org.apache.kafka.streams.TestTopicsTest > testRecordList PASSED

org.apache.kafka.streams.TestTopicsTest > testNonExistingInputTopic STARTED

org.apache.kafka.streams.TestTopicsTest > testNonExistingInputTopic PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMap STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValuesToMap PASSED

org.apache.kafka.streams.TestTopicsTest > testRecordsToList STARTED

org.apache.kafka.streams.TestTopicsTest > testRecordsToList PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValueListDuration STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValueListDuration PASSED

org.apache.kafka.streams.TestTopicsTest > testInputToString STARTED

org.apache.kafka.streams.TestTopicsTest > testInputToString PASSED

org.apache.kafka.streams.TestTopicsTest > testTimestamp STARTED

org.apache.kafka.streams.TestTopicsTest > testTimestamp PASSED

org.apache.kafka.streams.TestTopicsTest > testWithHeaders STARTED

org.apache.kafka.streams.TestTopicsTest > testWithHeaders PASSED

org.apache.kafka.streams.TestTopicsTest > testKeyValue STARTED

org.apache.kafka.streams.TestTopicsTest > testKeyValue PASSED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateTopicWithNullTopicName STARTED

org.apache.kafka.streams.TestTopicsTest > 
shouldNotAllowToCreateTopicWithNullTopicName PASSED

> Task :streams:upgrade-system-tests-0100:compileJava NO-SOURCE
> Task :streams:upgrade-system-tests-0100:processResources NO-SOURCE
> Task :streams:upgrade-system-tests-0100:classes UP-TO-DATE
> Task :streams:upgrade-system-tests-0100:checkstyleMain NO-SOURCE
> Task :streams:upgrade-system-tests-0100:compileTestJava
> Task 

[jira] [Resolved] (KAFKA-9553) Transaction state loading metric does not count total loading time

[Jason Gustafson (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-9553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Gustafson resolved KAFKA-9553.

Fix Version/s: 2.6.0
   Resolution: Fixed

> Transaction state loading metric does not count total loading time
> --
>
> Key: KAFKA-9553
> URL: https://issues.apache.org/jira/browse/KAFKA-9553
> Project: Kafka
>  Issue Type: Bug
>Reporter: Jason Gustafson
>Assignee: Agam Brahma
>Priority: Major
> Fix For: 2.6.0
>
>
> KIP-484 added a metric to track total loading time for internal topics: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-484%3A+Expose+metrics+for+group+and+transaction+metadata+loading+duration.
>  The value seems to be being updated incorrectly in TransactionStateManager. 
> Rather than recording the total loading time, it records the loading 
> separately after every read from the log.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Build failed in Jenkins: kafka-trunk-jdk8 #4343

[Apache Jenkins Server]

See 


Changes:

[github] KAFKA-8820: kafka-reassign-partitions.sh should support the KIP-455 API


--
[...truncated 2.91 MB...]

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKeyAndDefaultTimestamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithDefaultTimestamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithDefaultTimestamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKey STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKey PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithOtherTopicNameAndTimestampWithTimetamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithOtherTopicNameAndTimestampWithTimetamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullHeaders STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullHeaders PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithDefaultTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithDefaultTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKey STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKey PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecord STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecord PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecord STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecord PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > shouldAdvanceTime 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > shouldAdvanceTime 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairs STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairsAndCustomTimestamps
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairsAndCustomTimestamps
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairs 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicNameAndTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithOtherTopicNameAndTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairsAndCustomTimestamps
 STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithKeyValuePairsAndCustomTimestamps
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest >