Jenkins build is back to normal : kafka-2.5-jdk8 #164

2020-07-08 Thread Apache Jenkins Server
See

Build failed in Jenkins: kafka-2.3-jdk8 #218

2020-07-08 Thread Apache Jenkins Server
See Changes: [github] KAFKA-9144; Track timestamp from txn markers to prevent early producer -- [...truncated 2.99 MB...] kafka.log.LogCleanerTest > testPartialSegmentClean

Re: [DISCUSS] KIP-622 Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext

2020-07-08 Thread Matthias J. Sax
I think, we don't need a default implementation for the new methods. What would be the use-case to implement the `ProcessorContext` interface? In contract to, for example, `KeyValueStore`, `ProcessorContext` is a use-only interface because it's never passed into Kafka Streams, but only handed

Re: [VOTE] KIP-622 Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext

2020-07-08 Thread Matthias J. Sax
Thanks for the KIP. +1 (binding) -Matthias On 7/7/20 11:48 AM, William Bottrell wrote: > Hi everyone, > > I'd like to start a vote for adding two new time API's to ProcessorContext. > > Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext >

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2020-07-08 Thread Rajini Sivaram
Hi all, We would like to make a change related to this KIP to ensure that transient exceptions during reassignments are handled as retriable exceptions. Kafka brokers currently return REPLICA_NOT_AVAILABLE for Fetch requests if the replica is not available on the broker. But

Re: [VOTE] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

2020-07-08 Thread Matthias J. Sax
+1 (binding) -Matthias On 7/7/20 7:16 PM, John Roesler wrote: > Hi Badai, > > Thanks for picking this up. I've reviewed the KIP document and > the threads you linked. I think we may want to make more > improvements in the future to the printing of headers in particular, > but this KIP seems

[jira] [Resolved] (KAFKA-10245) Using vulnerable log4j version

2020-07-08 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bentley resolved KAFKA-10245. - Resolution: Duplicate > Using vulnerable log4j version > -- > >

Re: [VOTE] KIP-622 Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext

2020-07-08 Thread Bruno Cadonna
Thanks Will and Piotr, +1 (non-binding) Best, Bruno On Wed, Jul 8, 2020 at 8:12 AM Matthias J. Sax wrote: > > Thanks for the KIP. > > +1 (binding) > > > -Matthias > > On 7/7/20 11:48 AM, William Bottrell wrote: > > Hi everyone, > > > > I'd like to start a vote for adding two new time API's to

Re: [DISCUSS] KIP-616: Rename implicit Serdes instances in kafka-streams-scala

2020-07-08 Thread John Roesler
Hi Yuriy, Once it seems like there’s general agreement in the discussion, you can start a voting thread. You can find examples on the mailing list of what to say in the first message. It’s basically just a message with the subject line changed from “[DISCUSS]...” to “[VOTE]...”, and then

Re: [VOTE] KIP-632: Add DirectoryConfigProvider

2020-07-08 Thread Manikumar
+1 (bindig) Thanks for the KIP. On Tue, Jul 7, 2020 at 10:30 PM David Jacot wrote: > +1 (non-binding). Thanks for the KIP! > > On Tue, Jul 7, 2020 at 12:54 PM Tom Bentley wrote: > > > Hi, > > > > I'd like to start a vote on KIP-632, which is about making the config > > provider mechanism more

Re: [VOTE] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

2020-07-08 Thread Manikumar
+1 (binding) Thanks for the KIP. Thanks, Manikumar On Wed, Jul 8, 2020 at 11:47 AM Matthias J. Sax wrote: > +1 (binding) > > -Matthias > > On 7/7/20 7:16 PM, John Roesler wrote: > > Hi Badai, > > > > Thanks for picking this up. I've reviewed the KIP document and > > the threads you linked.

Re: [VOTE] KIP-621: Deprecate and replace DescribeLogDirsResult.all() and .values()

2020-07-08 Thread Manikumar
+1 (binding) Thanks for the KIP. On Tue, Jul 7, 2020 at 8:19 PM Colin McCabe wrote: > Thanks, Tom. > > I tried to think of a better way to do this, but I think you're right that > we probably just need different methods. > > +1 (binding). > > best, > Colin > > On Mon, Jul 6, 2020, at 01:14,

Re: [DISCUSS] KIP-638: Deprecate DescribeLogDirsResponse.[LogDirInfo, ReplicaInfo]

2020-07-08 Thread Dongjin Lee
Hi Mickael, Okay. I will close my KIP. Let's continue the discussion on KIP-621 thread. Thanks, Dongjin On Tue, Jul 7, 2020 at 11:06 PM Mickael Maison wrote: > Hi Dongjin, > > It looks like this KIP is addressing the same issue as KIP-621: >

[jira] [Resolved] (KAFKA-8794) Deprecate DescribeLogDirsResponse.[LogDirInfo, ReplicaInfo]

2020-07-08 Thread Dongjin Lee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjin Lee resolved KAFKA-8794. Resolution: Duplicate Duplication of KAFKA-10120. > Deprecate

Re: [DISCUSS] KIP-621: Deprecate and replace DescribeLogDirsResult.all() and .values()

2020-07-08 Thread Dongjin Lee
Hi Tom, Thanks for taking this issue. I opened a PR for this issue earlier, but your KIP was submitted first. So I closed my one . I have a question: for consistency with other methods, how about maintaining the

Re: [DISCUSS] KIP-363

2020-07-08 Thread Magnus Edenhill
Hi Tom, I think it would be useful with some real world (or made up!) numbers on how much relative/% space is saved for the most error-dense protocol requests. E.g., an OffsetCommitResponse with 10 topics and 100 failing partitions would reduce the overall size by % bytes. Thanks, Magnus Den

[jira] [Created] (KAFKA-10247) Streams may attempt to process after closing a task

2020-07-08 Thread John Roesler (Jira)
John Roesler created KAFKA-10247: Summary: Streams may attempt to process after closing a task Key: KAFKA-10247 URL: https://issues.apache.org/jira/browse/KAFKA-10247 Project: Kafka Issue

Re: [DISCUSS] Apache Kafka 2.6.0 release

2020-07-08 Thread John Roesler
Hi Randall, While developing system tests, I've just unearthed a new 2.6 regression: https://issues.apache.org/jira/browse/KAFKA-10247 I've got a PR in progress. Hoping to finish it up today: https://github.com/apache/kafka/pull/8994 Sorry for the trouble, -John On Mon, Jun 29, 2020, at 09:29,

[jira] [Resolved] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-07-08 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-10134. --- Resolution: Fixed > High CPU issue during rebalance in Kafka consumer after upgrading to

Build failed in Jenkins: kafka-trunk-jdk14 #276

2020-07-08 Thread Apache Jenkins Server
See Changes: [github] KAFKA-10134: Use long poll if we do not have fetchable partitions -- [...truncated 3.19 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [VOTE] KIP-632: Add DirectoryConfigProvider

2020-07-08 Thread Mickael Maison
+1 (binding) Thanks On Wed, Jul 8, 2020 at 11:31 AM Manikumar wrote: > > +1 (bindig) > > Thanks for the KIP. > > On Tue, Jul 7, 2020 at 10:30 PM David Jacot wrote: > > > +1 (non-binding). Thanks for the KIP! > > > > On Tue, Jul 7, 2020 at 12:54 PM Tom Bentley wrote: > > > > > Hi, > > > > > >

Re: [DISCUSS] KIP-621: Deprecate and replace DescribeLogDirsResult.all() and .values()

2020-07-08 Thread Colin McCabe
Hi Dongjin, Hmm. I'm not sure I follow. How does deprecating DescribeLogDirsResponse.LogDirInfo help here? The issue is not so much the class, but the fact that it's exposed as a public API. So it seems appropriate to deprecate the methods that return it, but not the class itself, since

[jira] [Resolved] (KAFKA-10225) Increase default zk session timeout for system tests

2020-07-08 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-10225. - Fix Version/s: 2.7.0 Resolution: Fixed merged the PR to trunk > Increase default zk session

Re: [DISCUSS] Apache Kafka 2.6.0 release

2020-07-08 Thread Sophie Blee-Goldman
Hey Randall, We just discovered another regression in 2.6: https://issues.apache.org/jira/browse/KAFKA-10249 The fix is extremely straightforward -- only about two lines of actual code -- and low risk. It is a new regression introduced in 2.6 and affects all Streams apps with any suppression or

[jira] [Created] (KAFKA-10248) Drop idempotent KTable source updates

2020-07-08 Thread John Roesler (Jira)
John Roesler created KAFKA-10248: Summary: Drop idempotent KTable source updates Key: KAFKA-10248 URL: https://issues.apache.org/jira/browse/KAFKA-10248 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-10249) In-memory stores are skipped when checkpointing but not skipped when reading the checkpoint

2020-07-08 Thread Sophie Blee-Goldman (Jira)
Sophie Blee-Goldman created KAFKA-10249: --- Summary: In-memory stores are skipped when checkpointing but not skipped when reading the checkpoint Key: KAFKA-10249 URL:

Build failed in Jenkins: kafka-2.2-jdk8 #49

2020-07-08 Thread Apache Jenkins Server
See Changes: [jason] KAFKA-9144; Track timestamp from txn markers to prevent early producer -- [...truncated 2.59 MB...] kafka.utils.CommandLineUtilsTest > testParseArgs

[jira] [Resolved] (KAFKA-10248) Drop idempotent KTable source updates

2020-07-08 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-10248. -- Resolution: Fixed > Drop idempotent KTable source updates >

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

2020-07-08 Thread Colin McCabe
On Tue, Jul 7, 2020, at 18:27, Ron Dagostino wrote: > HI Colin. Thanks for the KIP. Here is some feedback and various questions. > Hi Ron, Thanks for taking a look! > "*Controller processes will listen on a separate port from brokers. This > will be true even when the broker and controller

Build failed in Jenkins: kafka-trunk-jdk11 #1627

2020-07-08 Thread Apache Jenkins Server
See Changes: [github] KAFKA-10134: Use long poll if we do not have fetchable partitions -- [...truncated 6.38 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

[jira] [Created] (KAFKA-10250) Kafka broker shrinks the ISRs and disconnects from other brokers for few seconds

2020-07-08 Thread Rishank Chandra Puram (Jira)
Rishank Chandra Puram created KAFKA-10250: - Summary: Kafka broker shrinks the ISRs and disconnects from other brokers for few seconds Key: KAFKA-10250 URL:

Build failed in Jenkins: kafka-trunk-jdk14 #277

2020-07-08 Thread Apache Jenkins Server
See Changes: [github] MINOR; alterReplicaLogDirs should not fail all the futures when only one -- [...truncated 6.38 MB...]

[jira] [Created] (KAFKA-10251) Flaky Test kafka.api.TransactionsBounceTest.testWithGroupMetadata

2020-07-08 Thread Sophie Blee-Goldman (Jira)
Sophie Blee-Goldman created KAFKA-10251: --- Summary: Flaky Test kafka.api.TransactionsBounceTest.testWithGroupMetadata Key: KAFKA-10251 URL: https://issues.apache.org/jira/browse/KAFKA-10251

Jenkins build is back to normal : kafka-trunk-jdk11 #1628

2020-07-08 Thread Apache Jenkins Server
See

Re: Feedback: Print schemaId using bin/kafka-dump-log.sh

2020-07-08 Thread Mohanraj Nagasamy
Thanks Adam for providing the feedback! On 7/7/20, 5:05 AM, "Adam Bellemare" wrote: Hi Mohanraj While I see the usefulness of your suggestion, the main issue is that you're using the Confluent schema registry's conventions and hardwiring them into Kafka core. Given that

[DISCUSS] KIP-450: Sliding Windows

2020-07-08 Thread Leah Thomas
Hi all, I'd like to kick-off the discussion for KIP-450, adding sliding window aggregation support to Kafka Streams. https://cwiki.apache.org/confluence/display/KAFKA/KIP-450%3A+Sliding+Window+Aggregations+in+the+DSL Let me know what you think, Leah

Re: Kafka Streams: Dynamic Topic Routing & Nonexistent Topics

2020-07-08 Thread Rhys Anthony McCaig
Guozhang - thank you for your thoughts. You are right - this is more about the producer client than the streams client. caching the metadata outside producer, e.g. in an admin client would not > be a perfect solution since in either way your metadata cache inside the > producer or inside the

Jenkins build is back to normal : kafka-trunk-jdk14 #278

2020-07-08 Thread Apache Jenkins Server
See

Build failed in Jenkins: kafka-trunk-jdk8 #4701

2020-07-08 Thread Apache Jenkins Server
See Changes: [github] KAFKA-10134: Use long poll if we do not have fetchable partitions -- [...truncated 1.84 MB...] kafka.zookeeper.ZooKeeperClientTest >

[VOTE] KIP-616: Rename implicit Serdes instances in kafka-streams-scala

2020-07-08 Thread Yuriy Badalyantc
Hi everybody I would like to start a vote for KIP-616: https://cwiki.apache.org/confluence/display/KAFKA/KIP-616%3A+Rename+implicit+Serdes+instances+in+kafka-streams-scala This KIP fixes name clash in the org.apache.kafka.streams.scala.Serdes. -Yuriy