Re: [DISCUSS] KIP-612: Ability to Limit Connection Creation Rate on Brokers

Hi Anna, Thanks for the KIP, looks good overall. A couple of comments about per-IP connection quotas: 1) Should we consider making per-IP quota similar to other quotas? Configured as a dynamic quota for entity type IP, with per-IP limit as well as defaults? Perhaps that would fit better rather

Re: [VOTE] KIP-437: Custom replacement for MaskField SMT

Thanks for the KIP. This is a great improvement, +1(binding) -Bill On Fri, May 15, 2020 at 8:02 AM Andrew Schofield wrote: > +1 (non-binding). Looks like a good addition. > > Andrew Schofield > > On 14/05/2020, 06:19, "Konstantine Karantasis" > wrote: > > I think this improvement makes

Re: [VOTE] KIP-437: Custom replacement for MaskField SMT

+1 (non-binding). Looks like a good addition. Andrew Schofield On 14/05/2020, 06:19, "Konstantine Karantasis" wrote: I think this improvement makes total sense. It's interesting that it didn't accompany the initial version of this transformation. +1 (binding) Konstantine

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Hi, Randall's suggestion is really good. I think it gives the flexibility required and also keeps the interface the right way round. Thanks, Andrew Schofield On 15/05/2020, 02:07, "Aakash Shah" wrote: > Hi Randall, > > Thanks for the feedback. > > 1. This is a great suggestion, but I find

[jira] [Created] (KAFKA-10004) using kafka-configs.sh --describe for brokers will have error when querying default broker

Luke Chen created KAFKA-10004: - Summary: using kafka-configs.sh --describe for brokers will have error when querying default broker Key: KAFKA-10004 URL: https://issues.apache.org/jira/browse/KAFKA-10004

Re: [DISCUSS] KIP-554: Add Broker-side SCRAM Config API

Hi Colin, We have used different approaches for kafka-configs using ZooKeeper and using brokers until now. This is based on the fact that whatever you can access using kafka-configs with ZooKeeper, you can also access directly using ZooKeeper shell. For example, you can retrieve any config stored

Re: [DISCUSS] KIP-601: Configurable socket connection timeout

Hi Cheng, Let me rephrase my question. Let's say we didn't have the case of leastLoadedNode. We are only talking about connections to a specific node (i.e. leader or controller). We have a lot of these and I want to understand the benefits of the proposed timeout in this case alone. We currently

Re: [VOTE] KIP 585: Filter and conditional SMTs

+1 (non-binding) Thanks for the KIP. This will be very useful. Andrew Schofield On 13/05/2020, 10:14, "Tom Bentley" wrote: Hi, I'd like to start a vote on KIP-585: Filter and conditional SMTs

Re: [VOTE] KIP-607: Add Metrics to Record the Memory Used by RocksDB to Kafka Streams

Thanks for the KIP! +1 (binding) -Bill On Fri, May 15, 2020 at 11:12 AM Guozhang Wang wrote: > +1. > > Thanks！ > > On Fri, May 15, 2020 at 1:36 AM Bruno Cadonna wrote: > > > Hi all, > > > > I'd like to call for votes on > > > > KIP-607: Add Metrics to Record the Memory Used by RocksDB to

[jira] [Resolved] (KAFKA-9132) Refactor StreamThread to take advantage of new ConsumerRebalanceListener exception handling

[ https://issues.apache.org/jira/browse/KAFKA-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-9132. Fix Version/s: 2.6.0 Resolution: Fixed > Refactor StreamThread to take

Re: [VOTE] KIP-606: Add Metadata Context to MetricsReporter

Thanks for the proposal, Xavier. +1(binding) -Bill On Fri, May 15, 2020 at 1:41 PM John Roesler wrote: > Thanks for the KIP, Xavier! > > I'm +1 (binding) as well. > > Thanks, > -John > > On Thu, May 14, 2020, at 17:07, Randall Hauch wrote: > > +1 (binding) > > > > Thanks for the proposal,

Build failed in Jenkins: kafka-trunk-jdk8 #4532

See Changes: [github] KAFKA-9996: Upgrade zookeeper to 3.5.8 (#8674) -- [...truncated 3.07 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [DISCUSS] KIP-607: Add Metrics to Record the Memory Used by RocksDB to Kafka Streams

Thanks for the KIP, Bruno. Having sensible, easy to access RocksDB memory reporting will be a welcomed addition. FWIW I also agree to have the metrics reported on a store level. I'm glad you changed the KIP to that effect. -Bill On Wed, May 13, 2020 at 6:24 PM Guozhang Wang wrote: > Hi

Re: Consumer hang-up in case of unclean leader election

Hey Dmitry, Can you open a ticket at https://issues.apache.org/jira/issues/ and include all this information so we can track and look into it? Thanks! Sophie On Fri, May 15, 2020 at 2:26 AM Dmitry Sorokin wrote: > According to documentation, in case if `auto.offset.reset` is set > to none or

[jira] [Resolved] (KAFKA-8174) Can't call arbitrary SimpleBenchmarks tests from streams_simple_benchmark_test.py

[ https://issues.apache.org/jira/browse/KAFKA-8174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-8174. Resolution: Won't Do > Can't call arbitrary SimpleBenchmarks tests from >

Re: [VOTE] KIP-605 - Expand Connect Worker Internal Topic Settings

Thanks for clear KIP Randall, it clearly demonstrates the need for this change. +1 (binding) -Bill On Tue, May 12, 2020 at 2:02 PM Randall Hauch wrote: > FYI: I made a few clarifications to more carefully describe what happens > with extra topic settings that the broker doesn't accept. > >

Re: [VOTE] KIP-221: Enhance KStream with Connecting Topic Creation and Repartition Hint

Thanks Levani! I took the liberty to update the KIP accordingly, because so far no concerns were raised about the change. Also assigned the ticket to myself (let me know if you want to work on it instead). We should get the change into 2.6 release. -Matthias On 5/15/20 1:34 AM, Levani

[jira] [Resolved] (KAFKA-9521) Flaky Test DeleteConsumerGroupsTest#testDeleteCmdEmptyGroup

[ https://issues.apache.org/jira/browse/KAFKA-9521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-9521. Resolution: Duplicate > Flaky Test

Re: [VOTE] KIP-607: Add Metrics to Record the Memory Used by RocksDB to Kafka Streams

+1. Thanks！ On Fri, May 15, 2020 at 1:36 AM Bruno Cadonna wrote: > Hi all, > > I'd like to call for votes on > > KIP-607: Add Metrics to Record the Memory Used by RocksDB to Kafka Streams > > The KIP can be found here > >

[jira] [Resolved] (KAFKA-9001) Flaky Test KStreamAggregationIntegrationTest#shouldReduceSessionWindows

[ https://issues.apache.org/jira/browse/KAFKA-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman resolved KAFKA-9001. Resolution: Cannot Reproduce > Flaky Test

Re: [VOTE] KIP-596 Safely abort Producer transactions during application shutdown

+1 (binding) -Matthias On 5/15/20 8:20 AM, Boyang Chen wrote: > Thanks for the KIP Xiang, +1 (non-binding) from me! > > On Fri, May 15, 2020 at 2:05 AM 张祥 wrote: > >> Hi all， >> >> I'd like to start a vote for KIP-586. You can find the link for this KIP >> here： >> >>

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Small correction. I didn't mean to declare the new method `abstract`. I agree with Randall's suggestion to give it a default implementation that will call the old `put` and at the same time deprecate the old `put`. Konstantine On Fri, May 15, 2020 at 10:19 AM Konstantine Karantasis <

Re: [DISCUSS] KIP-613: Add end-to-end latency metrics to Streams

Thanks for the KIP, Sophie, this will be a useful metric to add. Regarding tracking min, I think it could be valuable for users to discern which part of their topologies are more efficient since this is a task-level metric. I realize everyone seems to be on board with including min anyway, but

Re: [VOTE] KIP-607: Add Metrics to Record the Memory Used by RocksDB to Kafka Streams

Thanks Bruno! +1 (non-binding) Sophie On Fri, May 15, 2020 at 8:15 AM Bill Bejeck wrote: > Thanks for the KIP! > > +1 (binding) > > -Bill > > On Fri, May 15, 2020 at 11:12 AM Guozhang Wang wrote: > > > +1. > > > > Thanks！ > > > > On Fri, May 15, 2020 at 1:36 AM Bruno Cadonna > wrote: > > > >

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

I was on the fence between the various overloading methods myself, liking `start(...)` the least. Initially, I thought we were interested in offering the ability to call the reporter out of band, outside `put`. But after your replies I understand you don't think that's the case, and I also agree

Re: [VOTE] KIP-606: Add Metadata Context to MetricsReporter

Thanks for the KIP, Xavier! I'm +1 (binding) as well. Thanks, -John On Thu, May 14, 2020, at 17:07, Randall Hauch wrote: > +1 (binding) > > Thanks for the proposal, Xavier. > > On Wed, May 13, 2020 at 4:10 PM Gwen Shapira wrote: > > > +1 (binding) > > Thanks for the proposal, Xavier. > > >

Re: [VOTE] KIP-596 Safely abort Producer transactions during application shutdown

Thanks for the KIP Xiang, +1 (non-binding) from me! On Fri, May 15, 2020 at 2:05 AM 张祥 wrote: > Hi all， > > I'd like to start a vote for KIP-586. You can find the link for this KIP > here： > >

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Konstantine said: > I notice Randall also used BiFunction in his example, I wonder if it's for > similar reasons. > Nope. Just a typo on my part. There appear to be three outstanding questions. First, Konstantine suggested calling this "failedRecordReporter". I think this is minor, but using

[jira] [Resolved] (KAFKA-7174) Improve version probing of subscription info

[ https://issues.apache.org/jira/browse/KAFKA-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7174. -- Resolution: Fixed > Improve version probing of subscription info >

Re: [VOTE] KIP 585: Filter and conditional SMTs

+1 (binding) Thanks Tom. Konstantine On Fri, May 15, 2020 at 5:03 AM Andrew Schofield wrote: > +1 (non-binding) > > Thanks for the KIP. This will be very useful. > > Andrew Schofield > > On 13/05/2020, 10:14, "Tom Bentley" wrote: > > Hi, > > I'd like to start a vote on KIP-585:

Re: [DISCUSS] KIP-608: Add a new method to AuthorizerServerInfo Interface

Hey Jeff, Thanks for the updated KIP. The interface looks good to me. I've got couple of comments: 1. You say: "Other broker or client plugins could potentially implement the interface and get the broker's or client's Metrics instance to add additional metrics from sub-components." Does it imply

Re: [DISCUSS] KIP-608: Add a new method to AuthorizerServerInfo Interface

Hi David, Thanks for your suggestions. I updated KIP based on your comments 1,2 and 4. For comment 3), I think the approach which Authorizer extend Monitorable would be simple for broker calling monitor function without checking whether Object implemented Monitorable function or not and make

[VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

Hi, I would like to start the vote on KIP-572: https://cwiki.apache.org/confluence/display/KAFKA/KIP-572%3A+Improve+timeouts+and+retries+in+Kafka+Streams -Matthias signature.asc Description: OpenPGP digital signature

Re: [VOTE] KIP-605 - Expand Connect Worker Internal Topic Settings

Thanks for the KIP, Randall! I've just taken a look, and I'm +1 (binding) as well. Thanks, -John On Fri, May 15, 2020, at 11:50, Bill Bejeck wrote: > Thanks for clear KIP Randall, it clearly demonstrates the need for this > change. > > +1 (binding) > > -Bill > > On Tue, May 12, 2020 at 2:02

Re: [VOTE] KIP-605 - Expand Connect Worker Internal Topic Settings

This KIP is now accepted. The vote has been open for >72 hours (actually 8 days), and we have four binding +1s (Konstantine, Bill, John, me), one non-binding +1 (Chris), and no -1 votes. Thanks, everyone! On Fri, May 15, 2020 at 12:13 PM John Roesler wrote: > Thanks for the KIP, Randall! > >

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Thanks for the additional feedback. I see the benefits of adding an overloaded put(...) over alternatives and I am on board going forward with this approach. It will definitely set forth a contract of where the reporter will be used with better aesthetics. The original idea of going with a

Re: [VOTE] KIP-437: Custom replacement for MaskField SMT

This KIP is now accepted. The vote has been open for >72 hours (actually almost 5 months), and we have three binding +1s (Konstantine, Bill, me), two non-binding +1s (Yu, Andrew), and no -1 votes. Thanks, everyone! On Fri, May 15, 2020 at 9:07 AM Bill Bejeck wrote: > Thanks for the KIP. > >

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Thanks for the quick response Aakash. To your last point, modern APIs like this tend to be asynchronous (see admin, producer in Kafka) and such definition results in more expressive and well defined APIs. What you describe is easily an opt-in feature for the connector developer. At the same

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

Hi Boyang, It is a separate thread, and you have just revealed yourself as a gmail user ;) (Gmail sometimes conflates vote and discuss threads for no apparent reason ) -John On Fri, May 15, 2020, at 13:39, Boyang Chen wrote: > Hey Matthias, should this be on a separate VOTE thread? > > On

Build failed in Jenkins: kafka-2.5-jdk8 #115

See Changes: [ismael] KAFKA-9996: Upgrade zookeeper to 3.5.8 (#8674) -- [...truncated 2.05 MB...] org.apache.kafka.streams.integration.GlobalThreadShutDownOrderTest >

Build failed in Jenkins: kafka-trunk-jdk11 #1458

See Changes: [github] KAFKA-9996: Upgrade zookeeper to 3.5.8 (#8674) -- [...truncated 4.93 MB...] kafka.admin.ConfigCommandTest >

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

On Fri, May 15, 2020 at 1:45 PM Konstantine Karantasis < konstant...@confluent.io> wrote: > Thanks for the quick response Aakash. > > To your last point, modern APIs like this tend to be asynchronous (see > admin, producer in Kafka) and such definition results in more expressive > and well

[jira] [Resolved] (KAFKA-9537) Abstract transformations in configurations cause unfriendly error message.

[ https://issues.apache.org/jira/browse/KAFKA-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-9537. -- Fix Version/s: 2.5.1 2.4.2 2.6.0 Reviewer:

[jira] [Resolved] (KAFKA-7041) Using RocksDB bulk loading for StandbyTasks

[ https://issues.apache.org/jira/browse/KAFKA-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-7041. Resolution: Won't Fix > Using RocksDB bulk loading for StandbyTasks >

Build failed in Jenkins: kafka-trunk-jdk8 #4533

See Changes: [github] KAFKA-9537 - Cleanup error messages for abstract transformations (#8090) -- [...truncated 6.14 MB...]

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

Thanks, Matthias! I’m +1 (binding) -John On Fri, May 15, 2020, at 11:55, Matthias J. Sax wrote: > Hi, > > I would like to start the vote on KIP-572: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-572%3A+Improve+timeouts+and+retries+in+Kafka+Streams > > > -Matthias > > >

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

Hey Matthias, should this be on a separate VOTE thread? On Fri, May 15, 2020 at 11:38 AM John Roesler wrote: > Thanks, Matthias! > > I’m +1 (binding) > > -John > > On Fri, May 15, 2020, at 11:55, Matthias J. Sax wrote: > > Hi, > > > > I would like to start the vote on KIP-572: > > > > >

Build failed in Jenkins: kafka-trunk-jdk11 #1459

See Changes: -- Started by an SCM change Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H28 (ubuntu) in workspace

Build failed in Jenkins: kafka-2.5-jdk8 #116

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H39 (ubuntu) in workspace

Build failed in Jenkins: kafka-trunk-jdk14 #87

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu) in workspace

Re: KAFKA-10000

We got 48 tickets closed since yesterday. Great start! I am sure there is more. Please continue with more cleanup work. -Matthias On 5/14/20 9:11 PM, Guozhang Wang wrote: > Hey Matthias, thanks for calling this out in the community. > > Maybe one way to do this is to encourage everyone who

Build failed in Jenkins: kafka-trunk-jdk11 #1462

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H28 (ubuntu) in workspace

Build failed in Jenkins: kafka-trunk-jdk14 #89

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu) in workspace

Re: [DISCUSSION] KIP-418: A method-chaining way to branch KStream

Hello, John, hello Matthias! Thank you very much for your detailed feedback! - John, > It looks like you missed my reply on Apr 23rd. For some unknown reason it didn't reach my inbox, fortunately we have all the emails on the web. > 1. Can you

Re: [VOTE] KIP-596 Safely abort Producer transactions during application shutdown

Thanks for the KIP, Xiang! I’m +1 (binding) -John On Fri, May 15, 2020, at 12:08, Matthias J. Sax wrote: > +1 (binding) > > -Matthias > > On 5/15/20 8:20 AM, Boyang Chen wrote: > > Thanks for the KIP Xiang, +1 (non-binding) from me! > > > > On Fri, May 15, 2020 at 2:05 AM 张祥 wrote: > > >

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

Called out! Seems like gmail struggles with [...] prefixed subjects. You'd think they would adapt all their practices to conform to the Apache Kafka mailing list standards, but no! +1 (non-binding) by the way On Fri, May 15, 2020 at 11:46 AM John Roesler wrote: > Hi Boyang, > > It is a

Re: [DISCUSS] KIP-613: Add end-to-end latency metrics to Streams

Hey Sophie, 90/99/min/max make sense to me. On Fri, May 15, 2020 at 12:20 PM Sophie Blee-Goldman wrote: > @Matthias > Regarding tracking the 50th percentile, I'll refer you to the 4:53 mark of > the video > *you* linked: https://youtu.be/lJ8ydIuPFeU?t=293 > > And that's what he says about the

[jira] [Created] (KAFKA-10007) Kafka consumer offset reset despite recent group activity

Raman Gupta created KAFKA-10007: --- Summary: Kafka consumer offset reset despite recent group activity Key: KAFKA-10007 URL: https://issues.apache.org/jira/browse/KAFKA-10007 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk14 #88

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu) in workspace

Re: [DISCUSS] KIP-608: Add a new method to AuthorizerServerInfo Interface

Hi Jeff, Thanks for the updated KIP, looks good. Regards, Rajini On Fri, May 15, 2020 at 5:57 PM Jeff Huang wrote: > Hi David, > > Thanks for your suggestions. I updated KIP based on your comments 1,2 and > 4. For comment 3), I think the approach which Authorizer extend Monitorable > would

Build failed in Jenkins: kafka-2.5-jdk8 #118

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H39 (ubuntu) in workspace

[VOTE] KIP-608: Expose Kafka Metrics in Authorizer

Thanks to everyone for their input. I've incorporated the changes, and I think this is ready for voting. To summarize, the KIP simply proposes to add a feature which expose instance of Kafka Metrics in Kafka Authorizer plugin.The KIP can be found here:

Re: [VOTE] KIP-607: Add Metrics to Record the Memory Used by RocksDB to Kafka Streams

Thanks, Bruno! I’m +1 (binding) -John On Fri, May 15, 2020, at 11:32, Sophie Blee-Goldman wrote: > Thanks Bruno! +1 (non-binding) > > Sophie > > On Fri, May 15, 2020 at 8:15 AM Bill Bejeck wrote: > > > Thanks for the KIP! > > > > +1 (binding) > > > > -Bill > > > > On Fri, May 15, 2020 at

Build failed in Jenkins: kafka-trunk-jdk14 #85

See Changes: [github] KAFKA-9996: Upgrade zookeeper to 3.5.8 (#8674) -- [...truncated 4.49 MB...] org.apache.kafka.common.metrics.MetricsTest > testRemoveInactiveMetrics

Build failed in Jenkins: kafka-trunk-jdk14 #86

See Changes: -- Started by an SCM change Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H38 (ubuntu) in workspace

Build failed in Jenkins: kafka-trunk-jdk11 #1460

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H28 (ubuntu) in workspace

[jira] [Created] (KAFKA-10006) Streams should not attempt to create internal topics that may exist

Sophie Blee-Goldman created KAFKA-10006: --- Summary: Streams should not attempt to create internal topics that may exist Key: KAFKA-10006 URL: https://issues.apache.org/jira/browse/KAFKA-10006

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

Good, good. Read through the discussions, the KIP looks good to me, +1 (non-binding) On Fri, May 15, 2020 at 11:51 AM Sophie Blee-Goldman wrote: > Called out! > > Seems like gmail struggles with [...] prefixed subjects. You'd think they > would adapt > all their practices to conform to the

Build failed in Jenkins: kafka-2.4-jdk8 #202

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H22 (ubuntu) in workspace

Re: [DISCUSS] KIP-601: Configurable socket connection timeout

Hi Cheng, I am fine with the rest of the KIP apart from the 10s default. If no one else has any concerns about this new default, let's go with it. Please go ahead and start vote. Regards, Rajini On Fri, May 15, 2020 at 8:21 PM Cheng Tan wrote: > Dear Rajini, > > > Thanks for the reply. > >

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Hi Randall, I think the problem here is that users aren't likely to implement a deprecated method, and may even remove that method from their connector after seeing that it is now deprecated. This would render that connector incompatible with older runtimes. Why deprecate a method that we don't

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Couple of thoughts: 1. If we add new parameters to put(..), and new connectors implement only this method, it makes them backward incompatible with older workers. I think newer connectors may only choose to only implement the latest method, and we are passing the compatibility problems back to

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Hi all, I think if we're going to avoid adding methods to the SinkTaskContext due to concerns over compatibility, we might also want to ensure that the approach we follow instead doesn't have similar pitfalls. An overloaded "put", "initialize", or "start" isn't quite as troublesome with regards

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

On Fri, May 15, 2020 at 3:13 PM Arjun Satish wrote: > Couple of thoughts: > > 1. If we add new parameters to put(..), and new connectors implement only > this method, it makes them backward incompatible with older workers. I > think newer connectors may only choose to only implement the latest

Re: [DISCUSS] KIP-613: Add end-to-end latency metrics to Streams

I am also happy with max/min/99/90. And I buy your naming argument about staleness vs latency. -Matthias On 5/15/20 12:24 PM, Boyang Chen wrote: > Hey Sophie, > > 90/99/min/max make sense to me. > > On Fri, May 15, 2020 at 12:20 PM Sophie Blee-Goldman > wrote: > >> @Matthias >> Regarding

Re: [VOTE] KIP-586: Deprecate commit records without record metadata

This KIP is now accepted. The vote has been open for >72 hours (actually ~4 weeks), and we have three binding +1s (Konstantine, Gwen, me), one non-binding +1 (Mario), and no -1 votes. Thanks, everyone! On Thu, May 14, 2020 at 6:42 PM Gwen Shapira wrote: > +1 (binding) > > Makes a lot of sense

Re: [DISCUSS] KIP-613: Add end-to-end latency metrics to Streams

@Matthias Regarding tracking the 50th percentile, I'll refer you to the 4:53 mark of the video *you* linked: https://youtu.be/lJ8ydIuPFeU?t=293 And that's what he says about the 95th percentile! Imagine what he would say about the 50th :P But seriously, since we can't seem to agree that the mean

Re: [DISCUSS] KIP-601: Configurable socket connection timeout

Dear Rajini, Thanks for the reply. > e have a lot of these and I want to > understand the benefits of the proposed timeout in this case alone. We > currently have a request timeout of 30s. Would you consider adding a 10s > connection timeout? A shorter timeout (10s) at the transportation

[jira] [Created] (KAFKA-10005) Decouple RestoreListener from RestoreCallback and not enable bulk loading for RocksDB

Guozhang Wang created KAFKA-10005: - Summary: Decouple RestoreListener from RestoreCallback and not enable bulk loading for RocksDB Key: KAFKA-10005 URL: https://issues.apache.org/jira/browse/KAFKA-10005

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Hi Konstantine, Thanks for the insight. In that case, I will make the appropriate changes to the KIP to support asynchronous functionality with a BiFunction. Best, Aakash On Fri, May 15, 2020 at 11:45 AM Konstantine Karantasis < konstant...@confluent.io> wrote: > Thanks for the quick response

Build failed in Jenkins: kafka-2.4-jdk8 #201

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H22 (ubuntu) in workspace

Build failed in Jenkins: kafka-2.5-jdk8 #117

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H39 (ubuntu) in workspace

Build failed in Jenkins: kafka-trunk-jdk11 #1461

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H28 (ubuntu) in workspace

[VOTE] KIP-613: Add end-to-end latency metrics to Streams

Hey all, I think we've reached a general consensus on all the non-implementation details of KIP-613 so I'd like to call for a vote. https://cwiki.apache.org/confluence/display/KAFKA/KIP-613%3A+Add+end-to-end+latency+metrics+to+Streams Thanks! Sophie

Re: [VOTE] KIP-607: Add Metrics to Record the Memory Used by RocksDB to Kafka Streams

+1 (binding) -Matthias On 5/15/20 11:48 AM, John Roesler wrote: > Thanks, Bruno! > > I’m +1 (binding) > > -John > > On Fri, May 15, 2020, at 11:32, Sophie Blee-Goldman wrote: >> Thanks Bruno! +1 (non-binding) >> >> Sophie >> >> On Fri, May 15, 2020 at 8:15 AM Bill Bejeck wrote: >> >>> Thanks

Build failed in Jenkins: kafka-2.4-jdk8 #203

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H22 (ubuntu) in workspace

Re: [DISCUSS] KIP-613: Add end-to-end latency metrics to Streams

Hi all, I'd like to clarify/modify one aspect of this KIP, which is to record the staleness at the *completion* of the record's processing by the operator/task in question, rather than on the intake. The task-level metrics will be recorded at the sink node instead of at the source, and the

Re: [DISCUSS] KIP-613: Add end-to-end latency metrics to Streams

Thanks, Sophie! I think this makes perfect sense. It will be much more intuitive to use the metric for the stated motivation this way. I’d be in favor of the proposal after this update. Thanks again for taking this on, -John On Fri, May 15, 2020, at 17:07, Sophie Blee-Goldman wrote: > Hi

Build failed in Jenkins: kafka-2.4-jdk8 #205

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H22 (ubuntu) in workspace

Build failed in Jenkins: kafka-trunk-jdk14 #91

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu) in workspace

Build failed in Jenkins: kafka-trunk-jdk11 #1465

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H28 (ubuntu) in workspace

Build failed in Jenkins: kafka-trunk-jdk11 #1466

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H28 (ubuntu) in workspace

Re: [DISCUSS] KIP-601: Configurable socket connection timeout

Dear Rajini, Thanks for all the feedbacks. They are very helpful for me to do the brainstorming. I’ve incorporated our discuss in the KIP and started a voting thread. Best, - Cheng Tan > On May 15, 2020, at 2:13 PM, Rajini Sivaram wrote: > > Hi Cheng, > > I am fine with the rest of the

Re: [DISCUSS] KIP-613: Add end-to-end latency metrics to Streams

Yeah, I had proposed to replace the "source + sink" task-level metric with "sink only" task-level metrics plus the stateful node-level ones. I completely agree that this proposal should and would look different once we complete those two proposals, ie decouple caching and consolidate suppression.

Build failed in Jenkins: kafka-trunk-jdk8 #4535

See Changes: [github] KAFKA-8611: update documentation for KIP-221 (#8558) -- [...truncated 6.14 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

[jira] [Resolved] (KAFKA-1793) Allow replacing ZooKeeper with a different distributed coordination system

[ https://issues.apache.org/jira/browse/KAFKA-1793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-1793. --- Resolution: Won't Do This is superseeded by KIP-500

Build failed in Jenkins: kafka-trunk-jdk14 #90

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H23 (ubuntu) in workspace

Build failed in Jenkins: kafka-2.5-jdk8 #120

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H39 (ubuntu) in workspace

Build failed in Jenkins: kafka-trunk-jdk11 #1464

See Changes: -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on H28 (ubuntu) in workspace

[jira] [Created] (KAFKA-10008) Symbol not found when running Kafka Streams with RocksDB dependency on MacOS 10.13.6

Victoria Xia created KAFKA-10008: Summary: Symbol not found when running Kafka Streams with RocksDB dependency on MacOS 10.13.6 Key: KAFKA-10008 URL: https://issues.apache.org/jira/browse/KAFKA-10008

  1   2   >