Build failed in Jenkins: Kafka » kafka-2.5-jdk8 #11

See Changes: [github] Backport Jenkinsfile to 2.5 (#9327) -- [...truncated 3.11 MB...] org.apache.kafka.streams.test.ConsumerRecordFactoryTest >

Re: [DISCUSS] Release Deadlines

I added one week between KIP freeze and feature freeze as my observation was that we tend to vote some last minutes KIPs and than rush the implementation within one week. Having two weeks to implement last minute KIPs should lead the a more stable release branch when we cut it after feature

Re: [VOTE] KIP-673: Emit JSONs with new auto-generated schema

Thank you, this will be quite helpful. +1 (binding) On Wed, Sep 30, 2020 at 11:19 AM Anastasia Vela wrote: > > Hi everyone, > > Thanks again for the discussion. I'd like to start the vote for this KIP. >

Re: [DISCUSS] KIP-630: Kafka Raft Snapshot

Hello Jose, Thanks for the replies and the KIP updates. Just want to clarify one more thing regarding my previous comment 3): I understand that when a snapshot has completed loading, then we can use it in our handling logic of vote request. And I understand that: 1) Before a snapshot has been

Re: [DISCUSS] Release Deadlines

Thanks for proposing this Matthias. A couple of questions: 1. How did you decide where to increase the time? 2. Do you think there's a risk that having more time won't necessarily help, we will just try to fit more things? I've seen that happen in similar circumstances. Ismael On Tue, Sep 29,

[jira] [Resolved] (KAFKA-10326) Both serializer and deserializer should be able to see the generated client id

[ https://issues.apache.org/jira/browse/KAFKA-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-10326. --- Fix Version/s: 2.7.0 Resolution: Fixed > Both serializer and deserializer should be

Re: [DISCUSS] Release Deadlines

Thanks for this, Matthias! I’m in favor. I’m glad we have all been working hard to stabilize our releases and offer a high quality project, even if it means delays. However, the last several releases have demonstrated that we consistently need more time than we have allocated. Rather than risk

Re: [DISCUSS] KIP-674: API to Aggregate Metrics in Kafka Streams

Hello Bruno, Thanks for the proposed KIP! Here are some thoughts: 1) Regarding Sophie's comment 1), I think there may be one merit to defining the aggregated metric on a different level (though only on a higher level would generally make sense). Although in order for the aggregated metric to

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #101

See Changes: [cshapi] KAFKA-10390; Remove ignore case option when grep process info to be more specific [github] KAFKA-10277: Allow null keys with non-null mappedKey in KStreamKGlobalTable join

Re: [DISCUSS] KIP-674: API to Aggregate Metrics in Kafka Streams

Thanks for the KIP! This looks really useful 1) > If the recording level for the application is set to INFO, a DEBUG-level > metric that should be aggregated will > not record values even if the metrics that records the aggregation is on > recording level INFO I think I'm missing something

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #103

See Changes: [github] KAFKA-10351: Add tests for IOExceptions for GlobalStateManagerImpl/OffsetCheckpoint (#9121) [cshapi] KAFKA-10390; Remove ignore case option when grep process info to be more

Build failed in Jenkins: Kafka » kafka-trunk-jdk15 #135

See Changes: [github] KAFKA-10351: Add tests for IOExceptions for GlobalStateManagerImpl/OffsetCheckpoint (#9121) [cshapi] KAFKA-10390; Remove ignore case option when grep process info to be more

Jenkins build is back to normal : Kafka » kafka-trunk-jdk8 #100

See

Re: [DISCUSS] KIP-631: The Quorum-based Kafka Controller

On Tue, Sep 29, 2020, at 17:43, Jason Gustafson wrote: > Hey Colin, > > Thanks for the hard work on this proposal. > > I'm gradually coming over to the idea of the controllers having separate > IDs. One of the benefits is that it allows us to separate the notion of > controller liveness from

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #102

See Changes: [github] KAFKA-6585: Consolidate duplicated logic on reset tools (#9255) -- [...truncated 6.70 MB...]

[jira] [Created] (KAFKA-10561) Support microseconds precision for Timestamps

Daniel Petisme created KAFKA-10561: -- Summary: Support microseconds precision for Timestamps Key: KAFKA-10561 URL: https://issues.apache.org/jira/browse/KAFKA-10561 Project: Kafka Issue

[jira] [Created] (KAFKA-10560) Request for AK bin command to get AK cluster ID

Yeva Byzek created KAFKA-10560: -- Summary: Request for AK bin command to get AK cluster ID Key: KAFKA-10560 URL: https://issues.apache.org/jira/browse/KAFKA-10560 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-516: Topic Identifiers

Hi, Justine, Thanks for the summary. Just a few minor comments blow. 30. {"name": "id", "type": "string", "doc": "version id"}}: The doc should say UUID. The issue seems unfixed. 40. Since UUID is public facing, could you include its definition? 41. StopReplicaResponse still includes the

Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

Thanks Dongjin, It typically is nicer to be able to see usage examples, so I'd certainly be in favor if you're willing to add it to the KIP. I'm wondering if it's possible to implement the whole ReadOnlyKeyValueStore interface as proposed, if we really go ahead and just internally query into the

Build failed in Jenkins: Kafka » kafka-2.6-jdk8 #21

See Changes: [John Roesler] MINOR: Do not swallow exception when collecting PIDs (#8914) -- [...truncated 6.31 MB...]

[jira] [Created] (KAFKA-10559) Don't shutdown the entire app upon TimeoutException during internal topic validation

Sophie Blee-Goldman created KAFKA-10559: --- Summary: Don't shutdown the entire app upon TimeoutException during internal topic validation Key: KAFKA-10559 URL:

Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #99

See Changes: [github] KAFKA-10469: Resolve logger levels hierarchically (#9266) -- [...truncated 6.65 MB...] org.apache.kafka.streams.TopologyTestDriverTest

Re: [VOTE] KIP-673: Emit JSONs with new auto-generated schema

Hi Anastasia, This looks like a great change that will make debugging cluster behavior much easier. +1 non binding. Lucas On Wed, Sep 30, 2020 at 11:19 AM Anastasia Vela wrote: > Hi everyone, > > Thanks again for the discussion. I'd like to start the vote for this KIP. > >

[VOTE] KIP-673: Emit JSONs with new auto-generated schema

Hi everyone, Thanks again for the discussion. I'd like to start the vote for this KIP. https://cwiki.apache.org/confluence/display/KAFKA/KIP-673%3A+Emit+JSONs+with+new+auto-generated+schema Thanks, Anastasia

Build failed in Jenkins: Kafka » kafka-trunk-jdk11 #101

See Changes: [github] KAFKA-10469: Resolve logger levels hierarchically (#9266) -- [...truncated 6.71 MB...] org.apache.kafka.streams.TopologyTestDriverTest

[jira] [Created] (KAFKA-10558) Fetch Session Cache Performance Improvement

Alyssa Huang created KAFKA-10558: Summary: Fetch Session Cache Performance Improvement Key: KAFKA-10558 URL: https://issues.apache.org/jira/browse/KAFKA-10558 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-630: Kafka Raft Snapshot

Hi, Jose, Thanks for the updated KIP. A few more comments below. 40. LBO: Code wise, logStartOffset is used in so many places like Log, ReplicaManager, LogCleaner, ReplicaFetcher, checkpoint files, etc. I am not if it's worth renaming in all those places. If the main concern is to avoid

Re: [DISCUSS] KIP-516: Topic Identifiers

Hello again, I've taken some time to discuss some of the remaining points brought up by the previous emails offline. Here are some of the conclusions. 1. Directory Structure: There was some talk about whether the directory structure should be changed to replace all topic names with topic IDs.

[jira] [Resolved] (KAFKA-6585) Consolidate duplicated logic on reset tools

[ https://issues.apache.org/jira/browse/KAFKA-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-6585. Fix Version/s: 2.7.0 Resolution: Fixed > Consolidate duplicated logic on reset

Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

> It seems like it must be a ReadOnlyKeyValueStore. Does that sound right? Yes, it is. Would it be better to add a detailed description of how this feature effects interactive query, with examples? Best, Dongjin On Tue, Sep 29, 2020 at 10:31 AM John Roesler wrote: > Hi Dongjin, > > Thanks!

[jira] [Created] (KAFKA-10557) Missing docs when describing topic configs including documentation

Mickael Maison created KAFKA-10557: -- Summary: Missing docs when describing topic configs including documentation Key: KAFKA-10557 URL: https://issues.apache.org/jira/browse/KAFKA-10557 Project:

Re: [DISCUSS] KIP-406: GlobalStreamThread should honor custom reset policy

I guess we need to have some cleanup mechanism for this case anyway, because, the global thread can enter RESTORING state at any point in time, and thus, even if we set a flag to pause processing on the StreamThreads we are subject to a race condition. Beside that, on a high level I am fine with

[jira] [Created] (KAFKA-10556) NPE if sasl.mechanism is unrecognized

Ron Dagostino created KAFKA-10556: - Summary: NPE if sasl.mechanism is unrecognized Key: KAFKA-10556 URL: https://issues.apache.org/jira/browse/KAFKA-10556 Project: Kafka Issue Type: Task

[jira] [Created] (KAFKA-10555) Improve client state machine

Matthias J. Sax created KAFKA-10555: --- Summary: Improve client state machine Key: KAFKA-10555 URL: https://issues.apache.org/jira/browse/KAFKA-10555 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] KIP-567: Kafka Cluster Audit (new discussion)

Hi Daniel, I think in this sense we can use the precedence set with the KAfkaAdminClient. It has *Result and *Options classes which in this interpretation are similar in versioning and usage as they transform and convey the responses of the protocol in a minimalistic API. I've modified the KIP a

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

Thanks, Matthias! I can certainly document it. I didn't bother because the old Processor, Supplier, and Context will themselves be deprecated, so any method that handles them won't be able to avoid the deprecation warning. Nevertheless, it doesn't hurt just to explicitly deprecated those methods.

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

Thanks John. I like the proposal. Btw: I was just going over the KIP and realized that we add new methods to `StreamBuilder`, `Topology`, and `KStream` that take the new `ProcessorSupplier` class -- should we also deprecate the corresponding existing ones that take the old `ProcessorSupplier`?

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

Thanks Paul and Sophie, Your feedback certainly underscores the need to be explicit in the javadoc about why that parameter is Optional. Getting this kind of feedback before the release is exactly the kind of outcome we hope to get from the KIP process! Thanks, -John On Tue, 2020-09-29 at 22:32

Re: [VOTE] KIP-671: Add method to Shutdown entire Streams Application

Bruno Cadonna 4:51 AM (2 hours ago) to dev Thank you all for voting! This KIP is accepted with +3 binding (Guozhang, Bill, Matthias) and +2 non-binding (Bruno, Leah). Matthias, we will take care of the global threads, and for the replacement that will depend on Kip-663. Best, On Wed, Sep 30,

Re: [VOTE] KIP-671: Add method to Shutdown entire Streams Application

Thanks a lot Walker! +1 (non-binding) Best, Bruno On 30.09.20 03:10, Matthias J. Sax wrote: Thanks Walker. The proposed API changes LGTM. +1 (binding) One minor nit: you should also mention the global-thread that also needs to be shutdown if requested by the user. Minor side question:

Re: [VOTE] KIP-663: API to Start and Shut Down Stream Threads and to Request Closing of Kafka Streams Clients

Thank you all for voting! This KIP is accepted with 3 binding +1 (Guozhang, John, Matthias). Best, Bruno On 29.09.20 22:24, Matthias J. Sax wrote: +1 (binding) I am not super happy with the impact on the client state. For example, I don't understand why it's ok to scale out if we lose one

[DISCUSS] KIP-674: API to Aggregate Metrics in Kafka Streams

Hi, I would like to propose the following KIP to add an API to the Kafka Streams client to aggregate metrics. https://cwiki.apache.org/confluence/display/KAFKA/KIP-674%3A+API+to+Aggregate+Metrics+in+Kafka+Streams Best, Bruno

[jira] [Created] (KAFKA-10554) Perform follower truncation based on epoch offsets returned in Fetch response

Rajini Sivaram created KAFKA-10554: -- Summary: Perform follower truncation based on epoch offsets returned in Fetch response Key: KAFKA-10554 URL: https://issues.apache.org/jira/browse/KAFKA-10554

Re: [DISCUSS] KIP-673: Emit JSONs with new auto-generated schema

Hi Anastasia, Is the concern that deserialization would have no guarantee as to the order > it appears? > No, it's simply that when reading the logs screeds of unformatted JSON can be hard to read, especially when more important fields come later in an object (or after an array). This is also a