[jira] [Resolved] (KAFKA-9720) Update gradle to 6.0+

[ https://issues.apache.org/jira/browse/KAFKA-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-9720. Assignee: Ismael Juma Resolution: Fixed > Update gradle to 6.0+ > -- >

Re: Help! Can't add reviewers for Github Kafka PR

Well, anybody familiar with the code can be a reviewer! Of course, a committer needs to merge the PR in the end. But committers are overloaded with reviews, and if others help reviewing, it reduced their workload. -Matthias On 5/17/20 3:36 PM, John Roesler wrote: > Ah, I just reread my

[jira] [Created] (KAFKA-10012) Reducing memory overhead associated with strings in MetricName

Navina Ramesh created KAFKA-10012: - Summary: Reducing memory overhead associated with strings in MetricName Key: KAFKA-10012 URL: https://issues.apache.org/jira/browse/KAFKA-10012 Project: Kafka

[GitHub] [kafka-site] showuon commented on pull request #265: MINOR: Update stream documentation for version 2.5

showuon commented on pull request #265: URL: https://github.com/apache/kafka-site/pull/265#issuecomment-629911823 Same change with the PR to `kafka` repo: https://github.com/apache/kafka/pull/8622/ per @bbejeck 's request. Thanks. :)

Build failed in Jenkins: kafka-trunk-jdk11 #1471

See Changes: [github] Update Gradle to 6.4.1 (#8678) -- [...truncated 4.86 MB...] kafka.controller.ReplicaStateMachineTest >

[GitHub] [kafka-site] showuon opened a new pull request #265: Update stream documentation

showuon opened a new pull request #265: URL: https://github.com/apache/kafka-site/pull/265 1. fix broken links 2. rephrase a sentence 3. update the version number This is an automated message from the Apache Git

Re: [DISCUSS] KIP-602 - Change default value for client.dns.lookup

Ismael What do you think of the PR and the explanation regarding the issue raised in KIP-235? Should I go ahead and build a proper PR? Thanks Badai On Mon, May 11, 2020 at 8:53 AM Badai Aqrandista wrote: > Ismael > > PR created: https://github.com/apache/kafka/pull/8644/files > > Also, as

[VOTE] KIP-610: Error Reporting in Sink Connectors

Hello all, I'd like to open a vote for KIP-610: https://cwiki.apache.org/confluence/display/KAFKA/KIP-610%3A+Error+Reporting+in+Sink+Connectors Thanks, Aakash

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

My apologies, had a typo. Meant to say "I will now open up a vote." Thanks, Aakash On Sun, May 17, 2020 at 4:55 PM Aakash Shah wrote: > Hi all, > > Thanks for all the feedback thus far. I've updated the KIP with all the > suggestions. I will not open up a vote. > > Thanks, > Aakash > > On Sun,

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Hi all, Thanks for all the feedback thus far. I've updated the KIP with all the suggestions. I will not open up a vote. Thanks, Aakash On Sun, May 17, 2020 at 3:45 PM Randall Hauch wrote: > All good points regarding `Future` instead of > `Future`, so +1 to that change. > > A few more nits.

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

All good points regarding `Future` instead of `Future`, so +1 to that change. A few more nits. The following sentences should be removed because they actually describe a change from the current DLQ functionality that already sets `max.in.flight.requests.per.connection=1` by default: "In order to

Re: Help! Can't add reviewers for Github Kafka PR

Ah, I just reread my message. I meant “committer”, of course. Thanks, John On Sun, May 17, 2020, at 16:10, Kowshik Prakasam wrote: > Thanks, John! > > > Cheers, > Kowshik > > On Sun, May 17, 2020 at 8:12 AM John Roesler wrote: > > Hi Kowshik, > > > > You just have to “@“ mention the

Re: [DISCUSS] KIP-553: Enable TLSv1.3 by default and disable all protocols except [TLSV1.2, TLSV1.3]

Hi Nikolay, Quick question, the following is meant to include TLSv1.3 as well, right? Change the value of the SslConfigs.DEFAULT_SSL_ENABLED_PROTOCOLS to > "TLSv1.2" In addition, two more questions: 1. `ssl.protocol` would remain TLSv1.2 with this change. It would be good to explain why

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Thanks for all the feedback, folks. re: having a callback as a parameter, I agree that at this point, it might not add much value to the proposal. re: synchronous vs asynchronous, is the motivation performance/higher throughput? Taking a step back, calling report(..) in the new interface does a

Re: Help! Can't add reviewers for Github Kafka PR

Thanks, John! Cheers, Kowshik On Sun, May 17, 2020 at 8:12 AM John Roesler wrote: > Hi Kowshik, > > You just have to “@“ mention the username of the person you want in a gh > comment. I think you have to be a committee to add labels, reviewers, etc. > > Hope this helps! > -John > > On Sun,

Build failed in Jenkins: kafka-trunk-jdk14 #97

See Changes: [github] Update Gradle to 6.4.1 (#8678) -- [...truncated 3.09 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Build failed in Jenkins: kafka-trunk-jdk8 #4538

See Changes: [github] Update Gradle to 6.4.1 (#8678) -- [...truncated 3.07 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Hi Konstantine, Given that the reporter interface is intentionally agnostic about how records are handled and does not necessarily entail writes to a DLQ, I'm also in favor of not specifying a return type for the reporting mechanism. I'm still unclear on how futures are going to provide any

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Hi all, I've updated the KIP to reflect all the new agreed-upon suggestions. Please let me know if you have any more suggestions. Thanks, Aakash On Sun, May 17, 2020 at 12:06 PM Konstantine Karantasis < konstant...@confluent.io> wrote: > Hi all, > > I'm on board with adding an interface in

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Hi all, I'm on board with adding an interface in the Connect API as Arjun suggested. Slightly higher commitment and maintenance but it also gives us an easier path to future extensions in this scope (error handling). The usage is equivalent to adding just a new method with known types to

[jira] [Created] (KAFKA-10011) lockedTaskDirectories should be cleared when task gets closed dirty in HandleLostAll

Boyang Chen created KAFKA-10011: --- Summary: lockedTaskDirectories should be cleared when task gets closed dirty in HandleLostAll Key: KAFKA-10011 URL: https://issues.apache.org/jira/browse/KAFKA-10011

Re: [DISCUSS] KIP-609: Use Pre-registration and Blocking Calls for Better Transaction Efficiency

My point here is only for the first AddPartitionToTxn request of the transaction, since only that request would potentially be blocked on the previous txn to complete. By deferring it we reduce the blocking time. I think StreamsConfigs override the linger.ms to 100ms not 10ms, so in the best case

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

If that's the case, I think framework should not commit if there are any outstanding records in teh reporter. That would prevent the scenario where we could potentially lose records frm being sent either to Sink/the reporter. WDYT about the KIP including that as part of the design? On Sun, May

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

On Sun, May 17, 2020 at 12:42 PM Magesh kumar Nandakumar < mage...@confluent.io> wrote: > Randall > > Thanks a lot for your thoughts. I was wondering if we would ever have to > make the API asynchronous, we could expose it as a new method right? If > that's a possibility would it be better if the

[jira] [Created] (KAFKA-10010) Should close standby task for safety during HandleLostAll

Boyang Chen created KAFKA-10010: --- Summary: Should close standby task for safety during HandleLostAll Key: KAFKA-10010 URL: https://issues.apache.org/jira/browse/KAFKA-10010 Project: Kafka

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Thanks, Aakash. After thinking about my previous proposal, I would like to retract the suggestion of adding the `report(SinkTask, Throwable, Callable)` method from the new interface for the following reasons: 1. The `Callable` interface requires the sink task developer to handle an error

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Randall Thanks a lot for your thoughts. I was wondering if we would ever have to make the API asynchronous, we could expose it as a new method right? If that's a possibility would it be better if the API explicitly has semantics of a synchronous API if the implementation is indeed going to be

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Hi Randall, Thanks for the suggestions. Now that we are adding an interface, I think it is a good idea to overload the report method to support both cases. > I guess it boils down to this question: do we know today that we will > *never* want the reporter to write asynchronously? Originally, I

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

On Sun, May 17, 2020 at 11:01 AM Magesh kumar Nandakumar < mage...@confluent.io> wrote: > Thanks Randall. The suggestion i made also has a problem when reporter > isn't enabled where it could potentially write records after error records > to sink before failing. > > The other concern i had with

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Thanks, Arjun! This has been very helpful. Looking in your POC and thinking in terms of the current KIP, it sounds like the suggestion is to keep the same method signature for reporting errors, but to change from the `BiFunction` to a new `ErrantRecordReporter` interface. More concretely, I'd

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Thanks Randall. The suggestion i made also has a problem when reporter isn't enabled where it could potentially write records after error records to sink before failing. The other concern i had with reporter being asynchronous. For some reason if the reporter is taking longer because of say a

Re: Help! Can't add reviewers for Github Kafka PR

Hi Kowshik, You just have to “@“ mention the username of the person you want in a gh comment. I think you have to be a committee to add labels, reviewers, etc. Hope this helps! -John On Sun, May 17, 2020, at 04:11, Kowshik Prakasam wrote: > Hi all, > > My intent is to create a PR for review

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Magesh, we have talked above overloading various existing SinkTask methods, and we concluded that this style of evolution complicates migration, whereas providing the reporter via the context follows existing patterns in the API and simplifies backward compatibility concerns. Arjun's research

Help! Can't add reviewers for Github Kafka PR

Hi all, My intent is to create a PR for review in https://github.com/apache/kafka . However I find that I'm unable to add reviewers to my PR. Does this need any specific permissions? If so, please could someone grant me access or help me understand what I need to do to get permissions to add

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Hi Satish, Thank you for your updates. I have some questions around potential use cases when unclean leader election is enabled. It is possible that a range of offsets of a segment which is already offloaded to a tier storage is included in the range of offsets to be truncated. A follower,

Re: [DISCUSS] KIP-610: Error Reporting in Sink Connectors

Have we considered returning error records by overriding flush/precommit? If we think aesthetics is important this on my opinion is one possible abstractions that could be cleaner. This would also mean that connector developers wouldn't have to worry about a new reporter or think if its