Build failed in Jenkins: kafka-trunk-jdk11 #772

2019-08-23 Thread Apache Jenkins Server
See Changes: [jason] KAFKA-8753; Expose controller topic deletion metrics (KIP-503) (#7156) [github] MINOR: Move the resetting from revoked to the thread loop (#7243) --

Re: [ DISCUSS ] KIP-512:Adding headers to RecordMetaData

2019-08-23 Thread Gwen Shapira
I am afraid I don't understand the proposal. The RecordMetadata is information returned from the broker regarding the record. The producer already has the record (including the headers), so why would the broker need to send the headers back as part of the metadata? On Fri, Aug 23, 2019 at 4:22 PM

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-23 Thread Arjun Satish
Jason, Thanks for your comments! I understand the usability issues with JMX that you mention. But it was chosen for the following reasons: 1. Cross-cutting functionality across different components (Kafka brokers, Connect workers and even with Streams jobs). If we go down the REST route, then

[DISCUSS] KIP-512:Adding headers to RecordMetaData

2019-08-23 Thread Renuka M
Hi All, I am starting this thread to discuss https://cwiki.apache.org/confluence/display/KAFKA/KIP-512%3AAdding+headers+to+RecordMetaData . Please provide the feedback. Thanks Renuka M >

[ DISCUSS ] KIP-512:Adding headers to RecordMetaData

2019-08-23 Thread Renuka M
Hi All, I am starting this thread to discuss https://cwiki.apache.org/confluence/display/KAFKA/KIP-512%3AAdding+headers+to+RecordMetaData . Please provide the feedback. Thanks Renuka M

[jira] [Created] (KAFKA-8831) Joining a new instance sometimes does not cause rebalancing

2019-08-23 Thread Chris Pettitt (Jira)
Chris Pettitt created KAFKA-8831: Summary: Joining a new instance sometimes does not cause rebalancing Key: KAFKA-8831 URL: https://issues.apache.org/jira/browse/KAFKA-8831 Project: Kafka

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-23 Thread Jason Gustafson
Let me elaborate a little bit. We made the decision early on for Connect to use HTTP instead of Kafka's custom RPC protocol. In exchange for losing some hygienic consistency with Kafka, we took easier integration with management tools. The scope of the connect REST APIs is really managing the

[jira] [Resolved] (KAFKA-8753) Add JMX for number of topics marked for deletion

2019-08-23 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8753. Fix Version/s: 2.4.0 Resolution: Fixed > Add JMX for number of topics marked for

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-23 Thread Ryanne Dolan
Thanks Colin, sgtm. Please make this clear in the KIP -- otherwise it is hard to nail down what we are voting for. Ryanne On Fri, Aug 23, 2019, 12:58 PM Colin McCabe wrote: > On Fri, Aug 23, 2019, at 06:24, Ryanne Dolan wrote: > > Colin, can you outline what specifically would be in scope for

Build failed in Jenkins: kafka-trunk-jdk8 #3867

2019-08-23 Thread Apache Jenkins Server
See Changes: [manikumar] KAFKA-8698: Fix typo in ListOffsetResponse v0 protocol field name -- [...truncated 5.91 MB...] org.apache.kafka.connect.json.JsonConverterTest >

[jira] [Created] (KAFKA-8830) KIP-512: Adding headers to RecordMetaData

2019-08-23 Thread Renuka Metukuru (Jira)
Renuka Metukuru created KAFKA-8830: -- Summary: KIP-512: Adding headers to RecordMetaData Key: KAFKA-8830 URL: https://issues.apache.org/jira/browse/KAFKA-8830 Project: Kafka Issue Type: New

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-23 Thread Jason Gustafson
Thanks Stan, good catch. I have updated the KIP. I will plan to close the vote Monday if there are no objections. -Jason On Fri, Aug 23, 2019 at 11:14 AM Colin McCabe wrote: > On Fri, Aug 23, 2019, at 11:08, Stanislav Kozlovski wrote: > > Thanks for the KIP, this is very helpful > > > > I had

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-23 Thread Colin McCabe
On Fri, Aug 23, 2019, at 11:08, Stanislav Kozlovski wrote: > Thanks for the KIP, this is very helpful > > I had an offline discussion with Jason and we discussed the semantics of > the underMinIsr/atMinIsr metrics. The current proposal would expose a gap > where we could report URP but no MinIsr.

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-23 Thread Stanislav Kozlovski
Thanks for the KIP, this is very helpful I had an offline discussion with Jason and we discussed the semantics of the underMinIsr/atMinIsr metrics. The current proposal would expose a gap where we could report URP but no MinIsr. A brief example: original replica set = [0,1,2] new replica set =

Re: [VOTE] KIP-352: Distinguish URPs caused by reassignment

2019-08-23 Thread Colin McCabe
+1 (binding). cheers, Colin On Tue, Aug 20, 2019, at 10:55, Jason Gustafson wrote: > Hi All, > > I'd like to start a vote on KIP-352, which is a follow-up to KIP-455 to > fix > a long-known shortcoming of URP reporting and to improve reassignment > monitoring: >

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-23 Thread Colin McCabe
On Fri, Aug 23, 2019, at 06:24, Ryanne Dolan wrote: > Colin, can you outline what specifically would be in scope for this KIP vs > deferred to the follow-on KIPs you've mentioned? Maybe a Future Work > section? Is the idea to get to the bridge release with this KIP, and then > go from there? > >

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-23 Thread Jason Gustafson
Hi Arjun, Thanks for the KIP. Do we really need a JMX-based API? Is there literally anyone in the world that wants to use JMX if they don't have to? I thought one of the major motivations of KIP-412 was how much of a pain JMX is. Thanks, Jason On Mon, Aug 19, 2019 at 5:28 PM Arjun Satish

Build failed in Jenkins: kafka-trunk-jdk11 #771

2019-08-23 Thread Apache Jenkins Server
See Changes: [manikumar] KAFKA-8698: Fix typo in ListOffsetResponse v0 protocol field name -- [...truncated 2.59 MB...] org.apache.kafka.trogdor.common.WorkerUtilsTest >

[jira] [Created] (KAFKA-8829) Provide factories for creating Window instances

2019-08-23 Thread Marcos Passos (Jira)
Marcos Passos created KAFKA-8829: Summary: Provide factories for creating Window instances Key: KAFKA-8829 URL: https://issues.apache.org/jira/browse/KAFKA-8829 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-8828) [BC Break] Global store returns a TimestampedKeyValueStore in 2.3

2019-08-23 Thread Marcos Passos (Jira)
Marcos Passos created KAFKA-8828: Summary: [BC Break] Global store returns a TimestampedKeyValueStore in 2.3 Key: KAFKA-8828 URL: https://issues.apache.org/jira/browse/KAFKA-8828 Project: Kafka

[jira] [Created] (KAFKA-8827) kafka followers intermittent traffic

2019-08-23 Thread frid (Jira)
frid created KAFKA-8827: --- Summary: kafka followers intermittent traffic Key: KAFKA-8827 URL: https://issues.apache.org/jira/browse/KAFKA-8827 Project: Kafka Issue Type: Bug Affects Versions:

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

2019-08-23 Thread Ryanne Dolan
Colin, can you outline what specifically would be in scope for this KIP vs deferred to the follow-on KIPs you've mentioned? Maybe a Future Work section? Is the idea to get to the bridge release with this KIP, and then go from there? Ryanne On Thu, Aug 22, 2019, 11:58 AM Colin McCabe wrote: >

[jira] [Resolved] (KAFKA-8698) ListOffsets Response protocol documentation

2019-08-23 Thread Manikumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-8698. -- Fix Version/s: 2.4.0 Resolution: Fixed > ListOffsets Response protocol documentation >

Re: [DISCUSS] KIP-511: Collect and Expose Client's Name and Version in the Brokers

2019-08-23 Thread Magnus Edenhill
Great proposal, this feature is well overdue! 1) >From an operator's perspective I don't think the kafka client implementation name and version are sufficient, I also believe the application name and version are of interest. You could have all applications in your cluster run the same kafka