Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

Hi, Tom, Thanks for the updated KIP. A few more comments below. 10. The proposal now stores the reassignment for all partitions in /admin/reassignment_requests/request_xxx. If the number of reassigned partitions is larger, the ZK write may hit the default 1MB limit and fail. An alternative

[GitHub] kafka-site issue #112: Migrate Streams Dev Guide content to AK

Github user derrickdoo commented on the issue: https://github.com/apache/kafka-site/pull/112 @joel-hamill just took a look at your branch. Content looks good, but I'd suggest adding that horizontal subnav on the developer guide page for consistency and make sure users can easily jump

[GitHub] kafka pull request #4335: MINOR: improve JavaDocs for KafkaStreams, KafkaPro...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/4335 MINOR: improve JavaDocs for KafkaStreams, KafkaProducer, KafkaConsumer You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka

[GitHub] kafka pull request #4334: [KAFKA-6265] GlobalKTable missing #queryableStoreN...

GitHub user ConcurrencyPractitioner opened a pull request: https://github.com/apache/kafka/pull/4334 [KAFKA-6265] GlobalKTable missing #queryableStoreName() You can merge this pull request into a Git repository by running: $ git pull

[GitHub] kafka pull request #4333: MINOR: Use TopicPartition in ConsumerGroupCommand ...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/4333 MINOR: Use TopicPartition in ConsumerGroupCommand instead of TopicAndPartition where possible ### Committer Checklist (excluded from commit message) - [ ] Verify design and

[GitHub] kafka pull request #4332: KAFKA-6302: Improve AdmintClient JavaDocs

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/4332 KAFKA-6302: Improve AdmintClient JavaDocs You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka kafka-6302-update-admin-client-javadoc

[GitHub] kafka pull request #3990: KAFKA-5473: handle ZK session expiration properly ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3990 ---

[GitHub] kafka pull request #4331: [WIP] MINOR: Improve Join integration test coverag...

GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/4331 [WIP] MINOR: Improve Join integration test coverage, PART I 1. Merge `KStreamRepartitionJoinTest.java` into `JoinIntegrationTest.java` with augmented stream-stream join. 2. Make the

[jira] [Created] (KAFKA-6375) Follower replicas can never catch up to be ISR due to creating ReplicaFetcherThread failed.

Rong Tang created KAFKA-6375: Summary: Follower replicas can never catch up to be ISR due to creating ReplicaFetcherThread failed. Key: KAFKA-6375 URL: https://issues.apache.org/jira/browse/KAFKA-6375

Jenkins build is back to normal : kafka-trunk-jdk8 #2279

See

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

Jan, Thanks for the updated KIP, and the raised questions. Here are my thoughts around the "back and forth mapper" approach on your wiki: 1) regarding the key-value types of KTableValueGetter, we do not necessarily enforce its template K, V to be the same as its calling Processor, although

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

+1 On Fri, Dec 15, 2017 at 3:57 PM, Guozhang Wang wrote: > +1 (binding), thanks! > > On Fri, Dec 15, 2017 at 11:56 AM, Ted Yu wrote: > > > Hi, > > Here is the discussion thread: > > > > http://search-hadoop.com/m/Kafka/uyzND12QnH514pPO9?subj= > >

[jira] [Created] (KAFKA-6374) Constant Consumer Errors after replacing a broker

Aravind Velamur Srinivasan created KAFKA-6374: - Summary: Constant Consumer Errors after replacing a broker Key: KAFKA-6374 URL: https://issues.apache.org/jira/browse/KAFKA-6374 Project:

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

+1 (binding), thanks! On Fri, Dec 15, 2017 at 11:56 AM, Ted Yu wrote: > Hi, > Here is the discussion thread: > > http://search-hadoop.com/m/Kafka/uyzND12QnH514pPO9?subj= > Re+DISCUSS+KIP+239+Add+queryableStoreName+to+GlobalKTable > > Please vote on this KIP. > > Thanks >

Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API

If there are no other feedback or suggestion on this KIP, I'll start a vote early next week. Thanks. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 11/29/2017 03:18 PM Subject:Re: [DISCUSS] KIP-231: Improve the Required ACL of

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

Regarding the record context inheritance: I agree it may be a better idea for now to drop the information when we cannot come up with a consensus about how the record context should be inherited. Like Bill I was a bit worried about the lacking of such data lineage information for trouble shooting

Re: [VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

+1 On 12/15/17 11:56 AM, Ted Yu wrote: > Hi, > Here is the discussion thread: > > http://search-hadoop.com/m/Kafka/uyzND12QnH514pPO9?subj=Re+DISCUSS+KIP+239+Add+queryableStoreName+to+GlobalKTable > > Please vote on this KIP. > > Thanks > signature.asc Description: OpenPGP digital signature

Jenkins build is back to normal : kafka-trunk-jdk7 #3041

See

[VOTE] KIP-239 Add queryableStoreName() to GlobalKTable

Hi, Here is the discussion thread: http://search-hadoop.com/m/Kafka/uyzND12QnH514pPO9?subj=Re+DISCUSS+KIP+239+Add+queryableStoreName+to+GlobalKTable Please vote on this KIP. Thanks

[jira] [Created] (KAFKA-6373) Log end offset of input table changing during restore

Charles Crain created KAFKA-6373: Summary: Log end offset of input table changing during restore Key: KAFKA-6373 URL: https://issues.apache.org/jira/browse/KAFKA-6373 Project: Kafka Issue

Re: [DISCUSS] KIP-239 Add queryableStoreName() to GlobalKTable

Ted, Thanks for creating the KIP. I have made a pass over it and it looks good to me. Note that since our current KafkaStreams#store(storeName, ..) already checks for global stores first with the given storeName (this is exactly the queryableStoreName). Having this KIP would naturally allow

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

On Fri, Dec 15, 2017, at 08:56, dan wrote: > i didn't think it would be appropriate to return `num.partitions` as a > Config for a Topic since that is not a valid config for that resource > type. > > yes, that is another option i have on the wiki in the Rejected > Alternatives section (though,

Re: Permission to create KIP

Hi, Vincent, Thanks for your interest. Just gave you the wiki permission. Jun On Fri, Dec 15, 2017 at 10:37 AM, Qiao Meng wrote: > Hi, > > Can I get the permission to create KIP? my user handle is > "suanmeiguo", or using email "suanmei...@gmail.com". > > Thank you! > >

Re: [VOTE] KIP-238: Expose Kafka cluster ID in Connect REST API

+1 On Fri, Dec 15, 2017 at 10:49 AM, Ewen Cheslack-Postava wrote: > Discussion seems to have tapered off, so I'd like to start the vote on > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 238%3A+Expose+Kafka+cluster+ID+in+Connect+REST+API > > Obviously +1 (binding)

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

Please create corresponding JIRA. For class Reassignment, it seems you forgot to include set of brokers. For class DescribeReassignmentsResult: public KafkaFuture reassignments(); the return value should be a Collection. On Fri, Dec 15, 2017 at 10:16 AM, Tom Bentley

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

Tom, I think it would be useful to be able to subscribe yourself on updates of reassignment changes. Our internal kafka supervisor and monitoring tools are currently subscribed to these changes in zookeeper so they can babysit our clusters. I think it would be nice if we could receive these

[VOTE] KIP-238: Expose Kafka cluster ID in Connect REST API

Discussion seems to have tapered off, so I'd like to start the vote on https://cwiki.apache.org/confluence/display/KAFKA/KIP-238%3A+Expose+Kafka+cluster+ID+in+Connect+REST+API Obviously +1 (binding) from me :) -Ewen

Permission to create KIP

Hi, Can I get the permission to create KIP? my user handle is "suanmeiguo", or using email "suanmei...@gmail.com". Thank you! -Vincent

[jira] [Created] (KAFKA-6372) Trogdor should use LogContext for log messages

Colin P. McCabe created KAFKA-6372: -- Summary: Trogdor should use LogContext for log messages Key: KAFKA-6372 URL: https://issues.apache.org/jira/browse/KAFKA-6372 Project: Kafka Issue Type:

[GitHub] kafka pull request #4271: KAFKA-5526: Additional `--describe` views for Cons...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4271 ---

[GitHub] kafka pull request #4330: [WIP] KAFKA-6359: KIP-236 interruptible reassignme...

GitHub user tombentley opened a pull request: https://github.com/apache/kafka/pull/4330 [WIP] KAFKA-6359: KIP-236 interruptible reassignments This is a WIP for KIP-236. All the existing tests (via the `/admin/reassign_partitions` path) still pass, and I've added a couple of tests

[GitHub] kafka pull request #4329: MINOR: stabilize flaky system tests

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/4329 MINOR: stabilize flaky system tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka hotfix-system-tests Alternatively you can

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

Just wanted to mention that I've started KIP-240, which builds on top of this one to provide an AdminClient API for listing and describing reassignments. On 15 December 2017 at 14:34, Tom Bentley wrote: > > Should we seek to improve this algorithm in this KIP, or leave

[DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

Hi, KIP-236 lays the foundations for AdminClient APIs to do with partition reassignment. I'd now like to start discussing KIP-240, which adds APIs to the AdminClient to list and describe the current reassignments.

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

i didn't think it would be appropriate to return `num.partitions` as a Config for a Topic since that is not a valid config for that resource type. yes, that is another option i have on the wiki in the Rejected Alternatives section (though, its really just an alternative until something else

Re: [VOTE] Allowing write access to GitHub repositories (aka GitBox)

Thanks to everyone who voted and contributed to the discussion. The vote passes with 7 binding votes (Damian, Rajini, Jason, Gwen, Guozhang, Sriram, Ismael) and 2 non-binding votes (Manikumar and Tom). I will file a JIRA ticket in the Apache Infra project requesting the migration to GitBox.

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

> Should we seek to improve this algorithm in this KIP, or leave that as a later optimisation? I've updated the KIP with a proposed algorithm. On 14 December 2017 at 09:57, Tom Bentley wrote: > Thanks Ted, now fixed. > > On 13 December 2017 at 18:38, Ted Yu

Build failed in Jenkins: kafka-trunk-jdk8 #2278

See Changes: [damian.guy] KAFKA-6086: Provide for custom error handling when Kafka Streams fails -- [...truncated 404.40 KB...] kafka.utils.CoreUtilsTest >

Build failed in Jenkins: kafka-trunk-jdk7 #3040

See Changes: [damian.guy] KAFKA-6086: Provide for custom error handling when Kafka Streams fails -- [...truncated 404.02 KB...] kafka.log.BrokerCompressionTest >

[GitHub] kafka pull request #4165: KAFKA-6086: Provide for custom error handling when...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4165 ---

Re: [VOTE] KIP-225 - Use tags for consumer “records.lag” metrics

Thank you very much for the votes. The vote is now closed with 3 binding +1 (Becket Qin, Jun Rao and Jason Gustafson). On Mon, Dec 11, 2017 at 11:21 PM, Jason Gustafson wrote: > +1. Thanks for the KIP. > > On Mon, Dec 11, 2017 at 1:54 AM, charly molter

[GitHub] kafka pull request #4328: KAFKA-6371 Fix DelayedFetch toString

GitHub user mayt opened a pull request: https://github.com/apache/kafka/pull/4328 KAFKA-6371 Fix DelayedFetch toString You can merge this pull request into a Git repository by running: $ git pull https://github.com/mayt/kafka KAFKA-6371 Alternatively you can review and

[jira] [Created] (KAFKA-6371) FetchMetadata creates unneeded Strings on instantiation

Maytee Chinavanichkit created KAFKA-6371: Summary: FetchMetadata creates unneeded Strings on instantiation Key: KAFKA-6371 URL: https://issues.apache.org/jira/browse/KAFKA-6371 Project: Kafka

[GitHub] kafka pull request #4327: KAFKA-6370: KafkaMetricsGroup.toScope should filte...

GitHub user huxihx opened a pull request: https://github.com/apache/kafka/pull/4327 KAFKA-6370: KafkaMetricsGroup.toScope should filter out tags null value KafkaMetricsGroup.toScope should filter out tags with value of `null` to avoid NullPointerException thrown. *More