[GitHub] kafka pull request #2894: [KAFKA-5092] [WIP] changed ProducerRecord interfac...

2017-12-13 Thread simplesteph
Github user simplesteph closed the pull request at: https://github.com/apache/kafka/pull/2894 ---

[GitHub] kafka pull request #4323: KAFKA-5849: Add process stop, round trip workload,...

2017-12-13 Thread cmccabe
GitHub user cmccabe opened a pull request: https://github.com/apache/kafka/pull/4323 KAFKA-5849: Add process stop, round trip workload, partitioned test * Implement process stop faults via SIGSTOP / SIGCONT * Implement RoundTripWorkload, which both sends messages, and

[GitHub] kafka pull request #4106: KAFKA-5849: Add partitioned produce consume test

2017-12-13 Thread cmccabe
Github user cmccabe closed the pull request at: https://github.com/apache/kafka/pull/4106 ---

[jira] [Created] (KAFKA-6361) Fast leader fail over can lead to log divergence between replica and follower

2017-12-13 Thread Jason Gustafson (JIRA)
Jason Gustafson created KAFKA-6361: -- Summary: Fast leader fail over can lead to log divergence between replica and follower Key: KAFKA-6361 URL: https://issues.apache.org/jira/browse/KAFKA-6361

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

2017-12-13 Thread Guozhang Wang
Metrics: this is a good point. Note that currently we have two metrics for `skipped-records` on different levels: 1) on the highest level, the thread-level, we have a `skipped-records`, that records all the skipped records due to deserialization errors. 2) on the lower processor-node level, we

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

2017-12-13 Thread Matthias J. Sax
One more after thought: should we add a metric for this? We also have a metric for `skippedDueToDeserializationError-rate` ? -Matthias On 12/6/17 7:54 AM, Bill Bejeck wrote: > Thanks for the clearly written KIP, no further comments from my end. > > -Bill > > On Wed, Dec 6, 2017 at 9:52 AM,

[GitHub] kafka pull request #4322: KAFKA-6126: Remove unnecessary topics created chec...

2017-12-13 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/4322 KAFKA-6126: Remove unnecessary topics created check ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build

[GitHub] kafka pull request #4308: catch and log exceptions thrown in waiters added t...

2017-12-13 Thread xvrl
Github user xvrl closed the pull request at: https://github.com/apache/kafka/pull/4308 ---

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-13 Thread Ted Yu
Tom: bq. create a znode /admin/reassignments/$topic-$partition Looks like the tree structure above should be: /admin/reassignments/$topic/$partition bq. The controller removes /admin/reassignment/$topic/$partition Note the lack of 's' for reassignment. It would be good to make zookeeper paths

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

2017-12-13 Thread Colin McCabe
On Wed, Dec 13, 2017, at 10:00, dan wrote: > > Why not just return > > org.apache.kafka.clients.admin.Config like describeConfigs does? > > brokers have a `num.partitions` config that does not map to a valid > `Config` entry for a topic. Hi Dan, Sorry if I'm misunderstanding something, but why

Re: [DISCUSS] KIP-222 - Add "describe consumer group" to KafkaAdminClient

2017-12-13 Thread Colin McCabe
On Tue, Dec 12, 2017, at 09:39, Jason Gustafson wrote: > Hi Colin, > > They do share the same namespace. We have a "protocol type" field in the > JoinGroup request to make sure that all members are of the same kind. Hi Jason, Thanks. That makes sense. > Very roughly what I was thinking is

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

2017-12-13 Thread dan
> Why not just return > org.apache.kafka.clients.admin.Config like describeConfigs does? brokers have a `num.partitions` config that does not map to a valid `Config` entry for a topic. another added benefit to using `NewTopic` may be (future kip) having the cluster return the actual replica

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

2017-12-13 Thread Colin McCabe
On Tue, Dec 12, 2017, at 19:02, Ewen Cheslack-Postava wrote: > re: API versions, I actually wasn't sure if we needed it or not. I'm fine > if people would prefer just bumping it, but I was actually curious if we > could get away without bumping it. I don't know the behavior of the > broker code

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

2017-12-13 Thread Tom Bentley
Hi Jun and Ted, Jun, you're right that needing one watcher per reassigned partition presents a scalability problem, and using a separate notification path solves that. I also agree that it makes sense to prevent users from using both methods on the same reassignment. Ted, naming the

[jira] [Created] (KAFKA-6360) RocksDB segments not removed when store is closed causes re-initialization to fail

2017-12-13 Thread Damian Guy (JIRA)
Damian Guy created KAFKA-6360: - Summary: RocksDB segments not removed when store is closed causes re-initialization to fail Key: KAFKA-6360 URL: https://issues.apache.org/jira/browse/KAFKA-6360 Project:

[jira] [Created] (KAFKA-6359) Work for KIP-236

2017-12-13 Thread Tom Bentley (JIRA)
Tom Bentley created KAFKA-6359: -- Summary: Work for KIP-236 Key: KAFKA-6359 URL: https://issues.apache.org/jira/browse/KAFKA-6359 Project: Kafka Issue Type: Improvement Reporter: Tom

[GitHub] kafka pull request #4321: KAFKA-6342 : Move workaround for JSON parsing of n...

2017-12-13 Thread umesh9794
GitHub user umesh9794 opened a pull request: https://github.com/apache/kafka/pull/4321 KAFKA-6342 : Move workaround for JSON parsing of non-escaped strings This PR moves the JSON parsing workaround of [this PR](https://github.com/apache/kafka/pull/4303) to new method and uses this

[GitHub] kafka pull request #4320: [WIP] Add logs to debug testHighConcurrencyModific...

2017-12-13 Thread omkreddy
GitHub user omkreddy opened a pull request: https://github.com/apache/kafka/pull/4320 [WIP] Add logs to debug testHighConcurrencyModificationOfResourceAcls test case Not able to reproduce locally. add few logs to check on jenkins looks like some synchronization issue

[jira] [Created] (KAFKA-6358) Per topic producer/fetch_consumer/fetch_follower metrics

2017-12-13 Thread Ricardo Bartolome (JIRA)
Ricardo Bartolome created KAFKA-6358: Summary: Per topic producer/fetch_consumer/fetch_follower metrics Key: KAFKA-6358 URL: https://issues.apache.org/jira/browse/KAFKA-6358 Project: Kafka

[jira] [Created] (KAFKA-6357) Return nonzero code in kafka-consumer-groups.sh tool in case of error

2017-12-13 Thread Rinat Shigapov (JIRA)
Rinat Shigapov created KAFKA-6357: - Summary: Return nonzero code in kafka-consumer-groups.sh tool in case of error Key: KAFKA-6357 URL: https://issues.apache.org/jira/browse/KAFKA-6357 Project: Kafka

[jira] [Created] (KAFKA-6356) UnknownTopicOrPartitionException & NotLeaderForPartitionException and log deletion happening with retention bytes kept at -1.

2017-12-13 Thread kaushik srinivas (JIRA)
kaushik srinivas created KAFKA-6356: --- Summary: UnknownTopicOrPartitionException & NotLeaderForPartitionException and log deletion happening with retention bytes kept at -1. Key: KAFKA-6356 URL: