Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

I have added the timeout I mentioned before, because it makes the implementation of topic alteration more symmetric with the topic creation APIs. I have also added a section ("Policy") on retrofitting the CreateTopicPolicy's rules to topic alteration requests, and made a few other minor fixes.

[jira] [Created] (KAFKA-5691) ZKUtils.CreateRecursive should handle NOAUTH

Ryan P created KAFKA-5691: - Summary: ZKUtils.CreateRecursive should handle NOAUTH Key: KAFKA-5691 URL: https://issues.apache.org/jira/browse/KAFKA-5691 Project: Kafka Issue Type: Bug

[GitHub] kafka pull request #3608: KAFKA-5691- handle NOAUTH in zkUtils.CreateRecursi...

GitHub user rnpridgeon opened a pull request: https://github.com/apache/kafka/pull/3608 KAFKA-5691- handle NOAUTH in zkUtils.CreateRecursive Currently old consumers are unable to register with secure ZK installations. Handling NOAUTH prevents failures when checking /consumers You

[GitHub] kafka pull request #3607: [DO NOT MERGE] Existing StreamThread exception han...

GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/3607 [DO NOT MERGE] Existing StreamThread exception handling issues This is for @dguy as a reference while working on the first step of KAFKA-5152, as a list of existing issues that need to be

[GitHub] kafka pull request #3606: MINOR: Read commit ID from env if set.

GitHub user maxzheng opened a pull request: https://github.com/apache/kafka/pull/3606 MINOR: Read commit ID from env if set. This allows a build system to set the correct commit ID when .git/HEAD would be wrong if there are local commits for build purposes. You can merge this

[GitHub] kafka pull request #3589: KAFKA-3623: KStreamTestDriver extends ExternalReso...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3589 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #3574: HOTFIX: handle commit failed exception on stream t...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3574 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Handler Threads in SocketServer

Hello, I have recently started looking into the Kafka Server code to get a deeper understanding of the internals and I have a few questions about the SocketServer implementation. The documentation for the SocketServer mentions: /** * The threading model is * 1 Acceptor thread that

Re: Handler Threads in SocketServer

Hi Haseeb, Yes, KafkaRequestHandler is a handler thread indeed. The `Handler` singleton object you refer to is unused. I submitted a PR to remove it. Thanks, Ismael On Tue, Aug 1, 2017 at 10:10 PM, Javed, Haseeb wrote: > Hello, > > > I have recently started

[jira] [Created] (KAFKA-5690) kafka-acls command should be able to list per principal

Koelli Mungee created KAFKA-5690: Summary: kafka-acls command should be able to list per principal Key: KAFKA-5690 URL: https://issues.apache.org/jira/browse/KAFKA-5690 Project: Kafka Issue

Jenkins build is back to normal : kafka-trunk-jdk7 #2592

See

Jenkins build is back to normal : kafka-trunk-jdk8 #1866

See

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

Hi, Regarding the double API I agree on having only one. Compared to an HTTP REST API you could have an "already executed" (fast) operation so the HTTP response status code could be just 200 OK while a "long running" operation could have an HTTP response status code as 202 ACCEPTED (but the

Re: Command tools : from Scala to Java, from Zookeeper utils to Admin Client API

Hi Paolo, Another option is to write the new tool in Java without support for `--zookeeper` and include some logic in the shell script to pick the implementation based on the presence of `--bootstrap-server` or `--zookeeper`. This would mean that we can deprecate the Scala tool while still

[jira] [Created] (KAFKA-5687) Retention settings are inconsistent between broker and topic

Ivan Babrou created KAFKA-5687: -- Summary: Retention settings are inconsistent between broker and topic Key: KAFKA-5687 URL: https://issues.apache.org/jira/browse/KAFKA-5687 Project: Kafka

[jira] [Created] (KAFKA-5688) Add a modifier to the REST endpoint to only show errors

Dustin Cote created KAFKA-5688: -- Summary: Add a modifier to the REST endpoint to only show errors Key: KAFKA-5688 URL: https://issues.apache.org/jira/browse/KAFKA-5688 Project: Kafka Issue

[GitHub] kafka pull request #3605: KAFKA-2111: Add help arguments and required fields

GitHub user johnma14 opened a pull request: https://github.com/apache/kafka/pull/3605 KAFKA-2111: Add help arguments and required fields This patch addresses the less invasive standardization of command line arguments as an offshoot of KIP-14. The following changes have been

[jira] [Created] (KAFKA-5685) Interface of ReadOnlyKeyValueStore should provide a non blocking interface for stores with non blocking capabilities

Chris Wewerka created KAFKA-5685: Summary: Interface of ReadOnlyKeyValueStore should provide a non blocking interface for stores with non blocking capabilities Key: KAFKA-5685 URL:

Re: Kafka Streams debugging with "no fluent" API choice

Hi Paolo, The change would require a KIP as it is a public API change. I don't see any harm in making the change, but I also don't think it is that difficult to use peek to achieve the same thing. Thanks, Damian On Tue, 1 Aug 2017 at 13:52 Paolo Patierno wrote: > Thanks

[jira] [Created] (KAFKA-5686) Documentation inconsistency on the "Compression"

Seweryn Habdank-Wojewodzki created KAFKA-5686: - Summary: Documentation inconsistency on the "Compression" Key: KAFKA-5686 URL: https://issues.apache.org/jira/browse/KAFKA-5686 Project:

[jira] [Resolved] (KAFKA-4314) Kafka Streams documentation needs definitive rework and improvement

[ https://issues.apache.org/jira/browse/KAFKA-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seweryn Habdank-Wojewodzki resolved KAFKA-4314. --- Resolution: Done > Kafka Streams documentation needs definitive

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

Hi Tom, A high-level point for discussion before going into the details. The proposed protocol API `alterTopics` has 2 types of operations: 1. Operations that cause data movement (or deletion): increase/decrease of replication factor and partition reassignment. These are currently done by

Re: Kafka Streams debugging with "no fluent" API choice

Hi Damian, changing the print() method for sure needs a KIP but I guess there is some reason we don't know why they decided to not have a fluent API for that. Regarding my JIRA I don't think a KIP is required, it's just internal stuff ... no ? Thanks Paolo Patierno Senior Software

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

Hi Ismael, Thanks for taking the time to look at this. Currently the proposal around the ReplicaStatusRequest/Response just allows you to see the lag for the given replicas. It's not something that's tied to a prior request to alter the topic at all, though obviously you can use it to monitor

Re: Kafka Streams debugging with "no fluent" API choice

Yeah, right. Sorry i missed that. The JIRA doesn't need a KIP On Tue, 1 Aug 2017 at 15:20 Paolo Patierno wrote: > Hi Damian, > > > changing the print() method for sure needs a KIP but I guess there is some > reason we don't know why they decided to not have a fluent API for

Re: Command tools : from Scala to Java, from Zookeeper utils to Admin Client API

Hi Tom, thanks for your reply. I think that you are right and what you have proposed should be the way to go. Until today I have been working on refactoring the TopicCommand tool in Java using the AdminClient getting rid of the Zookeeper usage in only "one step" and maybe it's wrong. I'd

Re: Kafka Streams debugging with "no fluent" API choice

I don't know specifically why this is removed, however if you want to get the same functionality you can use peek, i.e: stream.map(...).peek(...).filter(..) You can log the key values out in the peek call. On Tue, 1 Aug 2017 at 11:48 Paolo Patierno wrote: > Hi guys, > > >

[GitHub] kafka pull request #3604: MINOR: add memory management section to streams do...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/3604 MINOR: add memory management section to streams docs You can merge this pull request into a Git repository by running: $ git pull https://github.com/dguy/kafka memory-management-docs

Re: Command tools : from Scala to Java, from Zookeeper utils to Admin Client API

Hi Paolo, If the old tool is deprecated, then it's OK to add functionality to the new tool without adding it to the old tool as long as the old tool still works. So, there is some maintenance overhead to keep it working (but not to add new features). In practice, the code used by the old tool is

Build failed in Jenkins: kafka-trunk-jdk8 #1864

See Changes: [jason] KAFKA-5630; Consumer should block on corrupt records and keep throwing -- [...truncated 1.01 MB...] kafka.integration.AutoOffsetResetTest >

Re: Command tools : from Scala to Java, from Zookeeper utils to Admin Client API

Hi Ismael, in this case I embrace your way :-) To be more specific on what I was doing, It means that I can continue to develop the TopicCommand tool in Java using AdminClient API and then adding the deprecation for --zookeeper and the "tool switch" in the shell script. It doesn't need a

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

Hi both, Thanks for the replies. A few points: 1. It's worth emphasising that reassigning partitions is a different process than what happens when a topic is created, so not sure trying to make it symmetric is beneficial. In addition to what was already discussed, one should also enable

[jira] [Created] (KAFKA-5689) Refactor WindowStore hierarchy so that Metered Store is the outermost store

Damian Guy created KAFKA-5689: - Summary: Refactor WindowStore hierarchy so that Metered Store is the outermost store Key: KAFKA-5689 URL: https://issues.apache.org/jira/browse/KAFKA-5689 Project: Kafka

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

> Regarding adding the possibility to alter the topic config through the > AlterTopic API, the current TopicCommand implementation provides a warning > on doing this suggesting to use the ConfigCommand tool. So it would be a > step back allowing to do the configs change with the alter topic as

[GitHub] kafka pull request #3573: KAFKA-5630; The consumer should block on courrupt ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3573 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: Command tools : from Scala to Java, from Zookeeper utils to Admin Client API

Hi Ismael, your third option is really appealing for me :-) Just one thought about that ... If we decide to go in this way we could have an 1.0 release with both Scala (zookeeper) and Java (bootstrap-server) implementations but then the shell script to chose one of them with the --zookeeper

[jira] [Resolved] (KAFKA-5154) Kafka Streams throws NPE during rebalance

[ https://issues.apache.org/jira/browse/KAFKA-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-5154. --- Resolution: Fixed > Kafka Streams throws NPE during rebalance >

[GitHub] kafka pull request #3595: MINOR: Request log should be enabled by debug leve...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3595 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Jenkins build is back to normal : kafka-0.11.0-jdk7 #253

See

Re: [DISCUSS] KIP-179: Change ReassignPartitionsCommand to use AdminClient

Hi Ismael, 1. It's worth emphasising that reassigning partitions is a different > process than what happens when a topic is created, so not sure trying to > make it symmetric is beneficial. In addition to what was already discussed, > one should also enable replication throttling before moving

[GitHub] kafka pull request #3352: KAFKA-5454: Add a new Kafka Streams example IoT or...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3352 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Build failed in Jenkins: kafka-trunk-jdk7 #2591

See Changes: [jason] MINOR: Request log should be enabled by debug level -- [...truncated 1.01 MB...] kafka.log.ProducerStateManagerTest >

Kafka Streams debugging with "no fluent" API choice

Hi guys, I was thinking about Kafka Streams debug features and why the print() method overloads didn't return a KStream, in order to have a fluent DSL construction even during debugging, but just void. Then I came across this PR : https://github.com/apache/kafka/pull/1187 May I ask why

Re: Kafka Streams debugging with "no fluent" API choice

Thanks Damian, I knew about that but you have to write the code for printing by yourself. Of course you can do that even with the print() without using the default keyvalue mapper but passing a custom one. At same time if you want to print you should use a Serdes for key and value if they

[jira] [Created] (KAFKA-5684) KStreamPrintProcessor as customized KStreamPeekProcessor

Paolo Patierno created KAFKA-5684: - Summary: KStreamPrintProcessor as customized KStreamPeekProcessor Key: KAFKA-5684 URL: https://issues.apache.org/jira/browse/KAFKA-5684 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk8 #1865

See Changes: [jason] MINOR: Request log should be enabled by debug level -- [...truncated 5.05 MB...] org.apache.kafka.connect.runtime.distributed.DistributedHerderTest