Re: [DISCUSS] KIP-338 Support to exclude the internal topics in kafka-topics.sh command

Thanks for the KIP. LGTM. On Wed, Jul 11, 2018 at 3:01 AM, Chia-Ping Tsai wrote: > hi Kafka, > > KIP-338 is trying to make us exclude internal topics easily when using the > kafka-topics.sh. An new option will be added to kafka-topics.sh > (TopicCommand) and user can use the option to exclude al

Re: [DISCUSS]: KIP-339: Create a new ModifyConfigs API

Hey Colin, Thanks for the KIP! It seems that the AlterConfigsResult is pretty much the same as ModifyConfigsResult. Instead of adding ModifyConfigs API and deprecating AlterConfigs API, would it be simpler to just add API alterConfigs( Map changes, Set removals, final ModifyConfigsOptions options

Re: KIP-327: Add describe all topics API to AdminClient

Good point. We should probably have a maximum number of results like 1000 or something. That can go in the request RPC as well... Cheers, Colin On Fri, Jul 13, 2018, at 18:15, Ted Yu wrote: > bq. describe topics by a regular expression on the server side > > Should caution be taken if the regex

Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

Hi Mike, Thanks a lot for reviewing the KIP and sharing your feedback. I agree that such an issue could surface with option 1, but the existing PR (that currently implements this option) checks for such duplicate assignments and ignores one in favor of the other. So at the end there will be val

Contribution request

Greetings, I'd like to become a contributor for Apache Kafka project. Could you please add me to the list of contributors in JIRA? My username is "jpact" Thanks in advance! Regards, J

Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

This is great! For the client side implementation, I think it’s still possible for there to be a duplication. I’ll try to walk through the example here. Let’s says there are 2 consumers, 1 topic with 2 partitions. After the initial rebalance, generation 0: Consumer A has partition 0 Consumer

Re: [DISCUSS]: KIP-339: Create a new ModifyConfigs API

I experimented with putting null value into ConcurrentHashMap which led me to this code: final V putVal(K key, V value, boolean onlyIfAbsent) { if (key == null || value == null) throw new NullPointerException(); I agree that getting NPE this way is not user friendly. Using Java 8, th

[jira] [Created] (KAFKA-7162) Flaky unit tests caused by record creation timestamps differ from validation time by more than timestampDiffMaxMs

Lucas Wang created KAFKA-7162: - Summary: Flaky unit tests caused by record creation timestamps differ from validation time by more than timestampDiffMaxMs Key: KAFKA-7162 URL: https://issues.apache.org/jira/browse/KAF

Re: KIP-327: Add describe all topics API to AdminClient

bq. describe topics by a regular expression on the server side Should caution be taken if the regex doesn't filter ("*") ? Cheers On Fri, Jul 13, 2018 at 6:02 PM Colin McCabe wrote: > As Jason wrote, this won't scale as the number of partitions increases. > We already have users who have tens

Re: KIP-327: Add describe all topics API to AdminClient

As Jason wrote, this won't scale as the number of partitions increases. We already have users who have tens of thousands of topics, or more. If you multiply that by 100x over the next few years, you end up with this API returning full information about millions of topics, which clearly doesn't

Re: [DISCUSS]: KIP-339: Create a new ModifyConfigs API

On Fri, Jul 13, 2018, at 17:45, Ted Yu wrote: > Looking at modifyConfigs API, it doesn't seem that ConcurrentHashMap > should be used as the underlying parameter type. I agree that there are other types of maps that do support null values. However, the fact that some official map implementations fr

Re: [DISCUSS]: KIP-339: Create a new ModifyConfigs API

Looking at modifyConfigs API, it doesn't seem that ConcurrentHashMap should be used as the underlying parameter type. Anyway, to signify that null value is supported, value type can be declared as Optional. FYI On Fri, Jul 13, 2018 at 5:35 PM Colin McCabe wrote: > Hi Ted, > > That’s a fair que

Re: [DISCUSS]: KIP-339: Create a new ModifyConfigs API

Hi Ted, That’s a fair question. I think the main reason I didn’t propose that originally is that many people find null values in maps confusing. Also, some newer java maps don’t support null values, such as ConcuurentHashMap. I’m curious what others think about this. Best, Colin On Wed, Jul 11,

[DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

Hi all, I create a short KIP to address an issue in Sticky Assignor assignment logic: https://cwiki.apache.org/confluence/display/KAFKA/KIP-341%3A+Update+Sticky+Assignor%27s+User+Data+Protocol Please take a look and share your feedback / comments. In particular, there is a Generation Marker sec

[jira] [Created] (KAFKA-7161) KTable Reduce should check for invalid conditions

John Roesler created KAFKA-7161: --- Summary: KTable Reduce should check for invalid conditions Key: KAFKA-7161 URL: https://issues.apache.org/jira/browse/KAFKA-7161 Project: Kafka Issue Type: Imp

[jira] [Resolved] (KAFKA-7156) Deleting topics with long names can bring all brokers to unrecoverable state

[ https://issues.apache.org/jira/browse/KAFKA-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian resolved KAFKA-7156. Resolution: Duplicate > Deleting topics with long names can bring all brokers to unrecovera

Re: [VOTE] 2.0.0 RC2

Hi Rajini, Thanks for generating the RC. It seems like the kafka-streams-scala 2.12 artifact is missing from the Maven repository: https://repository.apache.org/content/groups/staging/org/apache/kafka/ Since this is the first time we are publishing this artifact, it is possible that this never w

Request permission to assign JIRA

Hi, I'm trying to claim a bug producing to Kafka in Jira, could you please do me a favor to help me get the necessary permissions? my JIRA username is williamguan. Thanks for your time. -- williamguan

[jira] [Created] (KAFKA-7160) Add check for group ID length

lambdaliu created KAFKA-7160: Summary: Add check for group ID length Key: KAFKA-7160 URL: https://issues.apache.org/jira/browse/KAFKA-7160 Project: Kafka Issue Type: Improvement Compone

Re: KIP-327: Add describe all topics API to AdminClient

The KIP looks good to me. However, if there is willingness in the community to work on metadata request with patterns, the feature proposed here and filtering by '*' or '.*' would be redundant. Andras On Fri, Jul 13, 2018 at 12:38 AM Jason Gustafson wrote: > Hey Manikumar, > > As Kafka begins

Kafka Namespaces

Hi All, At Kafka Summit London 2018, Neha presented a thought experiment about namespaces in Apache Kafka. I'd like to know if work on this vision has started and if so, where I can find more information on it. KIP-37 seems to be related, but wa

Re: Builder Pattern for kafka-clients in 2.x ?

Hi, Tommy! thanks for the feeback - I've reopened the PR adding the ctor overload. I've drafted a ProducerBuilder, which can be used like: final Producer producer = new ProducerBuilder() .partitionerClass(DefaultPartitioner.class) .keySerializer(Intege