Re: [VOTE] KIP-932: Queues for Kafka
Hi Andrew + 1 (Non-Binding) This will be great addition to Kafka On Mon, Mar 18, 2024 at 8:27 AM Apoorv Mittal wrote: > Hi Andrew, > Thanks for writing the KIP. This is indeed going to be a valuable addition > to the Kafka, excited to see the KIP. > > + 1 (Non-Binding) > > Regards, > Apoorv Mittal > +44 7721681581 > > > On Sun, Mar 17, 2024 at 11:16 PM Andrew Schofield < > andrew_schofield_j...@outlook.com> wrote: > > > Hi, > > I’ve been working to complete KIP-932 over the past few months and > > discussions have quietened down. > > > > I’d like to open the voting for KIP-932: > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A+Queues+for+Kafka > > > > Thanks, > > Andrew >
Dynamic Metrics Reporter Producer reload
Hello, We currently use short lived TLS certs for our CruiseControlMetricsReporter which is creating issues when the cert reloads the producer client is failing to restart as well. A rolling restart resolves this but I was hoping to find out if there is a dynamic way to reload these producers or any client reallly in a similar pattern we do with brokers. Github issue on the cruise control repo for another reference: https://github.com/linkedin/cruise-control/issues/1148 Kenneth Eversole
Re: [PROPOSAL] Add commercial support page on website
I agree with Divji here and to be more pointed. I worry that if we go down the path of adding vendors to a list it comes off as supporting their product, not to mention could be a huge security risk for novice users. I would rather this be a callout to other purely open source tooling, such as cruise control. Divji brings up good question 1. What value does additional of this page bring to the users of Apache Kafka? I think the community would be a better service to have a more synchronous line of communication such as Slack/Discord and we call that out here. It would be more inline with other major open source projects. --- Kenneth Eversole On Wed, Jan 10, 2024 at 10:30 AM Divij Vaidya wrote: > I don't see a need for this. What additional information does this provide > over what can be found via a quick google search? > > My primary concern is that we are getting in the business of listing > vendors in the project site which brings it's own complications without > adding much additional value for users. In the spirit of being vendor > neutral, I would try to avoid this as much as possible. > > So, my question to you is: > 1. What value does additional of this page bring to the users of Apache > Kafka? > 2. When a new PR is submitted to add a vendor, what criteria do we have to > decide whether to add them or not? If we keep a blanket criteria of > accepting all PRs, then we may end up in a situation where the llink > redirects to a phishing page or nefarious website. Hence, we might have to > at least perform some basic due diligence which adds overhead to the > resources of the community. > > -- > Divij Vaidya > > > > On Wed, Jan 10, 2024 at 5:00 PM fpapon wrote: > > > Hi, > > > > After starting a first thread on this topic ( > > https://lists.apache.org/thread/kkox33rhtjcdr5zztq3lzj7c5s7k9wsr), I > > would like to propose a PR: > > > > https://github.com/apache/kafka-site/pull/577 > > > > The purpose of this proposal is to help users to find support for sla, > > training, consulting...whatever that is not provide by the community as, > > like we can already see in many ASF projects, no commercial support is > > provided by the foundation. I think it could help with the adoption and > the > > growth of the project because the users > > need commercial support for production issues. > > > > If the community is agree about this idea and want to move forward, I > just > > add one company in the PR but everybody can add some by providing a new > PR > > to complete the list. If people want me to add other you can reply to > this > > thread because it will be better to have several company at the first > > publication of the page. > > > > Just provide the company-name and a short description of the service > offer > > around Apache Kafka. The information must be factual and informational in > > nature and not be a marketing statement. > > > > regards, > > > > François > > > > > > >
Re: [DISCUSS] KIP-975 Docker Image for Apache Kafka
Hello, I think this would be a wonderful improvement to the ecosystem. While Viktor is correct that most Docker pipelines eventually lead to a kubernetes deployment, that should not stop us from creating an Official Docker Image. Creating a Docker image would allow us to ensure a level of quality and support for people who want to deploy Kafka as a container on baremetal machines, it could allow us to create a sandbox/developer environment for new contributors and developers to test and have a single agreed upon environment that kafka works in for future KIPs and would most likely spawn more contributions from people wanting to optimize kafka for k8s. I am 100% for this and will gladly help if approved. Kenneth On Thu, Sep 14, 2023 at 5:47 AM Ismael Juma wrote: > Hi Viktor, > > I disagree. Docker is a very popular deployment tool and it's not only used > with Kubernetes. > > Ismael > > On Thu, Sep 14, 2023, 1:14 AM Viktor Somogyi-Vass > wrote: > > > Hi Krishna, > > > > I think you should merge this KIP and KIP-974 as there are overlaps as > > Federico pointed out on KIP-974. I think you should keep that one as it > > has well defined goals (improve tests) while I feel this one is too > > generic. Docker is usually just a tool for either testing or Kubernetes, > so > > they have very well defined use-cases. In the case of Flink for instance > > the image is used for its kubernetes operator. The use case would > determine > > a lot of things and I think a generic image would likely not fit the > needs > > of all use-cases. > > > > Best, > > Viktor > > > > On Fri, Sep 8, 2023 at 9:58 AM Krishna Agarwal < > > krishna0608agar...@gmail.com> > > wrote: > > > > > Hi, > > > Apache Kafka does not have an official docker image currently. > > > I want to submit a KIP to publish a docker image for Apache Kafka. > > > > > > KIP-975: Docker Image for Apache Kafka > > > < > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka > > > > > > > > > > Regards, > > > Krishna > > > > > >
Re: Feedback of using the 'confluent-kafka-go' for reliablity and performance
Hello, Xinli, I can only speak for Cloudflare, but we use various versions of the Sarama Go Client for various production workloads and they seem to function quite well. I am curious what your reasoning for wanting to replace Sarama? Kenneth Eversole On Fri, Jul 28, 2023 at 1:58 PM Xinli shang wrote: > Hi all, > > We are considering replacing the Sarama Go client with confluent-kafka-go > <https://github.com/confluentinc/confluent-kafka-go> which is supported > and > seems promising. Regarding reliability and performance, does anybody want > to share the experience? Whether or not it is the experience in production > or testing, they are all appreciated. > > -- > Xinli Shang >
Requesting permission to contribute to Apache Kafka
Hello, I am requesting permission to contribute to Apache Kafka wiki id: keversole jira id: keversole Thank you, Kenneth Eversole
Re: [ANNOUNCE] New committer: Greg Harris
Congrats Greg! On Mon, Jul 10, 2023 at 10:55 AM Viktor Somogyi-Vass wrote: > Congrats Greg! > > On Mon, Jul 10, 2023, 17:54 Yash Mayya wrote: > > > Congrats Greg! > > > > On Mon, Jul 10, 2023 at 9:15 PM Chris Egerton > wrote: > > > > > Hi all, > > > > > > The PMC for Apache Kafka has invited Greg Harris to become a committer, > > and > > > we are happy to announce that he has accepted! > > > > > > Greg has been contributing to Kafka since 2019. He has made over 50 > > commits > > > mostly around Kafka Connect and Mirror Maker 2. His most notable > > > contributions include KIP-898: "Modernize Connect plugin discovery" > and a > > > deep overhaul of the offset syncing logic in MM2 that addressed several > > > technically-difficult, long-standing, high-impact issues. > > > > > > He has also been an active participant in discussions and reviews on > the > > > mailing lists and on GitHub. > > > > > > Thanks for all of your contributions, Greg. Congratulations! > > > > > >
Re: Max Multi Tenancy topic naming - Cloudflare
Hello, I apologize for the lack of clarity in the question, this is a new area we are exploring. Frankly speaking, we would love to know the answer to both. I have read that there is an upper limit to partitions, 200k with zookeeper and 2mill with Kraft, but I have not found anything if there is a correlation between topics. This also goes for the isolate permissions to name spaces is there a correlation between the partition limit here and the limit allowed for this feature? Kenneth Eversole
Re: Max Multi Tenancy topic naming - Cloudflare
Hello Everyone, I just wanted to bump this question. If this readily available information I apologized, if not I would love to capture it and add it to the documents. Kenneth On Thu, Apr 13, 2023 at 12:39 PM Kenneth Eversole wrote: > Hello, > > I am on the Kafka team at Cloudflare, and I was hoping to find an answer > on what the max value is for the following subject, > https://kafka.apache.org/documentation/#multitenancy-topic-naming. We are > looking this for a workload that would bring us in the 100,000s of topics. > > > I look forward to hearing from you. > > Best regards, > > Kenneth >
Re: [DISCUSS] Apache Kafka 3.4.1 release
+1, Thanks for leading the charge. On Thu, Apr 27, 2023 at 2:54 PM Sophie Blee-Goldman wrote: > +1, thanks Luke! > > On Wed, Apr 26, 2023 at 10:53 AM Bruno Cadonna wrote: > > > Thanks a lot, Luke! > > > > +1 > > > > Best, > > Bruno > > > > On 26.04.23 16:11, Bill Bejeck wrote: > > > Thanks for volunteering! > > > > > > +1 > > > > > > On Wed, Apr 26, 2023 at 8:33 AM Mickael Maison < > mickael.mai...@gmail.com > > > > > > wrote: > > > > > >> Thanks Luke > > >> +1 > > >> > > >> On Wed, Apr 26, 2023 at 2:12 PM Chia-Ping Tsai > > wrote: > > >>> > > >>> +1 to Luke! > > >>> > > David Jacot 於 2023年4月26日 下午6:44 寫道： > > > > +1 Thanks, Luke! > > > > > Le mer. 26 avr. 2023 à 12:37, Luke Chen a > écrit > > : > > > > > > Hi all, > > > > > > I'd like to volunteer as release manager for the Apache > > > Kafka 3.4.1 release. > > > If there are no objections, I'll start building a release plan in > the > > > wiki in the next couple of days. > > > > > > Thank you. > > > Luke > > > > > >> > > > > > >
Max Multi Tenancy topic naming - Cloudflare
Hello, I am on the Kafka team at Cloudflare, and I was hoping to find an answer on what the max value is for the following subject, https://kafka.apache.org/documentation/#multitenancy-topic-naming. We are looking this for a workload that would bring us in the 100,000s of topics. I look forward to hearing from you. Best regards, Kenneth
