afka as
for on-premises setup. If yes then what are recommendations to make it
working.
Because we are facing issue in name resolution after request landed onto
the Kafka broker server as it is not taking the same reverse path to
fulfill the request and name resolution as well.
Thanks
Nitin Sharma
On
t to know is , How important the DNS is in
the Kafka setup while accessing it from open internet and what are the must
have requirements to achieve this setup to work in the On-premises
environment.*
Regards
Nitin Sharma
.
On Fri, Jun 18, 2021 at 7:36 PM Mich Talebzadeh
wrote:
> Hi,
>
&g
suggest if it is possible to create the same setup in the Data
centre premises instead of Cloud environment.
Thanks
Nitin Sharma
Hi All,
I am trying to increase my Kafka producer client throughput and for same, i
was exploring server.properties file parameters.
I would like to better understand the relation between
"Socket.request.max.bytes" and "Socket.receive.buffer.bytes" parameter sets
in broker's server.properties
Hi Team,
I have couple of questions related to Log commit :
a) What Committed Logs means?
i understand that Consumer can pick messages when it is committed, i.e.
when message reached to all in-sync replicas. But it leaves doubt in my
mind.
does commit means when all replicas have flushed
pport partition reassignment yet. It is something
> we would like to support in the future.
>
> Ismael
>
> On Mon, Jul 17, 2017 at 12:16 AM, nitin sharma <
> kumarsharma.ni...@gmail.com>
> wrote:
>
> > Hi All,
> >
> > My requirement is to automate "
Hi All,
My requirement is to automate "Partition Rebalance" through java APIs,
rather than .sh scripts.
I am in assumption that Kafka AdminClient (0.11.0 v) have these features
and hence kindly let me know if it is TRUE.
Also, if AdminCLient API are document on net then kindly guide me to the
; large number of consumers for a given cluster, we actually had to increase
> the interval to 10 minutes for quite some time because the load on
> Zookeeper was just too high. Once we moved the ZK transaction logs to SSDs,
> we were able to drop that back to 60 seconds.
>
> -Todd
&g
rogress to implement other options as well(KAFKA-2435
> <https://issues.apache.org/jira/browse/KAFKA-2435>, KAFKA-1215
> <https://issues.apache.org/jira/browse/KAFKA-1215>, KAFKA-1736
> <https://issues.apache.org/jira/browse/KAFKA-1736>).
>
> Thanks,
> Grant
&
Hi All,
I would like to know what kind of strategy Kafka adopts in splitting
partitions of a topic to across the cluster.
Ex: if i have 10 broker in a cluster. 1 topic with 5 partition and each
with replication-factor of 2. then how would Kafka will assign the
partitions ( 5*2 =10) across the
anyone faced this issue?
Regards,
Nitin Kumar Sharma.
On Wed, Sep 2, 2015 at 5:32 PM, nitin sharma <kumarsharma.ni...@gmail.com>
wrote:
> Hi All,
>
> i have run into a weird issue with my Kafka setup.
>
> I see that it takes around 5-6 sec for Zookeeper to update the offs
Hi All,
i have run into a weird issue with my Kafka setup.
I see that it takes around 5-6 sec for Zookeeper to update the offset for
Kafka topics.
I am running "ConsumerOffsetChecker" tool to see that lag and what i found
that even when my consumer is not up, it takes 5-6 sec for Zookeeper to
Hi Team,
I would like to know under which all circumstances I can use MirrorMaker
process ?
i read on a website that : *it is not really intended as a fault-tolerance
mechanism* , so curious to know where all i can think of using MirrorMaker.
Also, how to know the version of MirrorMaker process
...@gmail.com wrote:
Metrics like Bytepersec, FetchRequestRateAndTimeMs can help you to check if
the consumer has problem processing messages
On Thu, Apr 9, 2015 at 2:40 AM, nitin sharma kumarsharma.ni...@gmail.com
wrote:
thanks, but can you please tell which metrics could highlight
PM, nitin sharma
kumarsharma.ni...@gmail.com
wrote:
hi Xiao,
i have finally got JMX monitoring enabled for my kafka nodes in test
envrionment and here is what i observed.
i was monitoring mbeans under kafka.consumer domain of JVM running
Kafka
Mirror Maker process
to
identify the root cause. metrics in kafka.consumer and kafka.producer
categories will help you find out the problems.
This link gives instruction how to read the metrics
http://kafka.apache.org/documentation.html#monitoring
On Wed, Apr 8, 2015 at 3:39 AM, nitin sharma kumarsharma.ni...@gmail.com
xiao xiaotao...@gmail.com wrote:
Can you attach your mirror maker log?
On Wed, Apr 1, 2015 at 12:28 AM, nitin sharma kumarsharma.ni...@gmail.com
wrote:
i tried with auto.offset.reset=smallest, but still not working..
there is data in my source cluster
Regards,
Nitin Kumar Sharma
to consume
data from N sources (N = 1) and transfer data to one destination cluster.
You are free to supply as many consumer properties as you want to instruct
mirror maker where to consumer data from.
On Thu, Mar 26, 2015 at 9:50 PM, nitin sharma kumarsharma.ni...@gmail.com
wrote
cluster.
2. Use replica assignment tool to reassign all the partitions to brokers
in new datacenter.
3. Perform controlled shutdown on the brokers in old datacenter.
Jiangjie (Becket) Qin
On 3/25/15, 2:01 PM, nitin sharma kumarsharma.ni...@gmail.com wrote:
Hi Team,
in my
Hi Team,
in my project, we have built a new datacenter for Kafka brokers and wants
to migrate from current datacenter to new one.
Switching producers and consumers wont be a problem provided New Datacenter
has all the messages of existing Datacenter.
i have only 1 topic with 2 partition that
or higher and are you using the kafka-topics tool to
see which leaders are on which brokers?
Did you also wait for more than 15 seconds to see if the leader election
takes place?
Are there any errors in the controller log?
On Sat, Feb 14, 2015 at 9:35 AM, nitin sharma kumarsharma.ni
Hi,
Anyone who knows how to solve my problem, please respond..
Regards,
Nitin Kumar Sharma.
On Thu, Feb 12, 2015 at 3:50 PM, nitin sharma kumarsharma.ni...@gmail.com
wrote:
Hi Team,
i need some help in solving my current issue related to
kafka-leadership-rebalance
I have 2 brokers.. i
hi All,
I am facing the same issue.. i have got Kafka_2.9.2-0.8.1.1.jar deployed
in my application.
My Operation team had reported significant increase in the directory where
Kafak-log folder exists . when i checked the size of parent directory is it
99GB (100%) utilized.. This was very
btw -- curious to know how well your kafka broker handles the load...
please do inform us your result.
Regards,
Nitin Kumar Sharma.
On Mon, Dec 22, 2014 at 9:52 AM, nitin sharma kumarsharma.ni...@gmail.com
wrote:
Hey Pramod,
few things:
a. You can keep 2 brokers but you can increase
Hey Pramod,
few things:
a. You can keep 2 brokers but you can increase the ProducerSend thread on
your producer side to push more messages. best way try to create more
threads that execute the loop where send is called.
b. try to avoid/reduce putting any logic computation in the while loop ..
You
, nitin sharma kumarsharma.ni...@gmail.com
wrote:
Hi Team,
I have to make a decision on whether i should go with Kafka producer test
utility or build my own java tool for my load testing .
Kindly let me know if anyone knows any limitation with
kafka-producer-per-test.sh when it come
hi,
Few things you have to plan for:
a. Ensure that from resilience point of view, you are having sufficient
follower brokers for your partitions.
b. In my testing of kafka (50TB/week) so far, haven't seen much issue with
CPU utilization or memory. I had 24 CPU and 32GB RAM.
c. 200,000 partitions
Hi Team,
I have to make a decision on whether i should go with Kafka producer test
utility or build my own java tool for my load testing .
Kindly let me know if anyone knows any limitation with
kafka-producer-per-test.sh when it come to simulation of messages under
load condition?
Regards,
Hi All,
I am trying to figure out best configuration for my Kafka brokers so that
in case of restarted, the new node catch up with Leader at quick pace.
My test environment has 2 kafka brokers and 1 Topic with one Partition.
I first ran the test (Test#1) with default setting, i.e.
Hi Team,
Is it possible to know how many Consumer Group connected to kafka broker Ids
and as well as how many Instances within a Group are fetching messages from
Kafka Brokers
Regards,
Nitin Kumar Sharma.
the number of results
On Mon, Dec 15, 2014 at 11:34 AM, nitin sharma
kumarsharma.ni...@gmail.com wrote:
Hi Team,
Is it possible to know how many Consumer Group connected to kafka
broker
Ids
and as well as how many Instances within a Group are fetching messages
from
Kafka Brokers
got it ... thanks a lot.
Regards,
Nitin Kumar Sharma.
On Mon, Dec 15, 2014 at 9:26 PM, Gwen Shapira gshap...@cloudera.com wrote:
Hi Nitin,
Go to where you installed zookeeper and run:
bin/zkCli.sh -server 127.0.0.1:2181
On Mon, Dec 15, 2014 at 6:09 PM, nitin sharma
kumarsharma.ni
Hi Kishore,
You can use Kafka Producer API for same. You can find the sample code on
Kafka Quick Start page : http://kafka.apache.org/07/quickstart.html
Regards,
Nitin Kumar Sharma.
On Wed, Dec 10, 2014 at 2:14 AM, kishore kumar akishore...@gmail.com
wrote:
Hi kafkars,
I want to write a
values
coming in negative.
Regards,
Nitin Kumar Sharma.
On Wed, Dec 10, 2014 at 4:34 PM, Otis Gospodnetic
otis.gospodne...@gmail.com wrote:
Hi,
On Wed, Dec 10, 2014 at 12:02 AM, nitin sharma
kumarsharma.ni...@gmail.com
wrote:
thanks for response. But my questions are still not answered
HI,
I need some help related to Kafka Consumer Lag monitoring.. Kindly let me
know how can i post my query to the forum.
Regards,
Nitin Kumar Sharma.
+io3_7xsqxxfc6hhf7pzap5sfgtzd22stb...@mail.gmail.com%3E
Regards,
Nitin Kumar Sharma.
On Tue, Dec 9, 2014 at 4:47 PM, Guozhang Wang wangg...@gmail.com wrote:
Nitin,
You have found the place :) Just send an email to users@ mailing list is
fine.
Guozhang
On Tue, Dec 9, 2014 at 1:26 PM, nitin sharma
before:
http://search-hadoop.com/?q=monitor+lagfc_project=Kafka
Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr Elasticsearch Support * http://sematext.com/
On Tue, Dec 9, 2014 at 4:59 PM, nitin sharma kumarsharma.ni...@gmail.com
wrote:
Hi Neha
37 matches
Mail list logo
