Re: Kafka connect Graceful stop of task failed

Hello Greg Thanks a *lot* for your help on this. Indeed the empty poll is not the issue for us. As mentioned, our setup is a poll every 24 hours. So the `stop()` being stuck due to the `poll()` is hitting us hard. I did a trace today on my dev environment, I can indeed see this waiting log entry

Re: Kafka connect Graceful stop of task failed

m also. > The problem is precisely what Greg described, now the stop signal comes > from the same thread. So any source task which is running in a blocking way > will not process the stop signal until the current poll finishes. > So would need to patch source jdbc connector. > > On M

Re: Kafka connect Graceful stop of task failed

How can I stop getting these updates ? On Mon, Aug 21, 2023 at 9:01 AM Robson Hermes wrote: > This email was sent from an external source so please treat with caution. > > No, it stops them also. > The problem is precisely what Greg described, now the stop signal comes > from

Re: Kafka connect Graceful stop of task failed

No, it stops them also. The problem is precisely what Greg described, now the stop signal comes from the same thread. So any source task which is running in a blocking way will not process the stop signal until the current poll finishes. So would need to patch source jdbc connector. On Mon, 21

Re: Kafka connect Graceful stop of task failed

I think when you delete connector it removes the task and workers continues to run. When you stop it actually stops the worker. Both different things. Point to be noted is Worker has connector. So connector should be removed before stopping the worker. Though I am not expert in this. On Mon, 21

Re: Kafka connect Graceful stop of task failed

Hello Sunil I'm not calling a stop, I'm straight deleting the connectors with the DELETE. Stopping the connector is done internally during deletion. Regards Robson On Mon, 21 Aug 2023 at 15:36, sunil chaudhari wrote: > You have to remove connectors first using delete api > and the

Re: Kafka connect Graceful stop of task failed

You have to remove connectors first using delete api and then stop the connector On Thu, 17 Aug 2023 at 2:51 AM, Robson Hermes wrote: > Hello > > I'm using kafka connect 7.4.0 to read data from Postgres views and write to > another Postgres tables. So using JDBC source and sin

Re: Kafka connect Graceful stop of task failed

cesses a `stop()` when it returns from the current `poll()`execution. There was a change in the past to fix a similar problem <https://github.com/confluentinc/kafka-connect-jdbc/pull/677>, but not involving `stop()` from the same thread. I've just raised one <https://github.com/confluentin

Re: Kafka connect Graceful stop of task failed

connector. I know some implementations have received patches to compensate for this behavior in the framework, so also consider upgrading or checking release notes for your connectors. As for the effects of this error: whenever a non-graceful stop occurs, the runtime will immediately close

Kafka connect Graceful stop of task failed

Hello I'm using kafka connect 7.4.0 to read data from Postgres views and write to another Postgres tables. So using JDBC source and sink connectors. All works good, but whenever I stop the source connectors via the rest api: DEL http://kafka-connect:8083/connectors/connector_name_here

Consumers silently stop receiving data: how to reassign partitions in local sandbox?

Kafka consumers in our application occasionally stop receiving data. This seems to be happening very infrequently, with weeks or even months between occurrences, and it seems to occur after Kafka broker maintenance operations. I've been trying to replicate the issue in a local setup where I'm

weird state - we stop receving messages

We are chasing a strange behavior for a while now - asking your help because we ran out of ideas... maybe someone has a few good ideas/pointers? Components we use: *  We are using Kafka 2.2.1 iirc cluster - 3 nodes setup *  Topics having 6 partitions *  We are working in Java and using

Blocking stop top of the world vs continual flow incremental sticky rebalancing for conumer group autoscaling.

Dear all, I am running a simple kafka consumer group reactive autoscaling experiment on kubernetes, while leveraging range stop of the world assignor in the first run, and next in the second run I used incremental cooperative assignor. My workload is shown below where x-axis is the time

(node:10600) Electron: Loading non context-aware native modules in the renderer process is deprecated and will stop working at some point in the future, please see https://github.com/electron/electron

modules in the renderer process is deprecated and will stop working at some point in the future, please see https://github.com/electron/electron/issues/18397 for more information I can’t even go on now, because I can’t find a solution to a similar problem. Please help me, any suggestions

Re: stop

s from the list, send a message to: > > > > Cheers, > -John > > On Wed, Jan 22, 2020, at 10:12, Sowjanya Karangula wrote: > > stop > > >

Re: stop

Hey Sowjanya, That won't work. The "welcome" email you got when you signed up for the mailing list has instructions for unsubscribing: > To remove your address from the list, send a message to: > Cheers, -John On Wed, Jan 22, 2020, at 10:12, Sowjanya Karangula wrote: > stop >

stop

stop

Re: How to list/stop all current partition assignment which is running

kafka partition reassignment is using kafka manager. It seems it > is > > stuck somewhere. How can I check what is current assignment which is > > running and how can I stop it ? its been more that 12 hours and some of > the > > partitions are under replicated. > > >

Re: How to list/stop all current partition assignment which is running

kafka manager. It seems it is > stuck somewhere. How can I check what is current assignment which is > running and how can I stop it ? its been more that 12 hours and some of the > partitions are under replicated. > > Really appreciate your help. > > > Thanks > Ash >

How to list/stop all current partition assignment which is running

Hello All, We ran kafka partition reassignment is using kafka manager. It seems it is stuck somewhere. How can I check what is current assignment which is running and how can I stop it ? its been more that 12 hours and some of the partitions are under replicated. Really appreciate your help

Broker does not stop gracefully with SIGTERM

Hello, hope you all are doing well, Am trying to stop gracefully a Broker with SIGTERM (-15). After almost 12 hours the process is still alive. I do not see any data/replication going in or out from this Broker. The following are the logs immediately after sending the SIGTERM signal

DisconnectException stop Kafka Stream app consuming from topic

Hello, hope you all are great today! am using a Kafka Stream application: ... *final* StreamsBuilder streamsBuilder = *new* StreamsBuilder(); *final* KStream, MyObject> myObjects = streamsBuilder .stream(inputTopicNames, Consumed.*with*( myObjectsWindowSerde,

??????RE: New increased partitions could not be rebalance, until stop allconsumers and start them

Thank you very much. it's very clear and useful. Ruiping Li -- -- ??: "??"; : 2018??10??19??(??) 10:09 ??: "users@kafka.apache.org"; : RE: New increased partitions could not be rebalance, unti

Re: New increased partitions could not be rebalance, until stop allconsumers and start them

Thanks a lot. This config works. Ruiping Li -- -- ??: "hacker win7"; : 2018??10??18??(??) 6:29 ??: "users"; : Re: New increased partitions could not be rebalance, until stop allconsumers and start

RE: New increased partitions could not be rebalance, until stop all consumers and start them

ns could not be rebalance, until stop all >consumers and start them > >Sorry for bothering. I don't know whether it is a bug. Maybe something wrong >in my test or there is explanation for it. Could any Kafka master help take a >look? Thanks lot. > > >Ruiping Li > > >-

Re: New increased partitions could not be rebalance, until stop all consumers and start them

test or there is explanation for it. Could any Kafka master > help take a look? Thanks lot. > > > Ruiping Li > > > -- 原始邮件 -- > 发件人: "526564746"<526564...@qq.com>; > 发送时间: 2018年10月12日(星期五) 下午4:42 > 收件人: "users";

Re: New increased partitions could not be rebalance, until stop all consumers and start them

??: 2018??10??12??(??) 4:42 ??: "users"; : New increased partitions could not be rebalance, until stop all consumers and start them Hi Kafka team, I meet a strange thing about Kafka rebalance. If I increase partitions of a topic which subscribed by some java c

New increased partitions could not be rebalance, until stop all consumers and start them

Hi Kafka team, I meet a strange thing about Kafka rebalance. If I increase partitions of a topic which subscribed by some java consumers(in same one group), there is no rebalance occur. Furthermore, if I start a new consumer (or stop one) to cause a rebalance, the increased partitions could

Re: How to gracefully stop Kafka

> > > Regards! > > > > Alex > > > > Inviato da BlueMail<http://www.bluemail.me/r?b=13090> Il giorno 1 giu > > 2018, alle ore 07:31, Raghav > raghavas...@gmail.com>> ha scritto: > > > > Hi > > > > We have a 3 Kafka brokers setup

Re: How to gracefully stop Kafka

es. > > Regards! > > Alex > > Inviato da BlueMail<http://www.bluemail.me/r?b=13090> Il giorno 1 giu > 2018, alle ore 07:31, Raghav raghavas...@gmail.com>> ha scritto: > > Hi > > We have a 3 Kafka brokers setup on 0.10.2.1. We have a requirement in our >

Re: How to gracefully stop Kafka

2.1. We have a requirement in our company environment that we have to first stop our 3 Kafka Broker setup, then do some operations stuff that takes about 1 hours, and then bring up Kafka (version 1.1) brokers again. In order to achieve this, we issue: 1. Run *bin/<http://kafka-server-stop.sh>kafka-s

How to gracefully stop Kafka

Hi We have a 3 Kafka brokers setup on 0.10.2.1. We have a requirement in our company environment that we have to first stop our 3 Kafka Broker setup, then do some operations stuff that takes about 1 hours, and then bring up Kafka (version 1.1) brokers again. In order to achieve this, we issue

Why?? Kafka High Level Consumer is 0.8.2.2 Stop Responding After Few Hours... Timeout in Broker Log

for the consumer.timeout.ms. Therefore assuming consumer will never timeout when no message is received from the producer. I have observed that when we do not receive any message from producer for sometime the consumer stop responding to any message that is received after say 10 min. I have set

How to stop the producer if I use the incorrect bootstrap.servers

method:org.apache.kafka.clients.NetworkClient.handleDisconnections(NetworkClient.java:463) Node -1 disconnected. Would you like to tell me how to stop the producer when Connection timed out? Thank you in advance:)

Re: Issue with kafka-server-stop on RedHat7

The issue might be due to https://unix.stackexchange.com/questions/343353/ps-only-prints-up-to-4096-characters-of-any-processs-command-line I guess the issue is with kafka version >0.10.0. More details: https://github.com/apache/kafka/pull/2515 Regards, Ravi On Tue, May 9, 2017 at 12:01 PM,

Re: Issue with kafka-server-stop on RedHat7

Hi Vedant, Just try to run kill -s TERM $KafkaProcessPID . On Wed, May 10, 2017 at 12:31 AM, Vedant Nighojkar wrote: > Hi Team, > > We are using Apache Kafka in one of our products. We support Windows, AIX > and Linux RedHat6 and above. > > I am seeing an issue with the

Issue with kafka-server-stop on RedHat7

Hi Team, We are using Apache Kafka in one of our products. We support Windows, AIX and Linux RedHat6 and above. I am seeing an issue with the kafka-server-stop.sh script on RedHat7 machines. This used to work with RedHat6. ps ax | grep -i 'kafka.Kafka' - this is not able to find any running

Re: Calculating time elapsed using event start / stop notification messages

Hi Ali, One starting point would be the low level Processor API, where you get each event and process it. You can also use a persistent state store to keep track of the events seen so far, it can probably be an in-memory store. An an entry can probably be deleted once both start and stop

Calculating time elapsed using event start / stop notification messages

'working' per hour, per day. Any ideas how this could be achieved, while accounting for messages arriving out of order due to latency, etc (e.g the stop notification may arrive before start)? Would the kafka streams local store be of any use here (all events by the same user will have the same

Re: How to stop Kafka Mirror Maker

should also stop all the mirror maker instances in a given consumer group in parallel, as this will minimize the number of rebalances and how long it takes for you to start passing messages again. -Todd On Tue, Feb 21, 2017 at 5:14 PM, Qian Zhu <qi...@trulia.com> wrote: > Hi, >F

How to stop Kafka Mirror Maker

Hi, For now, I am doing “kill –9 processID” to stop the Kafka Mirror Maker. I am wondering whether there is a better way (e.g. a command) to do so? I don’t expect to stop the mirror maker frequently but I would like to have a script to automate the start and stop. Thanks a lot! Qian Zhu

My consumer stop consuming messages

Good day. My name is Valeriy, and I have a problem with my Kafka Consumer. Detail my problem is described here: http://stackoverflow.com/questions/40651260/apache-kafka-consumer-stop-consuming-messages Briefly, I am sure that the messages continue to arrive in the topic in the amount of 100 per

/bin/zookeeper-server-stop.sh fails to stop zookeeper with `kill -SIGINT`

'{print $1}' 4752 Trying to stop the zookeeper _fails_ with $ bin/zookeeper-server-stop.sh $ ps ax | grep -i 'zookeeper' | grep -v grep | awk '{print $1}' 4752 It still runs ... (should have stopped) $ kill -SIGINT 4752 $ ps ax | grep -i 'zookeeper' | grep -v grep | awk '{print $1

Re: Replication stop working

am not sure this is the correct mailing list for sending question. If not, please let me know and I will stop. I am looking for help to resolve replication issue. Replication stopped working a while back. Kafka environment: Kafka 0.8.1.1, Centos 6.5, 7 node cluster, default replication-factor

Replication stop working

Hi, I am new to this forum and I am not sure this is the correct mailing list for sending question. If not, please let me know and I will stop. I am looking for help to resolve replication issue. Replication stopped working a while back. Kafka environment: Kafka 0.8.1.1, Centos 6.5, 7 node

Re: Is there a way to stop/pause producers from writing?

Hi, Wouldn't you want to tell your app to stop producing messages then? Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr Elasticsearch Support * http://sematext.com/ On Tue, Aug 5, 2014 at 7:45 PM, Stephen Chan sc...@visiblemeasures.com wrote: Is there a tool or way

Is there a way to stop/pause producers from writing?

Is there a tool or way using the Kafka admin api to tell a producer to stop/pause writing to a specific topic? The use case is basically we need to stop writing to a topic, let the consumers get caught up and then deploy some new code either for the producers or consumers. Our producers

Re: Kafka 0.8.0 server-stop.sh does not stop broker

Bryan, The server's shutdown hook should be able to trigger with any SIG except SIGKILL. I just tried the following process and the stop script works for me: 1. bin/zookeeper-server-start.sh config/zookeeper.properties 2. bin/kafka-server-start.sh config/server.properties 3. bin/kafka-server

Re: Kafka 0.8.0 server-stop.sh does not stop broker

are you on? Thanks, Jun On Tue, Dec 17, 2013 at 11:15 AM, Bryan Baugher bjb...@gmail.com wrote: Hi, We have been trying out the kafka 0.8.0 beta1 for awhile and recently attempted to upgrade to 0.8.0 but noticed that the stop server script doesn't seem to stop

Kafka 0.8.0 server-stop.sh does not stop broker

Hi, We have been trying out the kafka 0.8.0 beta1 for awhile and recently attempted to upgrade to 0.8.0 but noticed that the stop server script doesn't seem to stop the broker anymore. I noticed here[1] that a commit was made before the release to change the signal sent to stop the broker from

Re: Kafka 0.8.0 server-stop.sh does not stop broker

Which OS are you on? Thanks, Jun On Tue, Dec 17, 2013 at 11:15 AM, Bryan Baugher bjb...@gmail.com wrote: Hi, We have been trying out the kafka 0.8.0 beta1 for awhile and recently attempted to upgrade to 0.8.0 but noticed that the stop server script doesn't seem to stop the broker anymore

Re: Kafka 0.8.0 server-stop.sh does not stop broker

attempted to upgrade to 0.8.0 but noticed that the stop server script doesn't seem to stop the broker anymore. I noticed here[1] that a commit was made before the release to change the signal sent to stop the broker from SIGTERM to SIGINT. Changing this script back to using SIGTERM seems to fix

Re: High level consumer stop reading data

or lost for some time, then high level consumer stop reading data from kafka even after the network is restarted/working. -- *Thanks Regards* *Hanish Bansal* -- *Thanks Regards* *Hanish Bansal*

High level consumer stop reading data

Hi All, We are running kafka-0.8, If kafka node machine's network is restarted or lost for some time, then high level consumer stop reading data from kafka even after the network is restarted/working. -- *Thanks Regards* *Hanish Bansal*

Re: High level consumer stop reading data

Any exception/error from the consumer? Thanks, Jun On Tue, Oct 29, 2013 at 4:50 AM, Hanish Bansal hanish.bansal.agar...@gmail.com wrote: Hi All, We are running kafka-0.8, If kafka node machine's network is restarted or lost for some time, then high level consumer stop reading data from

Re: High level consumer stop reading data

hanish.bansal.agar...@gmail.com wrote: Hi All, We are running kafka-0.8, If kafka node machine's network is restarted or lost for some time, then high level consumer stop reading data from kafka even after the network is restarted/working. -- *Thanks Regards* *Hanish Bansal

Re: Best way to start / stop / restart a Consumer?

is the recommended way to start, stop and restart a ConsumerConnector in the same running JMV? Thanks, 2013/10/10 Jun Rao jun...@gmail.com Each time we create a new consumer connector, we assign a random consumer id by default. You can try setting consumer.id to use a fixed consumer id. In any

Re: Best way to start / stop / restart a Consumer?

. To restart, you need to createMessageStreams() Thanks, Neha On Oct 11, 2013 6:10 AM, Tanguy tlrx tlrx@gmail.com wrote: Thanks Jun, Jira issue has been filled: https://issues.apache.org/jira/browse/KAFKA-1083 By the way, what is the recommended way to start, stop and restart

Re: Best way to start / stop / restart a Consumer?

, Jira issue has been filled: https://issues.apache.org/jira/browse/KAFKA-1083 By the way, what is the recommended way to start, stop and restart a ConsumerConnector in the same running JMV? Thanks, 2013/10/10 Jun Rao jun...@gmail.com Each time we create a new

Best way to start / stop / restart a Consumer?

/KAFKA-1083 By the way, what is the recommended way to start, stop and restart a ConsumerConnector in the same running JMV? Thanks, 2013/10/10 Jun Rao jun...@gmail.com Each time we create a new consumer connector, we assign a random consumer id by default. You can try setting

Best way to start / stop / restart a Consumer?

and partition, as documented here https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example We need to stop (halt) and restart the consumer. Today, we just call: connector.shutdown() and wait for threads to terminate. To restart the consumer, we create a new connector: connector

Re: Best way to start / stop / restart a Consumer?

://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example We need to stop (halt) and restart the consumer. Today, we just call: connector.shutdown() and wait for threads to terminate. To restart the consumer, we create a new connector: connector = Consumer.createJavaConsumerConnector

Re: Best way to start / stop / restart a Consumer?

/confluence/display/KAFKA/Consumer+Group+Example We need to stop (halt) and restart the consumer. Today, we just call: connector.shutdown() and wait for threads to terminate. To restart the consumer, we create a new connector: connector = Consumer.createJavaConsumerConnector

Re: Best way to start / stop / restart a Consumer?

per topic and partition, as documented here https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example We need to stop (halt) and restart the consumer. Today, we just call: connector.shutdown() and wait for threads to terminate. To restart the consumer, we

Re: Consumer in the mirror maker stop fetching logs

in the mirror maker stopped fetching logs brokers。I have to stop the mirror maker and start it over， and it continued fetching logs。 I start consumers using only 1 consumer thread。 By the way，there are 16 mirror makers in the consume group。 I did some calculation and it can

Re: Consumer in the mirror maker stop fetching logs

to stop the mirror maker and start it over， and it continued fetching logs。 I start consumers using only 1 consumer thread。 By the way，there are 16 mirror makers in the consume group。 I did some calculation and it can be assured that the partition numbers are larger than or equal