How to prevent data loss in "read-process-write" application?

2019-06-02 Thread 1095193...@qq.com
solve my problem. Has any other recommended solution in addition to enable Transcation( I dont need exactly once process)? 1095193...@qq.com

Re: Partition Count Dilemma

2019-03-20 Thread 1095193...@qq.com
a consumer can provide is C, and the target throughput is T. Then the minimum number of partitions, that is, the number of consumers, is T/C. 1095193...@qq.com From: shalom sagges Date: 2019-03-21 06:43 To: users Subject: Partition Count Dilemma Hi All, I'm really new to Kafka and wanted

Re: Consumer poll stuck on

2019-03-20 Thread 1095193...@qq.com
Topic: __consumer_offsets Partition: 9 Leader: 2 Replicas: 2,1 Isr: 1,2 Topic: __consumer_offsets Partition: 10 Leader: 1 Replicas: 1,2 Isr: 1,2 If any partitions is not in Isr, you can fix it. 1095193...@qq.com From: Manu Jacob Date: 2019-03-21 09:39 To: users@kafka.apache.org; d

Re: Using kafka with RESTful API

2019-03-18 Thread 1095193...@qq.com
Hi, A request-response structure is more suitable for your scenario, you should still persist RESTful API rather than Kafka. 1095193...@qq.com From: Desmond Lim Date: 2019-03-19 09:52 To: users Subject: Using kafka with RESTful API Hi all, Just started using kafka yesterday and I have

Re: Whether kafka broker will have impact with 2 MB message size

2019-03-13 Thread 1095193...@qq.com
We have a use case where we want to produce data to kafka with max size of 2 MB rarely (That is, based on user operations message size will vary). Whether producing 2 Mb size will have any impact or we need to split the message to small chunk such as 100 KB and produce. If we produce into small

Visual tool for kafka?

2018-10-19 Thread 1095193...@qq.com
Hi I need a Visual tool for kafka. For example, A Web UI can display the detail of each topics、the offset of each consumer. Has any recommended visual tools? 1095193...@qq.com

Re: What's the use of timestamp in ProducerRecord?

2018-01-18 Thread 1095193...@qq.com
kafka does not delete message when message is consumed, it will purge message when this message is expired. I guess this timeStamp is for checking whether message is expired. 1095193...@qq.com From: Jake Yoon Date: 2018-01-19 11:46 To: users Subject: What's the use of timestamp

Need feedbacks to consumer if hangs due to some __consumer_offsets partitions failed

2018-01-08 Thread 1095193...@qq.com
with a response GROUP_COORDINATOR_NOT_AVAILABLE. I hope broker can feedback to consumer if hangs due to some __consumer_offsets partitions failed , it can help user improve expericence and debug issure fastly. 1095193...@qq.com