Send faild ,re-send 发件人: 孙刘昌(sunliuchang) 发送时间: 2019年12月9日 16:00 收件人: 'dev@pulsar.apache.org' <dev@pulsar.apache.org>; 'us...@pulsar.apache.org' <us...@pulsar.apache.org>; 'comm...@pulsar.apache.org' <comm...@pulsar.apache.org> 主题: The pulsar go client missing data when transfer
Hi Guys Now I found one problem relate to pulsar go client The whole process of mine is csv file --> filebeat-->pulsar--> flume--> hive At past time I verified the data count ,found that the data by this path missing about 0.46% compare with the path of logstash-->kafka . No idea the reason why ,so I tried below steps 1 Reduce the amount of data from 100% to 50%, excluding data loss caused by too much pressure in the pulsar cluster, the data loss is still 0.46% 2 Reduce the amount of data from 50% to 10%, make the pulsar cluster stress-free, and the data loss is still 0.46% 3 Use bin / pulsar-perf to quantitatively send test test data to the pulsar cluster, which is 100% acceptable, eliminating the problems of the pulsar cluster 4 Quantitatively send test data to the pulsar cluster with filebeat. 99.6% can receive it (send 36077, receive 35091) [cid:image001.png@01D5AEAC.3DFB3960] I put the data file in the attachment. Please take a look at this problem. Now the loss rate is about five thousandths, which is a little too large. Another thing to say, I use the asynchronous sending of the go client, which has been confirmed with @tuteng Hopefully your response 本邮件及其附件含有保密信息并受法律保护。如果您错收了本邮件，请勿出于任何目的使用、阅读、复制、打印、存储或向任何人披露本邮件及其附件所含信息，并请您即刻将此误发情形通知发件人，并将本邮件从系统中永久删除，谢谢。
