2019-09-16 14:32:04 UTC - David Kjerrumgaard: @vikash Are the consumers on the backlogged topic still active and consuming messages? Can you check the logs of the consumers? ---- 2019-09-16 14:46:19 UTC - vikash: I think mine issue related to Pulsar GUI ,i cant see any consumer connect there ---- 2019-09-16 14:46:28 UTC - vikash: looks normal now after few hour ---- 2019-09-16 15:20:59 UTC - Matteo Merli: You can double check the stats from the CLI tool : ---- 2019-09-16 15:21:17 UTC - Matteo Merli: `pulsar-admin topics stats $TOPIC` ---- 2019-09-16 15:41:55 UTC - Nicolas Ha: I have two questions: - (1) it seems it it possible to delete a topic but not individual messages, is that correct? - (2) Are there any tips with doing RPC with pulsar? Something like <https://www.rabbitmq.com/tutorials/tutorial-six-python.html#targetText=In%20general%20doing%20RPC%20over,queue%20address%20with%20the%20request> ---- 2019-09-16 18:30:59 UTC - Luke Lu: Hey guys, any one ran into brokers pegging multiple cores under light load (sum of pulsar_rate_in under 1000 msg/s), spending most of the time in netty epoll stuff (similar to this: <https://github.com/netty/netty/issues/5896>) and went into load shedding mode. Restart the brokers would fix the problem as CPU usage went back to a fraction of a core. For people who use pulsar in production, do you guys regularly restart brokers to resolve such issues? ---- 2019-09-16 19:21:18 UTC - Cory Davenport: Question. So in pulsar I have the retention_policy set to 1 minute.
I have been running it for about 20 minutes sending a message each minute yet it still seems to be keeping all messages. Am I understanding this correctly. Should it not be deleting these messages after 1 minute? ---- 2019-09-16 22:18:30 UTC - Ali Ahmed: <https://www.youtube.com/watch?v=3WbiQyAFpWQ> ---- 2019-09-16 23:12:05 UTC - Sijie Guo: I haven’t seen such behavior before. ---- 2019-09-16 23:13:47 UTC - Sijie Guo: (1) correct (2) I think the approach can be applied to pulsar as well. but you need to make sure the consumer is able to receive that message. hence you need to configure the right message router and use proper subscription mode. ---- 2019-09-16 23:16:06 UTC - Luke Lu: We saw this with pulsar 2.4.0 (using netty 4.1.32) with the official pulsar container. ---- 2019-09-16 23:16:33 UTC - Sijie Guo: the messages are deleted by segments. the segments are rolled based on rollover policy. hence if the segment is not rolled, the messages will not be deleted even they pass the retention period. that said, a better way to interpret “retention_policy” - the messages older than the configured retention period can be removed. but the messages will be lazily deleted by Pulsar. The behavior of “when to delete” is undefined. ---- 2019-09-16 23:19:35 UTC - Sijie Guo: okay can you file a github issue and attach some diagrams for the details, so that we can take a look ---- 2019-09-17 00:28:48 UTC - Matteo Merli: @Luke Lu the time on epoll_wait is most certainly fine. Can you share a thread profile with either JFR or YourKit? That will tell which thread is keeping the cpu busy ---- 2019-09-17 00:51:24 UTC - Matteo Merli: To expand a bit, you can control the min/max time for ledgers rollover. Rolling over ledgers more frequently will make the data deletion to happen sooner. <https://github.com/apache/pulsar/blob/master/conf/broker.conf#L544> ---- 2019-09-17 01:51:45 UTC - Cory Davenport: @Sijie Guo @Matteo Merli Thanks. ---- 2019-09-17 05:21:49 UTC - Mark Andreev: @Mark Andreev has joined the channel ---- 2019-09-17 07:25:05 UTC - Luke Lu: Looks like the official 2.4.0 image use openjdk 1.8 which doesn’t support JFR yet. But we did use jstack-profiler to capture a reasonable sampling profile. <https://github.com/apache/pulsar/issues/5200> ----
