2018-06-02 09:16:51 UTC - Sijie Guo: @Igor Zubchenok sorry was away. just come back. let me write my suggestion about this ok_hand : Igor Zubchenok ---- 2018-06-02 09:24:54 UTC - Sijie Guo: since you have 20K topics one namespace, I would suggest use 500-1000 bundles, so it keeps the number of topics per bundle around 20-40 topic, so the time to unload a bundle is much lesser, so you are able to shift topic in smaller batches. since we are using smaller bundles, so the number of bundles increased, so the time for a broker to do gracefully shutdown increase, so you might want to increase brokerShutdownTimeoutMs to a higher value, say minutes, so broker won’t kill the process until it gracefully unload all the bundles.
hope this make sense ---- 2018-06-02 09:26:41 UTC - Igor Zubchenok: start with 1024 bundles ok_hand : Sijie Guo ---- 2018-06-02 09:36:29 UTC - Igor Zubchenok: and brokerShutdownTimeoutMs=300000 ok_hand : Sijie Guo ---- 2018-06-02 10:00:37 UTC - Igor Zubchenok: I have to say we have so much stress today(( 4 BK nodes, 3/3/2 managed ledgers config, 1 BK node used 100% CPU and we’re down To start it again, we had to stop and delete everything. Now we’re working, but it will probably fail again in a few days.( Thank you for your support anyway, I hope it will help and I appreciate it so much!!! ---- 2018-06-02 10:03:04 UTC - Sijie Guo: yeah. no problem. hopefully all the optimizations work for your use case. see_no_evil : Igor Zubchenok ---- 2018-06-02 23:16:30 UTC - Igor Zubchenok: We have a topic per user and send all data/messages from thousands of other users via Pulsar. When a user goes offline for 10 minutes, we delete his topic and stop sending needless updates, until the user reconnects again and then a new session (topic) is established. ---- 2018-06-02 23:18:50 UTC - Igor Zubchenok: @Ali Ahmed imaging real-time location sharing between friends when they have app opened on mobile device ---- 2018-06-02 23:20:26 UTC - Ali Ahmed: are you trying to build a sns type service on top of pulsar ? ---- 2018-06-02 23:44:47 UTC - Ali Ahmed: I would recommend pre creating needed topics and bind them for use when needed ----
