Hi Thomas, On Wed, Apr 4, 2018 at 2:37 PM, Thomas Weise <t...@apache.org> wrote:
> Hi, > > I was looking at the documentation and understand that broker and bookies > can be added on the fly to scale up Pulsar. What wasn't clear is what > support Pulsar offers to change the partitioning of a topic? > A partition in pulsar is a logic entity that isn't tied to a specific (set) of bookie(s). Pulsar also support changing the number of partitions for a given topic. > > Let's say the user has identified that a partition became a bottleneck and > wants to split it. How would that work with Pulsar? > Pulsar currently only support increasing the number of partitions. It doesn't support decreasing the number of partitions. You can use pulsar-admin tool to adjust the number of partitions. See : https://pulsar.incubator.apache.org/docs/latest/admin-api/partitioned-topics/#Update-td8sjr > Does it require resharding of the state? > Currently Pulsar doesn't resharding the existing data. It just simply adds more partitions. That says it doesn't support *key* based ordering. For example, if you are increasing the number of partitions from 4 to 6, messages of same routing key might live at different partitions before and after changing the number of partitions. If you don't require key based ordering, current feature can meet your requirement. We were also discussing support key ordering and dynamically re-partitioning. We would like to hear your use case and incorporate any feedback. > Related question: Does Pulsar provide anything for the user to identify > such bottlenecks? > Yes. Pulsar provides a lot of stats for users to identify the bottleneck. These stats include message rate, throughput, backlog. You can get the stats by using pulsar-admin tool - https://pulsar.incubator.apache.org/docs/latest/admin-api/persistent-topics/#Getstats-4akfjf Or you can setup a dashboard to view this in provided UI - https://pulsar.incubator.apache.org/docs/latest/deployment/Monitoring/#Dashboards-au8nty Hope these answer your question! > > Thanks, > Thomas > >
