jerrypeng commented on issue #2474: Revert "Fix: Function assignment can support large number of topics (#2438)" URL: https://github.com/apache/incubator-pulsar/pull/2474#issuecomment-417105009 Also discussing with @merlimat @srkukarni I don't think (#2438) is the best design for solving the issue. Because even with #2438, we will still have the inefficiency of submitting/updating one function requiring the publishing of all assignments which can be large. After discussion we think, the best way to solve size and scalability issues is to use topic compaction. In the assignment topic each message will have a key that is the function id and the value with be the assignments of its instances. And we can use topic compaction to trim old entries. This solves the message size issue as well as preventing writing/sending every assignment on the update of a single assignment
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
