This is an automated email from the ASF dual-hosted git repository.
mattisonchao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new cb24ab01c35 [improve][pip] Replace reader with table view in the topic
policy service (#21033)
cb24ab01c35 is described below
commit cb24ab01c358870b14a71070ad989d7d85282cf4
Author: Qiang Zhao <[email protected]>
AuthorDate: Sun Sep 3 21:28:23 2023 +0800
[improve][pip] Replace reader with table view in the topic policy service
(#21033)
---
pip/pip-284.md | 114 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 114 insertions(+)
diff --git a/pip/pip-284.md b/pip/pip-284.md
new file mode 100644
index 00000000000..5d5b0adbfe3
--- /dev/null
+++ b/pip/pip-284.md
@@ -0,0 +1,114 @@
+# Background knowledge
+
+Apache Pulsar introduced the topic-level policies by
[PIP-39](https://github.com/apache/pulsar/wiki/PIP-39%3A-Namespace-Change-Events).
+It uses the reader API to read messages and relies on compaction to implement
table-like logic in the memory.
+
+[PIP-104](https://github.com/apache/pulsar/issues/12356) Introduced new
consumer type `TableView` to support the same logic by
+Public Stable API.
+
+# Motivation
+
+Due to a lot of problems caused by the complex topic policy logic is as
follows. we can use a new stable `TableView` API instead of
+previous one.
+
+- https://github.com/apache/pulsar/pull/20763
+- https://github.com/apache/pulsar/pull/20613
+- https://github.com/apache/pulsar/pull/19746
+- etc...
+
+# Goals
+
+## In Scope
+
+- Reduce complex logic.
+- Reuse the public stable API to improve the project quality.
+
+## Out of Scope
+
+none.
+
+# High Level Design
+
+```mermaid
+sequenceDiagram
+ participant client
+ participant topic_policy_service
+ participant system_event_topic
+ client ->> topic_policy_service: Hi, what's the topic policy of
`persistent://public/default/test`
+ loop policy update event fetching
+ topic_policy_service ->> system_event_topic: Do you have new events?;
+ alt new policy event
+ system_event_topic ->> topic_policy_service: Yes, Sure. here you
are.
+ else no policy event
+ system_event_topic ->> topic_policy_service: No, Please wait for a
moment.
+ end;
+ end
+ Note right of topic_policy_service: Update the local cache table by
keeping the latest one;
+ topic_policy_service -->> client: Here you are.
+```
+
+# Detailed Design
+
+## Design & Implementation Details
+
+The new implementation will continue reuse the previous system topic
`namespace/__change_event` to store the topic policy data.
+and it will change the reading logic from the raw reader to TableView to
enhance the robustness.
+
+## Public-facing Changes
+
+none.
+
+### Public API
+
+none.
+
+### Binary protocol
+
+none.
+
+### Configuration
+
+none.
+
+### CLI
+
+none.
+
+### Metrics
+
+none.
+
+# Monitoring
+
+none.
+
+# Security Considerations
+
+none.
+
+# Backward & Forward Compatibility
+
+Without backward compatibility since we will reuse the same system topic
`namesapce/__change_event`.
+
+## Revert
+
+none.
+
+## Upgrade
+
+none.
+
+# Alternatives
+
+none.
+
+# General Notes
+
+none.
+
+# Links
+
+none.
+
+* Mailing List discussion thread:
https://lists.apache.org/thread/9v00sfpfxjpm775vgltkjoxwnllsgskg
+* Mailing List voting thread:
https://lists.apache.org/thread/gx43mzh88xp5ttz2gqghfqpz1yq51k60
