Github user rdhabalia commented on a diff in the pull request:
https://github.com/apache/incubator-pulsar/pull/532#discussion_r124362085
--- Diff: docs/NonPersistentTopic.md ---
@@ -0,0 +1,96 @@
+
+# Non-persistent topic
+
+<!-- TOC depthFrom:2 depthTo:4 withLinks:1 updateOnSave:1 orderedList:0 -->
+
+- [Topic](#topic)
+- [Non-persistent topic](#non-persistent-topic)
+ - [Performance](#performance)
+- [Client API](#client-api)
+ - [Consumer API](#consumer-api)
+ - [Producer API](#producer-api)
+- [Non-persistent configuration](#non-persistent-configuration)
+ - [Namespace configuration](#namespace-configuration)
+ - [Broker configuration](#broker-configuration)
+
+<!-- /TOC -->
+
+## Topic
+
+In pulsar, topic is a key entity that used to connect producers and
consumers. Pulsar supports **persistent** and **non-persistent** topics which
can be chosen by application based on its usecases and requirements. Producers
and consumers can use the same client-api to connect to persistent and
non-persistent topic.
+
+## Non-persistent topic
+
+As name suggests, non-persist topic does not persist messages into any
durable storage disk unlike persistent topic where messages are durably
persisted on multiple disks.
+
+Therefore, if you are using persistent delivery, messages are persisted to
disk/database so that they will survive a broker restart or subscriber
failover. While using non-persistent delivery, if you kill a broker or
subscriber is disconnected then subscriber will lose all in-transit messages.
So, client may see message loss with non-persistent topic.
+
+- In non-persistent topic, as soon as broker receives published message,
it immediately delivers this message to all connected subscribers without
persisting them into any storage. so, if subscriber gets disconnected with
broker then broker will not be able to deliver those in-transit messages and
subscribers will never be able to receive those messages again.
+- Broker only allows configured number of in-flight messages per client
connection. So, if producer tries to publish messages higher than this rate,
then broker silently drops those new incoming messages without processing and
delivering them to the subscribers. However, broker acknowledges with special
message-id (`msg-id: -1:-1`) for those dropped messages to signal producer
about the message drop.
+
+### Performance
+
+Non-persistent messaging is usually faster than persistent messaging
because broker does not persist messages and immediately sends ack back to
producer as soon as that message received by the broker. Therefore, producer
sees comparatively low publish latency with non-persistent topic.
+
+
+## Client API
+
+
+A topic name will look like:
+
+```
+non-persistent://my-property/us-west/my-namespace/my-topic
+```
+
+Producer and consumer can connect to non-persistent topic in a similar way
as persistent topic except topic name must start with `non-persistent`.
+
+Non-persistent topic supports all 3 different subscription-modes :
**Exclusive**, **Shared**, **Failover** which are already explained in details
at [GettingStarted](GettingStarted.md).
+
+
+### Consumer API
+
+```
+PulsarClient client = PulsarClient.create("pulsar://localhost:6650");
+
+Consumer consumer = client.subscribe(
+ "non-persistent://sample/standalone/ns1/my-topic",
+ "my-subscribtion-name");
+```
+
+### Producer API
+
+```
+PulsarClient client = PulsarClient.create("pulsar://localhost:6650");
+
+Producer producer = client.createProducer(
+ "non-persistent://sample/standalone/ns1/my-topic");
+```
+
+## Non-persistent configuration
+
+### Namespace configuration
+
+
+Boker creates non-persistent topic only under a namespace which is
configured with non-persistency. Client can set non-persistency of a namespace
using admin API and tools:
+
+```
+bin/pulsar-admin namespaces set-non-persistency test-property/cl1/ns1
--non-persistent
--- End diff --
> What happens in case the namespace is already in use and have persistent
topics on it?
Before setting non-persistency, it should verify that namespace should not
have any persistent-topic under it else it should fail the request. and it can
flip from non-persistent to persistent if there is no np-topic is being loaded
under that namespace.
> For example, let say I'm doing a lookup for non-persistent://.... and the
namespace is not owned by any broker, then the load balancer will assign this
namespace to a broker that supports non-persistent topics.
Actually, idea behind marking namespace as non-persistent to just isolate
namespace for non-persistent topics and configured-broker (allow to own only
np-topics) owns only non-persistent topics at any time.
If we don't have have namespace level visibility and if broker assigns
topic at runtime on lookup eg: `non-persistent://..` then on first
`non-persistent (np)` lookup configured-broker (which should only owns np
topic) owns that namespace and if the same namespace has persistent topic also
then configured broker ends up owning mix of persistent and non-persistent
topics. Therefore, team decided to take namespace level approach to avoid mix
of topics under a configured broker.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---