sijie commented on a change in pull request #4079: PIP-34 Key_Shared
subscription core implementation.
URL: https://github.com/apache/pulsar/pull/4079#discussion_r277941483
##########
File path:
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/TypedMessageBuilder.java
##########
@@ -103,6 +103,15 @@
*/
TypedMessageBuilder<T> keyBytes(byte[] key);
+ /**
+ * Sets the ordering key of the message for message dispatch in {@link
SubscriptionType#Key_Shared} mode.
+ * Partition key Will be used if ordering key not specified
+ *
+ * @param orderingKey the ordering key for the message
+ * @return the message builder instance
+ */
+ TypedMessageBuilder<T> orderingKey(byte[] orderingKey);
Review comment:
> What would be the use case for having 2 keys?
@merlimat I have explained one of the use cases above - CDC for a
distributed database which usually has 2 kinds of keys. there might be other
use cases.
> If you have partitions and set the partition routing based on 1 key, you
will have ordering based on that key. If on the dispatcher you set a different
key that won't address that you'll get messages out-of-order for the same
orderingKey.
first of all, from use case perspective, I think how applications define a
partition key and an ordering key is really up to themselves. but having a way
to specify an ordering key provides a flexible mechanism for people to adopt to
their own use cases.
secondly, from the motivation of introducing key_shared, it allows scaling
out the consumption on a single partition while maintaining some sort of key
ordering. The ordering can be the ordering of the original key, or any other
sub keys, or a total different key.
The original "key" does already determines which partition to store message.
The key indicates the ordering of the messages within a same partition, and a
same "key". However that means in dispatching the messages, all the messages of
a same key can only be sent to one consumer. This can limit the capability of
key_shared subscription. the ordering key allows applications to configure a
finer granularity "ordering_key" for scaling out the consumption beyond the
original "key".
for example, in a social app, you have a conversation stream, where the
stream is partitioned by `from_user_id`, so that all the conversations of a
same "from_user" is in one partition, but you want the consumers to consume
events by `conversation_id` (which is comprised of from_user_id and
to_user_id). in this example, you are using "from_user_id" as the key,
"conversation" as the ordering key.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
