rdhabalia commented on a change in pull request #1903: Support pulsar source to
start consumer with topic patterns
URL: https://github.com/apache/incubator-pulsar/pull/1903#discussion_r192929309
##########
File path: pulsar-functions/proto/src/main/proto/Function.proto
##########
@@ -70,6 +70,7 @@ message SourceSpec {
SubscriptionType subscriptionType = 3;
map<string,string> topicsToSerDeClassName = 4;
uint64 timeoutMs = 6;
+ string topicsPattern = 7;
Review comment:
I thought about it earlier but then I feel we can combine it in
`topicsToSerDeClassName` only. in one of the usecase, topicPattern can be used
to start all consumers under a given namespace. now, there is a possibility
that some of the topics may need different deserializers so, PulsarSource can
first check deserializer for a specific topic in the map and then try to match
the pattern in the same map. in this way, we can save cpu by avoiding
pattern-matching on every message and will be easier for user to configure it
in function-config `topicToSerDeMap.put("pattern", deserializer)`
```
+ SerDe deserializer = null;
+ if (this.topicToSerDeMap.containsKey(topicName)) {
+ deserializer = this.topicToSerDeMap.get(topicName);
+ } else if (isTopicsPattern) {
+ deserializer =
this.topicToSerDeMap.get(this.pulsarSourceConfig.getTopicsPattern());
+ }
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
