ksankeerth opened a new pull request, #1011:
URL: https://github.com/apache/pulsar-client-go/pull/1011
Hi Team,
This PR introduces message payload processor feature to Golang client.
(PIP-96)
- added support for custom payload processor
- added default payload processor
- added a byte slice reader to process messages similar to Java Client
_(In Java, the methods for reading messages are implemented as part of
MessagePayloadContext
https://github.com/apache/pulsar/blob/master/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/MessagePayloadContext.java#L65-#L79
In this PR, I used MessagePayloadContext just to carry the information and
separated the above methods into a reader.)_
- added tests for pulsar message format and custom message format similar
to java implementation
<!--
### Contribution Checklist
- Name the pull request in the form "[Issue XYZ][component] Title of the
pull request", where *XYZ* should be replaced by the actual issue number.
Skip *Issue XYZ* if there is no associated github issue for this pull
request.
Skip *component* if you are unsure about which is the best component.
E.g. `[docs] Fix typo in produce method`.
- Fill out the template below to describe the changes contributed by the
pull request. That will give reviewers the context they need to do the review.
- Each pull request should address only one issue, not mix up code from
multiple issues.
- Each commit in the pull request has a meaningful commit message
- Once all items of the checklist are addressed, remove the above text and
this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
Fixes #962
### Motivation
As per PIP-96 proposal, it adds capabilities to process messages by the
client using pluggable message processor. Same functionalities are implemented
for Golang client.
### Modifications
- In Java Client implementation, checksum is only validated if the first
bytes matches with magic number. However, the current Golang client always try
to verify the checksum and return errors if checksum is not present. Due to
this reasons, If we KoP and Kafka entry.format, We'll get errors when parsing
MessageMeta. **To avoid that, Checksum validation was separated from reading
messages (We can log precise errors) and implemented similar logic we have in
Java Client.**
- A few unexposed struct and APIs were exposed to the client. Changed the
visibility as the client needs to use them when processing messages.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change added tests and can be verified as follows:
- added tests with Default Payload processor. It'll process pulsar
messages
- added tests with custom payload processor for different entry format.
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (no)
- The public API: (yes, not breaking changes)
1. New field in ConsumerOptions (MessagePayloadProcessor)
```go
consumer, err := client.Subscribe(ConsumerOptions{
Topic: topic,
SubscriptionName: "my-sub",
Type: Exclusive,
MessagePayloadProcessor: DefaultPayloadProcessor{},
})
```
2. previously `message` visibility was limited. It has changed
Old
```golang
type message struct {
```
New
```golang
type MessageImpl struct {
publishTime time.Time
```
- The schema: (no )
- The default values of configurations: (no)
- The wire protocol: (no)
### Documentation
This PR adds new feature. I have added Godoc comments in code. If additional
docs/separate PR required, I could contribute.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
