BewareMyPower commented on issue #12087:
URL: https://github.com/apache/pulsar/issues/12087#issuecomment-925850421
@eolivelli
> how can you apply this to KOP ?
Each entry produced by KoP is a buffer that can be converted to a
`MemoryRecords`.
```java
public static MemoryRecords readableRecords(ByteBuffer buffer) {
return new MemoryRecords(buffer);
}
```
Then we can reuse the method of `MemoryRecords` to create iterable records
```java
public Iterable<Record> records() {
return records;
}
```
Each `Record` is a single message, i.e. `MessageImpl<T>`.
> I though that this would be a simpler change, but in fact you have to deal
with the fact that the Converter may decide to split the single message into
multiple messages.
No, the converter did the same as the code before. The buffer is an entry
but not a single message.
The default converter's behavior is:
1. If the entry is a single message, use `newMessage` to create a
`Iterable<Message<T>>` that can only iterate once because it only contains one
message.
2. If the entry is a batched message, create a `Iterable<Message<T>>`,
during each iteration, use `newSingleMessage` to create single messages.
The difference is, the previous implementation uses a for loop, the current
implementation creates a iterator. The overhead is the allocation of the
iterator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
