mathieudruart commented on pull request #8017: URL: https://github.com/apache/pinot/pull/8017#issuecomment-1039617009
> @mathieudruart can you send me some doc/examples to produce batch message scenarios where the code fails. Need it to fix the test cases. Hi @KKcorps, here is an example scenario with which we reproduce the problem : - activate infinite [retention policy](https://pulsar.apache.org/docs/en/cookbooks-retention-expiry/) on a Pulsar namespace (this will make it easier to reproduce the problem) : `pulsar-admin namespaces set-retention my-tenant/my-ns --size -1 --time -1` - use a Pulsar client to push messages with batch activated into a topic of the previous namespace (the Pulsar java client activates batches by default) - create a table into Pinot linked to the previous topic (all messages will be loaded correctly) - send new batch messages into the topic => the first messages will be skipped (the number of skipped messages depends on the size of the last correctly consumed batch) Don't hesitate to ask me for clarification. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
