jadami10 commented on PR #17254: URL: https://github.com/apache/pinot/pull/17254#issuecomment-3560816682
cc @Jackie-Jiang @ankitsultana in case you have good reviewers in mind We are internally exploring low TTL deduplication for preventing duplicates from issues in upstream systems that deliver at least once to kafka. Given the scale of some streams, even a few hours of primary keys becomes unnecessarily expensive. We will continuously test this, but we're also exploring alternatives to realtime deduplication in Pinot realtime tables. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
