mikedias commented on PR #7865: URL: https://github.com/apache/paimon/pull/7865#issuecomment-4990779823
Thanks for the super quick review @JingsongLi , added the missing validation! With this configuration, the additional scan is now opt-in, removing the impact on existing tables. However, I feel it is still sub-optimal because it introduces a flag in a critical part of the system, which increases overall complexity and reduces maintainability... Am I being too harsh? The optimal solution, in my mind, would be to reuse existing manifest scans to derive the bucket count, so then we don't need a config or an additional scan. But I couldn't think of how to achieve that in the streaming case to distribute records to the right channels/buckets... Do you think it's possible to achieve that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
