stevenzwu commented on issue #2918: URL: https://github.com/apache/iceberg/issues/2918#issuecomment-907686809
> But CDC source parallelism (x) and job parallelism(y) are out of iceberg and define by user. Dose iceberg should restrict CDC source parallelism must equal to job parallelism? How does this scenario work for other sinks (like MySQL)? How can other sinks ensure the processing ordering? Do all sinks need to override the parallelism of upstream operator and force the upstream operator parallelism matches the source parallelism? To me, x=y is not a behavior imposed by Iceberg sink. Rather, it is a behavior imposed by the CDC source. > we need to discuss the relation between equality keys and partition keys. Are partition keys a subset of equality keys? Agree. My intuition is that they are independent. But I am also not very clear on that. cc @openinx @rdblue to get more inputs on the whole discussion -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
