ferenc-csaky commented on PR #29102: URL: https://github.com/apache/flink/pull/29102#issuecomment-5663783842
@raminqaf @gustavodemorais Thank you both for the review, appreciate it! Before I start to address any requested changes, I'd like to reflect on the higher level and strategic questions. About the mailing list conversation, I'm totally on board with that, my main reason about opening the PR was the impl itself pretty much explains what the motivation is and it helps the general understanding. About the use-case i'm trying to solve: I basically have a generalized Flink SQL job, that can run on a lot of tables which has different schema. It does some transformations that are orthogonal to this change, but it involves converting the source retract stream to a special upsert stream only keeping `+I` and `+U` (the delete flag is baked into the schema, there are historical reasons to do so, that's also orthogonal here). Originally I did that with a lot dumber PTF, and because of this `SELECT *` is the only reasonable way to achieve this AFAIK. I still have that custom UDF as a workaround, but I feel like `TO_CHANGELOG` should be able to keep the schema as-is. So technically `SELECT * EXCLUDE(op) from TO_CHANGELOG` would solve this specific problem, but IMO `TO_CHANGELOG` changing the output schema inevitably seems like an unnecessary restriction from a UX perspective. Configuration options that have reasonable defaults does not really add too much complexity, users not even necessarily need to know it exists. until they need them for some reason. I'm happy to take this conversation to the mailing list, let me know WDYT. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
