kbendick commented on pull request #3326: URL: https://github.com/apache/iceberg/pull/3326#issuecomment-980423958
> Hey @harshm-dev personally I am +0 on this change. I think it would be really good to rationalise the `type` and `catalog-*` parameters and this does so. I am not sure how other people are using these fields and if its disruptive or not. What do others think? I too would have to say that I have some concern about restricting the ability for people to make type specific implementation configs. I can see that generally this doesn't exactly do that, but I agree with @rymurr that it seems we're heading further away from the direction of a general `catalog-impl` and a standardized way to add specific configs. I also have some concerns because newer Flink connector APIs require configuration options to be declared (both required and optional). So I would move further in the direction of standardizing on a configuration format that can be specified by catalog-impl and then allow that to hook in at runtime to Flinks configuration declaration, etc. If we continue to have to support catalog-impl, it should probably be the first class concept so that things like that don't get missed as it's easy to simply list out the configs that Flink needs from this set of well-knowk types (which I have seen proposed in some PRs to upgrade to the newer way Flink configs are input). I really do appreciate the contribution @harshm-dev , and will admit for a while I felt this was the better way to go, but thinking on it more I'm in agreement with @rymurr and feel that our efforts would be better spent standardizing on ways for catalogs to propagate their set of configuration values to the right places in a standardized way. Standardizing configs and emphasising on catalog-impl being first-class in the code base, so that 3rd party connectors that can't be open sourced for whatever reason or where it wouldn't make sense to integrate them into the main repository due to licensing issues can still be fully supported. That said, I'd consider maybe waiting until the Thanksgiving holiday is over in the US for last thoughts from others @harshm-dev :D -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
