openinx commented on pull request #3270: URL: https://github.com/apache/iceberg/pull/3270#issuecomment-987509603
As we will introduce more and more vendor's integration modules, I think it's time to consider the strategy to handle the relationship between iceberg core/api/engines modules and vendor integration modules. Currently, we have introduced the aws & nessie modules and we added them into the engines runtime modules by default. As more and more integration come in, the engine runtime modules will be even more fatter. The correct way is making them to be pluggable modules, for example the apache hadoop release don't include all the cloud vendor's object storage implementations into the default classpath, but it makes those modules to be hadoop's plugin implementations and if people want to enable the specific vendor implementation then they just enable it. For our apache iceberg integration case, I think we can also make the `iceberg-aliyun` to be optional plugin modules and we build `iceberg-aliyun` module with including all dependencies jars. If people want to access the iceberg tables backed aliyun oss, then they just need to load the `iceberg-aliyun` jar. Does that make sense (also for future vendor integration ) ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
