xinbinhuang commented on issue #3117: URL: https://github.com/apache/iceberg/issues/3117#issuecomment-1133059573
> Since the case I > > > > @xinbinhuang a tricky case would be flink on k8s creating/writing/reading iceberg table in hadoop catalog. How hadoop dependencies should be resolved in this case? > > > > > > As long as Hadoop is on the classpath, it should just work. The change simply try to defer the loading of Hadoop dependencies so that catalog that doesn't require Hadoop (i.e. Glue catalog) doesn't need the Hadoop jar on the classpath > > @xinbinhuang Since the case I mentioned is flink on k8s, it's not likely there would be hadoop on classpath in flink image. Ah I see. If you're using Hadoop catalog with k8s, this PR has nothing to do with it. Since you're using Hadoop catalog, you'll need to supply hadoop classes regardless. I'm not sure if bundle `hadoop-common` in your fat jar is good enough. You may also need to specify [`HADOOP_CLASSPATH`](https://nightlies.apache.org/flink/flink-docs-release-1.11/ops/deployment/hadoop.html#providing-hadoop-classes) when you [build the image](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/resource-providers/native_kubernetes/#flink-on-kubernetes-reference) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
