mchades commented on code in PR #7298:
URL: https://github.com/apache/gravitino/pull/7298#discussion_r2131628382
##########
bundles/aliyun-bundle/build.gradle.kts:
##########
Review Comment:
> > gravitino-{specific-fs}-bundle.jar
> > when using this jar, we assume that the environment does not have hadoop
dependencies and specific FS dependencies, so we don't need to shade the hadoop
dependencies and specific FS dependencies, but need to shade third party
dependencies to avoid potential conflicts
>
> How do you handle SPI files,
>
> When you shade `GoogleHadoopFileSystem`, the `ServiceLoader` can't load it
successfully.
>
> I believe this problem would be a similar problem in other filesystems
As you quoted, what I meant is that `gravitino-{specific-fs}-bundle.jar`
**does not need** shade specific FS dependencies, while in your example,
`GoogleHadoopFileSystem` is a specific FS dependency.
> Another point is that shading Hadoop and the filesystem may cover cases
where there are more than one version of those dependencies. One version is the
jars you have shaded, and another is the jars that users have introduced
mistakenly.
Sorry, I don't get your point. The main purpose of shading is to avoid
conflicts with multiple Hadoop versions, so only the
`gravitino-{specific-fs}.jar` will shade the hadoop dependencies and specific
FS dependencies to avoid potential version conflicts
##########
bundles/aliyun-bundle/build.gradle.kts:
##########
Review Comment:
> > gravitino-{specific-fs}-bundle.jar
> > when using this jar, we assume that the environment does not have hadoop
dependencies and specific FS dependencies, so we don't need to shade the hadoop
dependencies and specific FS dependencies, but need to shade third party
dependencies to avoid potential conflicts
>
> How do you handle SPI files,
>
> When you shade `GoogleHadoopFileSystem`, the `ServiceLoader` can't load it
successfully.
>
> I believe this problem would be a similar problem in other filesystems
As you quoted, what I meant is that `gravitino-{specific-fs}-bundle.jar`
**does not need** shade specific FS dependencies, while in your example,
`GoogleHadoopFileSystem` is a specific FS dependency.
> Another point is that shading Hadoop and the filesystem may cover cases
where there are more than one version of those dependencies. One version is the
jars you have shaded, and another is the jars that users have introduced
mistakenly.
Sorry, I don't get your point. The main purpose of shading is to avoid
conflicts with multiple Hadoop versions, so only the
`gravitino-{specific-fs}.jar` will shade the hadoop dependencies and specific
FS dependencies to avoid potential version conflicts
@yuqi1129
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
