yuqi1129 commented on code in PR #8813:
URL: https://github.com/apache/gravitino/pull/8813#discussion_r2431948897
##########
docs/manage-jobs-in-gravitino.md:
##########
@@ -169,6 +169,19 @@ Note that:
4. The `className` is required for the Java and Scala Spark job template, it
is the main class of
the Spark application to be executed. For PySpark job template, this field
can be `null` instead.
+:::info
+If you want to use the job artifacts on the Hadoop Compatible FileSystems
(like `hdfs://`,
+`s3a://`, `gs://`, etc.), You need to make sure that:
+
+1. The Gravitino server has the corresponding Hadoop client libraries in its
classpath
+ `{GRAVITINO_HOME}/libs` directory. For example, if you want to use `s3a://`
scheme, you need to
+ put the `hadoop-aws` and its dependencies in the `{GRAVITINO_HOME}/libs`
directory. By
+ default, the HDFS client libraries are already included in the Gravitino
server package.
+2. The Gravitino server has the corresponding Hadoop configurations in its
classpath
+ `{GRAVITINO_HOME}/conf` directory. For example, if you want to use `s3a://`
scheme, you need to
Review Comment:
> The Gravitino server has the corresponding Hadoop configurations in its
classpath
`{GRAVITINO_HOME}/conf` directory
I can't get this sentence, as there are no `*.xml` in the folder
`{GRAVITINO_HOME}/conf`, why do you say that?
Should the word `has` in `Gravitino server has the corresponding` be `needs`?
##########
core/src/main/java/org/apache/gravitino/utils/IsolatedClassLoader.java:
##########
@@ -261,7 +262,8 @@ private boolean isCatalogClass(String name) {
*/
private boolean isBarrierClass(String name) {
// We need to add more later on when we have more catalog implementations.
- return barrierClasses.stream().anyMatch(name::startsWith);
+ return barrierClasses.stream().anyMatch(name::startsWith)
+ || name.startsWith("org.apache.hadoop");
Review Comment:
Why not just put `"org.apache.hadoop"` in the `barrierClasses`, Is the
`barrierClasses` designed to handle this cases?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
