1u0 commented on a change in pull request #9169: [FLINK-12998][docs] Update documentation for file systems loading as plugins URL: https://github.com/apache/flink/pull/9169#discussion_r305285980
########## File path: docs/ops/filesystems/index.md ########## @@ -83,17 +101,21 @@ fs.hdfs.hadoopconf: /path/to/etc/hadoop This registers `/path/to/etc/hadoop` as Hadoop's configuration directory and is where Flink will look for the `core-site.xml` and `hdfs-site.xml` files. -## Adding new File System Implementations +## Adding a new pluggable File System implementation -File systems are represented via the `org.apache.flink.core.fs.FileSystem` class, which captures the ways to access and modify files and objects in that file system. -Implementations are discovered by Flink through Java's service abstraction, making it easy to add new file system implementations. +File systems are represented via the `org.apache.flink.core.fs.FileSystem` class, which captures the ways to access and modify files and objects in that file system. To add a new file system: - Add the File System implementation, which is a subclass of `org.apache.flink.core.fs.FileSystem`. - Add a factory that instantiates that file system and declares the scheme under which the FileSystem is registered. This must be a subclass of `org.apache.flink.core.fs.FileSystemFactory`. - - Add a service entry. Create a file `META-INF/services/org.apache.flink.core.fs.FileSystemFactory` which contains the class name of your file system factory class. + - Add a service entry. Create a file `META-INF/services/org.apache.flink.core.fs.FileSystemFactory` which contains the class name of your file system factory class + (see the [Java Service Loader docs](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) for more details). + +During plugins discovery, the file system factory class would be loaded by a dedicated Java class loader to avoid class conflicts with other plugins and Flink components. +The same class loader should be used during file system instantiation and the file system operation calls. Review comment: This sentence is a recommendation for implementation, we can't really enforce it or have such guarantee. That's why the next sentence mentions the caveat case. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
