Xuanwo opened a new issue, #4939: URL: https://github.com/apache/opendal/issues/4939
### Feature Description The OpenDAL Python binding now releases all its services in a single package, which makes it difficult to use and extend. I propose splitting it into separate packages, similar to what I've done with [`opendalfs`](https://github.com/fsspec/opendalfs). ```mermaid graph TD; opendalfs.OpendalFileSystem -- import --> MemoryFileSystem; opendalfs.OpendalFileSystem -- import --> S3FileSystem; opendalfs.OpendalFileSystem -- import --> FsFileSystem; opendalfs.OpendalFileSystem -- import --> ...FileSystem; MemoryFileSystem -- use --> opendalfs-core; S3FileSystem -- use --> opendalfs-core; FsFileSystem -- use --> opendalfs-core; ...FileSystem -- use --> opendalfs-core; opendalfs-core -- use -->opendal["Apache OpenDAL"]; ``` ### Problem and Solution OpenDAL Python is large yet still doesn't cover all the services users need. We can divide it into multiple packages, making `opendal` a virtual meta-package that only provides the Python API and imports the correct service on demand when needed. For example, as I showed up in `opendalfs`: https://github.com/fsspec/opendalfs/blob/e19d28eb9f82e285685f91b3c80805146759b7d7/opendalfs/fs.py#L8-L25 ```python def __init__(self, scheme, *args, **kwargs): super().__init__(*args, **kwargs) try: # Load the module dynamically based on scheme module = importlib.import_module(f"opendalfs_service_{scheme}") # Get the file system class based on scheme fs_class = getattr(module, f"{scheme.capitalize()}FileSystem") # initialize the file system with the kwargs self.fs = fs_class(**kwargs) except ImportError: raise ImportError( f"Cannot import opendal_service_{scheme}, please check if the module exists" ) except AttributeError: raise AttributeError( f"Cannot find {scheme.capitalize()}FileSystem in opendal_service_{scheme}" ) ``` ### Additional Context _No response_ ### Are you willing to contribute to the development of this feature? - [ ] Yes, I am willing to contribute to the development of this feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
