Hi all, OBS is an Object-based Storage Service developed and maintained by HuaweiCloud. It provides large storage capacity and is capable of storing any type of file. OBS supports both S3-client and OBS-client to connect to the OBS server.
Currently in CarbonData we are supporting HDFS and S3 as FileSystems and I am proposing to support OBSFS as another FileSystem because of a few drawbacks of S3FileSystem. CarbonData needs OBSFS instead of S3 because of two main reasons: 1. Append: While doing append first we have to read the whole object and then append to the object which is quite slow. 2. Atomic Rename: No atomic rename is there in S3 as also mentioned in Jira [CARBONDATA-2670]. So both these issues can be fixed if we use OBSFileSystem. Any suggestions from the community will be greatly appreciated. I would be uploading the design doc shortly. Thanks and regards Manish Nalla EI BigData Kernel, Huawei Technologies India Pvt. Ltd
