Hi Manish, May i know which version of hadoop will provide OBSFS support. Is this file system already supported in recent hadoop releases?
Thanks Sujith. On Tue, 19 Feb 2019 at 4:18 PM, manish nalla <manishnalla1...@gmail.com> wrote: > Hi all, > > OBS is an Object-based Storage Service developed and maintained by > HuaweiCloud. It provides large storage capacity and is capable of storing > any type of file. OBS supports both S3-client and OBS-client to connect to > the OBS server. > > Currently in CarbonData we are supporting HDFS and S3 as FileSystems and I > am proposing to support OBSFS as another FileSystem because of a few > drawbacks of S3FileSystem. > > CarbonData needs OBSFS instead of S3 because of two main reasons: > 1. Append: While doing append first we have to read the whole object and > then append to the object which is quite slow. > 2. Atomic Rename: No atomic rename is there in S3 as also mentioned in Jira > [CARBONDATA-2670]. > > So both these issues can be fixed if we use OBSFileSystem. > > Any suggestions from the community will be greatly appreciated. I would be > uploading the design doc shortly. > > Thanks and regards > Manish Nalla > EI BigData Kernel, > Huawei Technologies India Pvt. Ltd >
