Hi jinhui, Thanks for driving this, can you provide some information about "When using the s3 protocol to access the fusion bucket, the traffic will pass through the access layer twice", is there an official website?
Best, Jingsong On Thu, Jan 9, 2025 at 10:56 AM 刘金辉 <965147...@qq.com.invalid> wrote: > > Introduction > Tencent COS is a famous object storage system provided by Tencent Corp. > Hadoop-COS is a client that makes the upper computing systems based on HDFS > be able to use the COS as its underlying storage system. tencent cloud is top > 2 cloud vendors in China market and the object store COS > （https://intl.cloud.tencent.com/product/cos） is widely used among China’s > cloud users but now it is hard for hadoop user to access data laid on COS > storage > > > We can use S3 SDK and COSN SDK to access COSN，but > 1. When using the s3 protocol to access the fusion bucket, the traffic will > pass through the access layer twice, the traffic will be doubled, and the > performance will be impaired. This problem will not occur when using cosn. > 2. Compared with cos ordinary buckets, fusion buckets have performance > advantages in list, rename, truncate and other operations. > 3. If you must use the s3 protocol to access cos, we do not recommend using > fusion buckets, then the s3 protocol + cos ordinary bucket will encounter > some of the above performance issues. > 4. Other data lake frameworks also support access to cosn, oss, S3, etc. > > > > > Therefore, in order to allow more users to use paimon with high performance, > it is necessary to support cosn access to paimon. > > > > > > > Please give your opinions. thanks > > > > > > 刘金辉 > 965147...@qq.com > > > >
