+1 binding. Best Regards, Zelin Yu
On Wed, Jan 15, 2025 at 3:58 PM 刘金辉 <965147...@qq.com.invalid> wrote: > hi.all， > Happy New Year to all the friends in the paimon community，start a vote on > Paimon to support Tencent Cloud Cosn。 > > Introduction > Tencent COS is a famous object storage system provided by Tencent Corp. > Hadoop-COS is a client that makes the upper computing systems based on HDFS > be able to use the COS as its underlying storage system. tencent cloud is > top 2 cloud vendors in China market and the object store COS （ > https://intl.cloud.tencent.com/product/cos） is widely used among China’s > cloud users but now it is hard for hadoop user to access data laid on COS > storage > > We can use S3 SDK and COSN SDK to access COSN，but > 1. When using the s3 protocol to access the fusion bucket, the traffic > will pass through the access layer twice, the traffic will be doubled, and > the performance will be impaired. This problem will not occur when using > cosn. > 2. Compared with cos ordinary buckets, fusion buckets have performance > advantages in list, rename, truncate and other operations. > 3. If you must use the s3 protocol to access cos, we do not recommend > using fusion buckets, then the s3 protocol + cos ordinary bucket will > encounter some of the above performance issues. > 4. Other data lake frameworks also support access to cosn, oss, S3, etc. > > > Optimization methods of cosn: > 1. Performance optimization: > a. Combined with the architecture of COS fusion bucket, separate metadata > flow and data flow to achieve high-performance file semantic access > b. Based on COS distributed metadata engine, provide high-performance > metadata access > c. Provide high-concurrency data flow through pre-reading strategy, block > processing and other means > 2. Integrate Tencent Cloud service role authentication capabilities to > improve access security > 3. Integrate Tencent Cloud log reporting capabilities, support task-level > log reporting, and improve observability > > S3 Architecture: > > > > > Cosn Architecture > > best，liujinhui > > The vote will be open for at least 72 hours or until necessary numbers of > votes are reached. > > Please vote accordingly: > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove with the reason > > > > ------------------------------ > 刘金辉 > 965147...@qq.com > > <https://wx.mail.qq.com/home/index?t=readmail_businesscard_midpage&nocheck=true&name=%E5%88%98%E9%87%91%E8%BE%89&icon=http%3A%2F%2Fthirdqq.qlogo.cn%2Fg%3Fb%3Dsdk%26k%3DFpV1f1q15qLoN9OdTof6icw%26s%3D100%26t%3D1586874456%3Frand%3D1647223281&mail=965147871%40qq.com&code=> > >
