mrdrivingduck opened a new pull request, #205: URL: https://github.com/apache/paimon-cpp/pull/205
### Purpose This PR adds a native, read-only OSS filesystem backed by [Alibaba Cloud OSS SDK V2](https://github.com/aliyun/alibabacloud-oss-cpp-sdk-v2). ## Motivation ### Static linking and distribution Some integration scenarios require Paimon-cpp to be linked statically into another library, such as a DuckDB extension. JindoSDK is delivered as a separate dynamic library, which makes this integration and distribution model less flexible. ### Shutdown reliability in PostgreSQL processes While integrating PostgreSQL with DuckDB, we observed an intermittent shutdown hang along the PostgreSQL → DuckDB → duckdb-paimon → paimon-cpp path when JindoSDK was loaded through the Paimon filesystem. The captured shutdown stack was: ```text PostgreSQL backend process proc_exit() → exit() → C++ atexit/static destructor → bvar::PassiveStatus::~PassiveStatus() → bvar::detail::Sampler::destroy() → pthread_mutex_lock() ← waits indefinitely ``` The process also had a `bvar_sampler` thread running. This points to a fork/exit lifecycle issue in JindoSDK's embedded bRPC/bvar dependencies. Since JindoSDK is closed source and brings a relatively large dependency set, it is difficult to investigate or address from Paimon-cpp. ### An open-source OSS filesystem Given that OSS SDK V2 is open source, adding an OSS implementation to Paimon-cpp would provide an open and self-contained option for OSS access. The existing object-store abstraction introduced for S3 keeps this addition small and avoids duplicated filesystem logic. ## Benchmark The benchmark directly reads a 133 MiB Parquet data file belonging to a Paimon table in a real OSS environment. Both filesystems were built in a clean Release build, and each workload was run six times. Values are medians. | Workload | OSS Median | Jindo Median | |---|---:|---:| | Open(path) | 10.63 ms (-8.3%) | 11.59 ms | | Open(known status) | 0.73 ms (-94.9%) | 14.25 ms | | 1 MiB range read | 47.37 ms (-27.3%) | 65.13 ms | | Single 8 MiB sequential read | 96.83 ms (-20.4%) | 121.70 ms | `Open(known status)` reopens a file with previously obtained metadata. It avoids an extra metadata request through the status propagation introduced in [#189](https://github.com/apache/paimon-cpp/pull/189). The range and sequential workloads each issue a single read request. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
