Hello community,
I would like to discuss a possible approach to supporting Go in Apache TsFile. The community previously discussed Go support in [Issue #73](https://github.com/apache/tsfile/issues/73). There was once a pure-Go implementation, but it is no longer maintained and is incompatible with the current TsFile format. As the C++ implementation and C wrapper have become more mature, we can now consider providing Go bindings without maintaining another independent implementation of the TsFile format: ```text Go API → cgo → C API → C++ TsFile Core ``` This approach would allow Go applications to read and write current TsFile files while sharing the format compatibility, bug fixes, and performance improvements of the C++ module. The initial Go binding design could have the following characteristics: - expose idiomatic Go Reader, Writer, Schema, Tablet, and ResultSet APIs; - keep cgo and `unsafe` code inside an internal native package; - use opaque handles for C/C++ objects without exposing their internal layouts to Go users; - provide explicit `Close` methods with clearly defined memory ownership across the language boundary; - map C API error codes to Go errors; - provide both a custom Tablet API and an Apache Arrow API; - support batch data exchange through the Arrow C Data Interface; - reuse the existing `_tsfile_*` interfaces and extend the low-level capabilities needed by multiple language bindings when necessary. For performance, the primary read and write paths should use batch-oriented, columnar interfaces. Crossing the cgo boundary once per cell could introduce significant overhead, so Tablet, Arrow RecordBatch, or another batch representation should be used as the main data exchange unit, requiring only one or a small number of cgo calls per batch. Row-oriented APIs could still be provided for convenience, but they could buffer data in Go and write it as a Tablet instead of calling the C API once for every value. Batch data exchange can currently use the Arrow C Data Interface path. If the layout and lifetime of the existing TsBlock representation can be exposed to Go safely and reliably, a TsBlock batch path can also be used. Both paths have costs: the Arrow path requires data conversion and copying between TsBlock, Arrow, and Tablet, while directly exposing TsBlock increases the complexity of cross-language memory ownership and lifetime management. The initial scope could include: - Go module and package structure; - table-model Reader and Writer; - Schema and Tablet APIs; - custom Tablet batch reads and writes; - Apache Arrow batch reads and writes; - common scalar types, null values, strings, and binary values; - deterministic resource cleanup; - compatibility tests against files produced by the Java and C++ implementations; - benchmarks comparing native C++, batched Go bindings, and row-oriented Go access; - support for major platforms, including Linux, macOS, Windows, amd64, and arm64. Tree-model support could be added incrementally after the first-stage design and implementation become stable. I would appreciate the community's feedback on the following questions: 1. Which existing `_tsfile_*` interfaces can be used directly as general binding APIs, and which ones need to be extended or adjusted? 2. Which compatibility and performance metrics should be used as acceptance criteria for the Go bindings? More background and the initial proposal are available in [Issue #924](https://github.com/apache/tsfile/issues/924). Feedback on the overall direction, initial scope, and low-level API design would be very welcome. Thank you!
