Hi all,
I would like to share recent work on integrating the TsFile C++ library with LabVIEW. The implementation is available here: https://github.com/Young-Leo/tsfile/tree/ly/labview-bulk-io-perf The goal is to provide a stable C ABI that can be called from LabVIEW through the Call Library Function Node (CLFN), without exposing the C++ ABI or C++ object lifetimes directly. The current implementation includes: - Schema and writer lifecycle APIs. - Bulk INT32, FLOAT, and DOUBLE writes. - Row-major/interleaved input support: data[row * column_count + column] - Column-major/channel-contiguous input support: data[column * row_count + row] - Reusable Tablets for continuous acquisition. - Bulk numeric read APIs. - Explicit flush and synchronous close. - A two-phase asynchronous close API for file rotation. - CMake integration, tests, and LabVIEW usage documentation. The row-major path fills Tablet column buffers directly in one fused loop. The column-major path copies each channel as a contiguous array. Both paths avoid the previous full-size transpose buffer. The intended LabVIEW streaming flow is: create schema -> add columns -> open writer -> repeatedly write acquisition blocks -> optional flush -> close or asynchronous close I also compared the write performance with NI TDMS using the same nine-channel DOUBLE input: - 60,000 rows per block - 9 channels - 25 blocks - 13,500,000 channel values in total - Uncompressed value columns - Full-path timing from file creation until close returned The median full-path throughput was: - NI TDMS: 14.78 M values/s - TsFile row-major: 13.72 M values/s - TsFile column-major: 13.50 M values/s The target acquisition workload is 9 channels at 60 kHz, or approximately 0.54 M values/s. The measured TsFile throughput is about 25 times the target workload. During optimization, TsFile full-path throughput increased from 8.58 M values/s to 13.77 M values/s, while median flush time decreased from approximately 0.622 seconds to 0.066 seconds. The main optimizations were: - Direct row-major Tablet filling. - Column-major APIs for channel-contiguous input. - Reusable Tablet buffers. - Bulk bitmap handling. - 64 KiB Chunk pages. - Direct writing of encoded Chunk pages, avoiding an additional file-sized intermediate copy. The branch has been rebased onto the latest develop branch. Feedback and suggestions on the LabVIEW integration and API design are welcome. Best regards, Leo
