Hi all,



I generally support the tiered C++ standard approach.




Since Arrow requires C++17 starting from 10.0, Arrow-related functionality 
should be treated as an optional extension rather than part of the TsFile core. 
The core library should explicitly remain at C++11 to preserve compatibility 
with embedded and legacy environments.




From an implementation perspective, Arrow support can live in a separate module 
(e.g. tsfile-arrow), built with C++17 and guarded by a dedicated build option 
(such as TSFILE_ENABLE_ARROW). The interface between the core and this module 
should be kept narrow and avoid exposing Arrow types or C++17 features in 
public headers.




This keeps the core stable while still allowing Arrow-based capabilities where 
the toolchain and deployment environment permit.







Best regards,
Hingzhi Gao


Hongzhi Gao
[email protected]



        



         原始邮件
         
       
发件人:shuolin_l <[email protected]&gt;
发件时间:2026年1月19日 14:18
收件人:dev <[email protected]&gt;
主题:Proposal on C++ Standard Strategy for Optional Arrow-based Featuresin TsFile



       Hi&nbsp;all,

During&nbsp;the&nbsp;recent&nbsp;work&nbsp;on&nbsp;enabling&nbsp;batch&nbsp;read/write&nbsp;support&nbsp;for&nbsp;TsFile&nbsp;based&nbsp;on&nbsp;Apache&nbsp;Arrow,&nbsp;I&nbsp;noticed&nbsp;a&nbsp;divergence&nbsp;between&nbsp;Arrow&nbsp;and&nbsp;our&nbsp;current&nbsp;project&nbsp;in&nbsp;terms&nbsp;of&nbsp;supported&nbsp;C++&nbsp;standards.&nbsp;This&nbsp;raises&nbsp;an&nbsp;important&nbsp;question&nbsp;regarding&nbsp;the&nbsp;technical&nbsp;strategy&nbsp;for&nbsp;TsFile&nbsp;as&nbsp;a&nbsp;common&nbsp;component&nbsp;across&nbsp;both&nbsp;embedded&nbsp;and&nbsp;cloud&nbsp;environments.
According&nbsp;to&nbsp;the&nbsp;official&nbsp;Arrow&nbsp;documentation,&nbsp;starting&nbsp;from&nbsp;version&nbsp;10.0.0,&nbsp;C++11&nbsp;is&nbsp;no&nbsp;longer&nbsp;supported&nbsp;and&nbsp;C++17&nbsp;is&nbsp;required&nbsp;(see&nbsp;issue&nbsp;[1]).&nbsp;Therefore,&nbsp;introducing&nbsp;Arrow-related&nbsp;capabilities&nbsp;inevitably&nbsp;requires&nbsp;a&nbsp;build&nbsp;environment&nbsp;that&nbsp;supports&nbsp;a&nbsp;newer&nbsp;C++&nbsp;standard.

In&nbsp;embedded&nbsp;scenarios,&nbsp;although&nbsp;we&nbsp;already&nbsp;provide&nbsp;C-based&nbsp;interfaces&nbsp;to&nbsp;minimize&nbsp;dependency&nbsp;on&nbsp;C++&nbsp;versions,&nbsp;in&nbsp;practice&nbsp;some&nbsp;embedded&nbsp;or&nbsp;legacy&nbsp;edge&nbsp;devices&nbsp;still&nbsp;only&nbsp;support&nbsp;relatively&nbsp;old&nbsp;C++&nbsp;standards.&nbsp;In&nbsp;contrast,&nbsp;cloud&nbsp;environments&nbsp;or&nbsp;platforms&nbsp;used&nbsp;for&nbsp;Python-based&nbsp;training&nbsp;and&nbsp;data&nbsp;processing&nbsp;usually&nbsp;have&nbsp;more&nbsp;modern&nbsp;toolchains,&nbsp;where&nbsp;C++17&nbsp;support&nbsp;is&nbsp;generally&nbsp;not&nbsp;an&nbsp;issue.

Given&nbsp;these&nbsp;differences&nbsp;in&nbsp;usage&nbsp;scenarios,&nbsp;I&nbsp;propose&nbsp;adopting&nbsp;a&nbsp;tiered&nbsp;strategy&nbsp;for&nbsp;C++&nbsp;standard&nbsp;support&nbsp;within&nbsp;the&nbsp;project:

&nbsp;&nbsp;*&nbsp;&nbsp;&nbsp;The&nbsp;core&nbsp;and&nbsp;basic&nbsp;functionality&nbsp;of&nbsp;the&nbsp;project&nbsp;will&nbsp;continue&nbsp;to&nbsp;use&nbsp;C++11&nbsp;as&nbsp;the&nbsp;minimum&nbsp;required&nbsp;standard,&nbsp;to&nbsp;ensure&nbsp;compatibility&nbsp;with&nbsp;resource-constrained&nbsp;or&nbsp;legacy&nbsp;platforms.
&nbsp;&nbsp;*&nbsp;&nbsp;&nbsp;Features&nbsp;that&nbsp;depend&nbsp;on&nbsp;newer&nbsp;capabilities&nbsp;or&nbsp;third-party&nbsp;libraries&nbsp;such&nbsp;as&nbsp;Arrow&nbsp;will&nbsp;be&nbsp;provided&nbsp;as&nbsp;optional&nbsp;extension&nbsp;modules,&nbsp;and&nbsp;implemented&nbsp;and&nbsp;built&nbsp;using&nbsp;C++17.
&nbsp;&nbsp;*&nbsp;&nbsp;&nbsp;Users&nbsp;can&nbsp;decide&nbsp;whether&nbsp;to&nbsp;enable&nbsp;these&nbsp;optional&nbsp;features&nbsp;based&nbsp;on&nbsp;their&nbsp;runtime&nbsp;environment&nbsp;and&nbsp;functional&nbsp;requirements,&nbsp;and&nbsp;accordingly&nbsp;choose&nbsp;the&nbsp;appropriate&nbsp;C++&nbsp;standard&nbsp;for&nbsp;compilation.
With&nbsp;this&nbsp;approach,&nbsp;we&nbsp;aim&nbsp;to&nbsp;preserve&nbsp;overall&nbsp;compatibility&nbsp;while&nbsp;still&nbsp;enabling&nbsp;the&nbsp;introduction&nbsp;of&nbsp;high-performance&nbsp;data&nbsp;processing&nbsp;features,&nbsp;and&nbsp;to&nbsp;better&nbsp;accommodate&nbsp;the&nbsp;differences&nbsp;between&nbsp;embedded&nbsp;and&nbsp;cloud&nbsp;environments&nbsp;in&nbsp;terms&nbsp;of&nbsp;toolchains&nbsp;and&nbsp;functional&nbsp;needs.

[1]&nbsp;https://github.com/apache/arrow/issues/32415

Best&nbsp;regards,
Colin.

Reply via email to