Hi Jingsong, I have created a google doc https://docs.google.com/document/d/1qV7lUW5GsZ72IuF9FV96fX3wtJO_gzEWY6aLG97caxU/edit?usp=sharing from cwiki, please help to review it
Best, FangYong On Tue, Jan 13, 2026 at 6:58 PM Jingsong Li <[email protected]> wrote: > Thanks Yong for driving this! > > Cwiki not good in my network... Can you send the google doc too? > > Best, > Jingsong > > On Tue, Jan 13, 2026 at 6:26 PM Aitozi <[email protected]> wrote: > > > > Hi Yong: > > Thanks for bringing this discussion. File level local sort + Global > > file path index looks attractive in the senior of point query. > > I think it's common ability to paimon community, Nice to see this > feature. > > > > Best, > > Aitozi > > > > > > > > > > Yong Fang <[email protected]> 于2026年1月13日周二 17:33写道： > > > > > Hi devs, > > > > > > I'd like to initiate a discussion on PIP-41: Introduce FilePath Global > > > Index And Optimizations For Lookup In Append Table [1]. > > > > > > We use Paimon as cold storage for sample data of businesses such as > search, > > > recommendation, and advertising. Given the extremely large volume of > sample > > > data, we adopt Append Tables for data storage. > > > > > > Batch jobs read and process this sample data, while lookup by key > > > capability for historical data is also required during the sample data > > > processing. To support hybrid queries for such ultra-high-dimensional > data > > > in Paimon, we introduce the FilePath Global Index, along with > optimizations > > > for reading Parquet metadata and Parquet file data in Append Tables > (Some > > > of the major optimization designs come from our partner teams cc > @lingpeng, > > > @guanziyue, thx), aiming to enhance the lookup capability of Paimon > Append > > > Tables. > > > > > > Looking forward to hearing from you, thanks > > > > > > > > > [1] > > > > > > > https://cwiki.apache.org/confluence/display/PAIMON/PIP-41%3A+Introduce+FilePath+Global+Index+And+Optimizations+For+Lookup+In+Append+Table > > > > > > Best, > > > Fang Yong > > > >
