Hi devs, I'd like to initiate a discussion on PIP-41: Introduce FilePath Global Index And Optimizations For Lookup In Append Table [1].
Currently, Paimon already supports global indexes, which can be passed during queries. Paimon performs rowid filtering based on the indexes, but does not yet support file filtering. In this PIP, we introduce the FilePath Global Index. Index data can be built using the key->filepath and file->meta system tables. Users can construct index filters from the query results based on the index data and pass them to Paimon tables, accelerating point query performance. Looking forward to hearing from you, thanks [1] https://cwiki.apache.org/confluence/display/PAIMON/PIP-41%3A+Introduce+FilePath+Global+Index+And+Optimizations+For+Lookup+In+Append+Table Best, Fang Yong
