Using a virtual file system to read and write files, or using the SQL computing
engine to treat it as a structured file list, seems like a very practical
solution and making it easy for users to access files in the object table.
Best regards,
Jiajia
------------------------------------------------------------------
发件人：Jingsong Li <jingsongl...@gmail.com>
发送时间：2025年7月7日(周一) 19:17
收件人：dev<dev@paimon.apache.org>
主 题：[DISCUSS] Make Object Table as a virtual table
Hi everyone,
After the object table was developed, almost no one used it. Object
Table is a real Paimon table, it requires a refresh service to refresh
the metadata of files, which is difficult to complete in operation.
In fact, we don't need to refresh it, and accessing the file system
directly during reading is not a bad thing.
This discussion discusses whether it is possible to transform an
object table into a virtual existence. Users can:
1. Use the virtual file system to read and write files.
2. Or use the SQL computing engine to read it as a structured file list.
On this basis, we can develop virtual file systems, such as through
similar approaches vfs://catalog_name/database_name/table_name/ Use
the path to access files in the specific object table.
By using this method, we can integrate file operations on object
tables with the permission system of data catalogs.
What do you think?
Best,
Jingsong
