Hi Xiaoyu, After looking at TimescaleDB, I think we can start with an extension. CustomScan and custom tuple slots give us enough to experiment. We can pass batches between nodes, do lazy projections, and avoid converting values to Datums too early.
The idea is something like DataFusion, but with PostgreSQL types. It would have shared interfaces, reusable code, and a few reference batch nodes. Other extensions could add their own nodes. I hacked together a rough prototype with AI [1]. The numbers looked good. Heap scans were 1.4-2.5x faster, hash joins in memory 2.5-4x, and grouped aggregates 1.7-2.15x. These were warm-cache tests. Specialized algorithms account for some of the gain too. One thing does seem worth adding to core first. Tuple deformation needs a more convenient incremental interface. We should be able to extract Datums in steps without starting over. The prototype used an extra helper for that. Both paths used the same patched build. The cleaner rewrite is in Tessera [2], still WIP. The plan is to get it working as a library and test it as an extension. Then we can discuss what might be worth bringing into core. By then, we should know more about both the gains and the problems. So I don't have an answer on the column-request API yet. For now, I'd like to get some experience using these pieces together. [1] https://github.com/darthunix/pg_batch/blob/main/benchmark/results.md [2] https://github.com/darthunix/tessera Best regards, Denis Smirnov
