Cribbee opened a new pull request, #10325: URL: https://github.com/apache/paimon/pull/10325
### Purpose Query Service currently loads lookup files on demand. Add an opt-in `query-service.cache=FULL` backend for workloads that prefer preloaded, shared dimension state. `PARTIAL` remains the default, and clients keep `lookup.cache=AUTO` and the existing service discovery and RPC protocol. The new backend reuses the local full-cache implementation and reads only the partition/bucket pairs assigned to each executor by `ChannelComputer.select`. All executors finish bootstrap before the service is registered. File-monitor notifications drive incremental refresh, including compaction-diff reads; overwrites and expired scan snapshots rebuild the cache. Partition columns are restored in primary-key order from the wire request. Refresh failures stop the executor from serving partially updated state, and shutdown releases the cache and its temporary directory. The SQL lookup guide and generated option reference document configuration and tradeoffs. This remains a fixed-bucket primary-key service. Executors plan snapshot metadata independently, refresh can block requests within an executor, and there is no cross-executor snapshot-consistency guarantee. No throughput or latency improvement is claimed without a workload benchmark. ### Tests - Flink 1.20.1, JDK 17: `FullCacheTableQueryTest`, `LookupTableTest`, `FileStoreLookupFunctionTest`, and `RemoteLookupJoinITCase`: 79 passed, 1 pre-existing disabled test. Checkstyle, Spotless, and Maven enforcer checks enabled. - New coverage includes sharded bootstrap, partitioned keys with different field types, inserts/updates/deletes, projection, empty tables, overwrite/restart recovery, expired snapshots, compaction-diff refresh, refresh failures, concurrent requests, and an actual Flink SQL lookup join against the FULL service. - Flink 2.2.0, JDK 17: clean recompilation and the same four test classes: 79 passed, 1 pre-existing disabled test, with Checkstyle, Spotless, and Maven enforcer checks enabled. - The Flink configuration reference was regenerated with `ConfigOptionsDocGenerator`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
