naivedogger opened a new issue, #3670: URL: https://github.com/apache/fluss/issues/3670
### Search before asking - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar. ### Motivation Fluss's Flink lookup join doesn't custom-shuffle the probe stream, so probe rows are distributed to lookup subtasks arbitrarily. This hurts lookup.cache (partial) hit rate and enlarges RPC fan-out, since each subtask may hit any bucket / any TabletServer. Flink 2.0 added SupportsLookupCustomShuffle ([FLIP-462](https://cwiki.apache.org/confluence/spaces/FLINK/pages/309496789/FLIP-462+Support+Custom+Data+Distribution+for+Input+Stream+of+Lookup+Join)), which lets a source define the probe-stream distribution. We can shuffle by bucket key so each subtask handles a stable set of buckets → better cache locality and lower RPC fan-out. ### Solution - Implement SupportsLookupCustomShuffle and provide an InputDataPartitioner that maps the lookup key to a Fluss bucket using the existing bucketing logic (consistent with the write path / PrimaryKeyLookuper). - Only enable when the lookup key covers the bucket key (PK lookup / prefix lookup on bucket key); otherwise fall back to default distribution. - Note: the interface is Flink 2.0+, so this targets fluss-flink-2.2 only (FlinkTableSource lives in fluss-flink-common which must stay Flink 1.18 compatible). ### Anything else? _No response_ ### Willingness to contribute - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
