xwmr-max opened a new issue, #1812: URL: https://github.com/apache/incubator-paimon/issues/1812
### Search before asking - [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar. ### Motivation Currently in paimon, flink look up join is leveraging rocksdb's backend storage capabilities. This is a conversion from the original memory to back-end storage, and the timeliness is not inferior. However, each taskmansger, that is, each process, loads one piece of dimension table data, and if the dimension table is at the level of billions and the task requires dozens or more Taskmanagers to process the data, then dozens or more taskmanager dimension table data will need to be loaded repeatedly. With the ordered storage capabilities of the current LSM, can this problem of repeated loading be solved while also meeting the timeliness requirements in general? ### Solution _No response_ ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
