lszskye commented on PR #8478: URL: https://github.com/apache/paimon/pull/8478#issuecomment-4899894555
> The optimizer rule is not registered in `PaimonSparkSessionExtensions` yet (there is only a TODO), so `pushedMapSelectedKeys` is never populated and the advertised Spark pushdown is not actually enabled. Could you either register the rule together with the reader support, or adjust the PR/title to make this explicitly preparatory? Also, when the rule is enabled, unsupported cases such as nested shared-shredding maps or literal keys containing the metadata delimiter should fall back without pushdown instead of failing a valid query during planning. Sure, thanks for pointing this out. Since Xinyu’s changes for the read/write framework have not been merged yet, I don’t plan to enable the actual reader path in this PR. I’ll follow your suggestion and update the PR title/description to make it clear that this is preparatory work rather than an enabled end-to-end pushdown feature. Also unsupported cases such as nested shared-shredding maps is fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
