wayneli-vt commented on PR #6827: URL: https://github.com/apache/paimon/pull/6827#issuecomment-3669649637
Thank you for the reply! @JingsongLi These optimizations are mainly for `computed-column backfill` scenarios. For example, users may have already ingested multi-modal data into a Paimon table and they want to use Spark to process the data, derive some features, and write them back to the same table. In this case, it should be possible to do `Scan -> Calculate -> Write` directly, since the scan output already satisfies the required layout, so no additional shuffle/sort/join is needed. This PR is focused on optimizing this scenario. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
