luoyuxia commented on issue #311: URL: https://github.com/apache/fluss-rust/issues/311#issuecomment-3900298265
@zuston I'm really glad to see this discussion! Regarding the scanner logic, I have a slightly different perspective. The log scanner essentially performs a straightforward read of the log. However, for the primary key table, the process is more complex as it requires merging the changelog to output deduplicated data. On a related note, the community is currently considering providing a KV scan interface. The idea is to have the server return pre-deduplicated data directly, rather than handling the merge and deduplication on the client side. Given this context, perhaps we could consider supporting the log table first. The KV table is indeed considerably more complex to handle at this stage. Furthermore, once your fluss-datafusion development is ready, we would be very happy to welcome contributions to the fluss-rust repository, similar to how iceberg-rust operates. We're looking forward to it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
