HeartSaVioR commented on PR #53930: URL: https://github.com/apache/spark/pull/53930#issuecomment-3788425325
TODO tickets: > yet to cover the case of regular join where there is no event time in both join condition and the value We have an idea to give a try (via replacing GET-and-PUT pattern of count in secondary index with blind MERGE), though it may require broader change due to the issue in below. https://issues.apache.org/jira/browse/SPARK-55131 After resolving the above, we can give a try with blind MERGE and check the performance improvement. > retrieving matched rows with the "scope" of timestamps (in time-interval join) https://issues.apache.org/jira/browse/SPARK-55147 > further optimizations with RocksDB offering: WriteBatch (for batched writes), MGET, etc. https://issues.apache.org/jira/browse/SPARK-55148 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
