janniklinde opened a new pull request, #2368: URL: https://github.com/apache/systemds/pull/2368
This patch introduces various bugfixes and improvements to stream handling to address synchronization issues leading to deadlocks in specific cases (we now reliably guarantee that the finalizer callback is invoked after the last consumer finished execution). Further, the `Tee` instruction was modified to be introduced by statement level rewrites to keep track of transient reads / writes which led to issues with the previous DAG rewrite. Finally, we introduced basic block tracking to remove items from the `OOCEvictionManager` once they have no pending consumers. To achieve that, we track stream references and once no variable holds the underlying `CachingStream` we mark it as deletable (because then we know, that no additional stream consumer will be added). Blocks from deletable streams can safely be removed from the cache once their consumption count reaches the final number of registered consumers. Note that we modified the `VariableCPInstruction` to access variable remove and copy events. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
