ywcb00 opened a new pull request #1574: URL: https://github.com/apache/systemds/pull/1574
Hi, This PR addresses the problem of differentiating between different slices from the same broadcast data object. At the moment we can only identify broadcast slices by its original data object, not by its slice indices. This leads to reuse collisions when broadcasting two different slices from the same original file (i.e., different slice indices). However, this PR resolves the problem by appending the slice indices in form of a RightIndex lineage item to the transferred lineage trace. Then, we can identify the slice range on the federatet worker, and hence differentiate between the different data slices. Thanks for review :) PS: Note that github shows the added test _FederatedReuseSlicesTest_ as if the _FederatedReadCacheTest_ would have been renamed. That is actually not the case. The _FederatedReadCacheTest_ was actually replaced by the _FederatedReuseReadTest_ when we added the lineage based reuse of read objects. However, somehow this test managed to get again into the system by a mistaken rebase, I am sorry for that. :bowtie: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
