ywcb00 opened a new pull request #1544: URL: https://github.com/apache/systemds/pull/1544
Hi, This PR introduces the mechanics for tranferring the lineage trace of a data object to the federated worker. For now, we are including only the lineage trace of matrices which come from the data gen operation (e.g. _rand()_), as they have the respective lineage item set in their _CacheableData_ objects. In order to include the lineage trace for all matrices, we need to pass it through the _broadcast_ and _broadcastSliced_ methods to the creation of the actual PUT_VAR FederatedRequest. I decided to introduce the mechanics at first, and change the _broadcast_ methods with the next PR. This will allow for a better discussion about where to include the lineage trace, since then we can decide per instruction whether to put the lineage trace to the federated site or not. However, I first tried to set the _LineageItem_ into the _CacheableData_ object at every instruction, but this leads to an increase in the size of the _CacheableData_ object. Therefore, I just removed it again and changed the tests such that they have only one _LineageItem_ (from _DataGen_) in the lineage trace of the _PUT_ object for now. Thanks for review :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
