phaniarnab opened a new pull request #1140:
URL: https://github.com/apache/systemds/pull/1140
This PR enables lineage-based reuse in federated workers.
- Trace PUT and READ federated requests
- Start worker threads with lineage tracing and reuse
TODO: find a robust way to trace PUT requests. Serializing the lineage DAG
is an option but that can bloat the message, and also adds serialization and
deserialization cost. For now, I added the hash of the corresponding lineage
DAGs in the PUT requests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
