ywcb00 commented on a change in pull request #1522:
URL: https://github.com/apache/systemds/pull/1522#discussion_r801644662
##########
File path:
src/main/java/org/apache/sysds/runtime/controlprogram/federated/FederatedWorkerHandler.java
##########
@@ -246,74 +246,43 @@ private FederatedResponse readData(String filename,
DataType dataType,
// early throwing of exception to avoid infinitely
waiting threads for data
throw new FederatedWorkerHandlerException("Could not
recognize datatype");
- CacheableData<?> cd = _frc.get(filename);
- if(cd == null) {
+ ExecutionContext ec = ecm.get(tid);
+ LineageItem linItem = new LineageItem(filename);
+
+ CacheableData<?> cd = null;
+
+ if(!LineageCache.reuseRead(Long.toString(id), dataType,
linItem, ec)) {
+ if(ReuseCacheType.isNone() || dataType !=
DataType.MATRIX)
+ // using the read cache if lineage reuse is
disabled or the data is not a matrix
+ cd = _frc.get(filename);
Review comment:
I don't think that we allow for a dynamic change of the reuse flags on
the federated worker yet. However, since I'm not 100% sure about that and we
might want to add the possibility for changing the reuse types during runtime
in the future, I will add the respective solution to the next PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
