phaniarnab commented on a change in pull request #1522:
URL: https://github.com/apache/systemds/pull/1522#discussion_r800668490
##########
File path:
src/main/java/org/apache/sysds/runtime/controlprogram/federated/FederatedWorkerHandler.java
##########
@@ -246,74 +246,43 @@ private FederatedResponse readData(String filename,
DataType dataType,
// early throwing of exception to avoid infinitely
waiting threads for data
throw new FederatedWorkerHandlerException("Could not
recognize datatype");
- CacheableData<?> cd = _frc.get(filename);
- if(cd == null) {
+ ExecutionContext ec = ecm.get(tid);
+ LineageItem linItem = new LineageItem(filename);
+
+ CacheableData<?> cd = null;
+
+ if(!LineageCache.reuseRead(Long.toString(id), dataType,
linItem, ec)) {
+ if(ReuseCacheType.isNone() || dataType !=
DataType.MATRIX)
+ // using the read cache if lineage reuse is
disabled or the data is not a matrix
+ cd = _frc.get(filename);
Review comment:
Correct me if I am wrong. It is possible that lineage-based reuse is
enabled after the worker is initiated. In that case, the read cache may not be
empty. Hence, we may want to lookup both the caches. However, the condition
above `_frc.get(filename)` restricts readcache lookup if lineage reuse is
enabled.
I tried to remove the condition but that led to a test failure probably due
to an uninitialized readcache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
