phaniarnab opened a new pull request #932:
URL: https://github.com/apache/systemml/pull/932
This patch contains handling of multiple cache entries pointing to the
same data (due to multilevel caching).
1) All the entries with the same values are connected with a linkedlist.
Even though
they output same data, they have different computation time.
2) Eviction logic marks an entry for deferred spilling/removal if other
entries are linked to that. If all the entries in a list are marked for
spilling or removal, only then we evict the item.
3) Disk write and read happen only once for all the items connected to a
single matrix. This way single read and write restores multiple entries
to cache and clears more space respectively.
4) Initial experiments show huge improvements in cache management. Now
the cache can store many more entries (this patch fixes duplicate size
calculations), need reduced number of disk I/O. These changes overall
improve
cache hit count.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
