[
https://issues.apache.org/jira/browse/APEXMALHAR-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15657850#comment-15657850
]
ASF GitHub Bot commented on APEXMALHAR-2335:
--------------------------------------------
Github user brightchen closed the pull request at:
https://github.com/apache/apex-malhar/pull/493
> Problems on StateTracker
> ------------------------
>
> Key: APEXMALHAR-2335
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2335
> Project: Apache Apex Malhar
> Issue Type: Bug
> Reporter: bright chen
> Assignee: bright chen
> Original Estimate: 144h
> Remaining Estimate: 144h
>
> The intension of StateTracker#bucketAccessed as I understand was to keep the
> last access time of each bucket so freeMemory can free the buckets which not
> accessed recently.
> But here are some problems
> - StateTracker#bucketAccessed() will be called each time access to the
> bucket. Current implementation use Concurrent map and set to handle
> it(bucketHeap and bucketAccessTimes). it could be pretty heavy.
> - StateTracker#run use bucketHeap to get the bucket which need to free
> memory, but at the same time bucketHeap was constantly changed by
> bucketAccessed(). It could cause concurrent issue event bucketHeap is a
> Current set. And even more the while loop could never end. Following two bug
> probably related to this:
> https://issues.apache.org/jira/browse/APEXMALHAR-2333
> https://issues.apache.org/jira/browse/APEXMALHAR-2334
> Following are my thought to solve these issue.
> - It's not necessary to keep very accurate of update time, it probably only
> need to be updated once for each stream window. But StateTracker don't
> support endWindow() (in fact endWindow() was triggered at end of application
> window). So, we can keep the bucket which updated in this period and update
> only at the end of the period.
> - keep two sets of bucketHeap, one for bucketAccessed() and one for run(). so
> the lock only need when switch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
