[
https://issues.apache.org/jira/browse/APEXMALHAR-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15657851#comment-15657851
]
ASF GitHub Bot commented on APEXMALHAR-2335:
--------------------------------------------
GitHub user brightchen reopened a pull request:
https://github.com/apache/apex-malhar/pull/493
APEXMALHAR-2335 APEXMALHAR-2333 APEXMALHAR-2334 #resolve #comment Pro…
…blems on StateTracker
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/brightchen/apex-malhar APEXMALHAR-2335-PR
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-malhar/pull/493.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #493
----
commit b7ca8ea5fd5a17e3be70c3117aec057776eb995c
Author: brightchen <[email protected]>
Date: 2016-11-10T22:59:52Z
APEXMALHAR-2335 APEXMALHAR-2333 APEXMALHAR-2334 #resolve #comment Problems
on StateTracker
----
> Problems on StateTracker
> ------------------------
>
> Key: APEXMALHAR-2335
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2335
> Project: Apache Apex Malhar
> Issue Type: Bug
> Reporter: bright chen
> Assignee: bright chen
> Original Estimate: 144h
> Remaining Estimate: 144h
>
> The intension of StateTracker#bucketAccessed as I understand was to keep the
> last access time of each bucket so freeMemory can free the buckets which not
> accessed recently.
> But here are some problems
> - StateTracker#bucketAccessed() will be called each time access to the
> bucket. Current implementation use Concurrent map and set to handle
> it(bucketHeap and bucketAccessTimes). it could be pretty heavy.
> - StateTracker#run use bucketHeap to get the bucket which need to free
> memory, but at the same time bucketHeap was constantly changed by
> bucketAccessed(). It could cause concurrent issue event bucketHeap is a
> Current set. And even more the while loop could never end. Following two bug
> probably related to this:
> https://issues.apache.org/jira/browse/APEXMALHAR-2333
> https://issues.apache.org/jira/browse/APEXMALHAR-2334
> Following are my thought to solve these issue.
> - It's not necessary to keep very accurate of update time, it probably only
> need to be updated once for each stream window. But StateTracker don't
> support endWindow() (in fact endWindow() was triggered at end of application
> window). So, we can keep the bucket which updated in this period and update
> only at the end of the period.
> - keep two sets of bucketHeap, one for bucketAccessed() and one for run(). so
> the lock only need when switch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
