[
https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581381#comment-15581381
]
Chaitanya commented on APEXMALHAR-2284:
---------------------------------------
[~csingh] I am proposing the below approaches:
(1) Maintaining the below 2 maps in ManagedTimeStateMultiValue:
Map<Slice, Slice> keyValue; // Consists of key value. This map will be
flushed into the Managed state at the endWindow() once all the getAsync() are
done.
Map<Slice, Long> key2TimeBucket; // Key to timebucket Id. Map will be
cleared at checkpoint()
Psuedocode for insert and getAsync() methods:
bool insert(Slice key, Slice, value, long time)
{
long timebucket = get the time bucket from TimebucketAssigner
if (timebucket < key2TimeBucket.get(key)) {
return false;
}
Insert into keyValue and key2TimeBucket maps.
return true;
}
Future getAsync(Slice key)
{
a) If the key is present in keyValue map then return value.
b) If not then getAsync() on managedstate.
}
(2) Rewrite the getAsync() method based on join requirement in
ManagedTimeStateMultiValue as follows:
- If the key is present in bucket memory then return value else
asynchronously search into time buckets which are on disk.
I think the performance of 2nd approach is better than 1st approach. So, I am
preferring the second approach.
Please share your thoughts on above approaches.
> POJOInnerJoinOperatorTest fails in Travis CI
> --------------------------------------------
>
> Key: APEXMALHAR-2284
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2284
> Project: Apache Apex Malhar
> Issue Type: Bug
> Reporter: Thomas Weise
> Assignee: Chaitanya
> Priority: Blocker
> Fix For: 3.6.0
>
>
> https://s3.amazonaws.com/archive.travis-ci.org/jobs/166322754/log.txt
> {code}
> Failed tests:
> POJOInnerJoinOperatorTest.testEmitMultipleTuplesFromStream2:337 Number of
> tuple emitted expected:<2> but was:<4>
> POJOInnerJoinOperatorTest.testInnerJoinOperator:184 Number of tuple emitted
> expected:<1> but was:<2>
> POJOInnerJoinOperatorTest.testMultipleValues:236 Number of tuple emitted
> expected:<2> but was:<3>
> POJOInnerJoinOperatorTest.testUpdateStream1Values:292 Number of tuple
> emitted expected:<1> but was:<2>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
