Chaitanya commented on APEXMALHAR-2284:

[~csingh] I am proposing the below approaches:

(1) Maintaining the below 2 maps in ManagedTimeStateMultiValue:
      Map<Slice, Slice> keyValue; // Consists of key value. This map will be 
flushed into the Managed state at the endWindow() once all the getAsync() are 
      Map<Slice, Long> key2TimeBucket; // Key to timebucket Id. Map will be 
cleared at checkpoint()
      Psuedocode for insert and getAsync() methods:
      bool insert(Slice key, Slice, value, long time)
          long timebucket = get the time bucket from TimebucketAssigner
          if (timebucket < key2TimeBucket.get(key)) {
              return false;
          Insert into keyValue and key2TimeBucket maps.
         return true;
     Future getAsync(Slice key)
       a) If the key is present in keyValue map then return value. 
       b) If not then getAsync() on managedstate.

(2) Rewrite the getAsync() method based on join requirement in 
ManagedTimeStateMultiValue as follows:
       - If the key is present in bucket memory then return value else 
asynchronously search into time buckets which are on disk. 

  I think the performance of 2nd approach is better than 1st approach. So, I am 
preferring the second approach.
  Please share your thoughts on above approaches.

> POJOInnerJoinOperatorTest fails in Travis CI
> --------------------------------------------
>                 Key: APEXMALHAR-2284
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2284
>             Project: Apache Apex Malhar
>          Issue Type: Bug
>            Reporter: Thomas Weise
>            Assignee: Chaitanya
>            Priority: Blocker
>             Fix For: 3.6.0
> https://s3.amazonaws.com/archive.travis-ci.org/jobs/166322754/log.txt
> {code}
> Failed tests: 
>   POJOInnerJoinOperatorTest.testEmitMultipleTuplesFromStream2:337 Number of 
> tuple emitted  expected:<2> but was:<4>
>   POJOInnerJoinOperatorTest.testInnerJoinOperator:184 Number of tuple emitted 
>  expected:<1> but was:<2>
>   POJOInnerJoinOperatorTest.testMultipleValues:236 Number of tuple emitted  
> expected:<2> but was:<3>
>   POJOInnerJoinOperatorTest.testUpdateStream1Values:292 Number of tuple 
> emitted  expected:<1> but was:<2>
> {code}

This message was sent by Atlassian JIRA

Reply via email to