[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581381#comment-15581381
 ] 

Chaitanya commented on APEXMALHAR-2284:
---------------------------------------

[~csingh] I am proposing the below approaches:

(1) Maintaining the below 2 maps in ManagedTimeStateMultiValue:
      Map<Slice, Slice> keyValue; // Consists of key value. This map will be 
flushed into the Managed state at the endWindow() once all the getAsync() are 
done.
      Map<Slice, Long> key2TimeBucket; // Key to timebucket Id. Map will be 
cleared at checkpoint()
      Psuedocode for insert and getAsync() methods:
      bool insert(Slice key, Slice, value, long time)
     {
          long timebucket = get the time bucket from TimebucketAssigner
          if (timebucket < key2TimeBucket.get(key)) {
              return false;
          }
          Insert into keyValue and key2TimeBucket maps.
         return true;
     }
  
     Future getAsync(Slice key)
    {
       a) If the key is present in keyValue map then return value. 
       b) If not then getAsync() on managedstate.
    }

(2) Rewrite the getAsync() method based on join requirement in 
ManagedTimeStateMultiValue as follows:
       - If the key is present in bucket memory then return value else 
asynchronously search into time buckets which are on disk. 

  I think the performance of 2nd approach is better than 1st approach. So, I am 
preferring the second approach.
  Please share your thoughts on above approaches.
  


> POJOInnerJoinOperatorTest fails in Travis CI
> --------------------------------------------
>
>                 Key: APEXMALHAR-2284
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2284
>             Project: Apache Apex Malhar
>          Issue Type: Bug
>            Reporter: Thomas Weise
>            Assignee: Chaitanya
>            Priority: Blocker
>             Fix For: 3.6.0
>
>
> https://s3.amazonaws.com/archive.travis-ci.org/jobs/166322754/log.txt
> {code}
> Failed tests: 
>   POJOInnerJoinOperatorTest.testEmitMultipleTuplesFromStream2:337 Number of 
> tuple emitted  expected:<2> but was:<4>
>   POJOInnerJoinOperatorTest.testInnerJoinOperator:184 Number of tuple emitted 
>  expected:<1> but was:<2>
>   POJOInnerJoinOperatorTest.testMultipleValues:236 Number of tuple emitted  
> expected:<2> but was:<3>
>   POJOInnerJoinOperatorTest.testUpdateStream1Values:292 Number of tuple 
> emitted  expected:<1> but was:<2>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to