[jira] [Commented] (APEXCORE-634) Apex Platform unable to set unifier attributes for modules in DAG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880106#comment-15880106
 ] 

ASF GitHub Bot commented on APEXCORE-634:
-

GitHub user deepak-narkhede opened a pull request:

https://github.com/apache/apex-core/pull/478

APEXCORE-634 Changes to unifier attribute test case for module.

Incorporated comments from Vlad on 
(https://github.com/apache/apex-core/pull/466)
 [APEXCORE-634] Apex Platform unable to set unifier attributes for modules

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/deepak-narkhede/apex-core APEXCORE-634

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/478.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #478


commit 976d4452660ae6c0b44d19885f375fc0c7fdd126
Author: deepak-narkhede 
Date:   2017-02-23T08:08:42Z

APEXCORE-634 Changes to unifier attribute test case for module.




> Apex Platform unable to set unifier attributes for modules in DAG
> -
>
> Key: APEXCORE-634
> URL: https://issues.apache.org/jira/browse/APEXCORE-634
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Deepak Narkhede
>Assignee: Deepak Narkhede
> Fix For: 3.6.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (APEXCORE-634) Apex Platform unable to set unifier attributes for modules in DAG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878031#comment-15878031
 ] 

ASF GitHub Bot commented on APEXCORE-634:
-

Github user asfgit closed the pull request at:

https://github.com/apache/apex-core/pull/466


> Apex Platform unable to set unifier attributes for modules in DAG
> -
>
> Key: APEXCORE-634
> URL: https://issues.apache.org/jira/browse/APEXCORE-634
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Deepak Narkhede
>Assignee: Deepak Narkhede
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (APEXCORE-634) Apex Platform unable to set unifier attributes for modules in DAG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868202#comment-15868202
 ] 

ASF GitHub Bot commented on APEXCORE-634:
-

GitHub user deepak-narkhede reopened a pull request:

https://github.com/apache/apex-core/pull/466

APEXCORE-634 Apex Platform unable to set unifier attributes for modules

Problem: Unable to set Unifier attributes of output port within modules
Description: When modules are flatten in logical Plan of DAG, only top 
level attributes are cloned of OperatorMeta. Unifier attributes are not copied 
in PortMapping for output ports.
Solution:Clone the unifier attributes while flattening DAG in logical 
Plan.

Testing done for custom application with and without modules also with app 
template HDFS to S3 module. Have had debug logs and stack traces to verify the 
fix.

One of the snip of logs had set the unifier attribute TIMEOUT_WINDOW_COUNT 
to 10 ( i.e 1000 millisec)

**Without fix:**

2017-02-06 15:51:31,539 WARN 
com.datatorrent.stram.StreamingContainerManager: UNIFIER operator 
PTOperator[id=4,name=**genmodule$gen.out#unifier]** committed window 
, recovery window , current time 1486376491539, 
last window id change time 0, **window processing timeout millis 6**


**With Fix:**

2017-02-06 14:22:49,602 WARN 
com.datatorrent.stram.StreamingContainerManager: UNIFIER operator 
PTOperator[id=4,name=**genmodule$gen.out#unifier]** committed window 
, recovery window , current time 1486371169602, 
last window id change time 0, **window processing timeout millis 1000**



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/deepak-narkhede/apex-core APEXCORE-634

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/466.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #466


commit 12ce7c5d8082289c4690d70b09e65033090c44e1
Author: deepak-narkhede 
Date:   2017-02-07T11:09:07Z

APEXCORE-634 Apex Platform unable to set unifier attributes for modules in 
DAG.

Problem: Unable to set Unifier attributes of output port within modules
Description: When modules are flatten in logical Plan of DAG, only top 
level attributes are cloned of OperatorMeta.
 Unifier attributes are not copied in PortMapping for output ports.
Solution:Clone the unifier attributes while flattening DAG in logical 
Plan.

commit 5df34a81670520cedc74b1ac3f14d1a2d02ca203
Author: deepak-narkhede 
Date:   2017-02-15T16:09:23Z

APEXCORE-634 Added unit test for testing unifier attribute for module.




> Apex Platform unable to set unifier attributes for modules in DAG
> -
>
> Key: APEXCORE-634
> URL: https://issues.apache.org/jira/browse/APEXCORE-634
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Deepak Narkhede
>Assignee: Deepak Narkhede
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (APEXCORE-634) Apex Platform unable to set unifier attributes for modules in DAG

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868166#comment-15868166
 ] 

ASF GitHub Bot commented on APEXCORE-634:
-

Github user deepak-narkhede closed the pull request at:

https://github.com/apache/apex-core/pull/466


> Apex Platform unable to set unifier attributes for modules in DAG
> -
>
> Key: APEXCORE-634
> URL: https://issues.apache.org/jira/browse/APEXCORE-634
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Deepak Narkhede
>Assignee: Deepak Narkhede
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (APEXCORE-634) Apex Platform unable to set unifier attributes for modules in DAG

[Deepak Narkhede (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855811#comment-15855811
 ] 

Deepak Narkhede commented on APEXCORE-634:
--

Hi Ram,

After further investigation the problem was with unifier attributes of 
operators within the modules when the DAG is flatten.

 Problem: Unable to set Unifier attributes of output port within modules
Description: When modules are flatten in logical Plan of DAG, only top level 
attributes are cloned of OperatorMeta. Unifier attributes are not copied in 
PortMapping for output ports.
Solution: Clone the unifier attributes while flattening DAG in logical Plan.

Thanks,
Deepak

> Apex Platform unable to set unifier attributes for modules in DAG
> -
>
> Key: APEXCORE-634
> URL: https://issues.apache.org/jira/browse/APEXCORE-634
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Deepak Narkhede
>Assignee: Deepak Narkhede
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)