[GitHub] incubator-gearpump issue #90: fix GEARPUMP-144 make the user for HDFS and Ya...

2016-12-13 Thread karol-brejna-i 
Github user karol-brejna-i commented on the issue:

https://github.com/apache/incubator-gearpump/pull/90
  
+1
works fine. 
also, in our new environments logs started to work properly (probably this 
and previous change helped). i closed the other ticket, too.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (GEARPUMP-144) Modify Gearpump's yarnclient so it can use provided user for HDFS and YARN

2016-12-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/GEARPUMP-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747527#comment-15747527
 ] 

ASF GitHub Bot commented on GEARPUMP-144:
-

Github user karol-brejna-i commented on the issue:

https://github.com/apache/incubator-gearpump/pull/90
  
+1
works fine. 
also, in our new environments logs started to work properly (probably this 
and previous change helped). i closed the other ticket, too.


> Modify Gearpump's yarnclient so it can use provided user for HDFS and YARN
> --
>
> Key: GEARPUMP-144
> URL: https://issues.apache.org/jira/browse/GEARPUMP-144
> Project: Apache Gearpump
>  Issue Type: Improvement
>  Components: yarn
>Reporter: Karol Brejna
>Assignee: Karol Brejna
> Fix For: 0.8.3
>
>
> We have dedicated users for operations like deploying applications to yarn, 
> accessing hdfs etc. 
> Currently, there is no way to provide yarnclient (Gearpump component for 
> spawning Gearpump to yarn) the user to be used in the operation.
> Gearpump code needs to be modified to overcome this shortcoming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-gearpump issue #123: [GEARPUMP-249] Fix function chains to enforce...

2016-12-13 Thread manuzhang 
Github user manuzhang commented on the issue:

https://github.com/apache/incubator-gearpump/pull/123
  
@huafengw please review


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump pull request #122: [GEARPUMP-245] Apply groupBy and fix g...

2016-12-13 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/incubator-gearpump/pull/122


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump pull request #90: fix GEARPUMP-144 make the user for HDFS...

2016-12-13 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/incubator-gearpump/pull/90


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump issue #122: [GEARPUMP-245] Apply groupBy and fix group im...

2016-12-13 Thread huafengw 
Github user huafengw commented on the issue:

https://github.com/apache/incubator-gearpump/pull/122
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (GEARPUMP-249) reduce is not executed when following flatMap in Stream DSL

2016-12-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/GEARPUMP-249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747278#comment-15747278
 ] 

ASF GitHub Bot commented on GEARPUMP-249:
-

Github user codecov-io commented on the issue:

https://github.com/apache/incubator-gearpump/pull/123
  
## [Current 
coverage](https://codecov.io/gh/apache/incubator-gearpump/pull/123?src=pr) is 
70.72% (diff: 100%)
> Merging 
[#123](https://codecov.io/gh/apache/incubator-gearpump/pull/123?src=pr) into 
[master](https://codecov.io/gh/apache/incubator-gearpump/branch/master?src=pr) 
will increase coverage by **0.40%**


```diff
@@ master   #123   diff @@
==
  Files   186186  
  Lines  5984   5985 +1   
  Methods5472   5475 +3   
  Messages  0  0  
  Branches510508 -2   
==
+ Hits   4208   4233+25   
+ Misses 1776   1752-24   
  Partials  0  0  
```


![Sunburst](https://codecov.io/gh/apache/incubator-gearpump/pull/123/graphs/sunburst.svg?src=pr=150)

> Powered by [Codecov](https://codecov.io?src=pr). Last update 
[914eb90...29c730e](https://codecov.io/gh/apache/incubator-gearpump/compare/914eb9008882835fbafd126085f3ebe73bd87c9f...29c730e6df064894798c763a45f45ffbd97c7f8a?src=pr)


> reduce is not executed when following flatMap in Stream DSL
> ---
>
> Key: GEARPUMP-249
> URL: https://issues.apache.org/jira/browse/GEARPUMP-249
> Project: Apache Gearpump
>  Issue Type: Bug
>  Components: streaming
>Affects Versions: 0.8.2
>Reporter: Manu Zhang
>Assignee: Manu Zhang
>
> The issue can be reproduced if a redundant {{map}} is added between 
> {{groupBy}} and {{sum}}
> {code}
> app.source(data.lines.toList, 1, "source").
>   // word => (word, count)
>   flatMap(line => line.split("[\\s]+")).map((_, 1)).
>   // (word, count1), (word, count2) => (word, count1 + count2)
>   groupByKey().sum.log
> {code}
> The problem is with {{AndThen#process}} method, where {{flatMap}} is lazily/ 
> never executed.
> {code}
>   override def process(value: IN): TraversableOnce[OUT] = {
> first.process(value).flatMap(second.process)
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (GEARPUMP-245) groupBy is not applied in JavaStream DSL

2016-12-13 Thread Manu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/GEARPUMP-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manu Zhang resolved GEARPUMP-245.
-
   Resolution: Fixed
Fix Version/s: 0.8.3

Issue resolved by pull request 122
[https://github.com/apache/incubator-gearpump/pull/122]

> groupBy is not applied in JavaStream DSL
> 
>
> Key: GEARPUMP-245
> URL: https://issues.apache.org/jira/browse/GEARPUMP-245
> Project: Apache Gearpump
>  Issue Type: Bug
>  Components: streaming
>Affects Versions: 0.8.2
>Reporter: Manu Zhang
>Assignee: Manu Zhang
> Fix For: 0.8.3
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (GEARPUMP-245) groupBy is not applied in JavaStream DSL

2016-12-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/GEARPUMP-245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747256#comment-15747256
 ] 

ASF GitHub Bot commented on GEARPUMP-245:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-gearpump/pull/122


> groupBy is not applied in JavaStream DSL
> 
>
> Key: GEARPUMP-245
> URL: https://issues.apache.org/jira/browse/GEARPUMP-245
> Project: Apache Gearpump
>  Issue Type: Bug
>  Components: streaming
>Affects Versions: 0.8.2
>Reporter: Manu Zhang
>Assignee: Manu Zhang
> Fix For: 0.8.3
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (GEARPUMP-144) Modify Gearpump's yarnclient so it can use provided user for HDFS and YARN

2016-12-13 Thread Manu Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/GEARPUMP-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manu Zhang resolved GEARPUMP-144.
-
Resolution: Fixed

Issue resolved by pull request 90
[https://github.com/apache/incubator-gearpump/pull/90]

> Modify Gearpump's yarnclient so it can use provided user for HDFS and YARN
> --
>
> Key: GEARPUMP-144
> URL: https://issues.apache.org/jira/browse/GEARPUMP-144
> Project: Apache Gearpump
>  Issue Type: Improvement
>  Components: yarn
>Reporter: Karol Brejna
>Assignee: Karol Brejna
> Fix For: 0.8.3
>
>
> We have dedicated users for operations like deploying applications to yarn, 
> accessing hdfs etc. 
> Currently, there is no way to provide yarnclient (Gearpump component for 
> spawning Gearpump to yarn) the user to be used in the operation.
> Gearpump code needs to be modified to overcome this shortcoming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (GEARPUMP-144) Modify Gearpump's yarnclient so it can use provided user for HDFS and YARN

2016-12-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/GEARPUMP-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747250#comment-15747250
 ] 

ASF GitHub Bot commented on GEARPUMP-144:
-

Github user manuzhang commented on the issue:

https://github.com/apache/incubator-gearpump/pull/90
  
+1 


> Modify Gearpump's yarnclient so it can use provided user for HDFS and YARN
> --
>
> Key: GEARPUMP-144
> URL: https://issues.apache.org/jira/browse/GEARPUMP-144
> Project: Apache Gearpump
>  Issue Type: Improvement
>  Components: yarn
>Reporter: Karol Brejna
>Assignee: Karol Brejna
> Fix For: 0.8.3
>
>
> We have dedicated users for operations like deploying applications to yarn, 
> accessing hdfs etc. 
> Currently, there is no way to provide yarnclient (Gearpump component for 
> spawning Gearpump to yarn) the user to be used in the operation.
> Gearpump code needs to be modified to overcome this shortcoming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (GEARPUMP-245) groupBy is not applied in JavaStream DSL

2016-12-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/GEARPUMP-245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747233#comment-15747233
 ] 

ASF GitHub Bot commented on GEARPUMP-245:
-

Github user huafengw commented on the issue:

https://github.com/apache/incubator-gearpump/pull/122
  
+1


> groupBy is not applied in JavaStream DSL
> 
>
> Key: GEARPUMP-245
> URL: https://issues.apache.org/jira/browse/GEARPUMP-245
> Project: Apache Gearpump
>  Issue Type: Bug
>  Components: streaming
>Affects Versions: 0.8.2
>Reporter: Manu Zhang
>Assignee: Manu Zhang
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-gearpump issue #122: [GEARPUMP-245] Apply groupBy and fix group im...

2016-12-13 Thread codecov-io 
Github user codecov-io commented on the issue:

https://github.com/apache/incubator-gearpump/pull/122
  
## [Current 
coverage](https://codecov.io/gh/apache/incubator-gearpump/pull/122?src=pr) is 
70.40% (diff: 89.47%)
> Merging 
[#122](https://codecov.io/gh/apache/incubator-gearpump/pull/122?src=pr) into 
[master](https://codecov.io/gh/apache/incubator-gearpump/branch/master?src=pr) 
will increase coverage by **0.08%**


```diff
@@ master   #122   diff @@
==
  Files   186186  
  Lines  5984   5984  
  Methods5472   5471 -1   
  Messages  0  0  
  Branches510511 +1   
==
+ Hits   4208   4213 +5   
+ Misses 1776   1771 -5   
  Partials  0  0  
```


![Sunburst](https://codecov.io/gh/apache/incubator-gearpump/pull/122/graphs/sunburst.svg?src=pr=150)

> Powered by [Codecov](https://codecov.io?src=pr). Last update 
[914eb90...3031f58](https://codecov.io/gh/apache/incubator-gearpump/compare/914eb9008882835fbafd126085f3ebe73bd87c9f...3031f58b58c3781f60308bd47e17669549baca5c?src=pr)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-gearpump pull request #122: [GEARPUMP-245] Apply groupBy and fix g...

2016-12-13 Thread manuzhang 
GitHub user manuzhang opened a pull request:

https://github.com/apache/incubator-gearpump/pull/122

[GEARPUMP-245] Apply groupBy and fix group implementations

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the commit message is formatted like:
   `[GEARPUMP-] Meaningful description of pull request` 
 - [ ] Make sure tests pass via `sbt clean test`.
 - [ ] Make sure old documentation affected by the pull request has been 
updated and new documentation added for new functionality. 




You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/incubator-gearpump GEARPUMP-245

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-gearpump/pull/122.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #122


commit 3031f58b58c3781f60308bd47e17669549baca5c
Author: manuzhang 
Date:   2016-12-14T03:39:02Z

[GEARPUMP-245] Apply groupBy and fix group implementations




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (GEARPUMP-251) Add Java friendly API for WindowFn and Trigger

2016-12-13 Thread Manu Zhang (JIRA) 
Manu Zhang created GEARPUMP-251:
---

 Summary: Add Java friendly API for WindowFn and Trigger
 Key: GEARPUMP-251
 URL: https://issues.apache.org/jira/browse/GEARPUMP-251
 Project: Apache Gearpump
  Issue Type: Improvement
Reporter: Manu Zhang






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (GEARPUMP-245) groupBy is not applied in JavaStream DSL

2016-12-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/GEARPUMP-245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747157#comment-15747157
 ] 

ASF GitHub Bot commented on GEARPUMP-245:
-

Github user codecov-io commented on the issue:

https://github.com/apache/incubator-gearpump/pull/122
  
## [Current 
coverage](https://codecov.io/gh/apache/incubator-gearpump/pull/122?src=pr) is 
70.40% (diff: 89.47%)
> Merging 
[#122](https://codecov.io/gh/apache/incubator-gearpump/pull/122?src=pr) into 
[master](https://codecov.io/gh/apache/incubator-gearpump/branch/master?src=pr) 
will increase coverage by **0.08%**


```diff
@@ master   #122   diff @@
==
  Files   186186  
  Lines  5984   5984  
  Methods5472   5471 -1   
  Messages  0  0  
  Branches510511 +1   
==
+ Hits   4208   4213 +5   
+ Misses 1776   1771 -5   
  Partials  0  0  
```


![Sunburst](https://codecov.io/gh/apache/incubator-gearpump/pull/122/graphs/sunburst.svg?src=pr=150)

> Powered by [Codecov](https://codecov.io?src=pr). Last update 
[914eb90...3031f58](https://codecov.io/gh/apache/incubator-gearpump/compare/914eb9008882835fbafd126085f3ebe73bd87c9f...3031f58b58c3781f60308bd47e17669549baca5c?src=pr)


> groupBy is not applied in JavaStream DSL
> 
>
> Key: GEARPUMP-245
> URL: https://issues.apache.org/jira/browse/GEARPUMP-245
> Project: Apache Gearpump
>  Issue Type: Bug
>  Components: streaming
>Affects Versions: 0.8.2
>Reporter: Manu Zhang
>Assignee: Manu Zhang
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (GEARPUMP-249) reduce is not executed when following flatMap in Stream DSL

2016-12-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/GEARPUMP-249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747126#comment-15747126
 ] 

ASF GitHub Bot commented on GEARPUMP-249:
-

GitHub user manuzhang opened a pull request:

https://github.com/apache/incubator-gearpump/pull/123

[GEARPUMP-249] Fix function chains to enforce execution

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the commit message is formatted like:
   `[GEARPUMP-] Meaningful description of pull request` 
 - [ ] Make sure tests pass via `sbt clean test`.
 - [ ] Make sure old documentation affected by the pull request has been 
updated and new documentation added for new functionality. 




You can merge this pull request into a Git repository by running:

$ git pull https://github.com/manuzhang/incubator-gearpump GEARPUMP-249

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-gearpump/pull/123.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #123


commit cd1124784ba95c4688ccd457c135e7904520e0c1
Author: manuzhang 
Date:   2016-12-14T03:36:39Z

[GEARPUMP-249] Fix function chains to enforce execution




> reduce is not executed when following flatMap in Stream DSL
> ---
>
> Key: GEARPUMP-249
> URL: https://issues.apache.org/jira/browse/GEARPUMP-249
> Project: Apache Gearpump
>  Issue Type: Bug
>  Components: streaming
>Affects Versions: 0.8.2
>Reporter: Manu Zhang
>Assignee: Manu Zhang
>
> The issue can be reproduced if a redundant {{map}} is added between 
> {{groupBy}} and {{sum}}
> {code}
> app.source(data.lines.toList, 1, "source").
>   // word => (word, count)
>   flatMap(line => line.split("[\\s]+")).map((_, 1)).
>   // (word, count1), (word, count2) => (word, count1 + count2)
>   groupByKey().sum.log
> {code}
> The problem is with {{AndThen#process}} method, where {{flatMap}} is lazily/ 
> never executed.
> {code}
>   override def process(value: IN): TraversableOnce[OUT] = {
> first.process(value).flatMap(second.process)
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)