[jira] [Resolved] (GOBBLIN-1097) ResultChainingIterator.add should check if the argument iterator is null

2020-03-24 Thread Hung Tran (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hung Tran resolved GOBBLIN-1097.

Fix Version/s: 0.15.0
   Resolution: Fixed

Issue resolved by pull request #2938
[https://github.com/apache/incubator-gobblin/pull/2938]

> ResultChainingIterator.add should check if the argument iterator is null
> 
>
> Key: GOBBLIN-1097
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1097
> Project: Apache Gobblin
>  Issue Type: Bug
>Reporter: Alex Li
>Priority: Major
> Fix For: 0.15.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> ResultChainingIterator.add should check if the argument iterator is null.
> It fails, if the argument is null.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (GOBBLIN-1097) ResultChainingIterator.add should check if the argument iterator is null

2020-03-24 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-1097?focusedWorklogId=409252=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409252
 ]

ASF GitHub Bot logged work on GOBBLIN-1097:
---

Author: ASF GitHub Bot
Created on: 25/Mar/20 01:01
Start Date: 25/Mar/20 01:01
Worklog Time Spent: 10m 
  Work Description: asfgit commented on pull request #2938: GOBBLIN-1097: 
ResultChainingIterator.add argument cannot be null
URL: https://github.com/apache/incubator-gobblin/pull/2938
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 409252)
Time Spent: 0.5h  (was: 20m)

> ResultChainingIterator.add should check if the argument iterator is null
> 
>
> Key: GOBBLIN-1097
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1097
> Project: Apache Gobblin
>  Issue Type: Bug
>Reporter: Alex Li
>Priority: Major
> Fix For: 0.15.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> ResultChainingIterator.add should check if the argument iterator is null.
> It fails, if the argument is null.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [incubator-gobblin] asfgit closed pull request #2938: GOBBLIN-1097: ResultChainingIterator.add argument cannot be null

2020-03-24 Thread GitBox
asfgit closed pull request #2938: GOBBLIN-1097: ResultChainingIterator.add 
argument cannot be null
URL: https://github.com/apache/incubator-gobblin/pull/2938
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Work logged] (GOBBLIN-1097) ResultChainingIterator.add should check if the argument iterator is null

2020-03-24 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-1097?focusedWorklogId=409248=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409248
 ]

ASF GitHub Bot logged work on GOBBLIN-1097:
---

Author: ASF GitHub Bot
Created on: 25/Mar/20 00:46
Start Date: 25/Mar/20 00:46
Worklog Time Spent: 10m 
  Work Description: codecov-io commented on issue #2938: GOBBLIN-1097: 
ResultChainingIterator.add argument cannot be null
URL: 
https://github.com/apache/incubator-gobblin/pull/2938#issuecomment-603578777
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2938?src=pr=h1)
 Report
   > Merging 
[#2938](https://codecov.io/gh/apache/incubator-gobblin/pull/2938?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-gobblin/commit/a63461257c3fcea8f4ff67087f8cb29be25d6baf=desc)
 will **decrease** coverage by `40.49%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/graphs/tree.svg?width=650=150=pr=4MgURJ0bGc)](https://codecov.io/gh/apache/incubator-gobblin/pull/2938?src=pr=tree)
   
   ```diff
   @@ Coverage Diff  @@
   ## master   #2938   +/-   ##
   
   - Coverage 44.60%   4.11%   -40.50% 
   + Complexity 8980 750 -8230 
   
 Files  19361936   
 Lines 73234   73235+1 
 Branches   80838083   
   
   - Hits  326693016-29653 
   - Misses37515   69899+32384 
   + Partials   3050 320 -2730 
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2938?src=pr=tree) 
| Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | 
[...che/gobblin/salesforce/ResultChainingIterator.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1zYWxlc2ZvcmNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NhbGVzZm9yY2UvUmVzdWx0Q2hhaW5pbmdJdGVyYXRvci5qYXZh)
 | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | 
[...c/main/java/org/apache/gobblin/util/FileUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvRmlsZVV0aWxzLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-4.00%)` | |
   | 
[...n/java/org/apache/gobblin/fork/CopyableSchema.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ZvcmsvQ29weWFibGVTY2hlbWEuamF2YQ==)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-2.00%)` | |
   | 
[...java/org/apache/gobblin/stream/ControlMessage.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vc3RyZWFtL0NvbnRyb2xNZXNzYWdlLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | |
   | 
[...va/org/apache/gobblin/dataset/DatasetResolver.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YXNldC9EYXRhc2V0UmVzb2x2ZXIuamF2YQ==)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-3.00%)` | |
   | 
[...va/org/apache/gobblin/converter/EmptyIterable.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9FbXB0eUl0ZXJhYmxlLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | |
   | 
[...org/apache/gobblin/ack/BasicAckableForTesting.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vYWNrL0Jhc2ljQWNrYWJsZUZvclRlc3RpbmcuamF2YQ==)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-3.00%)` | |
   | 
[.../org/apache/gobblin/yarn/HelixMessageSubTypes.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi15YXJuL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3lhcm4vSGVsaXhNZXNzYWdlU3ViVHlwZXMuamF2YQ==)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | |
   | 
[...va/org/apache/gobblin/cluster/SingleHelixTask.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvU2luZ2xlSGVsaXhUYXNrLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-5.00%)` | |
   | 
[...a/org/apache/gobblin/cluster/ContainerMetrics.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvQ29udGFpbmVyTWV0cmljcy5qYXZh)
 | 

[GitHub] [incubator-gobblin] codecov-io commented on issue #2938: GOBBLIN-1097: ResultChainingIterator.add argument cannot be null

2020-03-24 Thread GitBox
codecov-io commented on issue #2938: GOBBLIN-1097: ResultChainingIterator.add 
argument cannot be null
URL: 
https://github.com/apache/incubator-gobblin/pull/2938#issuecomment-603578777
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2938?src=pr=h1)
 Report
   > Merging 
[#2938](https://codecov.io/gh/apache/incubator-gobblin/pull/2938?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-gobblin/commit/a63461257c3fcea8f4ff67087f8cb29be25d6baf=desc)
 will **decrease** coverage by `40.49%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/graphs/tree.svg?width=650=150=pr=4MgURJ0bGc)](https://codecov.io/gh/apache/incubator-gobblin/pull/2938?src=pr=tree)
   
   ```diff
   @@ Coverage Diff  @@
   ## master   #2938   +/-   ##
   
   - Coverage 44.60%   4.11%   -40.50% 
   + Complexity 8980 750 -8230 
   
 Files  19361936   
 Lines 73234   73235+1 
 Branches   80838083   
   
   - Hits  326693016-29653 
   - Misses37515   69899+32384 
   + Partials   3050 320 -2730 
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2938?src=pr=tree) 
| Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | 
[...che/gobblin/salesforce/ResultChainingIterator.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1zYWxlc2ZvcmNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NhbGVzZm9yY2UvUmVzdWx0Q2hhaW5pbmdJdGVyYXRvci5qYXZh)
 | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | 
[...c/main/java/org/apache/gobblin/util/FileUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvRmlsZVV0aWxzLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-4.00%)` | |
   | 
[...n/java/org/apache/gobblin/fork/CopyableSchema.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ZvcmsvQ29weWFibGVTY2hlbWEuamF2YQ==)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-2.00%)` | |
   | 
[...java/org/apache/gobblin/stream/ControlMessage.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vc3RyZWFtL0NvbnRyb2xNZXNzYWdlLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | |
   | 
[...va/org/apache/gobblin/dataset/DatasetResolver.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YXNldC9EYXRhc2V0UmVzb2x2ZXIuamF2YQ==)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-3.00%)` | |
   | 
[...va/org/apache/gobblin/converter/EmptyIterable.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9FbXB0eUl0ZXJhYmxlLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | |
   | 
[...org/apache/gobblin/ack/BasicAckableForTesting.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vYWNrL0Jhc2ljQWNrYWJsZUZvclRlc3RpbmcuamF2YQ==)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-3.00%)` | |
   | 
[.../org/apache/gobblin/yarn/HelixMessageSubTypes.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi15YXJuL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3lhcm4vSGVsaXhNZXNzYWdlU3ViVHlwZXMuamF2YQ==)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-1.00%)` | |
   | 
[...va/org/apache/gobblin/cluster/SingleHelixTask.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvU2luZ2xlSGVsaXhUYXNrLmphdmE=)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-5.00%)` | |
   | 
[...a/org/apache/gobblin/cluster/ContainerMetrics.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvQ29udGFpbmVyTWV0cmljcy5qYXZh)
 | `0.00% <0.00%> (-100.00%)` | `0.00% <0.00%> (-4.00%)` | |
   | ... and [1143 
more](https://codecov.io/gh/apache/incubator-gobblin/pull/2938/diff?src=pr=tree-more)
 | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2938?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, 

[jira] [Work logged] (GOBBLIN-1097) ResultChainingIterator.add should check if the argument iterator is null

2020-03-24 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-1097?focusedWorklogId=409208=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409208
 ]

ASF GitHub Bot logged work on GOBBLIN-1097:
---

Author: ASF GitHub Bot
Created on: 25/Mar/20 00:16
Start Date: 25/Mar/20 00:16
Worklog Time Spent: 10m 
  Work Description: arekusuri commented on pull request #2938: 
GOBBLIN-1097: ResultChainingIterator.add argument cannot be null
URL: https://github.com/apache/incubator-gobblin/pull/2938
 
 
   
   Dear Gobblin maintainers,
   
   ### JIRA
   https://issues.apache.org/jira/browse/GOBBLIN-1097
   
   ### Description
   ResultChainingIterator.add should check if the argument iterator is null
   
   ### Tests
   https://ltx1-holdemaz05.grid.linkedin.com:8443/executor?execid=24779423
   
   
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 409208)
Remaining Estimate: 0h
Time Spent: 10m

> ResultChainingIterator.add should check if the argument iterator is null
> 
>
> Key: GOBBLIN-1097
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1097
> Project: Apache Gobblin
>  Issue Type: Bug
>Reporter: Alex Li
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> ResultChainingIterator.add should check if the argument iterator is null.
> It fails, if the argument is null.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [incubator-gobblin] arekusuri opened a new pull request #2938: GOBBLIN-1097: ResultChainingIterator.add argument cannot be null

2020-03-24 Thread GitBox
arekusuri opened a new pull request #2938: GOBBLIN-1097: 
ResultChainingIterator.add argument cannot be null
URL: https://github.com/apache/incubator-gobblin/pull/2938
 
 
   
   Dear Gobblin maintainers,
   
   ### JIRA
   https://issues.apache.org/jira/browse/GOBBLIN-1097
   
   ### Description
   ResultChainingIterator.add should check if the argument iterator is null
   
   ### Tests
   https://ltx1-holdemaz05.grid.linkedin.com:8443/executor?execid=24779423
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Created] (GOBBLIN-1097) ResultChainingIterator.add should check if the argument iterator is null

2020-03-24 Thread Alex Li (Jira)
Alex Li created GOBBLIN-1097:


 Summary: ResultChainingIterator.add should check if the argument 
iterator is null
 Key: GOBBLIN-1097
 URL: https://issues.apache.org/jira/browse/GOBBLIN-1097
 Project: Apache Gobblin
  Issue Type: Bug
Reporter: Alex Li


ResultChainingIterator.add should check if the argument iterator is null.

It fails, if the argument is null.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (GOBBLIN-1096) Work with DST change in compaction watermark

2020-03-24 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-1096?focusedWorklogId=409130=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409130
 ]

ASF GitHub Bot logged work on GOBBLIN-1096:
---

Author: ASF GitHub Bot
Created on: 24/Mar/20 22:18
Start Date: 24/Mar/20 22:18
Worklog Time Spent: 10m 
  Work Description: zxcware commented on pull request #2937: [GOBBLIN-1096] 
Work with DST change in compaction watermark
URL: https://github.com/apache/incubator-gobblin/pull/2937
 
 
   Dear Gobblin maintainers,
   
   Please accept this PR. I understand that it will not be reviewed until I 
have checked off all the steps below!
   
   
   ### JIRA
   - [x] My PR addresses the following [Gobblin 
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references 
them in the PR title. 
   - https://issues.apache.org/jira/browse/GOBBLIN-1096
   
   
   ### Description
   - [x] Here are some details about my PR:
 - Increment existing compaction watermark by 1 day with DST in mind. 
Luckily, since 1.8, java time handles DST in PST time zone
   
   ### Tests
   - [x] My PR adds the following unit tests:
 - `CompactionWatermarkActionTest.testWatermarkWithDST` verified that DST 
change on 03/08/2020 was handled correctly in PST timezone
   
   ### Commits
   - [x] My commits all reference JIRA issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
   1. Subject is separated from body by a blank line
   2. Subject is limited to 50 characters
   3. Subject does not end with a period
   4. Subject uses the imperative mood ("add", not "adding")
   5. Body wraps at 72 characters
   6. Body explains "what" and "why", not "how"
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 409130)
Remaining Estimate: 0h
Time Spent: 10m

> Work with DST change in compaction watermark
> 
>
> Key: GOBBLIN-1096
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1096
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zhixiong Chen
>Assignee: Zhixiong Chen
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [incubator-gobblin] zxcware opened a new pull request #2937: [GOBBLIN-1096] Work with DST change in compaction watermark

2020-03-24 Thread GitBox
zxcware opened a new pull request #2937: [GOBBLIN-1096] Work with DST change in 
compaction watermark
URL: https://github.com/apache/incubator-gobblin/pull/2937
 
 
   Dear Gobblin maintainers,
   
   Please accept this PR. I understand that it will not be reviewed until I 
have checked off all the steps below!
   
   
   ### JIRA
   - [x] My PR addresses the following [Gobblin 
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references 
them in the PR title. 
   - https://issues.apache.org/jira/browse/GOBBLIN-1096
   
   
   ### Description
   - [x] Here are some details about my PR:
 - Increment existing compaction watermark by 1 day with DST in mind. 
Luckily, since 1.8, java time handles DST in PST time zone
   
   ### Tests
   - [x] My PR adds the following unit tests:
 - `CompactionWatermarkActionTest.testWatermarkWithDST` verified that DST 
change on 03/08/2020 was handled correctly in PST timezone
   
   ### Commits
   - [x] My commits all reference JIRA issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
   1. Subject is separated from body by a blank line
   2. Subject is limited to 50 characters
   3. Subject does not end with a period
   4. Subject uses the imperative mood ("add", not "adding")
   5. Body wraps at 72 characters
   6. Body explains "what" and "why", not "how"
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Created] (GOBBLIN-1096) Work with DST change in compaction watermark

2020-03-24 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1096:
--

 Summary: Work with DST change in compaction watermark
 Key: GOBBLIN-1096
 URL: https://issues.apache.org/jira/browse/GOBBLIN-1096
 Project: Apache Gobblin
  Issue Type: Task
Reporter: Zhixiong Chen
Assignee: Zhixiong Chen






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [incubator-gobblin] codecov-io commented on issue #2936: Fail YarnAppLaunch if there's existing AM with configuration

2020-03-24 Thread GitBox
codecov-io commented on issue #2936: Fail YarnAppLaunch if there's existing AM 
with configuration
URL: 
https://github.com/apache/incubator-gobblin/pull/2936#issuecomment-603487068
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2936?src=pr=h1)
 Report
   > Merging 
[#2936](https://codecov.io/gh/apache/incubator-gobblin/pull/2936?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-gobblin/commit/a63461257c3fcea8f4ff67087f8cb29be25d6baf=desc)
 will **increase** coverage by `0.00%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-gobblin/pull/2936/graphs/tree.svg?width=650=150=pr=4MgURJ0bGc)](https://codecov.io/gh/apache/incubator-gobblin/pull/2936?src=pr=tree)
   
   ```diff
   @@Coverage Diff@@
   ## master#2936   +/-   ##
   =
 Coverage 44.60%   44.61%   
 Complexity 8980 8980   
   =
 Files  1936 1936   
 Lines 7323473238+4 
 Branches   8083 8085+2 
   =
   + Hits  3266932676+7 
   + Misses3751537508-7 
   - Partials   3050 3054+4 
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2936?src=pr=tree) 
| Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | 
[...rg/apache/gobblin/yarn/GobblinYarnAppLauncher.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2936/diff?src=pr=tree#diff-Z29iYmxpbi15YXJuL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3lhcm4vR29iYmxpbllhcm5BcHBMYXVuY2hlci5qYXZh)
 | `20.96% <0.00%> (-0.21%)` | `8.00 <0.00> (ø)` | |
   | 
[...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2936/diff?src=pr=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=)
 | `85.71% <0.00%> (-7.15%)` | `3.00% <0.00%> (ø%)` | |
   | 
[...ava/org/apache/gobblin/fsm/FiniteStateMachine.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2936/diff?src=pr=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ZzbS9GaW5pdGVTdGF0ZU1hY2hpbmUuamF2YQ==)
 | `73.48% <0.00%> (-3.04%)` | `18.00% <0.00%> (-3.00%)` | |
   | 
[...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2936/diff?src=pr=tree#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=)
 | `70.00% <0.00%> (-2.23%)` | `13.00% <0.00%> (ø%)` | |
   | 
[...lin/elasticsearch/writer/FutureCallbackHolder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2936/diff?src=pr=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tZWxhc3RpY3NlYXJjaC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9lbGFzdGljc2VhcmNoL3dyaXRlci9GdXR1cmVDYWxsYmFja0hvbGRlci5qYXZh)
 | `61.42% <0.00%> (-1.43%)` | `4.00% <0.00%> (ø%)` | |
   | 
[.../apache/gobblin/runtime/api/JobExecutionState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2936/diff?src=pr=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0pvYkV4ZWN1dGlvblN0YXRlLmphdmE=)
 | `80.37% <0.00%> (+0.93%)` | `24.00% <0.00%> (ø%)` | |
   | 
[...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2936/diff?src=pr=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh)
 | `41.73% <0.00%> (+3.47%)` | `15.00% <0.00%> (+1.00%)` | |
   | 
[...e/gobblin/runtime/app/ServiceBasedAppLauncher.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2936/diff?src=pr=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBwL1NlcnZpY2VCYXNlZEFwcExhdW5jaGVyLmphdmE=)
 | `49.51% <0.00%> (+9.70%)` | `12.00% <0.00%> (+2.00%)` | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2936?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2936?src=pr=footer).
 Last update 
[a634612...57ac9bf](https://codecov.io/gh/apache/incubator-gobblin/pull/2936?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For 

[GitHub] [incubator-gobblin] autumnust opened a new pull request #2936: Fail YarnAppLaunch if there's existing AM with configuration

2020-03-24 Thread GitBox
autumnust opened a new pull request #2936: Fail YarnAppLaunch if there's 
existing AM with configuration
URL: https://github.com/apache/incubator-gobblin/pull/2936
 
 
   Dear Gobblin maintainers,
   
   Please accept this PR. I understand that it will not be reviewed until I 
have checked off all the steps below!
   
   
   ### JIRA
   - [ ] My PR addresses the following [Gobblin 
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references 
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
   - https://issues.apache.org/jira/browse/GOBBLIN-XXX
   
   
   ### Description
   - [ ] Here are some details about my PR, including screenshots (if 
applicable):
   
   
   ### Tests
   - [ ] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   
   
   ### Commits
   - [ ] My commits all reference JIRA issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
   1. Subject is separated from body by a blank line
   2. Subject is limited to 50 characters
   3. Subject does not end with a period
   4. Subject uses the imperative mood ("add", not "adding")
   5. Body wraps at 72 characters
   6. Body explains "what" and "why", not "how"
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


Hadoop 3 support

2020-03-24 Thread Jay Sen
Hi Dev Team,

What do you guys think about support for Hadoop 3 to run the Gobblin on ?

Thanks
Jay