Re: Pre-Apache Pull Requests

[Zhixiong Chen]

Done by adding the associated JIRA issue.


From: Tamas Nemeth 
Sent: Wednesday, August 16, 2017 12:30:44 PM
To: Abhishek Tiwari; dev@gobblin.incubator.apache.org; 
u...@gobblin.incubator.apache.org
Cc: Chavdar Botev; Aditya Sharma; Issac Buenrostro; Lorand Bendig; Pradhan 
Cadabam; Eric Ogren; Kuai Yu; Zhixiong Chen; Vasanth Rajamani; Shirshanka Das; 
Hung Tran
Subject: Re: Pre-Apache Pull Requests


You don't often get email from tamas.nem...@prezi.com. Learn why this is 
important
Feedback
Hi Abishek,

You can ignore my json module pull request as I took it apart and it's first 
part is the Support eventual consistent filesystems like S3 pull request. It 
made more sense in that format. I will create a new json module pull request 
once we are done with the Support eventual consistent filesystem one.

Sorry about the confusion.

Thanks,
Tamas

On Wed, Aug 16, 2017 at 9:08 PM Abhishek Tiwari 
> wrote:
Wiki link: 
https://cwiki.apache.org/confluence/display/GOBBLIN/Pre-Apache+Pull+Requests

Hi all,


The following PRs have not seen any updates in months / years. So, to help us 
get to a resolution please:

  1.  Preferably update the PR to resolve code conflict / address PR comments
  2.  If you are busy right now but want to continue working on it - please 
update the PR title with corresponding JIRA issue number (I have pasted JIRA 
link as a comment in each PR)
  3.  Close the PR if you believe it is not relevant anymore

If we do not hear back from the original contributor within a week, or the 
contributor is not interested in working on it anymore: we will close the PR 
and attach it's diff to the corresponding JIRA issue (for reference of anyone 
who wants to work on it in the future)


Chavdar

Create cluster build: https://github.com/apache/incubator-gobblin/pull/1542
Adding test/common/Log4jUtils to simplify logging setup in tests: 
https://github.com/apache/incubator-gobblin/pull/571
Make build compatible with Gradle 2: 
https://github.com/apache/incubator-gobblin/pull/269

Aditya
Added skipping feature to MultiWorkUnits: 
https://github.com/apache/incubator-gobblin/pull/1521

Issac
Implemented Avro field replacer converter: 
https://github.com/apache/incubator-gobblin/pull/1412
Config checker: https://github.com/apache/incubator-gobblin/pull/750

Lorand
IllegalArgumentException in JdbcPublisher when staging tables are skipped: 
https://github.com/apache/incubator-gobblin/pull/1177

Pradhan
Create key annotation to auto generate docs: 
https://github.com/apache/incubator-gobblin/pull/1114

Kaiserchen
change TaskDone notification for forks to be not racy: 
https://github.com/apache/incubator-gobblin/pull/841

Min
Config management regression test: 
https://github.com/apache/incubator-gobblin/pull/758
client API/impl check in: https://github.com/apache/incubator-gobblin/pull/627

Eric
Encryption skeleton: capabilities and encoders: 
https://github.com/apache/incubator-gobblin/pull/1607

Tamas
Support eventual consistent filesystems like S3: 
https://github.com/apache/incubator-gobblin/pull/1993
Gobblin Json module: https://github.com/apache/incubator-gobblin/pull/1686

Kuai
Create workunitstream for CompactionSource: 
https://github.com/apache/incubator-gobblin/pull/1826

Zhixiong
new salesforce writer: https://github.com/apache/incubator-gobblin/pull/1835

Regards,
Abhishek

[jira] [Updated] (GOBBLIN-212) Exception handling of TaskStateCollectorServiceHandler

[Lei Sun (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GOBBLIN-212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei Sun updated GOBBLIN-212:

Sprint: Apache Gobblin 170807

> Exception handling of TaskStateCollectorServiceHandler
> --
>
> Key: GOBBLIN-212
> URL: https://issues.apache.org/jira/browse/GOBBLIN-212
> Project: Apache Gobblin
>  Issue Type: Bug
>Reporter: Lei Sun
>Assignee: Lei Sun
>
> Current if the TaskStateCollectorServiceHandler failed, the whole job won't 
> proceed. It is not the correct behavior sometimes if we would like the job to 
> proceed and finish dataset commit even there's something wrong happen in the 
> TaskStateCollectorServiceHandler. Should catch the exception carefully and 
> make it configurable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Work started] (GOBBLIN-205) Fix Replication bug in Push Mode

[Lei Sun (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GOBBLIN-205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on GOBBLIN-205 started by Lei Sun.
---
> Fix Replication bug in Push Mode
> 
>
> Key: GOBBLIN-205
> URL: https://issues.apache.org/jira/browse/GOBBLIN-205
> Project: Apache Gobblin
>  Issue Type: Bug
>Reporter: Lei Sun
>Assignee: Lei Sun
>
> Fix bugs In DistcpNG (Gobblin/copy) in terms of EndPoint comparison.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (GOBBLIN-211) Implement new salesforce writer

[Zhixiong Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GOBBLIN-211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhixiong Chen updated GOBBLIN-211:
--
Sprint:   (was: Apache Gobblin 170807)

> Implement new salesforce writer
> ---
>
> Key: GOBBLIN-211
> URL: https://issues.apache.org/jira/browse/GOBBLIN-211
> Project: Apache Gobblin
>  Issue Type: Bug
>Reporter: Zhixiong Chen
>Assignee: Zhixiong Chen
>
> The current implementation of salesforce writer, 
> `org.apache.gobblin.writer.http.SalesForceRestWriterBuilder`, deeply coupled 
> with `CloseableHttpClient`.
> The new implementation will decouple the http client from the http writer 
> based on the gobblin http framework.  An example implementation is 
> `org.apache.gobblin.writer.AsyncHttpWriter`, which is mainly composed by:
> {code:java}
> org.apache.gobblin.http.HttpClient
> org.apache.gobblin.async.AsyncRequestBuilder
> org.apache.gobblin.http.ResponseHandler
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (GOBBLIN-36) new salesforce writer

[Abhishek Tiwari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GOBBLIN-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Tiwari closed GOBBLIN-36.
--
Resolution: Duplicate

> new salesforce writer
> -
>
> Key: GOBBLIN-36
> URL: https://issues.apache.org/jira/browse/GOBBLIN-36
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zhixiong Chen
>Assignee: Zhixiong Chen
>
> *Overview*
> An async salesforce writer built on top of the async http writer framework. 
> Old suite of `HttpWriter` and `SalesforceRestWriter` are marked as deprecated.
> *Auth*
> `OAuthAccessToken` provides a standard way to authenticate using OAuth. 
> `SalesforceAuth` is a `OAuthAccessToken` with specific way to extract 
> information from config.
> *Client*
> `ApacheClient` provides a standard way to send apache `HttpUriRequest` and 
> get `CloseableHttpResponse`. `SalesforceClient` is a `ApacheClient` 
> integrated with authentication.
> *Writer*
> `SalesforceWriterBuilder` instantiates `SalesforceClient`, 
> `SalesforceRequestBuilder`, `SalesforceResponseHandler` and plugs them into 
> an instance of `AsyncHttpWriter`. It builds an `AsyncWriterManager` that 
> wraps the `AsyncHttpWriter` instance.
>  
> *Github Url* : https://github.com/linkedin/gobblin/pull/1835 
> *Github Reporter* : [~zxc] 
> *Github Created At* : 2017-05-05T17:28:21Z 
> *Github Updated At* : 2017-05-06T01:20:59Z



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (GOBBLIN-211) Implement new salesforce writer

[Zhixiong Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GOBBLIN-211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhixiong Chen updated GOBBLIN-211:
--
Summary: Implement new salesforce writer  (was: Implement new salesforce 
writer based on http framework)

> Implement new salesforce writer
> ---
>
> Key: GOBBLIN-211
> URL: https://issues.apache.org/jira/browse/GOBBLIN-211
> Project: Apache Gobblin
>  Issue Type: Bug
>Reporter: Zhixiong Chen
>Assignee: Zhixiong Chen
>
> The current implementation of salesforce writer, 
> `org.apache.gobblin.writer.http.SalesForceRestWriterBuilder`, deeply coupled 
> with `CloseableHttpClient`.
> The new implementation will decouple the http client from the http writer 
> based on the gobblin http fr



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (GOBBLIN-211) Implement new salesforce writer based on http framework

[Zhixiong Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GOBBLIN-211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhixiong Chen updated GOBBLIN-211:
--
Labels:   (was: Bug:Generic)

> Implement new salesforce writer based on http framework
> ---
>
> Key: GOBBLIN-211
> URL: https://issues.apache.org/jira/browse/GOBBLIN-211
> Project: Apache Gobblin
>  Issue Type: Bug
>Reporter: Zhixiong Chen
>Assignee: Zhixiong Chen
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (GOBBLIN-211) Implement new salesforce writer based on http framework

[Zhixiong Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GOBBLIN-211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhixiong Chen updated GOBBLIN-211:
--
Labels: Bug:Generic  (was: Bug:Generic Core:JobManagement Core:Publisher)

> Implement new salesforce writer based on http framework
> ---
>
> Key: GOBBLIN-211
> URL: https://issues.apache.org/jira/browse/GOBBLIN-211
> Project: Apache Gobblin
>  Issue Type: Bug
>Reporter: Zhixiong Chen
>Assignee: Zhixiong Chen
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Pre-Apache Pull Requests

[Tamas Nemeth]

Hi Abishek,

You can ignore my json module pull request as I took it apart and it's
first part is the Support eventual consistent filesystems like S3 pull
request. It made more sense in that format. I will create a new json module
pull request once we are done with the Support eventual consistent
filesystem one.

Sorry about the confusion.

Thanks,
Tamas

On Wed, Aug 16, 2017 at 9:08 PM Abhishek Tiwari  wrote:

> Wiki link:
> https://cwiki.apache.org/confluence/display/GOBBLIN/Pre-Apache+Pull+Requests
>
> Hi all,
>
> The following PRs have not seen any updates in months / years. So, to help
> us get to a resolution please:
>
>1. Preferably update the PR to resolve code conflict / address PR
>comments
>2. If you are busy right now but want to continue working on it -
>please update the PR title with corresponding JIRA issue number (I have
>pasted JIRA link as a comment in each PR)
>3. Close the PR if you believe it is not relevant anymore
>
> If we do not hear back from the original contributor within a week, or the
> contributor is not interested in working on it anymore: we will close the
> PR and attach it's diff to the corresponding JIRA issue (for reference of
> anyone who wants to work on it in the future)
>
>
> *Chavdar*
>
> Create cluster build:
> https://github.com/apache/incubator-gobblin/pull/1542
> Adding test/common/Log4jUtils to simplify logging setup in tests:
> https://github.com/apache/incubator-gobblin/pull/571
> Make build compatible with Gradle 2:
> https://github.com/apache/incubator-gobblin/pull/269
>
> *Aditya*
> Added skipping feature to MultiWorkUnits:
> https://github.com/apache/incubator-gobblin/pull/1521
>
> *Issac*
> Implemented Avro field replacer converter:
> https://github.com/apache/incubator-gobblin/pull/1412
> Config checker: https://github.com/apache/incubator-gobblin/pull/750
>
> *Lorand*
> IllegalArgumentException in JdbcPublisher when staging tables are skipped:
> https://github.com/apache/incubator-gobblin/pull/1177
>
> *Pradhan*
> Create key annotation to auto generate docs:
> https://github.com/apache/incubator-gobblin/pull/1114
>
> *Kaiserchen*
> change TaskDone notification for forks to be not racy:
> https://github.com/apache/incubator-gobblin/pull/841
>
> *Min*
> Config management regression test:
> https://github.com/apache/incubator-gobblin/pull/758
> client API/impl check in:
> https://github.com/apache/incubator-gobblin/pull/627
>
> *Eric*
> Encryption skeleton: capabilities and encoders:
> https://github.com/apache/incubator-gobblin/pull/1607
>
> *Tamas*
> Support eventual consistent filesystems like S3:
> https://github.com/apache/incubator-gobblin/pull/1993
> Gobblin Json module: https://github.com/apache/incubator-gobblin/pull/1686
>
> *Kuai*
> Create workunitstream for CompactionSource:
> https://github.com/apache/incubator-gobblin/pull/1826
>
> *Zhixiong*
> new salesforce writer:
> https://github.com/apache/incubator-gobblin/pull/1835
>
>
> Regards,
> Abhishek
>

[jira] [Created] (GOBBLIN-210) Implement a source based on Dataset Finder

[Issac Buenrostro (JIRA)]

Issac Buenrostro created GOBBLIN-210:


 Summary: Implement a source based on Dataset Finder
 Key: GOBBLIN-210
 URL: https://issues.apache.org/jira/browse/GOBBLIN-210
 Project: Apache Gobblin
  Issue Type: Bug
  Components: gobblin-core
Reporter: Issac Buenrostro
Assignee: Issac Buenrostro






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (GOBBLIN-204) Add a service that fetches GaaS flow configs from a git repository

[Hung Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/GOBBLIN-204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hung Tran resolved GOBBLIN-204.
---
Resolution: Fixed

Issue resolved by pull request #2055
[https://github.com/apache/incubator-gobblin/pull/2055]

> Add a service that fetches GaaS flow configs from a git repository
> --
>
> Key: GOBBLIN-204
> URL: https://issues.apache.org/jira/browse/GOBBLIN-204
> Project: Apache Gobblin
>  Issue Type: Bug
>Reporter: Hung Tran
>Assignee: Hung Tran
>
> Add a service that polls a git repository for flow configuration. The service 
> is integrated with GaaS alongside the REST-based API for flow spec creation. 
> Both methods of flow management can co-exist, but should be used to manage 
> disjoint flows.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)