[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-11 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=327067=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327067
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 11/Oct/19 20:45
Start Date: 11/Oct/19 20:45
Worklog Time Spent: 10m 
  Work Description: asfgit commented on pull request #2753: 
[GOBBLIN-899]Add config in replication config to determine wheter schema cehck 
ena…
URL: https://github.com/apache/incubator-gobblin/pull/2753
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327067)
Time Spent: 1h 40m  (was: 1.5h)

> Add a key in dataset config to disable schema check for a specific dataset
> --
>
> Key: GOBBLIN-899
> URL: https://issues.apache.org/jira/browse/GOBBLIN-899
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zihan Li
>Priority: Major
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-11 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=327028=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327028
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 11/Oct/19 18:21
Start Date: 11/Oct/19 18:21
Worklog Time Spent: 10m 
  Work Description: yukuai518 commented on issue #2753: [GOBBLIN-899]Add 
config in replication config to determine wheter schema cehck ena…
URL: 
https://github.com/apache/incubator-gobblin/pull/2753#issuecomment-541171925
 
 
   +1 LGTM
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 327028)
Time Spent: 1.5h  (was: 1h 20m)

> Add a key in dataset config to disable schema check for a specific dataset
> --
>
> Key: GOBBLIN-899
> URL: https://issues.apache.org/jira/browse/GOBBLIN-899
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zihan Li
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-08 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=325377=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325377
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 08/Oct/19 22:54
Start Date: 08/Oct/19 22:54
Worklog Time Spent: 10m 
  Work Description: codecov-io commented on issue #2753: [GOBBLIN-899]Add 
config in replication config to determine wheter schema cehck ena…
URL: 
https://github.com/apache/incubator-gobblin/pull/2753#issuecomment-538532040
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2753?src=pr=h1)
 Report
   > Merging 
[#2753](https://codecov.io/gh/apache/incubator-gobblin/pull/2753?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-gobblin/commit/5b8af8cf6e932e8031ef060c79260a4a8f47974c?src=pr=desc)
 will **increase** coverage by `<.01%`.
   > The diff coverage is `26.66%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/graphs/tree.svg?width=650=4MgURJ0bGc=150=pr)](https://codecov.io/gh/apache/incubator-gobblin/pull/2753?src=pr=tree)
   
   ```diff
   @@ Coverage Diff  @@
   ## master#2753  +/-   ##
   
   + Coverage 45.27%   45.27%   +<.01% 
   - Complexity 8828 8829   +1 
   
 Files  1889 1889  
 Lines 7063470648  +14 
 Branches   7757 7759   +2 
   
   + Hits  3197931986   +7 
   - Misses3570135709   +8 
   + Partials   2954 2953   -1
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2753?src=pr=tree) 
| Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | 
[.../FileAwareInputStreamExtractorWithCheckSchema.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvZXh0cmFjdG9yL0ZpbGVBd2FyZUlucHV0U3RyZWFtRXh0cmFjdG9yV2l0aENoZWNrU2NoZW1hLmphdmE=)
 | `60% <0%> (-15%)` | `4 <1> (ø)` | |
   | 
[...pache/gobblin/data/management/copy/CopySource.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvQ29weVNvdXJjZS5qYXZh)
 | `68.22% <0%> (-0.72%)` | `22 <0> (ø)` | |
   | 
[...anagement/copy/replication/ConfigBasedDataset.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcmVwbGljYXRpb24vQ29uZmlnQmFzZWREYXRhc2V0LmphdmE=)
 | `70.54% <0%> (-0.49%)` | `10 <0> (ø)` | |
   | 
[...ent/copy/replication/ReplicationConfiguration.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcmVwbGljYXRpb24vUmVwbGljYXRpb25Db25maWd1cmF0aW9uLmphdmE=)
 | `86.28% <80%> (-0.19%)` | `8 <0> (ø)` | |
   | 
[...rg/apache/gobblin/yarn/GobblinYarnAppLauncher.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi15YXJuL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3lhcm4vR29iYmxpbllhcm5BcHBMYXVuY2hlci5qYXZh)
 | `20.41% <0%> (-0.11%)` | `7% <0%> (ø)` | |
   | 
[.../apache/gobblin/runtime/api/JobExecutionState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0pvYkV4ZWN1dGlvblN0YXRlLmphdmE=)
 | `80.37% <0%> (+0.93%)` | `24% <0%> (ø)` | :arrow_down: |
   | 
[...lin/elasticsearch/writer/FutureCallbackHolder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tZWxhc3RpY3NlYXJjaC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9lbGFzdGljc2VhcmNoL3dyaXRlci9GdXR1cmVDYWxsYmFja0hvbGRlci5qYXZh)
 | `62.85% <0%> (+1.42%)` | `4% <0%> (ø)` | :arrow_down: |
   | 
[...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=)
 | `92.85% <0%> (+7.14%)` | `3% <0%> (ø)` | :arrow_down: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2753?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > 

[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-08 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=325268=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325268
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 08/Oct/19 18:29
Start Date: 08/Oct/19 18:29
Worklog Time Spent: 10m 
  Work Description: ZihanLi58 commented on pull request #2753: 
[GOBBLIN-899]Add config in replication config to determine wheter schema cehck 
ena…
URL: https://github.com/apache/incubator-gobblin/pull/2753#discussion_r332666140
 
 

 ##
 File path: 
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/extractor/FileAwareInputStreamExtractorWithCheckSchema.java
 ##
 @@ -66,14 +67,19 @@ protected FileAwareInputStream buildStream(FileSystem 
fsFromFile) throws DataRec
* @throws IOException
*/
   protected boolean schemaChecking(FileSystem fsFromFile) throws IOException {
+if( 
!this.state.getPropAsBoolean(ReplicationConfiguration.COPY_SCHEMA_CHECK_ENABLED,
 false) ) {
+  return true;
+}
 DatumReader datumReader = new GenericDatumReader<>();
 DataFileReader dataFileReader =
 new DataFileReader(new FsInput(this.file.getFileStatus().getPath(), 
new Configuration()), datumReader);
 
 Review comment:
   There is no existing generic logic to read schema because the schema need to 
be read by specific reader. And the next PR should enable schema check for ORC 
schema as well. But it will be implemented by an if else statement which is 
also not generic.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 325268)
Time Spent: 1h 10m  (was: 1h)

> Add a key in dataset config to disable schema check for a specific dataset
> --
>
> Key: GOBBLIN-899
> URL: https://issues.apache.org/jira/browse/GOBBLIN-899
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zihan Li
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-08 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=325224=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325224
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 08/Oct/19 18:05
Start Date: 08/Oct/19 18:05
Worklog Time Spent: 10m 
  Work Description: yukuai518 commented on pull request #2753: 
[GOBBLIN-899]Add config in replication config to determine wheter schema cehck 
ena…
URL: https://github.com/apache/incubator-gobblin/pull/2753#discussion_r332652123
 
 

 ##
 File path: 
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/extractor/FileAwareInputStreamExtractorWithCheckSchema.java
 ##
 @@ -66,14 +67,19 @@ protected FileAwareInputStream buildStream(FileSystem 
fsFromFile) throws DataRec
* @throws IOException
*/
   protected boolean schemaChecking(FileSystem fsFromFile) throws IOException {
+if( 
!this.state.getPropAsBoolean(ReplicationConfiguration.COPY_SCHEMA_CHECK_ENABLED,
 false) ) {
+  return true;
+}
 DatumReader datumReader = new GenericDatumReader<>();
 DataFileReader dataFileReader =
 new DataFileReader(new FsInput(this.file.getFileStatus().getPath(), 
new Configuration()), datumReader);
 
 Review comment:
   The overall logic looks good to me. However I don't understand why we would 
assume this.file is an avro file. Should this read schema logic be generic? I'm 
assuming this is an existing logic, so you can leave it for now, but still want 
to point it out.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 325224)
Time Spent: 40m  (was: 0.5h)

> Add a key in dataset config to disable schema check for a specific dataset
> --
>
> Key: GOBBLIN-899
> URL: https://issues.apache.org/jira/browse/GOBBLIN-899
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zihan Li
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-08 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=325225=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325225
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 08/Oct/19 18:05
Start Date: 08/Oct/19 18:05
Worklog Time Spent: 10m 
  Work Description: yukuai518 commented on pull request #2753: 
[GOBBLIN-899]Add config in replication config to determine wheter schema cehck 
ena…
URL: https://github.com/apache/incubator-gobblin/pull/2753#discussion_r332653741
 
 

 ##
 File path: 
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/replication/ReplicationConfiguration.java
 ##
 @@ -71,6 +71,8 @@
   public static final String REPLICATION_DATA_CATETORY_TYPE = 
"replicationDataCategoryType";
   public static final String REPLICATION_DATA_FINITE_INSTANCE = 
"replicationDataFiniteInstance";
 
+  public static final String COPY_SCHEMA_CHECK_ENABLED = 
"gobblin.selected.schemaCheck.enabled";
 
 Review comment:
   Why the constant name and value doesn't match? 
gobblin.selected.schemaCheck.enabled => gobblin.copy.schemaCheck.enabled ?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 325225)
Time Spent: 50m  (was: 40m)

> Add a key in dataset config to disable schema check for a specific dataset
> --
>
> Key: GOBBLIN-899
> URL: https://issues.apache.org/jira/browse/GOBBLIN-899
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zihan Li
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-08 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=325226=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325226
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 08/Oct/19 18:05
Start Date: 08/Oct/19 18:05
Worklog Time Spent: 10m 
  Work Description: yukuai518 commented on pull request #2753: 
[GOBBLIN-899]Add config in replication config to determine wheter schema cehck 
ena…
URL: https://github.com/apache/incubator-gobblin/pull/2753#discussion_r332652960
 
 

 ##
 File path: 
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/extractor/FileAwareInputStreamExtractorWithCheckSchema.java
 ##
 @@ -66,14 +67,19 @@ protected FileAwareInputStream buildStream(FileSystem 
fsFromFile) throws DataRec
* @throws IOException
*/
   protected boolean schemaChecking(FileSystem fsFromFile) throws IOException {
+if( 
!this.state.getPropAsBoolean(ReplicationConfiguration.COPY_SCHEMA_CHECK_ENABLED,
 false) ) {
 
 Review comment:
   Replace false with DEFAULT_COPY_SCHEMA_CHECK_ENABLED
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 325226)
Time Spent: 50m  (was: 40m)

> Add a key in dataset config to disable schema check for a specific dataset
> --
>
> Key: GOBBLIN-899
> URL: https://issues.apache.org/jira/browse/GOBBLIN-899
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zihan Li
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-08 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=325227=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325227
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 08/Oct/19 18:05
Start Date: 08/Oct/19 18:05
Worklog Time Spent: 10m 
  Work Description: yukuai518 commented on pull request #2753: 
[GOBBLIN-899]Add config in replication config to determine wheter schema cehck 
ena…
URL: https://github.com/apache/incubator-gobblin/pull/2753#discussion_r332654552
 
 

 ##
 File path: 
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/replication/ReplicationConfiguration.java
 ##
 @@ -193,6 +201,11 @@ public Builder 
withEnforceFileSizeMatchFromConfigStore(Config config) {
   return this;
 }
 
+public Builder withSchemaCheckEnabled(Config config) {
+  this.schemaCheckEnabled = !config.hasPath(COPY_SCHEMA_CHECK_ENABLED) || 
config.getBoolean(COPY_SCHEMA_CHECK_ENABLED);
 
 Review comment:
   Consider to use ConfigUtils.getConfig(config, COPY_SCHEMA_CHECK_ENABLED, 
DEFAULT_COPY_SCHEMA_CHECK_ENABLED)
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 325227)
Time Spent: 1h  (was: 50m)

> Add a key in dataset config to disable schema check for a specific dataset
> --
>
> Key: GOBBLIN-899
> URL: https://issues.apache.org/jira/browse/GOBBLIN-899
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zihan Li
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-04 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=323663=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323663
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 04/Oct/19 19:34
Start Date: 04/Oct/19 19:34
Worklog Time Spent: 10m 
  Work Description: ZihanLi58 commented on issue #2753: [GOBBLIN-899]Add 
config in replication config to determine wheter schema cehck ena…
URL: 
https://github.com/apache/incubator-gobblin/pull/2753#issuecomment-538532663
 
 
   @yukuai518 Can you help take a look at this change? Thanks
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323663)
Time Spent: 0.5h  (was: 20m)

> Add a key in dataset config to disable schema check for a specific dataset
> --
>
> Key: GOBBLIN-899
> URL: https://issues.apache.org/jira/browse/GOBBLIN-899
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zihan Li
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-04 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=323661=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323661
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 04/Oct/19 19:32
Start Date: 04/Oct/19 19:32
Worklog Time Spent: 10m 
  Work Description: codecov-io commented on issue #2753: [GOBBLIN-899]Add 
config in replication config to determine wheter schema cehck ena…
URL: 
https://github.com/apache/incubator-gobblin/pull/2753#issuecomment-538532040
 
 
   # 
[Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2753?src=pr=h1)
 Report
   > Merging 
[#2753](https://codecov.io/gh/apache/incubator-gobblin/pull/2753?src=pr=desc)
 into 
[master](https://codecov.io/gh/apache/incubator-gobblin/commit/5b8af8cf6e932e8031ef060c79260a4a8f47974c?src=pr=desc)
 will **decrease** coverage by `0.01%`.
   > The diff coverage is `20%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/graphs/tree.svg?width=650=4MgURJ0bGc=150=pr)](https://codecov.io/gh/apache/incubator-gobblin/pull/2753?src=pr=tree)
   
   ```diff
   @@ Coverage Diff  @@
   ## master#2753  +/-   ##
   
   - Coverage 45.27%   45.26%   -0.02% 
   + Complexity 8828 8827   -1 
   
 Files  1889 1889  
 Lines 7063470646  +12 
 Branches   7757 7760   +3 
   
   - Hits  3197931977   -2 
   - Misses3570135715  +14 
 Partials   2954 2954
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2753?src=pr=tree) 
| Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | 
[.../FileAwareInputStreamExtractorWithCheckSchema.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvZXh0cmFjdG9yL0ZpbGVBd2FyZUlucHV0U3RyZWFtRXh0cmFjdG9yV2l0aENoZWNrU2NoZW1hLmphdmE=)
 | `60% <0%> (-15%)` | `4 <1> (ø)` | |
   | 
[...pache/gobblin/data/management/copy/CopySource.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvQ29weVNvdXJjZS5qYXZh)
 | `68.22% <0%> (-0.72%)` | `22 <0> (ø)` | |
   | 
[...anagement/copy/replication/ConfigBasedDataset.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcmVwbGljYXRpb24vQ29uZmlnQmFzZWREYXRhc2V0LmphdmE=)
 | `70.54% <0%> (-0.49%)` | `10 <0> (ø)` | |
   | 
[...ent/copy/replication/ReplicationConfiguration.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcmVwbGljYXRpb24vUmVwbGljYXRpb25Db25maWd1cmF0aW9uLmphdmE=)
 | `85.71% <60%> (-0.76%)` | `8 <0> (ø)` | |
   | 
[...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh)
 | `32.71% <0%> (-6.55%)` | `11% <0%> (-2%)` | |
   | 
[...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=)
 | `70% <0%> (-2.23%)` | `13% <0%> (ø)` | |
   | 
[.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==)
 | `64.81% <0%> (+0.92%)` | `28% <0%> (ø)` | :arrow_down: |
   | 
[.../apache/gobblin/runtime/api/JobExecutionState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0pvYkV4ZWN1dGlvblN0YXRlLmphdmE=)
 | `80.37% <0%> (+0.93%)` | `24% <0%> (ø)` | :arrow_down: |
   | 
[...lin/elasticsearch/writer/FutureCallbackHolder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2753/diff?src=pr=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tZWxhc3RpY3NlYXJjaC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9lbGFzdGljc2VhcmNoL3dyaXRlci9GdXR1cmVDYWxsYmFja0hvbGRlci5qYXZh)
 | `62.85% 

[jira] [Work logged] (GOBBLIN-899) Add a key in dataset config to disable schema check for a specific dataset

2019-10-03 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/GOBBLIN-899?focusedWorklogId=323135=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323135
 ]

ASF GitHub Bot logged work on GOBBLIN-899:
--

Author: ASF GitHub Bot
Created on: 04/Oct/19 01:05
Start Date: 04/Oct/19 01:05
Worklog Time Spent: 10m 
  Work Description: ZihanLi58 commented on pull request #2753: 
[GOBBLIN-899]Add config in replication config to determine wheter schema cehck 
ena…
URL: https://github.com/apache/incubator-gobblin/pull/2753
 
 
   …ble for the dataset
   
   Dear Gobblin maintainers,
   
   Please accept this PR. I understand that it will not be reviewed until I 
have checked off all the steps below!
   
   
   ### JIRA
   - [ ] My PR addresses the following [Gobblin 
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references 
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
   - https://issues.apache.org/jira/browse/GOBBLIN-899
   
   
   ### Description
   - [ ] Here are some details about my PR, including screenshots (if 
applicable):
   Add config in replication config to determine wheter schema cehck enable for 
the dataset
   
   ### Tests
   - [ ] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   Test on parallel pipeline
   
   ### Commits
   - [ ] My commits all reference JIRA issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
   1. Subject is separated from body by a blank line
   2. Subject is limited to 50 characters
   3. Subject does not end with a period
   4. Subject uses the imperative mood ("add", not "adding")
   5. Body wraps at 72 characters
   6. Body explains "what" and "why", not "how"
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 323135)
Remaining Estimate: 0h
Time Spent: 10m

> Add a key in dataset config to disable schema check for a specific dataset
> --
>
> Key: GOBBLIN-899
> URL: https://issues.apache.org/jira/browse/GOBBLIN-899
> Project: Apache Gobblin
>  Issue Type: Task
>Reporter: Zihan Li
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)