[
https://issues.apache.org/jira/browse/GOBBLIN-968?focusedWorklogId=504796&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504796
]
ASF GitHub Bot logged work on GOBBLIN-968:
------------------------------------------
Author: ASF GitHub Bot
Created on: 26/Oct/20 16:34
Start Date: 26/Oct/20 16:34
Worklog Time Spent: 10m
Work Description: codecov-io edited a comment on pull request #2816:
URL:
https://github.com/apache/incubator-gobblin/pull/2816#issuecomment-555263057
#
[Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=h1)
Report
> Merging
[#2816](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=desc)
into
[master](https://codecov.io/gh/apache/incubator-gobblin/commit/a714c943bddfe49c19830444434aa64501294b2d?el=desc)
will **increase** coverage by `0.07%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #2816 +/- ##
============================================
+ Coverage 45.47% 45.55% +0.07%
- Complexity 8908 8950 +42
============================================
Files 1899 1900 +1
Lines 71096 71217 +121
Branches 7833 7858 +25
============================================
+ Hits 32332 32440 +108
- Misses 35784 35790 +6
- Partials 2980 2987 +7
```
| [Impacted
Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=tree)
| Coverage Δ | Complexity Δ | |
|---|---|---|---|
|
[...source/extractor/hadoop/HadoopFileInputSource.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvaGFkb29wL0hhZG9vcEZpbGVJbnB1dFNvdXJjZS5qYXZh)
| `59.32% <0.00%> (-4.32%)` | `6.00 <0.00> (ø)` | |
|
[...ava/org/apache/gobblin/fsm/FiniteStateMachine.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ZzbS9GaW5pdGVTdGF0ZU1hY2hpbmUuamF2YQ==)
| `73.48% <0.00%> (-3.04%)` | `18.00% <0.00%> (-3.00%)` | |
|
[...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh)
| `32.71% <0.00%> (-2.81%)` | `11.00% <0.00%> (-1.00%)` | |
|
[...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=)
| `70.00% <0.00%> (-2.23%)` | `13.00% <0.00%> (ø%)` | |
|
[.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==)
| `63.42% <0.00%> (-1.86%)` | `27.00% <0.00%> (-1.00%)` | |
|
[.../apache/gobblin/runtime/api/JobExecutionState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0pvYkV4ZWN1dGlvblN0YXRlLmphdmE=)
| `79.43% <0.00%> (-0.94%)` | `24.00% <0.00%> (ø%)` | |
|
[...ement/copy/replication/SourceHadoopFsEndPoint.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcmVwbGljYXRpb24vU291cmNlSGFkb29wRnNFbmRQb2ludC5qYXZh)
| `19.04% <0.00%> (-0.63%)` | `6.00% <0.00%> (ø%)` | |
|
[...main/java/org/apache/gobblin/util/HadoopUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvSGFkb29wVXRpbHMuamF2YQ==)
| `30.20% <0.00%> (-0.34%)` | `24.00% <0.00%> (ø%)` | |
|
[...n/java/org/apache/gobblin/runtime/api/JobSpec.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0pvYlNwZWMuamF2YQ==)
| `80.13% <0.00%> (-0.01%)` | `17.00% <0.00%> (ø%)` | |
|
[...verter/avro/AvroRecursionEliminatingConverter.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9hdnJvL0F2cm9SZWN1cnNpb25FbGltaW5hdGluZ0NvbnZlcnRlci5qYXZh)
| `75.00% <0.00%> (ø)` | `5.00% <0.00%> (?%)` | |
| ... and [9
more](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree-more)
| |
------
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=footer).
Last update
[a714c94...993efd9](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 504796)
Remaining Estimate: 23h 20m (was: 23.5h)
Time Spent: 40m (was: 0.5h)
> Honor file split size for HadoopFileInputSource
> -----------------------------------------------
>
> Key: GOBBLIN-968
> URL: https://issues.apache.org/jira/browse/GOBBLIN-968
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-core
> Affects Versions: 0.14.0
> Reporter: Chirag Todarka
> Assignee: Abhishek Tiwari
> Priority: Major
> Labels: easyfix
> Original Estimate: 24h
> Time Spent: 40m
> Remaining Estimate: 23h 20m
>
> For HadoopFileInputSource it is splitting the source file only in default
> chunk size (which is 128MB) and not honoring the min and mix split size
> mentioned in pull file.
> For example even if below entires are present in pull file it will still
> chunk the source file in smaller 134217728 bytes files and not in 268435456
> bytes files.
> {{source.hadoop.file.input.split.minsize=268435456}}
> {{source.hadoop.file.input.split.maxsize=268435456}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
