[
https://issues.apache.org/jira/browse/GOBBLIN-968?focusedWorklogId=345670&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-345670
]
ASF GitHub Bot logged work on GOBBLIN-968:
------------------------------------------
Author: ASF GitHub Bot
Created on: 18/Nov/19 23:43
Start Date: 18/Nov/19 23:43
Worklog Time Spent: 10m
Work Description: codecov-io commented on issue #2816: [GOBBLIN-968]
Honor file split size for HadoopFileInputSource
URL:
https://github.com/apache/incubator-gobblin/pull/2816#issuecomment-555263057
#
[Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=h1)
Report
> Merging
[#2816](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=desc)
into
[master](https://codecov.io/gh/apache/incubator-gobblin/commit/e24bc26a28434bfee9ea4926e1b7d59901e63ec8?src=pr&el=desc)
will **decrease** coverage by `0.01%`.
> The diff coverage is `0%`.
[](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #2816 +/- ##
============================================
- Coverage 45.56% 45.55% -0.02%
+ Complexity 8954 8950 -4
============================================
Files 1900 1900
Lines 71213 71217 +4
Branches 7856 7858 +2
============================================
- Hits 32451 32440 -11
- Misses 35780 35790 +10
- Partials 2982 2987 +5
```
| [Impacted
Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=tree)
| Coverage Δ | Complexity Δ | |
|---|---|---|---|
|
[...source/extractor/hadoop/HadoopFileInputSource.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvaGFkb29wL0hhZG9vcEZpbGVJbnB1dFNvdXJjZS5qYXZh)
| `59.32% <0%> (-4.32%)` | `6 <0> (ø)` | |
|
[...ava/org/apache/gobblin/fsm/FiniteStateMachine.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ZzbS9GaW5pdGVTdGF0ZU1hY2hpbmUuamF2YQ==)
| `73.48% <0%> (-3.04%)` | `18% <0%> (-3%)` | |
|
[...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh)
| `32.71% <0%> (-2.81%)` | `11% <0%> (-1%)` | |
|
[.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==)
| `63.42% <0%> (-1.39%)` | `27% <0%> (-1%)` | |
|
[.../apache/gobblin/runtime/api/JobExecutionState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2816/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0pvYkV4ZWN1dGlvblN0YXRlLmphdmE=)
| `79.43% <0%> (-0.94%)` | `24% <0%> (ø)` | |
------
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=footer).
Last update
[e24bc26...993efd9](https://codecov.io/gh/apache/incubator-gobblin/pull/2816?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 345670)
Remaining Estimate: 23h 40m (was: 23h 50m)
Time Spent: 20m (was: 10m)
> Honor file split size for HadoopFileInputSource
> -----------------------------------------------
>
> Key: GOBBLIN-968
> URL: https://issues.apache.org/jira/browse/GOBBLIN-968
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-core
> Affects Versions: 0.14.0
> Reporter: Chirag Todarka
> Assignee: Abhishek Tiwari
> Priority: Major
> Labels: easyfix
> Original Estimate: 24h
> Time Spent: 20m
> Remaining Estimate: 23h 40m
>
> For HadoopFileInputSource it is splitting the source file only in default
> chunk size (which is 128MB) and not honoring the min and mix split size
> mentioned in pull file.
> For example even if below entires are present in pull file it will still
> chunk the source file in smaller 134217728 bytes files and not in 268435456
> bytes files.
> {{source.hadoop.file.input.split.minsize=268435456}}
> {{source.hadoop.file.input.split.maxsize=268435456}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
