[
https://issues.apache.org/jira/browse/GOBBLIN-968?focusedWorklogId=522516&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-522516
]
ASF GitHub Bot logged work on GOBBLIN-968:
------------------------------------------
Author: ASF GitHub Bot
Created on: 10/Dec/20 03:08
Start Date: 10/Dec/20 03:08
Worklog Time Spent: 10m
Work Description: asfgit closed pull request #2816:
URL: https://github.com/apache/incubator-gobblin/pull/2816
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 522516)
Remaining Estimate: 23h 10m (was: 23h 20m)
Time Spent: 50m (was: 40m)
> Honor file split size for HadoopFileInputSource
> -----------------------------------------------
>
> Key: GOBBLIN-968
> URL: https://issues.apache.org/jira/browse/GOBBLIN-968
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-core
> Affects Versions: 0.14.0
> Reporter: Chirag Todarka
> Assignee: Abhishek Tiwari
> Priority: Major
> Labels: easyfix
> Original Estimate: 24h
> Time Spent: 50m
> Remaining Estimate: 23h 10m
>
> For HadoopFileInputSource it is splitting the source file only in default
> chunk size (which is 128MB) and not honoring the min and mix split size
> mentioned in pull file.
> For example even if below entires are present in pull file it will still
> chunk the source file in smaller 134217728 bytes files and not in 268435456
> bytes files.
> {{source.hadoop.file.input.split.minsize=268435456}}
> {{source.hadoop.file.input.split.maxsize=268435456}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
