Chirag Todarka created GOBBLIN-968:
--------------------------------------
Summary: Honor file split size for HadoopFileInputSource
Key: GOBBLIN-968
URL: https://issues.apache.org/jira/browse/GOBBLIN-968
Project: Apache Gobblin
Issue Type: Bug
Components: gobblin-core
Affects Versions: 0.14.0
Reporter: Chirag Todarka
Assignee: Abhishek Tiwari
For HadoopFileInputSource it is splitting the source file only in default chunk
size (which is 128MB) and not honoring the min and mix split size mentioned in
pull file.
For example even if below entires are present in pull file it will still chunk
the source file in smaller 134217728 bytes files and not in 268435456 bytes
files.
{{source.hadoop.file.input.split.minsize=268435456}}
{{source.hadoop.file.input.split.maxsize=268435456}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
