Arina Ielchiieva created DRILL-5106:
---------------------------------------
Summary: Refactor SkipRecordsInspector to exclude check for
predefined file formats
Key: DRILL-5106
URL: https://issues.apache.org/jira/browse/DRILL-5106
Project: Apache Drill
Issue Type: Improvement
Components: Storage - Hive
Affects Versions: 1.9.0
Reporter: Arina Ielchiieva
Priority: Minor
After changes introduced in DRILL-4982, SkipRecordInspector is used only for
predefined formats (using hasHeaderFooter: false / true). But
SkipRecordInspector has its own check for formats where skip strategy can be
applied. Acceptable file formats are stored in private final Set<Object>
fileFormats and initialized in constructor, currently it contains only one
format - TextInputFormat. Now this check is redundant and may lead to ignoring
hasHeaderFooter setting true for any other format except of Text.
To do:
1. remove private final Set<Object> fileFormats
2. remove if block from SkipRecordsInspector.retrievePositiveIntProperty:
{code}
if
(!fileFormats.contains(tableProperties.get(hive_metastoreConstants.FILE_INPUT_FORMAT)))
{
return propertyIntValue;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
