[
https://issues.apache.org/jira/browse/CRUNCH-331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13889803#comment-13889803
]
Micah Whitacre commented on CRUNCH-331:
---------------------------------------
I'm personally leaning towards disabling for the formatted files explicitly.
The Parquet source is already disabling itself and that seems like a good
policy to reduce "complexity" or magic in Crunch.
> Change default settings for CombineFileInputFormat
> --------------------------------------------------
>
> Key: CRUNCH-331
> URL: https://issues.apache.org/jira/browse/CRUNCH-331
> Project: Crunch
> Issue Type: Bug
> Components: IO
> Affects Versions: 0.9.0, 0.8.2
> Reporter: Josh Wills
>
> Currently, we default to enabling the CombineFileInputFormat settings for any
> extensions of FileSourceImpl b/c it tends to improve performance for common
> file formats like text, sequence files, and Avro files. However, this default
> has caused problems for formats like Parquet and for custom file formats that
> have complex split logic.
> This JIRA is to track modifying the default combine file settings in at least
> some contexts, such as with From.formattedFile for custom input formats.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
