[
https://issues.apache.org/jira/browse/FLINK-8243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336365#comment-16336365
]
ASF GitHub Bot commented on FLINK-8243:
---------------------------------------
GitHub user fhueske opened a pull request:
https://github.com/apache/flink/pull/5344
[FLINK-8243] [orc] OrcTableSource reads input path recursively by default.
## What is the purpose of the change
- enable recursive directory enumeration for `OrcTableSource` (Hive's
default behavior)
- expose the configuration in the `OrcTableSource` builder.
## Brief change log
* see purpose above
## Verifying this change
* Unit test to validate configuration of `FileInputFormat` has been added
in `OrcTableSourceTest.testBuilder()`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): **no**
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: **no**
- The serializers: **no**
- The runtime per-record code paths (performance sensitive): **no**
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
- The S3 file system connector: **no**
## Documentation
- Does this pull request introduce a new feature? **yes**
- If yes, how is the feature documented? Java docs of builder + web
documentation.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fhueske/flink orcRecursive
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5344.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5344
----
commit e8ff9948d16bf4f0884644ba123f067e697e5a73
Author: Fabian Hueske <fhueske@...>
Date: 2017-12-12T11:18:23Z
[FLINK-8243] [orc] OrcTableSource reads input path recursively by default.
----
> OrcTableSource should recursively read all files in nested directories of the
> input path.
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-8243
> URL: https://issues.apache.org/jira/browse/FLINK-8243
> Project: Flink
> Issue Type: Improvement
> Components: Batch Connectors and Input/Output Formats
> Affects Versions: 1.4.0
> Reporter: Fabian Hueske
> Assignee: Fabian Hueske
> Priority: Critical
> Fix For: 1.5.0, 1.4.1
>
>
> The {{OrcTableSource}} only reads files on the first level of the provided
> input path.
> Hive's default behavior is to recursively read all nested files in the input
> path. We should follow this behavior and add a switch to disable recursive
> reading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
