[
https://issues.apache.org/jira/browse/HIVE-22980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056825#comment-17056825
]
Hive QA commented on HIVE-22980:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996315/HIVE-22980.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 18097 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestScheduledReplicationScenarios.testAcidTablesReplLoadBootstrapIncr
(batchId=270)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21064/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21064/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21064/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996315 - PreCommit-HIVE-Build
> Support custom path filter for ORC tables
> -----------------------------------------
>
> Key: HIVE-22980
> URL: https://issues.apache.org/jira/browse/HIVE-22980
> Project: Hive
> Issue Type: New Feature
> Components: ORC
> Reporter: Oleksiy Sayankin
> Assignee: Oleksiy Sayankin
> Priority: Major
> Attachments: HIVE-22980.1.patch, HIVE-22980.2.patch,
> HIVE-22980.3.patch
>
>
> The customer is looking for an option to specify custom path filter for ORC
> tables. Please find the details below from customer requirement.
> Problem Statement/Approach in customer words :
> {quote}
> Currently, Orc file input format does not take in path filters set in the
> property "mapreduce.input.pathfilter.class" OR "
> mapred.input.pathfilter.class ". So, we cannot use custom filters with Orc
> files.
> AcidUtils class has a static filter called "hiddenFilters" which is used by
> ORC to filter input paths. If we can pass the custom filter classes(set in
> the property mentioned above) to AcidUtils and replace hiddenFilter with a
> filter that does an "and" operation over hiddenFilter+customFilters, the
> filters would work well.
> On local testing, mapreduce.input.pathfilter.class seems to be working for
> Text tables but not for ORC tables.
> {quote}
> Our analysis:
> {{OrcInputFormat}} and {{FileInputFormat}} are different implementations for
> {{Inputformat}} interface. Property "{{mapreduce.input.pathfilter.class}}" is
> only respected by {{FileInputFormat}}, but not by any other implementations
> of {{InputFormat}}. The customer wants to have the ability to filter out rows
> based on path/filenames, current ORC features like bloomfilters and indexes
> are not good enough for them to minimize number of disk read operations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
