[jira] [Commented] (HADOOP-12328) Hadoop/MapReduce ignores hidden (dot) files

[Ajith S (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HADOOP-12328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699061#comment-14699061
 ] 

Ajith S commented on HADOOP-12328:
----------------------------------

Hi

. are hidden files internal to system and by default, in 
org.apache.hadoop.mapred.FileInputFormat<K, V> we have a hiddenFileFilter

{code}
  private static final PathFilter hiddenFileFilter = new PathFilter(){
      public boolean accept(Path p){
        String name = p.getName(); 
        return !name.startsWith("_") && !name.startsWith("."); 
      }
    };
{code}

so in case you use TextInputFormat, SequenceFileInputFormat etc, its expected 
to ignore it
If you want to list it for some reason, you can override the 
org.apache.hadoop.mapred.FileInputFormat.listStatus(JobConf) in your custom 
format class

> Hadoop/MapReduce ignores hidden (dot) files
> -------------------------------------------
>
>                 Key: HADOOP-12328
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12328
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Naga
>
> For some reason Hadoop/MapReduce does not pick up hidden (dot) files for 
> processing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)