> On April 23, 2015, 7:34 p.m., Rohini Palaniswamy wrote: > > core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java, line > > 81 > > <https://reviews.apache.org/r/33400/diff/1/?file=938254#file938254line81> > > > > Please get rid of this check. You have already done the globbing. To > > throw an error after that is meaningless. NN and DistributedFileSystem are > > already built to handle this by fetching in batches. Need to remove this > > check everywhere - https://issues.apache.org/jira/browse/OOZIE-1682 .
Actually more efficient one is to go with the PathFilter suggestion by Puru. Throw some specific exception in the accept() method and catch the exception and return true. You don't have to do any limit check. Even if you get one Path that means it exists and you need to return ture. If it did not go into catch block, you return false. This would minimize the calls to namenode to one listStatus call for first pattern. It would be good to avoid the multiple batch calls fs.listStatus (DistributedFileSystem.listStatusInternal) makes to NN in case of big list of paths, but that is not possible. Filed https://issues.apache.org/jira/browse/HDFS-8234 to fix it in hdfs code. - Rohini ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33400/#review81348 ----------------------------------------------------------- On April 21, 2015, 5:17 p.m., Ryota Egashira wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33400/ > ----------------------------------------------------------- > > (Updated April 21, 2015, 5:17 p.m.) > > > Review request for oozie. > > > Repository: oozie-git > > > Description > ------- > > https://issues.apache.org/jira/browse/OOZIE-2215 > > > Diffs > ----- > > core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java > f8af61c > core/src/test/java/org/apache/oozie/action/hadoop/TestFsELFunctions.java > c81fc61 > docs/src/site/twiki/WorkflowFunctionalSpec.twiki e3790a4 > > Diff: https://reviews.apache.org/r/33400/diff/ > > > Testing > ------- > > > Thanks, > > Ryota Egashira > >
