[jira] [Created] (HIVE-16806) Utilities isEmptyPath Loads All Files

[BELUGA BEHR (JIRA)]

BELUGA BEHR created HIVE-16806:
----------------------------------

             Summary: Utilities isEmptyPath Loads All Files
                 Key: HIVE-16806
                 URL: https://issues.apache.org/jira/browse/HIVE-16806
             Project: Hive
          Issue Type: Improvement
    Affects Versions: 2.1.1, 3.0.0
            Reporter: BELUGA BEHR
            Priority: Minor


{code:title=org.apache.hadoop.hive.ql.exec.Utilities.isEmptyPath(Configuration, 
Path)}
  public static boolean isEmptyPath(Configuration job, Path dirPath) throws 
IOException {
    FileSystem inpFs = dirPath.getFileSystem(job);
    try {
      FileStatus[] fStats = inpFs.listStatus(dirPath, 
FileUtils.HIDDEN_FILES_PATH_FILTER);
      if (fStats.length > 0) {
        return false;
      }
    } catch(FileNotFoundException fnf) {
      return true;
    }
    return true;
  }
{code}

You can see here that the code is loading every instance of {{FileStatus}} even 
though all we care about here is if there are any.  I propose adding a new 
filter which stops collecting files into this array once it has found at least 
one. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)