[ https://issues.apache.org/jira/browse/HIVE-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954324#comment-14954324 ]
niklaus xiao commented on HIVE-12154: ------------------------------------- Use {quote} fs.globStatus(pattern); {quote} instead of {quote} fs.listStatus(path); {quote} Attached the initial patch. > Load data inpath 'PATTERN' into table should only check files match the > PATTERN > ------------------------------------------------------------------------------- > > Key: HIVE-12154 > URL: https://issues.apache.org/jira/browse/HIVE-12154 > Project: Hive > Issue Type: Bug > Components: SQLStandardAuthorization > Affects Versions: 0.13.1, 1.0.0, 1.2.0, 1.1.0, 1.2.1 > Reporter: niklaus xiao > Priority: Minor > Attachments: HIVE-12154.1.patch > > > We are using flume to sink data to HDFS directory '/tmp/test/', temporal > files that flume actively writes into has a suffix .tmp, after writes finish, > the file will be renamed to SAMPLE.data. > Hive periodic task execute script like > {quote} > load data inpath '/tmp/test/*.data' into table t1; > {quote} > This exception happens sometimes > {quote} > 2015-10-12 19:38:00,133 | ERROR | HiveServer2-Handler-Pool: Thread-57 | > FAILED: HiveAuthzPluginException Error getting permissions for > hdfs://hacluster/tmp/test/*.data: null > org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: > Error getting permissions for hdfs://hacluster/tmp/test/*.data: null > ... > Caused by: java.io.FileNotFoundException: Path not found > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAccess(FSNamesystem.java:8175) > {quote} > I digged into the code, and found that SQLStdHiveAuthorizationValidator > checks all the files in /tmp/test/ directory, but when checks the permission > of .tmp file, the file is renamed to .data, hdfs cannot find this file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)