Alex, Thank you for the response.
I do see that all of the directories in question are owned by hive:hive, which makes sense, and indeed, writing to them from HCatWriter seems to be no problem. The problem arises in the org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer.constructPartition function when applyGroupAndPerms is called. This generates the "user X is not the owner of inode 'partition'" error. That's why I'm thinking HCatWriter simply cannot write partitions under Sentry. I guess a potential fix would be to have the constructPartition call check for appropriate permissions before blindly trying to set them. In my case, at least, the directory can be created no problem and has the hive:hive permissions the same as any other table directory. Nathan ________________________________ From: Alexander Kolbasov <ak...@cloudera.com> Sent: Monday, September 24, 2018 1:30:58 PM To: user@hive.apache.org Subject: Re: unable to write partitions with HCatWriter on Cloudera with Sentry and HDFS ACL plugin Nathan, when Sentry is enabled, all files are supposed to be owned by hive:hive or impala:impala and permissions are managed via HDFS ACLs which are coordinated with Sentry. - Alex On Mon, Sep 24, 2018 at 1:15 PM Nathan Bamford <nathan.bamf...@redpointglobal.com<mailto:nathan.bamf...@redpointglobal.com>> wrote: Hi, We use HCatWriter to write records to Hive, and I've recently run into a problem with HCatWriter that seems intractable. We can write tables without partitions all the live-long day, but any attempt to write to a partition results in the following error: "net/redpoint/hiveclient/DMHCatWriter.closeWriter:org.apache.hive.hcatalog.common.HCatException : 2004 : HCatOutputFormat not initialized, setOutput has to be called. Cause : org.apache.hive.hcatalog.common.HCatException : 2006 : Error adding partition to metastore. Cause : org.apache.hadoop.security.AccessControlException: Permission denied. user=nbamford is not the owner of inode=state=CO" Digging into the source for org.apache.hive.hcatalog.mapreduce.FileOutputCommitterContainer.constructPartition, I find the following lines: for (FieldSchema partKey : table.getPartitionKeys()) { if (i++ != 0) { fs.mkdirs(partPath); // Attempt to make the path in case it does not exist before we check applyGroupAndPerms(fs, partPath, perms, grpName, false); } partPath = constructPartialPartPath(partPath, partKey.getName().toLowerCase(), partKVs); } } The error is thrown from the applyGroupAndPerms function, which you will note does not check for the directory existing, and having the right permissions (in this case, it does). I am at a complete loss for how to proceed. I can't even think of a workaround. It seems to me HCatWriter simply cannot write partitions when Sentry and the HDFS ACL plugin are in force.
