Somes to me you encountered this bug? HDFS-10481
<https://issues.apache.org/jira/browse/HDFS-10481>
If you’re using CDH, this is fixed in CDH5.5.5, CDH5.7.2 and CDH5.8.2
Wei-Chiu Chuang
A very happy Clouderan
> On Oct 11, 2016, at 8:38 AM, Benjamin Ross <br...@lattice-engines.com> wrote:
>
> All,
> I'm trying to use httpfs to write to an encryption zone with security off. I
> can read from an encryption zone, but I can't write to one.
>
> Here's the applicable namenode logs. httpfs and root both have all possible
> privileges in the KMS. What am I missing?
>
>
> 2016-10-07 15:48:16,164 DEBUG ipc.Server
> (Server.java:authorizeConnection(2095)) - Successfully authorized userInfo {
> effectiveUser: "root"
> realUser: "httpfs"
> }
> protocol: "org.apache.hadoop.hdfs.protocol.ClientProtocol"
>
> 2016-10-07 15:48:16,164 DEBUG ipc.Server (Server.java:processOneRpc(1902)) -
> got #2
> 2016-10-07 15:48:16,164 DEBUG ipc.Server (Server.java:run(2179)) - IPC Server
> handler 9 on 8020: org.apache.hadoop.hdfs.protocol.ClientProtocol.create from
> 10.41.1.64:47622 Call#2 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER
> 2016-10-07 15:48:16,165 DEBUG security.UserGroupInformation
> (UserGroupInformation.java:logPrivilegedAction(1751)) - PrivilegedAction
> as:root (auth:PROXY) via httpfs (auth:SIMPLE)
> from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2205)
> 2016-10-07 15:48:16,166 DEBUG hdfs.StateChange
> (NameNodeRpcServer.java:create(699)) - *DIR* NameNode.create: file
> /tmp/cryptotest/hairyballs for DFSClient_NONMAPREDUCE_-1005188439_28 at
> 10.41.1.64
> 2016-10-07 15:48:16,166 DEBUG hdfs.StateChange
> (FSNamesystem.java:startFileInt(2411)) - DIR* NameSystem.startFile:
> src=/tmp/cryptotest/hairyballs, holder=DFSClient_NONMAPREDUCE_-1005188439_28,
> clientMachine=10.41.1.64, createParent=true, replication=3, createFlag=[CREATE
> , OVERWRITE], blockSize=134217728,
> supportedVersions=[CryptoProtocolVersion{description='Encryption zones',
> version=2, unknownValue=null}]
> 2016-10-07 15:48:16,167 DEBUG security.UserGroupInformation
> (UserGroupInformation.java:logPrivilegedAction(1751)) - PrivilegedAction
> as:hdfs (auth:SIMPLE)
> from:org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:484)
> 2016-10-07 15:48:16,171 DEBUG client.KerberosAuthenticator
> (KerberosAuthenticator.java:authenticate(205)) - Using fallback authenticator
> sequence.
> 2016-10-07 15:48:16,176 DEBUG security.UserGroupInformation
> (UserGroupInformation.java:doAs(1728)) - PrivilegedActionException as:hdfs
> (auth:SIMPLE)
> cause:org.apache.hadoop.security.authentication.client.AuthenticationException:
> Authentication failed, status: 403, messag
> e: Forbidden
> 2016-10-07 15:48:16,176 DEBUG ipc.Server (ProtobufRpcEngine.java:call(631)) -
> Served: create queueTime= 2 procesingTime= 10 exception= IOException
> 2016-10-07 15:48:16,177 DEBUG security.UserGroupInformation
> (UserGroupInformation.java:doAs(1728)) - PrivilegedActionException as:root
> (auth:PROXY) via httpfs (auth:SIMPLE) cause:java.io.IOException:
> java.util.concurrent.ExecutionException: java.io.IOException: org.apach
> e.hadoop.security.authentication.client.AuthenticationException:
> Authentication failed, status: 403, message: Forbidden
> 2016-10-07 15:48:16,177 INFO ipc.Server (Server.java:logException(2299)) -
> IPC Server handler 9 on 8020, call
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create from 10.41.1.64:47622
> Call#2 Retry#0
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException:
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> Authentication failed, status: 403, message: Forbidden
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.generateEncryptedKey(KMSClientProvider.java:750)
> at
> org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:371)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.generateEncryptedDataEncryptionKey(FSNamesystem.java:2352)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2478)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2377)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:716)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:405)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2211)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2207)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2205)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOException:
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> Authentication failed, status: 403, message: Forbidden
> at
> com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:289)
> at
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:276)
> at
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111)
> at
> com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:132)
> at
> com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2381)
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2351)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
> at
> com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
> at
> com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
> at
> org.apache.hadoop.crypto.key.kms.ValueQueue.getAtMost(ValueQueue.java:266)
> at
> org.apache.hadoop.crypto.key.kms.ValueQueue.getNext(ValueQueue.java:226)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.generateEncryptedKey(KMSClientProvider.java:745)
> ... 15 more
> Caused by: java.io.IOException:
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> Authentication failed, status: 403, message: Forbidden
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:495)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.access$100(KMSClientProvider.java:84)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider$EncryptedQueueRefiller.fillQueueForKey(KMSClientProvider.java:133)
> at
> org.apache.hadoop.crypto.key.kms.ValueQueue$1.load(ValueQueue.java:181)
> at
> org.apache.hadoop.crypto.key.kms.ValueQueue$1.load(ValueQueue.java:175)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
> ... 23 more
> Caused by:
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> Authentication failed, status: 403, message: Forbidden
> at
> org.apache.hadoop.security.authentication.client.AuthenticatedURL.extractToken(AuthenticatedURL.java:274)
>
>
>
> This message has been scanned for malware by Websense. www.websense.com
> <http://www.websense.com/>
