> On Feb. 12, 2015, 3:07 p.m., kturner wrote: > > core/src/main/java/org/apache/accumulo/core/client/admin/SecurityOperations.java, > > line 360 > > <https://reviews.apache.org/r/30817/diff/5/?file=861176#file861176line360> > > > > Do you think this method would ever need options? Or does it need any > > options now? Just thinking that passing in a `DelegationTokenOptions` will > > make this new API much more flexible over time. Of course it does not make > > sense if there are no options to pass in now. > > Josh Elser wrote: > One option would be to further restrict the lifetime of the DT down from > the system config. For example, the system config would let the toke be valid > for N days, but you know you only need N hours. I think this is a good > addition which I hadn't previously considered. Thanks for mentioning it. > > kturner wrote: > That sounds like a useful option. What will you do if the timeout > specified for `getDelegationToken()` exceeds that of the system configured > timeout?
Throw an exception of course :) - Josh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30817/#review72162 ----------------------------------------------------------- On Feb. 11, 2015, 11:15 p.m., Josh Elser wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/30817/ > ----------------------------------------------------------- > > (Updated Feb. 11, 2015, 11:15 p.m.) > > > Review request for accumulo and Christopher Tubbs. > > > Bugs: ACCUMULO-3513 > https://issues.apache.org/jira/browse/ACCUMULO-3513 > > > Repository: accumulo > > > Description > ------- > > DelegationTokens are a way to enable MapReduce jobs (or any distributed task > which does not have direct access to the user's kerberos credentials) to > identify themself and communicate with Accumulo. This is implemented in the > same style of Hadoop which does this to provide HDFS and YARN access in YARN > jobs, as well as HBase (HBase also served as the base when I started coding). > > A DelegationToken (nothing more than a short-lived, on-demand password > generated by Accumulo) is provided to users who request one through the > SecurityOperations API. Secret keys are created, distributed and expired > within Accumulo itself (Master creates and expires keys, distributes via ZK > to tservers), which allow the master to create delegation tokens upon request > and tabletservers to validate delegation tokens provided as the > authentication mechanism by users. Tokens are valid for 7 days by default and > new secret keys are rolled every day (configurable). > > RPC is implemented SASL's DIGEST-MD5 mechanism (in addition to the GSSAPI > support we already have for "normal" kerberos -- thrift makes this really > easy). YARN provides a secure way to pass delegation tokens from the client > JVM to each YARN task (mapper/reducer) to prevent unwanted parties from > stealing the token (password) and using it for their own (nefarious) > purposes. The feature is transparent for users as long as they as using our > MapReduce JCommander options classes. > > User documentation has been added to the kerberos chapter. > > > Diffs > ----- > > core/src/main/java/org/apache/accumulo/core/Constants.java 0229d4e > core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java 216f32d > > core/src/main/java/org/apache/accumulo/core/cli/MapReduceClientOnDefaultTable.java > 0cf081f > > core/src/main/java/org/apache/accumulo/core/cli/MapReduceClientOnRequiredTable.java > 7719e92 > core/src/main/java/org/apache/accumulo/core/cli/MapReduceClientOpts.java > 4b3b7ed > > core/src/main/java/org/apache/accumulo/core/client/admin/SecurityOperations.java > efeafc0 > core/src/main/java/org/apache/accumulo/core/client/impl/ClientContext.java > 8470da4 > > core/src/main/java/org/apache/accumulo/core/client/impl/SecurityOperationsImpl.java > feb1ee7 > > core/src/main/java/org/apache/accumulo/core/client/impl/ThriftTransportKey.java > 072724b > > core/src/main/java/org/apache/accumulo/core/client/mapred/AbstractInputFormat.java > b83a024 > > core/src/main/java/org/apache/accumulo/core/client/mapred/AccumuloOutputFormat.java > f877ec6 > > core/src/main/java/org/apache/accumulo/core/client/mapreduce/AbstractInputFormat.java > 5c7b780 > > core/src/main/java/org/apache/accumulo/core/client/mapreduce/AccumuloOutputFormat.java > 5e0aa73 > > core/src/main/java/org/apache/accumulo/core/client/mapreduce/impl/DelegationTokenStub.java > PRE-CREATION > > core/src/main/java/org/apache/accumulo/core/client/mapreduce/lib/impl/ConfiguratorBase.java > b2b5150 > > core/src/main/java/org/apache/accumulo/core/client/mapreduce/lib/impl/InputConfigurator.java > 5405ac0 > > core/src/main/java/org/apache/accumulo/core/client/mock/MockSecurityOperations.java > db88cfb > > core/src/main/java/org/apache/accumulo/core/client/security/tokens/DelegationToken.java > PRE-CREATION > core/src/main/java/org/apache/accumulo/core/conf/Property.java 68fac73 > > core/src/main/java/org/apache/accumulo/core/master/thrift/MasterClientService.java > 4b90a34 > > core/src/main/java/org/apache/accumulo/core/rpc/SaslClientDigestCallbackHandler.java > PRE-CREATION > core/src/main/java/org/apache/accumulo/core/rpc/SaslConnectionParams.java > e067e23 > > core/src/main/java/org/apache/accumulo/core/rpc/SaslDigestCallbackHandler.java > PRE-CREATION > core/src/main/java/org/apache/accumulo/core/rpc/ThriftUtil.java d880fb3 > > core/src/main/java/org/apache/accumulo/core/security/AuthenticationTokenIdentifier.java > PRE-CREATION > core/src/main/java/org/apache/accumulo/core/security/SystemPermission.java > b998179 > > core/src/main/java/org/apache/accumulo/core/security/thrift/TAuthenticationKey.java > PRE-CREATION > > core/src/main/java/org/apache/accumulo/core/security/thrift/TAuthenticationTokenIdentifier.java > PRE-CREATION > > core/src/main/java/org/apache/accumulo/core/security/thrift/TDelegationToken.java > PRE-CREATION > core/src/main/java/org/apache/accumulo/core/util/ThriftMessageUtil.java > PRE-CREATION > core/src/main/thrift/master.thrift d89e381 > core/src/main/thrift/security.thrift 66235a8 > > core/src/test/java/org/apache/accumulo/core/client/impl/ThriftTransportKeyTest.java > 2723273 > > core/src/test/java/org/apache/accumulo/core/client/security/tokens/DelegationTokenTest.java > PRE-CREATION > > core/src/test/java/org/apache/accumulo/core/rpc/SaslClientDigestCallbackHandlerTest.java > PRE-CREATION > > core/src/test/java/org/apache/accumulo/core/rpc/SaslConnectionParamsTest.java > 3910f34 > > core/src/test/java/org/apache/accumulo/core/security/AuthenticationTokenIdentifierTest.java > PRE-CREATION > core/src/test/java/org/apache/accumulo/core/util/ThriftMessageUtilTest.java > PRE-CREATION > docs/src/main/asciidoc/chapters/kerberos.txt 05d7384 > fate/src/main/java/org/apache/accumulo/fate/zookeeper/IZooReader.java > 610b1bd > fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooReader.java > 5706cf3 > fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooUtil.java 805bfff > proxy/src/main/java/org/apache/accumulo/proxy/Proxy.java f9039be > > server/base/src/main/java/org/apache/accumulo/server/AccumuloServerContext.java > 84c3853 > > server/base/src/main/java/org/apache/accumulo/server/master/state/MetaDataStateStore.java > bf56a7a > > server/base/src/main/java/org/apache/accumulo/server/rpc/SaslServerConnectionParams.java > PRE-CREATION > > server/base/src/main/java/org/apache/accumulo/server/rpc/SaslServerDigestCallbackHandler.java > PRE-CREATION > > server/base/src/main/java/org/apache/accumulo/server/rpc/TCredentialsUpdatingInvocationHandler.java > f85505d > server/base/src/main/java/org/apache/accumulo/server/rpc/TServerUtils.java > f1f8963 > > server/base/src/main/java/org/apache/accumulo/server/rpc/UGIAssumingProcessor.java > ab106a6 > > server/base/src/main/java/org/apache/accumulo/server/security/AuditedSecurityOperation.java > cc7a7cd > > server/base/src/main/java/org/apache/accumulo/server/security/SecurityOperation.java > 7adb46e > > server/base/src/main/java/org/apache/accumulo/server/security/SystemCredentials.java > 51d50a1 > > server/base/src/main/java/org/apache/accumulo/server/security/delegation/AuthenticationKey.java > PRE-CREATION > > server/base/src/main/java/org/apache/accumulo/server/security/delegation/AuthenticationTokenKeyManager.java > PRE-CREATION > > server/base/src/main/java/org/apache/accumulo/server/security/delegation/AuthenticationTokenSecretManager.java > PRE-CREATION > > server/base/src/main/java/org/apache/accumulo/server/security/delegation/ZooAuthenticationKeyDistributor.java > PRE-CREATION > > server/base/src/main/java/org/apache/accumulo/server/security/delegation/ZooAuthenticationKeyWatcher.java > PRE-CREATION > > server/base/src/main/java/org/apache/accumulo/server/security/handler/KerberosAuthenticator.java > 08fa55b > > server/base/src/test/java/org/apache/accumulo/server/AccumuloServerContextTest.java > 49a60a6 > > server/base/src/test/java/org/apache/accumulo/server/rpc/SaslDigestCallbackHandlerTest.java > PRE-CREATION > > server/base/src/test/java/org/apache/accumulo/server/rpc/SaslServerConnectionParamsTest.java > PRE-CREATION > > server/base/src/test/java/org/apache/accumulo/server/security/delegation/AuthenticationKeyTest.java > PRE-CREATION > > server/base/src/test/java/org/apache/accumulo/server/security/delegation/AuthenticationTokenKeyManagerTest.java > PRE-CREATION > > server/base/src/test/java/org/apache/accumulo/server/security/delegation/AuthenticationTokenSecretManagerTest.java > PRE-CREATION > > server/base/src/test/java/org/apache/accumulo/server/security/delegation/ZooAuthenticationKeyDistributorTest.java > PRE-CREATION > > server/base/src/test/java/org/apache/accumulo/server/security/delegation/ZooAuthenticationKeyWatcherTest.java > PRE-CREATION > server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java > da0b07c > > server/gc/src/test/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogsTest.java > 1d7f90f > > server/gc/src/test/java/org/apache/accumulo/gc/SimpleGarbageCollectorTest.java > 6fcdd37 > > server/gc/src/test/java/org/apache/accumulo/gc/replication/CloseWriteAheadLogReferencesTest.java > 120692a > server/master/src/main/java/org/apache/accumulo/master/Master.java be476de > > server/master/src/main/java/org/apache/accumulo/master/MasterClientServiceHandler.java > 72cba26 > server/tserver/src/main/java/org/apache/accumulo/tserver/TabletServer.java > a5675dc > > test/src/main/java/org/apache/accumulo/test/continuous/ContinuousBatchWalker.java > a2687bb > > test/src/main/java/org/apache/accumulo/test/continuous/ContinuousIngest.java > dba6ac9 > test/src/main/java/org/apache/accumulo/test/continuous/ContinuousMoru.java > 4b5c3e7 > test/src/main/java/org/apache/accumulo/test/continuous/ContinuousOpts.java > PRE-CREATION > test/src/main/java/org/apache/accumulo/test/continuous/ContinuousQuery.java > 73048f6 > > test/src/main/java/org/apache/accumulo/test/continuous/ContinuousScanner.java > f68377a > test/src/main/java/org/apache/accumulo/test/continuous/ContinuousWalk.java > 60f8ec2 > test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java > 06b4303 > test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 9296548 > test/src/test/java/org/apache/accumulo/test/functional/KerberosIT.java > 3d48657 > > Diff: https://reviews.apache.org/r/30817/diff/ > > > Testing > ------- > > Many new unit tests. Additional integration tests. Standalone mapreduce jobs > (continuous verify and terasort ingest) on a fully secure cluster. > > > Thanks, > > Josh Elser > >
