Just an anecdote from someone who has been bitten by mock more than a couple times. I would try to deprecate it in 1.8 and remove in 2.0 if at all possible. People really shouldn't write tests against it.
> On Jun 8, 2015, at 10:10 PM, Sean Busbey <[email protected]> wrote: > > Josh's comment below made me realize we still haven't formally deprecated > MockAccumulo. > > What do folks think about doing it soon-ish with an aim of removing it in > Accumulo 3.0? (that's version three, so that it can remain deprecated for > all of version 2). > > > -Sean > >> On Sun, Jun 7, 2015 at 12:37 PM, Josh Elser <[email protected]> wrote: >> >> MiniAccumulo, yes. MockAccumulo, no. In general, we've near completely >> moved away from MockAccumulo. I wouldn't be surprised if it gets deprecated >> and removed soon. >> >> >> https://github.com/apache/accumulo/blob/1.7/test/src/test/java/org/apache/accumulo/test/functional/KerberosIT.java >> >> Apache Directory provides a MiniKdc that can be used easily w/ >> MiniAccumulo. Many of the integration tests have already been altered to >> support running w/ or w/o kerberos. >> >> James Hughes wrote: >> >>> Hi all, >>> >>> For GeoMesa, stats writing is quite secondary and optional, so we can >>> sort that out as a follow-on to seeing GeoMesa work with Accumulo 1.7. >>> >>> I haven't had a chance to read in details yet, so forgive me if this is >>> covered in the docs. Does either Mock or MiniAccumulo provide enough >>> hooks to test out Kerberos integration effectively? I suppose I'm >>> really asking what kind of testing environment a project like GeoMesa >>> would need to use to test out Accumulo 1.7. >>> >>> Even though MockAccumulo has a number of limitations, it is rather >>> effective in unit tests which can be part of a quick build. >>> >>> Thanks, >>> >>> Jim >>> >>> On Sat, Jun 6, 2015 at 11:14 PM, Xu (Simon) Chen <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Nope, I am running the example as what the readme file suggested: >>> >>> java -cp ./target/geomesa-quickstart-1.0-SNAPSHOT.jar >>> org.geomesa.QuickStart -instanceId somecloud -zookeepers >>> "zoo1:2181,zoo2:2181,zoo3:2181" -user someuser -password somepwd >>> -tableName sometable >>> >>> I'll raise this question with the geomesa folks, but you're right that >>> I can ignore it for now... >>> >>> Thanks! >>> -Simon >>> >>> >>> On Sat, Jun 6, 2015 at 11:00 PM, Josh Elser <[email protected] >>> <mailto:[email protected]>> wrote: >>>> Are you running it via `mvn exec:java` by chance or netbeans? >>> >>> http://mail-archives.apache.org/mod_mbox/accumulo-user/201411.mbox/%[email protected]%3E >>>> >>>> If that's just a background thread writing in Stats, it might >>> just be a >>>> factor of how you're invoking the program and you can ignore it. >>> I don't >>>> know enough about the inner-workings of GeoMesa to say one way or >>> the other. >>>> >>>> >>>> Xu (Simon) Chen wrote: >>>>> >>>>> Josh, >>>>> >>>>> Everything works well, except for one thing :-) >>>>> >>>>> I am running geomesa-quickstart program that ingest some data >>> and then >>>>> perform a simple query: >>>>> https://github.com/geomesa/geomesa-quickstart >>>>> >>>>> For some reason, the following error is emitted consistently at >>> the >>>>> end of the execution, after outputting the correct result: >>>>> 15/06/07 00:29:22 INFO zookeeper.ZooCache: Zookeeper error, will >>> retry >>>>> java.lang.InterruptedException >>>>> at java.lang.Object.wait(Native Method) >>>>> at java.lang.Object.wait(Object.java:503) >>>>> at >>> org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342) >>>>> at >>> org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1036) >>>>> at >>> org.apache.accumulo.fate.zookeeper.ZooCache$2.run(ZooCache.java:264) >>>>> at >>> org.apache.accumulo.fate.zookeeper.ZooCache.retry(ZooCache.java:162) >>>>> at >>>>> org.apache.accumulo.fate.zookeeper.ZooCache.get(ZooCache.java:289) >>>>> at >>>>> org.apache.accumulo.fate.zookeeper.ZooCache.get(ZooCache.java:238) >>>>> at >>> >>> org.apache.accumulo.core.client.impl.Tables.getTableState(Tables.java:180) >>>>> at >>> >>> org.apache.accumulo.core.client.impl.ConnectorImpl.getTableId(ConnectorImpl.java:82) >>>>> at >>> >>> org.apache.accumulo.core.client.impl.ConnectorImpl.createBatchWriter(ConnectorImpl.java:128) >>>>> at >>> >>> org.locationtech.geomesa.core.stats.StatWriter$$anonfun$write$2.apply(StatWriter.scala:174) >>>>> at >>> >>> org.locationtech.geomesa.core.stats.StatWriter$$anonfun$write$2.apply(StatWriter.scala:156) >>>>> at >>> scala.collection.immutable.Map$Map1.foreach(Map.scala:109) >>>>> at >>> >>> org.locationtech.geomesa.core.stats.StatWriter$.write(StatWriter.scala:156) >>>>> at >>> >>> org.locationtech.geomesa.core.stats.StatWriter$.drainQueue(StatWriter.scala:148) >>>>> at >>> >>> org.locationtech.geomesa.core.stats.StatWriter$.run(StatWriter.scala:116) >>>>> at >>> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>>>> at >>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) >>>>> at >>> >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) >>>>> at >>> >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>>> at >>> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>> at >>> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>> at java.lang.Thread.run(Thread.java:745) >>>>> >>>>> This is more annoying than a real problem. I am new to both >>> accumulo >>>>> and geomesa, but I am curious what the problem might be. >>>>> >>>>> Thanks! >>>>> -Simon >>>>> >>>>> >>>>> On Sat, Jun 6, 2015 at 8:01 PM, Josh Elser<[email protected] >>> <mailto:[email protected]>> wrote: >>>>>> >>>>>> Great! Glad to hear it. Please let us know how it works out! >>>>>> >>>>>> >>>>>> Xu (Simon) Chen wrote: >>>>>>> >>>>>>> Josh, >>>>>>> >>>>>>> You're right again.. Thanks! >>>>>>> >>>>>>> My ansible play actually pushed client.conf to all the server >>> config >>>>>>> directories, but didn't do anything for the clients, and that's >>> my >>>>>>> problem. Now kerberos is working great for me. >>>>>>> >>>>>>> Thanks again! >>>>>>> -Simon >>>>>>> >>>>>>> On Sat, Jun 6, 2015 at 5:04 PM, Josh >>> Elser<[email protected] <mailto:[email protected]>> >>> >>>>>>> wrote: >>>>>>>> >>>>>>>> Simon, >>>>>>>> >>>>>>>> Did you create a client configuration file (~/.accumulo/config >>> or >>>>>>>> $ACCUMULO_CONF_DIR/client.conf)? You need to configure >>> Accumulo clients >>>>>>>> to >>>>>>>> actually use SASL when you're trying to use Kerberos >>> authentication. >>>>>>>> Your >>>>>>>> server is expecting that, but I would venture a guess that >>> your client >>>>>>>> isn't. >>>>>>>> >>>>>>>> See >>> >>> http://accumulo.apache.org/1.7/accumulo_user_manual.html#_configuration_3 >>>>>>>> >>>>>>>> >>>>>>>> Xu (Simon) Chen wrote: >>>>>>>>> >>>>>>>>> Josh, >>>>>>>>> >>>>>>>>> Thanks. It makes sense... >>>>>>>>> >>>>>>>>> I used a KerberosToken, but my program got stuck when >>> running the >>>>>>>>> following: >>>>>>>>> new ZooKeeperInstance(instance, zookeepers).getConnector(user, >>>>>>>>> krbToken) >>>>>>>>> >>>>>>>>> It looks like my client is stuck here: >>> >>> https://github.com/apache/accumulo/blob/master/core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java#L70 >>>>>>>>> failing in the receive part of >>> >>> org.apache.accumulo.core.client.impl.thrift.ClientService.Client.authenticate(). >>>>>>>>> >>>>>>>>> On my tservers, I see the following: >>>>>>>>> >>>>>>>>> 2015-06-06 18:58:19,616 [server.TThreadPoolServer] ERROR: >>> Error >>>>>>>>> occurred during processing of message. >>>>>>>>> java.lang.RuntimeException: >>>>>>>>> org.apache.thrift.transport.TTransportException: >>>>>>>>> java.net.SocketTimeoutException: Read timed out >>>>>>>>> at >>> >>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) >>>>>>>>> at >>> >>> org.apache.accumulo.core.rpc.UGIAssumingTransportFactory$1.run(UGIAssumingTransportFactory.java:51) >>>>>>>>> at >>> >>> org.apache.accumulo.core.rpc.UGIAssumingTransportFactory$1.run(UGIAssumingTransportFactory.java:48) >>>>>>>>> at >>> java.security.AccessController.doPrivileged(Native >>>>>>>>> Method) >>>>>>>>> at >>> javax.security.auth.Subject.doAs(Subject.java:356) >>>>>>>>> at >>> >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1622) >>>>>>>>> at >>> >>> org.apache.accumulo.core.rpc.UGIAssumingTransportFactory.getTransport(UGIAssumingTransportFactory.java:48) >>>>>>>>> at >>> >>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:208) >>>>>>>>> at >>> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>> at >>> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>> at >>> >>> org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35) >>>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>>> Caused by: org.apache.thrift.transport.TTransportException: >>>>>>>>> java.net.SocketTimeoutException: Read timed out >>>>>>>>> at >>> >>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129) >>>>>>>>> at >>> org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) >>>>>>>>> at >>> >>> org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182) >>>>>>>>> at >>> >>> org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125) >>>>>>>>> at >>> >>> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253) >>>>>>>>> at >>> >>> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) >>>>>>>>> at >>> >>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) >>>>>>>>> ... 11 more >>>>>>>>> Caused by: java.net.SocketTimeoutException: Read timed out >>>>>>>>> at java.net.SocketInputStream.socketRead0(Native >>> Method) >>>>>>>>> at >>>>>>>>> java.net.SocketInputStream.read(SocketInputStream.java:152) >>>>>>>>> at >>>>>>>>> java.net.SocketInputStream.read(SocketInputStream.java:122) >>>>>>>>> at >>> java.io.BufferedInputStream.read1(BufferedInputStream.java:273) >>>>>>>>> at >>>>>>>>> java.io.BufferedInputStream.read(BufferedInputStream.java:334) >>>>>>>>> at >>> >>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) >>>>>>>>> ... 17 more >>>>>>>>> >>>>>>>>> Any ideas why? >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> -Simon >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Jun 6, 2015 at 2:19 PM, Josh >>> Elser<[email protected] <mailto:[email protected]>> >>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Make sure you read the JavaDoc on DelegationToken: >>>>>>>>>> >>>>>>>>>> <snip> >>>>>>>>>> Obtain a delegation token by calling {@link >>> >>> SecurityOperations#getDelegationToken(org.apache.accumulo.core.client.admin.DelegationTokenConfig)} >>>>>>>>>> </snip> >>>>>>>>>> >>>>>>>>>> You cannot create a usable DelegationToken as the client >>> itself. >>>>>>>>>> >>>>>>>>>> Anyways, DelegationTokens are only relevant in cases where >>> the client >>>>>>>>>> Kerberos credentials are unavailable. The most common case >>> is running >>>>>>>>>> MapReduce jobs. If you are just interacting with Accumulo >>> through the >>>>>>>>>> Java >>>>>>>>>> API, the KerberosToken is all you need to use. >>>>>>>>>> >>>>>>>>>> The user-manual likely just needs to be updated. I believe >>> the >>>>>>>>>> DelegationTokenConfig was added after I wrote the initial >>>>>>>>>> documentation. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Xu (Simon) Chen wrote: >>>>>>>>>>> >>>>>>>>>>> Hi folks, >>>>>>>>>>> >>>>>>>>>>> The latest kerberos doc seems to indicate that >>> getDelegationToken >>>>>>>>>>> can >>>>>>>>>>> be >>>>>>>>>>> called without any parameters: >>> >>> https://github.com/apache/accumulo/blob/1.7/docs/src/main/asciidoc/chapters/kerberos.txt#L410 >>>>>>>>>>> >>>>>>>>>>> Yet the source code indicates a DelegationTokenConfig >>> object must be >>>>>>>>>>> passed in: >>> >>> https://github.com/apache/accumulo/blob/1.7/core/src/main/java/org/apache/accumulo/core/client/admin/SecurityOperations.java#L359 >>>>>>>>>>> >>>>>>>>>>> Any ideas on how I should construct the >>> DelegationTokenConfig >>>>>>>>>>> object? >>>>>>>>>>> >>>>>>>>>>> For context, I've been trying to get geomesa to work on my >>> accumulo >>>>>>>>>>> 1.7 >>>>>>>>>>> with kerberos turned on. Right now, the code is somewhat >>> tied to >>>>>>>>>>> password auth: >>> >>> https://github.com/locationtech/geomesa/blob/rc7_a1.7_h2.5/geomesa-core/src/main/scala/org/locationtech/geomesa/core/data/AccumuloDataStoreFactory.scala#L177 >>>>>>>>>>> My thought is that I should get a KerberosToken first, and >>> then try >>>>>>>>>>> generate a DelegationToken, which is passed back for later >>>>>>>>>>> interactions >>>>>>>>>>> between geomesa and accumulo. >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> -Simon > > > -- > Sean
