Dan, I was unable to get Christopher's example to work as posted without steps mentioned earlier in the thread. In the MiniAccumuloCluster code for version 1.6.2, if I am reading it correctly, you must do a couple of things.
1 - build the native libs 2 - call config.setNativeLibPaths(somedirectory) The ProcessBuilder that is used to run the MiniAccumuloCluster does not appear to reset the environment. But it does overwrite whatever is in DYLIB_LIBRARY_PATH and LD_LIBRARY_PATH with the value set in the config. So you must call config.setNativeLibPaths and then use that config when you create the MiniAccumuloCluster, as far as I can tell I created the following repo to show how I did it, https://github.com/mjwall/miniaccumulocluster-nativemaps. Hope that helps. Mike On Tue, Feb 23, 2016 at 5:14 PM, Christopher <ctubb...@apache.org> wrote: > Looking at the NativeMap, it looks like it will always log some message at > the INFO level if it successfully loaded the native maps, or at the ERROR > level if it failed to do so (with some extra DEBUG messages while it > searches the path). > > I thought maybe there was a class loading race condition where > NativeMap.isLoaded() returns false while it's still trying to load... that > might still be a possibility (I'm not sure if this can happen with static > initializer blocks?), but if it were, you'd still see the log messages > about loading or not. > > I can't see your code, so I don't know what's wrong, but something like > the following should work fine: > > 1. MiniAccumuloConfig config = new MiniAccumuloConfig(new > File("/path/to/miniDir"), "rootPassword"); > 2. HashMap<String,String> map = new HashMap<String,String>(); > 3. map.put(Property.TSERV_NATIVEMAP_ENABLED.getKey(), "true"); > 4. config.setSiteConfig(map); > 5. MiniAccumuloCluster mini = new MiniAccumuloCluster(config); > > > On Tue, Feb 23, 2016 at 2:21 PM Dan Blum <db...@bbn.com> wrote: > >> In fact, we are calling that (in Groovy which is why I missed it before, >> not being that familiar with Groovy). I verified that the path is correct – >> doesn’t help. >> >> >> >> *From:* Christopher [mailto:ctubb...@apache.org] >> *Sent:* Tuesday, February 23, 2016 2:06 PM >> >> >> *To:* user@accumulo.apache.org >> *Subject:* Re: Unable to get Mini to use native maps - 1.6.2 >> >> >> >> MiniAccumuloConfig has a method, called "setNativeLibPaths(String... >> nativePathItems)". >> >> You should call that method with the absolute path for your compiled >> native map shared library file (.so), before you start Mini. >> >> >> >> On Tue, Feb 23, 2016 at 2:03 PM Josh Elser <josh.el...@gmail.com> wrote: >> >> MiniAccumuloCluster spawns its own processes, though. Calling >> NativeMap.isLoaded() in your test JVM isn't proving anything. >> >> That's why you need to call these methods on MAC, you would need to >> check the TabletServer*.log file(s), and make sure that its >> configuration is set up properly to find the .so. >> >> Does that make sense? Did I misinterpret you? >> >> Dan Blum wrote: >> > I'll see what I can do, but there's no simple way to pull out something >> > small we can share (and it would have to be a gradle project). >> > >> > I confirmed that the path is not the immediate issue by adding an >> explicit >> > call to NativeMap.isLoaded() at the start of my test - that produces >> logging >> > from NativeMap saying it can't find the library, which is what I expect. >> > Without this call NativeMap still logs nothing so the setting that >> should >> > cause it to be referenced is getting overridden somewhere. Calling >> > InstanceOperations.getSiteConfiguration and getSystemConfiguration shows >> > that the native maps are enabled, however. >> > >> > -----Original Message----- >> > From: Josh Elser [mailto:josh.el...@gmail.com] >> > Sent: Tuesday, February 23, 2016 12:56 PM >> > To: user@accumulo.apache.org >> > Subject: Re: Unable to get Mini to use native maps - 1.6.2 >> > >> > Well, I'm near positive that 1.6.2 had native maps working, so there >> > must be something unexpected happening :). MAC should be very close to >> > what a real standalone instance is doing -- if you have the ability to >> > share some end-to-end project with where you are seeing this, that'd be >> > extremely helpful (e.g. a Maven project that we can just run would be >> > superb). >> > >> > Dan Blum wrote: >> >> I'll take a look but I don't think the path is the problem - NativeMap >> >> should try to load the library regardless of whether this path is set >> and >> >> will log if it can't find it. This isn't happening. >> >> >> >> -----Original Message----- >> >> From: Josh Elser [mailto:josh.el...@gmail.com] >> >> Sent: Tuesday, February 23, 2016 12:27 PM >> >> To: user@accumulo.apache.org >> >> Subject: Re: Unable to get Mini to use native maps - 1.6.2 >> >> >> >> Hi Dan, >> >> >> >> I'm seeing in our internal integration tests that we have some >> >> configuration happening which (at least, intends to) configure the >> >> native maps for the minicluster. >> >> >> >> If you're not familiar, the MiniAccumuloConfig and MiniAccumuloCluster >> >> classes are thin wrappers around MiniAccumuloConfigImpl and >> >> MiniAccumuloClusterImpl. There is a setNativeLibPaths method on >> >> MiniAccumuloConfigImpl which you can use to provide the path to the >> >> native library shared object (.so). You will probably have to switch >> >> from MiniAccumuloConfig/MiniAccumuloCluster to >> >> MiniAccumuloConfigImpl/MiniAccumuloClusterImpl to use the "hidden" >> > methods. >> >> You could also look at MiniClusterHarness.java in>=1.7 if you want a >> >> concrete example of how we initialize things for our tests. >> >> >> >> - Josh >> >> >> >> Dan Blum wrote: >> >>> In order to test to make sure we don't have more code that needs a >> >>> workaround for https://issues.apache.org/jira/browse/ACCUMULO-4148 I >> am >> >>> trying again to enable the native maps for Mini, which we use for >> > testing. >> >>> I set tserver.memory.maps.native.enabled to true in the site XML, and >> > this >> >>> is getting picked up since I see this in the Mini logs: >> >>> >> >>> [server.Accumulo] INFO : tserver.memory.maps.native.enabled = true >> >>> >> >>> However, NativeMap should log something when it tries to load the >> > library, >> >>> whether it succeeds or fails, but it logs nothing. The obvious >> conclusion >> >> is >> >>> that something about how MiniAccumuloCluster starts means that this >> >> setting >> >>> is ignored or overridden, but I am not finding it. (I see the >> mergeProp >> >> call >> >>> in MiniAccumuloConfigImpl.initialize which will set >> >> TSERV_NATIVEMAP_ENABLED >> >>> to false, but that should only set it if it's not already in the >> >> properties, >> >>> which it should be, and as far as I can tell the log message above is >> >> issued >> >>> after this.) >> >>> >> > >> >>
