Stephen: Do you have time to come up with standalone demo ? Cheers
On Fri, Jan 2, 2015 at 4:39 PM, Stephen Boesch <[email protected]> wrote: > We have a few classes and a not small codebase. I will try to put together > a standalone demo of this issue - will be Monday. > > thx for your help > > 2015-01-02 16:34 GMT-08:00 Ted Yu <[email protected]>: > > > In Spark, I found the following: > > examples/src/main/scala/org/apache/spark/examples/HBaseTest.scala > > which doesn't involve HBaseTestingUtility. > > > > Can you pastebin your code ? > > > > If you were running in Spark, please give the complete command line. > > > > Cheers > > > > On Fri, Jan 2, 2015 at 4:20 PM, Stephen Boesch <[email protected]> > wrote: > > > > > Hi Ted, > > > Also on macOs. I am running within a completely different project > > > (spark). Is there something else particular to the hbase testing env > > > (workspace) that needs to be replicated to the spark maven build to > have > > > this work properly? > > > > > > thx! > > > > > > 2015-01-02 16:11 GMT-08:00 Ted Yu <[email protected]>: > > > > > > > I was running test on Macbook. > > > > What OS do you use ? > > > > > > > > Were you running in workspace based in master branch ? > > > > > > > > Cheers > > > > > > > > > > > > > > > > On Fri, Jan 2, 2015 at 4:03 PM, Stephen Boesch <[email protected]> > > > wrote: > > > > > > > > > Hi Ted, > > > > > > > > > > I can not get the same behavior. > > > > > > > > > > The command line includes: > > > > > > > > > > -Dtest.build.data.basedirectory=/tmp/minihbase > > > > > > > > > > And then I have also tried to set the System property directly: > > > > > > > > > > val WorkDirProperty = "test.build.data.basedirectory" > > > > > val DefaultWorkDir = "/tmp/minihbase" > > > > > > > > > > System.setProperty(WorkDirProperty, workDir) > > > > > > > > > > testUtil = new HBaseTestingUtility > > > > > cluster = testUtil.startMiniCluster(nMasters, nRegionServers, > > > > > nDataNodes) > > > > > > > > > > It is not clear to me why we see different behaviors. > > > > > > > > > > 2015-01-02 14:43 GMT-08:00 Ted Yu <[email protected]>: > > > > > > > > > > > Here is the command I used (in workspace corresponding to master > > > > branch): > > > > > > > > > > > > mvn test -PrunAllTests > > > > > > -DfailIfNoTests=false -Dtest.build.data.basedirectory=/tmp/ted/ > > > > > > -Dtest=TestHFileOutputFormat2#testExcludeMinorCompaction > > > > > > > > > > > > Under /tmp/ted, I have: > > > > > > $ ls e47ced18-e1f8-4310-bbed-4684d401982c/ > > > > > > cache_data hadoop_logs mapred_local > > > > > > dfscluster_b2dbb460-3735-4813-a1d1-f18e082ee89b hadoop_tmp > > > > > > > > > > > > FYI > > > > > > > > > > > > On Fri, Jan 2, 2015 at 2:13 PM, Stephen Boesch < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > We are experiencing an issue that the MiniHBase cluster is > > creating > > > > > temp > > > > > > > directories/data under the same directory as the tests are run: > > > > > > > > > > > > > > Upon doing some research it appears there is a system property > to > > > > > control > > > > > > > that temp dir: > > > > > > > > > > > > > > test.build.data.basedirectory > > > > > > > > > > > > > > Also there is in the doc a command to set it: > > > > > > > > > > > > > > <command>mvn test -P runAllTests > > > > -Dsurefire.secondPartThreadCount=12 > > > > > - > > > > > > > Dtest.build.data.basedirectory=/ram2G</command> > > > > > > > > > > > > > > However, we find that does NOT work unless the following > > additional > > > > > maven > > > > > > > parameter is set: > > > > > > > > > > > > > > -DforkMode=never > > > > > > > > > > > > > > But that flag causes other testing issues and we would not be > > able > > > to > > > > > go > > > > > > > that route. > > > > > > > > > > > > > > How then can we redirect that testing data directory without > > > setting > > > > > the > > > > > > > System property? > > > > > > > > > > > > > > > > > > > > > > > > > > > >
