Hi Ulrich, Yes, I'm starting mini cluster inside @BeforeClass. There are 3 different test cases, and between 2 and 15 tests per test case.
Thanks! Best regards, Cristofer -----Mensagem original----- De: [email protected] [mailto:[email protected]] Em nome de Ulrich Staudinger Enviada em: sexta-feira, 31 de agosto de 2012 08:05 Para: [email protected] Assunto: Re: HBase and unit tests Hi Cristofer, > At least 15 seconds are spent on starting the mini cluster for each > test case. and you are sure that you are reusing your mini cluster across unit tests? HTH2, Ulrich On Fri, Aug 31, 2012 at 12:28 PM, Cristofer Weber < [email protected]> wrote: > Hi Sonal, Stack and Ulrich! > > Yes, I should provide more details :$ > > I reached the links you provided when I was searching for a way to > start HBase with JUnit. From default, the only params I have changed > are Zookeeper port and the amount of nodes, which is 1 in my case. > Based on logs I suspect that most of time are spent with HDFS and > that's why I asked if there is a way to start a standalone instance of > HBase. The amount of data written at each test case would probably fit > in memstore anyway, and table cleansing between each test method is managed > by a loop of deletes. > > At least 15 seconds are spent on starting the mini cluster for each > test case. > > Right now I reminded that I should turn off WAL when running unit > tests :-), but this will not reflect on startup time. > > Thanks!! > > Best regards, > Cristofer > > ________________________________________ > De: Ulrich Staudinger [[email protected]] > Enviado: sexta-feira, 31 de agosto de 2012 2:21 > Para: [email protected] > Assunto: Re: HBase and unit tests > > As a general advice, although you probably do take care of this, > instantiate the mini cluster only once in your junit test constructor > and not in every test method. at the end of each test, either cleanup > your hbase or use a different "area" per test. > > best regards, > ulrich > > > -- > connect on xing or linkedin. sent from my tablet. > > On 31.08.2012, at 06:46, Stack <[email protected]> wrote: > > > On Thu, Aug 30, 2012 at 4:44 PM, Cristofer Weber > > <[email protected]> wrote: > >> Hi there! > >> > >> After I started studying HBase, I've searched for open source > >> projects > backed by HBase and I found Titan distributed graph database (you > probably heard about it). As soon as I read in their documentation > that HBase adapter is experimental and suboptimal (disclaimer here: > https://github.com/thinkaurelius/titan/wiki/Using-HBase) I volunteered > to help improving this adapter and since then I made a few changes to > improve on running tests (reduced from hours to minutes) and also an > improvement on search feature. > >> > >> Now I'm trying to break the dependency on a pre-installed HBase for > unit tests and found miniCluster inside HBase tests, but minicluster > demands too much time to start and I don't know if tweaking on configs > will improve significantly. Is there a way to start a 'lightweight' > instance, like programatically starting a standalone instance? > >> > > > > How much is 'too much time' Cristofer? Do you want a standalone > > cluster > at all? > > St.Ack > > P.S. If digging in this area, you might find the blog post by the > > sematextians of use: > > > http://blog.sematext.com/2010/08/30/hbase-case-study-using-hbasetestin > gutility-for-local-testing-development/ > -- Ulrich Staudinger <http://goog_958005736>http://www.activequant.com Connect online: https://www.xing.com/profile/Ulrich_Staudinger
