Hi Steve, Thank you for the update. Yes, --block_cache_capacity_mb should be in MiBytes (and yes, I messed things up with --block_cache_type, of course -- it seems I copy-pasted the wrong flag name in there).
That's a good call: I don't know any valid reason why not to include javadocs at least for KuduTestHarness into the API docs published at the kudu.apache.org site. I think adding it there would be a great improvement. Are you interested in making a contribution, posting a patch for that? You can find the guidelines for contributions at https://kudu.apache.org/docs/contributing.html Patches are always welcome! :) Thanks, Alexey On Wed, Oct 7, 2020 at 10:19 AM <stephen.hindma...@bt.com> wrote: > Thanks Alexey, > > > > I solved this in the end by reading the source code. My mistake was to > assume the test harness managed the builder as a singleton and the get > method was an accessor. Instead it creates a fresh one every time and the > get method is factory. You have to inject your customised builder when you > call the harness rule. I also corrected the fact I was providing a value in > bytes instead of MB to the flag. This is the solution. > > > > public static MiniKuduClusterBuilder builder; > > > > @BeforeClass > > public static void classInit(){ > > builder=KuduTestHarness.getBaseClusterBuilder() > > .addMasterServerFlag("--block_cache_capacity_mb=475") > > .addTabletServerFlag("--block_cache_capacity_mb=475"); > > } > > > > @Rule > > public KuduTestHarness harness=new KuduTestHarness(builder); > > > > Is there a reason why the javadocs for the test cluster classes are not > available on the main kudu.apache.org site? > > > > *Cheers* > > *Steve* > > > > *From:* Alexey Serbin [mailto:aser...@cloudera.com] > *Sent:* 07 October 2020 18:01 > *To:* user@kudu.apache.org > *Subject:* Re: Cache error running KuduTestHarness > > > > Hi, > > > > I haven't looked at the issue with the builder ignoring the settings you > added, but as a working example of adding custom flags to Kudu master and > tablet servers you can take a look at: > https://github.com/apache/kudu/blob/90aa4fa7d1527f376803440a4642668e3d798748/java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiation.java#L48-L52 > <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fkudu%2Fblob%2F90aa4fa7d1527f376803440a4642668e3d798748%2Fjava%2Fkudu-client%2Fsrc%2Ftest%2Fjava%2Forg%2Fapache%2Fkudu%2Fclient%2FTestNegotiation.java%23L48-L52&data=02%7C01%7Cstephen.hindmarch%40bt.com%7C6503391bd9a04c2dc13008d86ae2bad3%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637376869750326403&sdata=xJrv%2BhEh7e36QLg16bJhp4nlCrEThTeQ2t9nDl7jV7o%3D&reserved=0> > and > https://github.com/apache/kudu/blob/90aa4fa7d1527f376803440a4642668e3d798748/java/kudu-client/src/test/java/org/apache/kudu/client/TestTimeouts.java#L94 > <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fkudu%2Fblob%2F90aa4fa7d1527f376803440a4642668e3d798748%2Fjava%2Fkudu-client%2Fsrc%2Ftest%2Fjava%2Forg%2Fapache%2Fkudu%2Fclient%2FTestTimeouts.java%23L94&data=02%7C01%7Cstephen.hindmarch%40bt.com%7C6503391bd9a04c2dc13008d86ae2bad3%7Ca7f356889c004d5eba4129f146377ab0%7C0%7C0%7C637376869750336362&sdata=B3phwkG3omjGU62Q4rAPnybvNqIj8msEjDVeSRxjKos%3D&reserved=0> > (I guess in this context a custom MiniKuduClusterBuilder isn't necessary). > > > > It seems in case of running tests at machines with less memory it's worth > setting --block_cache_type to something low for both master and tablet > servers (like 128MB: --block_cache_type=134217728) because master and > tserver will eat up some memory once started. An alternative option is to > add --force_block_cache_capacity flag to both master's and tserver's flags. > > > > > > HTH, > > > > Alexey > > > > On Wed, Oct 7, 2020 at 4:14 AM <stephen.hindma...@bt.com> wrote: > > I am trying to follow the guide to using the KuduTestHarness in the > Getting Started guide. I have created the following simple test case. > > === > import org.apache.kudu.test.KuduTestHarness; > > import static org.junit.Assert.assertTrue; > import org.junit.Rule; > import org.junit.Test; > > public class DemoTest{ > @Rule > public KuduTestHarness harness=new KuduTestHarness(); > > @Test > public void testDemo(){ > assertTrue(true); > } > } > === > > But I get the following errors in the console log. > > === > 2020-10-07 11:50:01,060 [cluster stderr printer] INFO > org.apache.kudu.test.cluster.MiniKuduCluster - E1007 11:50:01.059237 17257 > block_cache.cc:99] Block cache capacity exceeds the memory pressure > threshold (536870912 bytes vs. 498776800 bytes). This will cause > instability and harmful flushing behavior. Lower --block_cache_capacity_mb > or raise --memory_limit_hard_bytes. > > 2020-10-07 11:50:01,060 [cluster stderr printer] INFO > org.apache.kudu.test.cluster.MiniKuduCluster - E1007 11:50:01.059262 17257 > flags.cc:441] Detected inconsistency in command-line flags; exiting > > 2020-10-07 11:50:01,100 [main] DEBUG > org.apache.kudu.test.cluster.MiniKuduCluster - Response: error { > code: RUNTIME_ERROR > message: "failed to start masters: Unable to start Master at index 0: > /tmp/kudu-binary-jar1893943400146501302/kudu-binary-1.13.0-linux-x86_64/bin/kudu-master: > process exited with non-zero status 1" > } > === > > I tried adding a flag to the base builder, but it does not have any > affect. The new flag does not show up in the list of flags in the logs. > > === > import org.apache.kudu.test.cluster.MiniKuduCluster.MiniKuduClusterBuilder; > > ... > > static{ > MiniKuduClusterBuilder > builder=KuduTestHarness.getBaseClusterBuilder(); > builder.addMasterServerFlag("--block_cache_capacity_mb=498776800"); > } > ... > === > > Can someone point me in the right direction for solving this problem. > > Thanks > Steve Hindmarch > >
