Hi , I went ahead with Mockito for testing StreamingAppMasterService to check blacklisting of consecutively failing nodes. Here is a pull request for the same: https://github.com/apache/incubator-apex-core/pull/98
Mockito was fairly easy to use for for mocking responses from Hadoop APIs and some of StreamingAppMasterService methods. I had captured some basic Mocktio APIs that I used in this document (in case someone has a use for it): https://docs.google.com/document/d/1WBdWceHNIVg4UeTL06pqzoJLQQ-Q7yLNzTV1IDoHFyo/edit?usp=sharing Thanks, Isha On Wed, Sep 30, 2015 at 3:49 PM, Isha Arkatkar <[email protected]> wrote: > I was thinking of mocking the calls that we make during allocation. E.g. > RegisterApplicationMasterResponse, AMRMClient.allocate call, > AllocateResponse returned etc.. > But yes, if there is any supporting library for the same, it would be very > useful! > > Isha > > On Wed, Sep 30, 2015 at 1:26 PM, Pramod Immaneni <[email protected]> > wrote: > >> Would going to Mockito route mean we have to keep up with the RMClient >> protocol and handle generation of responses ourselves, are there any >> supporting libraries in hadoop that would help us with that. >> >> On Wed, Sep 30, 2015 at 12:36 PM, Isha Arkatkar <[email protected]> >> wrote: >> >> > Hi all, >> > >> > I want to add some unit testing for container allocation flow in >> > StreamingAppMasterService class. Some of the scenarios like consecutive >> > node failures etc. are hard to simulate manually and would be nice to >> cover >> > that through unit testing. >> > >> > I am right now exploring 2 options: Mocking RMClient responses using >> > Mockito and via MiniYARNCluster. >> > >> > Mockito route requires some refactoring in >> StreamingAppMasterService. If >> > we divide execute method in multiple functions, then those can be tested >> > individually. It gives more flexibility to simulate failure scenarios. >> > >> > With MiniYARNCluster, we have a few tests in StreamMiniClusterTest. >> These >> > are probably good end-to-end tests. But I don't know if we can set >> > container launch failures. >> > >> > Please suggest which is best way to go :) >> > >> > Thanks! >> > Isha >> > >> > >
