Chandni, Agreed. See whether the tests work with the synchronous storage agent. If yes, change them. The default needs to remain async.
Thomas On Wed, Sep 2, 2015 at 11:05 PM, Chandni Singh <[email protected]> wrote: > Hi, > > I would like to know what was the reason to use AsyncFSStorageAgent with > StramLocalCluster? > StramLocalCluster is mainly for testing in a non-distributed mode and I am > unclear how AsyncFSStorageAgent is helpful in this mode. > > Thanks, > Chandni > > On Wed, Sep 2, 2015 at 10:45 PM, Chandni Singh <[email protected]> > wrote: > >> This is because of recent changes to StramLocalCluster where >> AsyncFSStorageAgent is used for checkpointing >> >> dag.setAttribute(OperatorContext.STORAGE_AGENT, new AsyncFSStorageAgent(new >> Path(pathUri, LogicalPlan.SUBDIR_CHECKPOINTS).toString(), null)); >> >> The AsyncFSStorageAgent(String path, Configuration conf) uses "." as >> localBasePath and therefore creates sub-directories per operator in the >> current working directory. >> >> I am going to create a ticket to address this and will fix it. >> >> -Chandni >> >> >> On Wed, Sep 2, 2015 at 7:13 PM, Chandni Singh <[email protected]> >> wrote: >> >>> Hi, >>> >>> I can see empty folders getting created under Malhar/lib called '1' and >>> '2'. >>> I think this is because of using LocalMode to run a test application. >>> >>> >>> If anyone has checked in such cases please do check and let us know. >>> >>> Thanks, >>> Chandni >>> >> >> >
