Hi, I would like to know what was the reason to use AsyncFSStorageAgent with StramLocalCluster? StramLocalCluster is mainly for testing in a non-distributed mode and I am unclear how AsyncFSStorageAgent is helpful in this mode.
Thanks, Chandni On Wed, Sep 2, 2015 at 10:45 PM, Chandni Singh <[email protected]> wrote: > This is because of recent changes to StramLocalCluster where > AsyncFSStorageAgent is used for checkpointing > > dag.setAttribute(OperatorContext.STORAGE_AGENT, new AsyncFSStorageAgent(new > Path(pathUri, LogicalPlan.SUBDIR_CHECKPOINTS).toString(), null)); > > The AsyncFSStorageAgent(String path, Configuration conf) uses "." as > localBasePath and therefore creates sub-directories per operator in the > current working directory. > > I am going to create a ticket to address this and will fix it. > > -Chandni > > > On Wed, Sep 2, 2015 at 7:13 PM, Chandni Singh <[email protected]> > wrote: > >> Hi, >> >> I can see empty folders getting created under Malhar/lib called '1' and >> '2'. >> I think this is because of using LocalMode to run a test application. >> >> >> If anyone has checked in such cases please do check and let us know. >> >> Thanks, >> Chandni >> > >
