As Thomas mentioned as default remains to be async. You can either change the storage agent or set the APPLICATION_PATH.
When container runs in cluster, "." specifies the containers local path on the node where container specific jars and other resources resides. It creates a folder under that which is live as long as container lives. So there are no vagrant folders anywhere Thanks -Gaurav On Wed, Sep 2, 2015 at 11:33 PM, Chandni Singh <[email protected]> wrote: > I think there is a problem in the default Async as well. It also uses the > working directory as its local base path. > > In the Async -> copyToHdfs() method, we delete the window files but the > folder with the operator name never gets deleted. > So on the cluster there will be such vagrant folders in the working > directory? > > On Wed, Sep 2, 2015 at 11:17 PM, Thomas Weise <[email protected]> > wrote: > >> Chandni, >> >> Agreed. See whether the tests work with the synchronous storage agent. If >> yes, change them. The default needs to remain async. >> >> Thomas >> >> >> On Wed, Sep 2, 2015 at 11:05 PM, Chandni Singh <[email protected]> >> wrote: >> >>> Hi, >>> >>> I would like to know what was the reason to use AsyncFSStorageAgent with >>> StramLocalCluster? >>> StramLocalCluster is mainly for testing in a non-distributed mode and I >>> am unclear how AsyncFSStorageAgent is helpful in this mode. >>> >>> Thanks, >>> Chandni >>> >>> On Wed, Sep 2, 2015 at 10:45 PM, Chandni Singh <[email protected]> >>> wrote: >>> >>>> This is because of recent changes to StramLocalCluster where >>>> AsyncFSStorageAgent is used for checkpointing >>>> >>>> dag.setAttribute(OperatorContext.STORAGE_AGENT, new >>>> AsyncFSStorageAgent(new Path(pathUri, >>>> LogicalPlan.SUBDIR_CHECKPOINTS).toString(), null)); >>>> >>>> The AsyncFSStorageAgent(String path, Configuration conf) uses "." as >>>> localBasePath and therefore creates sub-directories per operator in the >>>> current working directory. >>>> >>>> I am going to create a ticket to address this and will fix it. >>>> >>>> -Chandni >>>> >>>> >>>> On Wed, Sep 2, 2015 at 7:13 PM, Chandni Singh <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I can see empty folders getting created under Malhar/lib called '1' >>>>> and '2'. >>>>> I think this is because of using LocalMode to run a test application. >>>>> >>>>> >>>>> If anyone has checked in such cases please do check and let us know. >>>>> >>>>> Thanks, >>>>> Chandni >>>>> >>>> >>>> >>> >> >
