Chaitanya, Are you creating NFS for naming purposes for users to find out? At a very high level is the idea to have the following command return something in Malhar directory
find . | grep -i nfs | grep java We do need the above commands to common sources, destinations, protocols to return something to get folks in. If so this could be as simple as changing default values/properties, or a particular attributes (say partitioning) in the Module that in your view makes it better suited for NFS. On somewhat more complex side it could be java code. Thks Amol On Thu, May 5, 2016 at 12:08 AM, Chaitanya Chebolu < [email protected]> wrote: > Hi Chandni, > > Its a good point. I created the hierarchy based on user perspective and > especially for non Java users. If I return FileSplitter and BlockReader > from FS Input Module, then this module works for NFS. But, for users > perspective it would be difficult, whether this module works for NFS or any > other fileSystem. > > Regards, > Chaitanya > > On Thu, May 5, 2016 at 11:05 AM, Chandni Singh <[email protected]> > wrote: > > > I am sorry Chaitanya but I have more questions about this > > > > 1. why is the FS Input Module abstract when by default it can return > > FileSplitter & BlockReader in com.datatorrent.lib.io.fs? > > These implementations are not specific to NFS. > > > > 2. In the NFS module that you have suggested to create, what is specific > to > > NFS? > > > > Please note: I have created a ticket APEXMALHAR-2081 to remove > > FSFileSplitter from library and move its feature to the base operator. > > > > Thanks, > > Chandni > > > > On Wed, May 4, 2016 at 10:29 PM, Chaitanya Chebolu < > > [email protected]> wrote: > > > > > FSFileSplitter & BlockReader are available in com.datatorrent.lib.io.fs > > > package. > > > > > > On Thu, May 5, 2016 at 10:47 AM, Chandni Singh < > [email protected]> > > > wrote: > > > > > > > Ok. What is specific about the fileSplitter and blockReader returned > by > > > > this implementation? > > > > > > > > > > > > On May 4, 2016 9:43 PM, "Chaitanya Chebolu" < > [email protected] > > > > > > > wrote: > > > > > > > > > Hi Chandni, > > > > > > > > > > Properties wise nothing specific. FS Input Module is an abstract > > Module > > > > and > > > > > NFS Module implements the abstract methods - createFileSplitter() > and > > > > > createBlockReader(). > > > > > > > > > > Regards, > > > > > Chaitanya > > > > > > > > > > On Wed, May 4, 2016 at 9:45 PM, Chandni Singh < > > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > Hi Chaitanya, > > > > > > > > > > > > What will be specific in NFS Input Module that is not provided by > > FS > > > > > Input > > > > > > Module? > > > > > > > > > > > > Thanks, > > > > > > Chandni > > > > > > > > > > > > On Wed, May 4, 2016 at 7:12 AM, Amol Kekre <[email protected] > > > > > > wrote: > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > Thks > > > > > > > Amol > > > > > > > > > > > > > > On Tue, May 3, 2016 at 10:06 PM, Sandeep Deshmukh < > > > > > > [email protected] > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > Regards, > > > > > > > > Sandeep > > > > > > > > > > > > > > > > On Fri, Apr 29, 2016 at 3:26 PM, Mohit Jotwani < > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > Mohit > > > > > > > > > > > > > > > > > > On Fri, Apr 29, 2016 at 2:09 PM, Chaitanya Chebolu < > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > > > > > I am proposing NFS Input Module. Use case is to read > > large > > > > > files > > > > > > > from > > > > > > > > > NFS > > > > > > > > > > in parallel. > > > > > > > > > > > > > > > > > > > > Design of NFS input module: > > > > > > > > > > > > > > > > > > > > There is a common interface "FSInputModule" in Malhar > > for > > > > the > > > > > > > input > > > > > > > > > > Modules. NFS input Module extends from FSInputModule and > > can > > > be > > > > > > > > achieved > > > > > > > > > by > > > > > > > > > > using FSFileSplitter and BlockReader operators. > > > > > > > > > > > > > > > > > > > > Please share your thoughts on this. > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > Chaitanya > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
