FileSplitterBase Operator

[Vishal Agrawal]

Hi,

I am planning to use below Dag configuration.



public void populateDAG(DAG dag, Configuration configuration){

    DagInput input = dag.addOperator("Input", new DagInput());

    FileSplitterBase splitter = dag.addOperator("Splitter", new
FileSplitterBase());

    FSSliceReader blockReader = dag.addOperator("BlockReader", new
FSSliceReader());

    dag.addStream("file-info", input.output, splitter.input);

    dag.addStream("block-metadata", splitter.blocksMetadataOutput,
blockReader.blocksMetadataInput);

    ...

  }


Here DagInput will lookup the source files path and will pass it to
FileSplitterBase operator in FileInfo Object.

Now as Splitter already has Absolute path of source file in FileInfo Object,
I didn’t understand the significance of
com.datatorrent.lib.io.fs.FileSplitterBase.file field.


Thanks,

Vishal