Hi Direct runner is able to read from unbounded source. The next transforms of your pipeline have to deal with unbounded pcollection.
Regards JB On Feb 7, 2017, 22:16, at 22:16, Wyatt Frelot <[email protected]> wrote: >Good evening all, > >I am working on project whose goal is to show the strength of beam and >build a "pipeline" that ingests both bounded and unbounded data. > >I use NiFi to both "batch up" and stream the same dataset. I am running >this using the Direct Runner > >Bounded: I can setup the pipeline to read and process the files. Works >as >suspected > >Unbounded: *Doesn't work*, but I am certain it's because I don't >understand >how the pipeline needs to be created to create the PCollection. I can't >find any good examples > >#*I KNOW THIS IS WRONG* > > > > > > >*StreamingOptions streamingOptions =PipelineOptions.class.;Pipeline >stream = Pipeline.create(streamingOptions);stream.apply("WordStream", >Read.from(new >UnboundedReadFromBoundedSource.BoundedToUnboundedSourceAdapter<>())) > .apply("ExtraLinesStream",ParDo.of(new AnomalyAlertDetector())) > .apply("WriteToAccumuloStream", ParDo.of(new >WriteToAccumuloStream()));*
