Good evening all,
I am working on project whose goal is to show the strength of beam and
build a "pipeline" that ingests both bounded and unbounded data.
I use NiFi to both "batch up" and stream the same dataset. I am running
this using the Direct Runner
Bounded: I can setup the pipeline to read and process the files. Works as
suspected
Unbounded: *Doesn't work*, but I am certain it's because I don't understand
how the pipeline needs to be created to create the PCollection. I can't
find any good examples
#*I KNOW THIS IS WRONG*
*StreamingOptions streamingOptions =PipelineOptions.class.;Pipeline
stream = Pipeline.create(streamingOptions);stream.apply("WordStream",
Read.from(new
UnboundedReadFromBoundedSource.BoundedToUnboundedSourceAdapter<>()))
.apply("ExtraLinesStream",ParDo.of(new AnomalyAlertDetector()))
.apply("WriteToAccumuloStream", ParDo.of(new
WriteToAccumuloStream()));*
