Hi Emanuele

KafkaIO  supports withMaxNumRecords(X) support which will create a bounded
source from Kafka.  However, the pipeline will finish once X number of
records are read.

Regards
Sumit Chawla


On Sun, Sep 18, 2016 at 2:00 PM, Emanuele Cesena <emanu...@shopkick.com>
wrote:

> Hi,
>
> Thanks for the hint - I’ll debug better but I thought I did that:
> https://github.com/ecesena/beam-starter/blob/master/src/
> main/java/com/dataradiant/beam/examples/StreamWordCount.java#L140
>
> Best,
>
>
> > On Sep 18, 2016, at 1:54 PM, Jean-Baptiste Onofré <j...@nanthrax.net>
> wrote:
> >
> > Hi Emanuele
> >
> > You have to use a window to create a bounded collection from an
> unbounded source.
> >
> > Regards
> > JB
> >
> > On Sep 18, 2016, at 21:04, Emanuele Cesena <emanu...@shopkick.com>
> wrote:
> > Hi,
> >
> > I wrote a while ago about a simple example I was building to test
> KafkaIO:
> > https://github.com/ecesena/beam-starter/blob/master/src/
> main/java/com/dataradiant/beam/examples/StreamWordCount.java
> >
> > Issues with Flink should be fixed now, and I’m try to run the example on
> master and Flink 1.1.2.
> > I’m currently getting:
> > Caused by: java.lang.IllegalArgumentException: Write can only be
> applied to a Bounded PCollection
> >
> > What is the recommended way to go here?
> > - is there a way to create a bounded collection from an unbounded one?
> > - is there a plat to let TextIO write unbounded collections?
> > - is there another recommended “simple sink” to use?
> >
> > Thank you much!
> >
> > Best,
>
> --
> Emanuele Cesena, Data Eng.
> http://www.shopkick.com
>
> Il corpo non ha ideali
>
>
>
>
>

Reply via email to