So to be clear-

The following currently merged examples aren't supposed to be runnable
https://github.com/apache/incubator-beam/blob/f62d04e22679ee2ac19e3ae37dec487d953d51c1/runners/flink/examples/src/main/java/org/apache/beam/runners/flink/examples/streaming/WindowedWordCount.java
https://github.com/apache/incubator-beam/blob/f62d04e22679ee2ac19e3ae37dec487d953d51c1/runners/flink/examples/src/main/java/org/apache/beam/runners/flink/examples/streaming/JoinExamples.java

Because those throw the same error re: Write / Bounded collections.

tg

Trevor Grant
Data Scientist
https://github.com/rawkintrevo
http://stackexchange.com/users/3002022/rawkintrevo
http://trevorgrant.org

*"Fortunate is he, who is able to know the causes of things."  -Virgil*


On Fri, Oct 14, 2016 at 8:09 PM, Kenneth Knowles <k...@google.com> wrote:

> Hi Trevor,
>
> The problem is that "Write" is an old name that should be changed to
> "BoundedWrite" (actually it is even more specific). In fact, it re-windows
> into the global window and removes all triggering, so it is suitable only
> for bounded PCollections where this will ensure all the data arrives at
> once. Hence the error message.
>
> For a streaming write, I'd recommend just performing the write in a
> ParDo(DoFn) with probably some prep to get the data ready for idempotent
> writing. If you look around the codebase you'll see some examples of this.
> Also note that Write isn't a primitive but just a certain pattern of how
> you can use ParDo and side inputs.
>
> Kenn
>
> On Fri, Oct 14, 2016, 17:16 Trevor Grant <trevor.d.gr...@gmail.com> wrote:
>
>> Hi,
>>
>> Trying to write a Twitter exampe on the Flink streamer-
>>
>> https://github.com/rawkintrevo/incubator-beam/blob/flink-twitter-example/
>> runners/flink/examples/src/main/java/org/apache/beam/
>> runners/flink/examples/streaming/TwitterWindowedWordCountExamples.java
>>
>> I'm getting an error
>> Caused by: java.lang.IllegalArgumentException: Write can only be applied
>> to a Bounded PCollection
>>
>> I was nearly copy and pasting from the other word count example, every
>> thing looks good in intellij. Can't figure out for the life of me what I'm
>> doing wrong here.
>>
>> I am running the job against a flink cluster (uploaded via web-UI) if
>> that is informative...
>>
>> Thanks, I plan on adding the example back in once it is done (still have
>> to parse the tweets).
>>
>> tg
>>
>>
>> Trevor Grant
>> Data Scientist
>> https://github.com/rawkintrevo
>> http://stackexchange.com/users/3002022/rawkintrevo
>> http://trevorgrant.org
>>
>> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
>>
>>

Reply via email to