Hi Trevor,

The examples used to be working but they are not anymore due to SDK
changes. Unfortunately, they are not automatically tested. We should
change that.

You can make the examples work if you use a bounded source, e.g. set
`.withMaxNumRecords(100)` on the socket source.

Cheers,
Max

On Sun, Oct 16, 2016 at 10:43 PM, Trevor Grant <trevor.d.gr...@gmail.com> wrote:
> So to be clear-
>
> The following currently merged examples aren't supposed to be runnable
> https://github.com/apache/incubator-beam/blob/f62d04e22679ee2ac19e3ae37dec487d953d51c1/runners/flink/examples/src/main/java/org/apache/beam/runners/flink/examples/streaming/WindowedWordCount.java
> https://github.com/apache/incubator-beam/blob/f62d04e22679ee2ac19e3ae37dec487d953d51c1/runners/flink/examples/src/main/java/org/apache/beam/runners/flink/examples/streaming/JoinExamples.java
>
> Because those throw the same error re: Write / Bounded collections.
>
> tg
>
> Trevor Grant
> Data Scientist
> https://github.com/rawkintrevo
> http://stackexchange.com/users/3002022/rawkintrevo
> http://trevorgrant.org
>
> "Fortunate is he, who is able to know the causes of things."  -Virgil
>
>
> On Fri, Oct 14, 2016 at 8:09 PM, Kenneth Knowles <k...@google.com> wrote:
>>
>> Hi Trevor,
>>
>> The problem is that "Write" is an old name that should be changed to
>> "BoundedWrite" (actually it is even more specific). In fact, it re-windows
>> into the global window and removes all triggering, so it is suitable only
>> for bounded PCollections where this will ensure all the data arrives at
>> once. Hence the error message.
>>
>> For a streaming write, I'd recommend just performing the write in a
>> ParDo(DoFn) with probably some prep to get the data ready for idempotent
>> writing. If you look around the codebase you'll see some examples of this.
>> Also note that Write isn't a primitive but just a certain pattern of how you
>> can use ParDo and side inputs.
>>
>> Kenn
>>
>>
>> On Fri, Oct 14, 2016, 17:16 Trevor Grant <trevor.d.gr...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> Trying to write a Twitter exampe on the Flink streamer-
>>>
>>>
>>> https://github.com/rawkintrevo/incubator-beam/blob/flink-twitter-example/runners/flink/examples/src/main/java/org/apache/beam/runners/flink/examples/streaming/TwitterWindowedWordCountExamples.java
>>>
>>> I'm getting an error
>>> Caused by: java.lang.IllegalArgumentException: Write can only be applied
>>> to a Bounded PCollection
>>>
>>> I was nearly copy and pasting from the other word count example, every
>>> thing looks good in intellij. Can't figure out for the life of me what I'm
>>> doing wrong here.
>>>
>>> I am running the job against a flink cluster (uploaded via web-UI) if
>>> that is informative...
>>>
>>> Thanks, I plan on adding the example back in once it is done (still have
>>> to parse the tweets).
>>>
>>> tg
>>>
>>>
>>> Trevor Grant
>>> Data Scientist
>>> https://github.com/rawkintrevo
>>> http://stackexchange.com/users/3002022/rawkintrevo
>>> http://trevorgrant.org
>>>
>>> "Fortunate is he, who is able to know the causes of things."  -Virgil
>>>
>

Reply via email to