Streaming experts, any clues how to achieve this?
After extracting few variables, I need to run them through a REST API for
verification and decision making.
Thanks for your help.
Nick
On Fri, Jan 17, 2020, 6:27 PM Nick Dawes <nickdawe...@gmail.com> wrote:
> I need to extract a value from a PySpark structured streaming Dataframe to
> a string variable to check something.
>
> I tried this code.
>
> agentName =
> kinesisDF.select(kinesisDF.agentName.getItem(0).alias("agentName")).collect()[0][0]
>
> This works on a non-streaming Dataframe only. In a streaming Dataframe,
> collect is not supported.
>
> Any workaround for this?
>
> Nick
>
>
>
