Re: Convert DStream into Streaming Dataframe

2017-05-12 Thread Michael Armbrust
Are there any particular things that the DataFrame or Dataset API are missing? On Fri, May 12, 2017 at 9:49 AM, Tejinder Aulakh wrote: > Hi, > > Is there any way to convert a DStream to a streaming dataframe? I want to > use Structured streaming in a new common module

Re: Convert DStream into Streaming Dataframe

2017-05-12 Thread Tathagata Das
Unfortunately, no. DStreams and streaming DataFrames are so different in their abstractions and implementations that there is no way to convert them. On Fri, May 12, 2017 at 9:49 AM, Tejinder Aulakh wrote: > Hi, > > Is there any way to convert a DStream to a streaming

Convert DStream into Streaming Dataframe

2017-05-12 Thread Tejinder Aulakh
Hi, Is there any way to convert a DStream to a streaming dataframe? I want to use Structured streaming in a new common module that I'm developing. The existing code uses DStream so trying to figure out how to convert a DStream to a Streaming Dataframe. The documentation only describes how to read