No, streaming dataframe needs to be written to disk or similar (or an in-memory backend) then when the next stream arrive join them - create graph and store the next stream together with the existing stream on disk etc.
> On 14. Jul 2018, at 17:19, kant kodali <kanth...@gmail.com> wrote: > > The question now would be can it be done in streaming fashion? Are you > talking about the union of two streaming dataframes and then constructing a > graphframe (also during streaming) ? > >> On Sat, Jul 14, 2018 at 8:07 AM, Jörn Franke <jornfra...@gmail.com> wrote: >> For your use case one might indeed be able to work simply with incremental >> graph updates. However they are not straight forward in Spark. You can union >> the new Data with the existing dataframes that represent your graph and >> create from that a new graph frame. >> >> However I am not sure if this will fully fulfill your requirement for >> incremental graph updates. >> >>> On 14. Jul 2018, at 15:59, kant kodali <kanth...@gmail.com> wrote: >>> >>> "You want to update incrementally an existing graph and run incrementally a >>> graph algorithm suitable for this - you have to implement yourself as far >>> as I am aware" >>> >>> I want to update the graph incrementally and want to run some graph queries >>> similar to Cypher like give me all the vertices that are connected by a >>> specific set of edges and so on. Don't really intend to run graph >>> algorithms like ConnectedComponents or anything else at this point but of >>> course, it's great to have. >>> >>> If we were to do this myself should I extend the GraphFrame? any >>> suggestions? >>> >>> >>>> On Sun, Apr 29, 2018 at 3:24 AM, Jörn Franke <jornfra...@gmail.com> wrote: >>>> What is the use case you are trying to solve? >>>> You want to load graph data from a streaming window in separate graphs - >>>> possible but requires probably a lot of memory. >>>> You want to update an existing graph with new streaming data and then >>>> fully rerun an algorithms -> look at Janusgraph >>>> You want to update incrementally an existing graph and run incrementally a >>>> graph algorithm suitable for this - you have to implement yourself as far >>>> as I am aware >>>> >>>> > On 29. Apr 2018, at 11:43, kant kodali <kanth...@gmail.com> wrote: >>>> > >>>> > Do GraphFrames support streaming? >>> >
