Re: [StructuredStreaming] multiple queries of the socket source: only one query works.

2017-08-13 Thread Gerard Maas
Hi Shixiong, Thanks for the explanation. In my view, this is different from the intuitive understanding of the Structured Streaming model [1], where incoming data is appended to an 'unbounded table' and queries are run on that. I had expected that all queries would run on that 'unbounded table

[BlockMatrix] multiply is an action or a transformation ?

2017-08-13 Thread Jose Francisco Saray Villamizar
Hi Everyone, Sorry if the question can be simple, or confusing, but I have not see anywhere in documentation the anwser: Is multiply method in BlockMatrix a transformation or an action. I mean, in order that the multiplication is effectively done it is enough with calling : m1.multiply(m2), Or

Re: [StructuredStreaming] multiple queries of the socket source: only one query works.

2017-08-13 Thread Silvio Fiorito
Hi Gerard, Each query has its own distinct query plan and tracks offsets independently from other queries. Also, each query will generate a dynamic group id to ensure it gets all events and appears as a new consumer from Kafka’s perspective, that’s done internally to the Kafka source. That’s