Re: SQL materialized upsert tables

2018-02-21 Thread Fabian Hueske
Hi Elias, it would be great if you could let us know if the approach works. Btw. I should point out that the join in a query like: SELECT s.tstamp, s.item, s.score, t.source FROM( SELECT item, source, MAX(tstamp), LAST_VAL(score, tstamp) FROM Telemetry GROUP BY item, source ) INNER JOIN

Re: SQL materialized upsert tables

2018-02-21 Thread Elias Levy
On Wed, Feb 21, 2018 at 3:24 AM, Fabian Hueske wrote: > Hi Elias, > > Flink does not have built-in support for upsert stream -> table > conversions, yet. However, the community is working on that (see FLINK-8545 > [1]). > With a workaround, you can also solve the issue with

Re: SQL materialized upsert tables

2018-02-20 Thread Elias Levy
[ Adding the list back in, as this clarifies my question ] On Tue, Feb 20, 2018 at 3:42 PM, Darshan Singh wrote: > I am no expert in Flink but I will try my best. Issue you mentioned will > be with all streaming systems even with Kafka KTable I use them a lot for >

SQL materialized upsert tables

2018-02-20 Thread Elias Levy
I noticed that has been significant work on the SQL / Table subsystem and decided to evaluate it for one of our use cases. The use case requires the joining of two streams, which can be considered a stream of table upserts. Critically, when joining the streams, we only want to join against the