Hi Renjie,
it would be great if we can reuse my QueryableStateTableSink
implementation. As far as I know, the queryable state interfaces changed
a bit in the last months and a new queryable state client as been added.
So my branch needs to be rebased to the current APIs.
We don't need a big design document but at least some paragraphs how the
APIs should look like. How do you want to extend the SQL client for
that? We can also introduce the QueryableStateTableSink in one pull
request and do the API in a follow-up PR.
What do you think?
Regards,
Timo
Am 3/5/18 um 3:52 AM schrieb Renjie Liu:
Hi, Timo:
I've read your QueryableStateTableSink implementation and that basically
implementes what I want to do. I also want to extend SQL client so that
user can do point query against the table sink. Do we still need a design
doc for that? It seems that I just need to finish the left part and do some
test against it.
Hi, Stefano:
Your requirement needs some changes to the flink table implementation but I
don't know why you need that? For debugging? The operator state is internal
and subject to optisimation logic, so I think it maybe meanless to expose
that.
On Fri, Mar 2, 2018 at 9:37 PM Stefano Bortoli <stefano.bort...@huawei.com>
wrote:
Hi Timo, Renjie,
What I was thinking did not include the QueryableStateTableSink, but
rather tap in directly into the state of a streaming operator. Perhaps it
is the same thing, but just it sounds not intuitive to consider it a sink.
So, we would need a way to configure the environment for the query to
share the "state name" before the query is executed, and then use this to
create the hook for the queriable state in the operator. Perhaps extend the
current codegen and operator implementations to get as a parameter the
StateDescriptor to be inquired.
Looking forward for the design document, will be happy to give you
feedback.
Best,
Stefano
-----Original Message-----
From: Renjie Liu [mailto:liurenjie2...@gmail.com]
Sent: Friday, March 02, 2018 11:42 AM
To: dev@flink.apache.org
Subject: Re: StreamSQL queriable state
Great, thank you.
I'll start by writing a design doc.
On Fri, Mar 2, 2018 at 6:40 PM Timo Walther <twal...@apache.org> wrote:
I gave you contributor permissions in Jira. You should be able to
assign it to yourself now.
Am 3/2/18 um 11:33 AM schrieb Renjie Liu:
Hi, Timo:
It seems that I can't assign it to myself. Could you please help to
assign
that to me?
My jira username is liurenjie1024 and my email is
liurenjie2...@gmail.com
On Fri, Mar 2, 2018 at 6:24 PM Timo Walther <twal...@apache.org>
wrote:
Hi Renjie,
that would be great. There is already a Jira issue for it:
https://issues.apache.org/jira/browse/FLINK-6968
Feel free to assign it to yourself. You can reuse parts of my code
if you want. But maybe it would make sense to have a little design
document first about what we want to support.
Regards,
Timo
Am 3/2/18 um 11:10 AM schrieb Renjie Liu:
Hi, Timo, I've been planning on the same thing and would like to
contribute
that.
On Fri, Mar 2, 2018 at 6:05 PM Timo Walther <twal...@apache.org>
wrote:
Hi Stefano,
yes there are plan in this direction. Actually, I already worked
on
such
a QueryableStateTableSink [1] in the past but never finished it
because
of priority shifts. Would be great if somebody wants to
contribute
this
functionality :)
Regards,
Timo
[1] https://github.com/twalthr/flink/tree/QueryableTableSink
Am 3/2/18 um 10:58 AM schrieb Stefano Bortoli:
Hi guys,
I am checking out the queriable state API, and it seems that
most of
the
tooling is already available. However, the queriable state is
available
just for the streaming API, not at the StreamSQL API level. In
principle,
as the flink-table is aware of the query semantic and data output
type,
it
should be possible to configure the query compilation to nest
queriable
state in the process/window functions. Is there any plan in this
direction?
Best,
Stefano
--
Liu, Renjie
Software Engineer, MVAD
--
Liu, Renjie
Software Engineer, MVAD
--
Liu, Renjie
Software Engineer, MVAD
