The design looks good to me now. +1 to start the vote if there are no more
comments..
Best
Yun Tang
From: Dian Fu
Sent: Tuesday, January 5, 2021 13:32
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-153: Support state access in Python DataStream API
link supports to migrate state automatically if
>>> new provided serializer is compatible with old serializer[4]. I'm afraid
>> if
>>> python data stream API wraps user's serializer as
>>> BytePrimitiveArraySerializer, we will lose such functionality. Moreover,
>>> RocksDB will migrate state au
this will
> > break if python related bytes involved.
> > >>> 5. Queryable state client. Currently, we only have java-based
> > queryable state client [6], and we need another python-based queryable
> > state client if involved python bytes.
> > >>>
t; https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/experimental.html#reinterpreting-a-pre-partitioned-data-stream-as-keyed-stream
> >>> [3]
> https://github.com/apache/flink/blob/58cc2a5fbd419d6a9e4f9c251ac01ecf59a8c5a2/flink-streaming-java/src/main/java/org/apache/flink/s
t;>> https://github.com/apache/flink/blob/58cc2a5fbd419d6a9e4f9c251ac01ecf59a8c5a2/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/InternalTimerServiceImpl.java#L203
>>> [4]
>>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/schema_evolution.html#evolving-state-schema
>
https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/stream/state/custom_serialization.html#off-heap-state-backends-eg-rocksdbstatebackend
>> [6]
>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#example
>>
>> Be
html#example
>
> Best
> Yun Tang
>
>
> ________________
> From: Shuiqiang Chen
> Sent: Wednesday, December 16, 2020 17:32
> To: dev@flink.apache.org
> Subject: Re: [DISCUSS] FLIP-153: Support state access in Python DataStream API
>
> H
From: Shuiqiang Chen
Sent: Wednesday, December 16, 2020 17:32
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-153: Support state access in Python DataStream API
Hi Xingbo,
Thank you for your valuable suggestions.
Indeed, we need to provide clearer abstractions
Hi Xingbo,
Thank you for your valuable suggestions.
Indeed, we need to provide clearer abstractions for StateDescriptor and State
APIs, I have updated the FLIP accordingly. Looking forward to your feedbacks!
Best,
Shuiqiang
> 在 2020年12月14日，上午11:27，Xingbo Huang 写道：
>
> Thanks Shuiqiang for
Thanks Shuiqiang for starting this discussion.
Big +1 for this feature. State access support can further improve the
functionality of our existing Python DataStream.
I have 2 comments regarding to the design doc:
a) I think that `StateDescriptor` needs to hold the variable `typeInfo`
instead of
Hi devs,
In FLIP-130, we have already supported Python DataStream stateless APIs so
that users are able to perform some basic data transformations. To
implement more complex data processing, we need to provide state access
support. So I would propose to add state access APIs in Python DataStream
11 matches
Mail list logo
