退订
Hi all,
We are trying to ingest large amounts of data (20TB) from S3 using Flink
filesystem connector to bootstrap a Hudi table. Data are well partitioned
in S3 by date/time, but we have been facing OOM issues in Flink jobs, so we
wanted to update the Flink job to ingest the data chunk by chuck
It sounds like the better option is to make a Flink bridge release that
runs both Kryo v2 and Kryo v5 side-by-side.
The code base for the Flink bridge release should only use Kryo v2 for
deserializing legacy data, and use Kryo v5 for serializing+deserializing
new data.
User APIs registering
> Can't programmers just code up migration tools to the current version
of Kryo or whatever serialization platform you choose?
Well yes, if someone writes a tool to implement a reasonable migration
path than we may be able to upgrade Kryo.
Until that happens we are blocked on upgrading Kryo.
What do you mean you are blocked? Can't programmers just code up migration
tools to the current version of Kryo or whatever serialization platform you
choose?
Can't you follow the Kryo migration guide that supports loading data
serialized with Kryo v2 and reserializing with Kryo v5?
Hi all, I’m using the Flink k8s operator to run a SQL stream to/from
various connectors, and just added a Parquet format. I customized the image
a bit per the example (mostly by adding maven downloads of flink-connector*
jars). If I do that for flink-parquet-1.16.1 it fails on missing
Hi Emarotti,
It sounds like you want to process your streams based on the event time. This
means, all your processing, windowing, timer and co are based on a timestamp
that is provided by individual events. You can read more about here:
> you can't reasonably stay on the 2015 version forever, refuse to
adopt any of the updates or fixes in the 8 years since then, and
reasonably expect things to continue to work well.
We are well aware that Kryo is a ticking time bomb.
> Is there any possibility a future release of Flink can
退订
weijie guo 于2023年2月8日周三 05:29写道：
> Hi,
>
> 你需要发送邮件到 user-zh-unsubscr...@flink.apache.org
> 而不是 user-zh@flink.apache.org.
>
>
> Best regards,
>
> Weijie
>
>
> wujunxi <462329...@qq.com.invalid> 于2023年2月7日周二 16:52写道：
>
> > 退订
>
