Re: KeyedProcessFunction within an iteration

2023-02-18 Thread Zhipeng Zhang
Hi Lorenzo, Could you provide some code example to reproduce your question? As I understand, IterativeStream#keyBy is supported since it is a subclass of DataStream. Moreover, we have implemented an unified iteration module for Flink [1] in Flink ML [2], which relies on Flink 1.15.2. Probably

[ANNOUNCE] Apache Flink ML 2.1.0 released

2022-07-12 Thread Zhipeng Zhang
The Apache Flink community is excited to announce the release of Flink ML 2.1.0! This release focuses on improving Flink ML's infrastructure, such as Python SDK, memory management, and benchmark framework, to facilitate the development of performant, memory-safe, and easy-to-use algorithm

[ANNOUNCE] Apache Flink ML 2.1.0 released

2022-07-12 Thread Zhipeng Zhang
The Apache Flink community is excited to announce the release of Flink ML 2.1.0! This release focuses on improving Flink ML's infrastructure, such as Python SDK, memory management, and benchmark framework, to facilitate the development of performant, memory-safe, and easy-to-use algorithm

Re: flink-ml algorithms

2022-06-05 Thread Zhipeng Zhang
Hi Natia, As I understand, the processing order of onlineKmeans is the same the input data. Are you running OnlineKmeans with using one data point with random initial KmeansModel? Could you use a fixed initial model following [1] and try out? [1]

Re: Flink 1.12.x DataSet --> Flink 1.14.x DataStream

2022-02-15 Thread Zhipeng Zhang
Hi Saravanan, One solution could be using a streamOperator to implement `BoundedOneInput` interface. An example code could be found here [1]. [1]

Re: Flink-ML: Sink model data in online training

2022-01-27 Thread Zhipeng Zhang
Hi thekingofcity, Thanks for your interest! Unfortunately we don't have an example for online learning for now. We are working on an online machine learning example. Hopefully it will be added here [1] in the next three weeks. [1] https://github.com/apache/flink-ml thekingofcity

Re: [DISCUSS] Drop Gelly

2022-01-04 Thread Zhipeng Zhang
> On Tue, 4 Jan 2022 at 02:57, Zhipeng Zhang > wrote: > >> Hi everyone, >> >> Thanks for starting the discussion :) >> >> We (Alink team [1]) are actually using part of the Gelly library to >> support graph algorithms (connected component, single source shor

Re: [DISCUSS] Drop Gelly

2022-01-03 Thread Zhipeng Zhang
Hi everyone, Thanks for starting the discussion :) We (Alink team [1]) are actually using part of the Gelly library to support graph algorithms (connected component, single source shortest path, etc.) for users in Alibaba Inc. As DataSet API is going to be dropped, shall we also provide a new

Re: flink-table-api-scala-bridge missing source files

2021-12-26 Thread Zhipeng Zhang
Hi Yuval, It seems that scala code is not included in `flink-table-api-scala-bridge_2.12-1.14.2-sources.jar` for now. You can find all the compiled code in the compiled jar (flink-table-api-scala-bridge_2.12-1.14.2.jar) for debugging. If we need to also include scala code in the sources.jar, we

Re: Kryo EOFException: No more bytes left

2021-12-19 Thread Zhipeng Zhang
Hi Dan, Could you provide the code snippet such that we can reproduce the bug here? Dan Hill 于2021年12月20日周一 07:18写道: > Hi. > > I was curious if anyone else has hit this exception. I'm using the > IntervalJoinOperator to two streams of protos. I registered the protos > with a kryo serializer.

ubsubscribe

2021-06-06 Thread Zhipeng Zhang
-- best, Zhipeng