Hi everyone, I would like to start a discussion thread on "Support Cython Optimizing Python User Defined Function"
Scalar Python UDF FLIP-58[1] has already been supported in release 1.10 and Python UDTF will be supported in the coming release of 1.11. In release 1.10, we focused on supporting UDF features and did not make many optimizations in terms of performance. Although we have made a lot of optimizations in master[2], Cython can further greatly improve the performance of Python UDF. Robert Metzger, Jincheng Sun and I have discussed offline and have drafted the FLIP-121[3]. It includes the following items: - Introduces Cython implementation of coder and operations - Doc changes for building sdist and wheel packages from source code - Solutions for packages building Looking forward to your feedback! Best, Xingbo [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table [2] https://issues.apache.org/jira/browse/FLINK-16747 [3] https://cwiki.apache.org/confluence/display/FLINK/FLIP-121%3A+Support+Cython+Optimizing+Python+User+Defined+Function
