Sxnan commented on code in PR #800: URL: https://github.com/apache/flink-web/pull/800#discussion_r2199825296
########## docs/content/posts/2025-07-31-release-2.1.0.md: ########## @@ -0,0 +1,438 @@ +--- +authors: + - reswqa: + name: "Ron Liu" + twitter: "Ron999" + +date: "2025-07-31T08:00:00Z" +subtitle: "" +title: Announcing the Release of Apache Flink 2.1 +aliases: + - /news/2025/07/31/release-2.1.0.html +--- + +The Apache Flink PMC is pleased to announce the release of Apache Flink 2.1.0. As usual, we are +looking at a packed release with a wide variety of improvements and new features. Overall, 116 +people contributed to this release completing 15 FLIPs and 200+ issues. Thank you! + +Let's dive into the highlights. + +# Flink SQL Improvements + +## Realtime AI Function + +Since Flink 2.0, we have introduced dedicated syntax for AI models, enabling users to define models as easily as creating catalog objects +and invoke them like standard functions or table functions in SQL statements. + +In Flink 2.1, we expanded the `ML_PREDICT` table-valued function (TVF) to perform realtime model inference in SQL queries, applying machine learning models to data streams seamlessly. +The implementation supports both embedded models (including OpenAI) and custom model providers, accelerating Flink's evolution from a real-time data processing engine to a unified realtime AI platform. +Looking ahead, we plan to introduce more AI functions such as `ML_EVALUATE`, `VECTOR_SEARCHOR` to unlock end-to-end experience for real-time data processing, model training, and inference. + +Take the following SQL statements as an example: +```sql +-- Declare a AI model +CREATE MODEL `my_model` +INPUT (f1 INT, f2 STRING) +OUTPUT (label STRING, probs ARRAY<FLOAT>) +WITH( + 'task' = 'classification', + 'type' = 'remote', + 'provider' = 'openai', + 'openai.endpoint' = 'https://api.openai.com/v1/llm/v1/chat', + 'openai.api_key' = 'abcdefg' +); + +-- Basic usage +SELECT * FROM ML_PREDICT( + TABLE input_table, + MODEL my_model, + DESCRIPTOR(feature1, feature2) +); + +-- With configuration options +SELECT * FROM ML_PREDICT( + TABLE input_table, + MODEL my_model, + DESCRIPTOR(feature1, feature2), + MAP['async', 'true', 'timeout', '100s'] +); + +-- Using named parameters +SELECT * FROM ML_PREDICT( + INPUT => TABLE input_table, + MODEL => MODEL my_model, + ARGS => DESCRIPTOR(feature1, feature2), + CONFIG => MAP['async', 'true'] +); +``` + +**More Information** +* [FLINK-34992](https://issues.apache.org/jira/browse/FLINK-34992) +* [FLINK-37777](https://issues.apache.org/jira/browse/FLINK-37777) +* [FLIP-437](https://cwiki.apache.org/confluence/display/FLINK/FLIP-437%3A+Support+ML+Models+in+Flink+SQL) +* [FLIP-525](https://cwiki.apache.org/confluence/display/FLINK/FLIP-525%3A+Model+ML_PREDICT%2C+ML_EVALUATE+Implementation+Design) +* [Model Inference](https://nightlies.apache.org/flink/flink-docs-release-2.1/docs/dev/table/sql/queries/model-inference/) + +## Variant Type + +Variant is a new data type for semi-structured data(e.g. JSON), it supports storing any +semi-structured data, including ARRAY, MAP(with STRING keys), and scalar types—while preserving +field type information in a JSON-like structure. Unlike ROW and STRUCTURED types, VARIANT provides +superior flexibility for handling deeply nested and evolving schemas. + +Users can use `PARSE_JSON` or`TRY_PARSE_JSON` to convert JSON-formatted VARCHAR data to VARIANT. In +addition, table formats like Apache Paimon and Iceberg now support the VARIANT type, this enable Review Comment: Iceberg doesn't support VARIANT at the moment. And there will be some development to be made to Paimon to allow writing Variant with Flink. Maybe we can say "table formats like Apache Paimon and Iceberg will support the VARIANT type" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
