This is an automated email from the ASF dual-hosted git repository.
aloyszhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new a5b5d3f7eed [INLONG-1096][Release] Add blog for the 2.1.0 release
(#1099)
a5b5d3f7eed is described below
commit a5b5d3f7eed38128fc9e026dcca17a7c122e202e
Author: justinwwhuang <[email protected]>
AuthorDate: Fri Jan 17 10:03:27 2025 +0800
[INLONG-1096][Release] Add blog for the 2.1.0 release (#1099)
---
blog/2025-01-03-release-2.1.0.md | 167 +++++++++++++++++++++
blog/img/2.1.0/2.1.0-agent-node.png | Bin 0 -> 126513 bytes
blog/img/2.1.0/2.1.0-agent-param.png | Bin 0 -> 43470 bytes
blog/img/2.1.0/2.1.0-agent-type.png | Bin 0 -> 68534 bytes
blog/img/2.1.0/2.1.0-dashboard-operate.png | Bin 0 -> 104451 bytes
blog/img/2.1.0/2.1.0-dashboard-select.png | Bin 0 -> 150822 bytes
blog/img/2.1.0/2.1.0-sort-dirty.png | Bin 0 -> 840861 bytes
.../2025-01-03-release-2.1.0.md | 151 +++++++++++++++++++
.../img/2.1.0/2.1.0-agent-node.png | Bin 0 -> 126217 bytes
.../img/2.1.0/2.1.0-agent-param.png | Bin 0 -> 153087 bytes
.../img/2.1.0/2.1.0-agent-type.png | Bin 0 -> 146030 bytes
.../img/2.1.0/2.1.0-dashboard-operate.png | Bin 0 -> 422959 bytes
.../img/2.1.0/2.1.0-dashboard-select.png | Bin 0 -> 143806 bytes
.../img/2.1.0/2.1.0-sort-dirty.png | Bin 0 -> 840861 bytes
14 files changed, 318 insertions(+)
diff --git a/blog/2025-01-03-release-2.1.0.md b/blog/2025-01-03-release-2.1.0.md
new file mode 100644
index 00000000000..a58acb4cdf3
--- /dev/null
+++ b/blog/2025-01-03-release-2.1.0.md
@@ -0,0 +1,167 @@
+---
+title: Release 2.1.0
+author: justinwwhuang
+author_url: https://github.com/justinwwhuang
+author_image_url: https://avatars.githubusercontent.com/justinwwhuang
+tags: [Apache InLong, Version]
+---
+
+Apache InLong has recently released version 2.1.0, which has closed over 120
issue, including more than 4 major features and over 110 optimizations.
+The main accomplishments include Dashboard supports batch operation of nodes,
Manager supports multiple scheduling engines, Agent supports COS data sources,
+Sort supports archiving dirty data through the InLong SDK. Simultaneously
optimize the user experience of Apache InLong operations and maintenance. In
Apache InLong 2.1.0 version, a large number of other features have also been
completed.
+<!--truncate-->
+
+## About Apache InLong
+As the industry's first one-stop, all-scenario massive data integration
framework, Apache InLong provides automated, secure, reliable,
+and high-performance data transmission capabilities, enabling businesses to
quickly build stream-based data analysis, modeling, and applications.
+Currently, InLong is widely used in various industries including advertising,
payment, social networking, gaming, and artificial intelligence,
+serving thousands of businesses, with high-performance scenarios processing
over hundreds of billions of records per day and highly reliable scenarios
+handling over tens of trillions of records per day.
+
+The core keywords for InLong's project positioning are "one-stop,"
"all-scenario," and "massive data." For "one-stop,"
+we aim to shield technical details, provide complete data integration and
supporting services, and achieve out-of-the-box usability;
+for "all-scenario," we aim to offer comprehensive solutions covering common
data integration scenarios in the big data field;
+for "massive data," we hope to leverage architectural advantages such as
layered data links, fully extensible components,
+and built-in multi-cluster management to stably support even larger data
volumes based on hundreds of billions of records per day.
+
+## 2.1.0 Overview
+
+Apache InLong has recently released version 2.1.0, which has closed over 120
issue, including more than 4 major features and over 110 optimizations.
+The main accomplishments include
+- Dashboard supports batch operation of nodes
+- Manager supports multiple scheduling engines
+- Agent supports COS data sources
+- Sort supports archiving dirty data through the InLong SDK.
+
+Simultaneously optimize the user experience of Apache InLong operations and
maintenance. In Apache InLong 2.1.0 version, a large number of other features
have also been completed.
+
+### Dashboard Module
+- Support COS data source
+- Support batch operation of agents: restart, upgrade
+- Support exporting audit data as CSV files
+- Support sorting of audit data and comparison of differences
+- Support queries for all types of indicators
+- Support data preview field segmentation
+### Manager Module
+- Support COS data source
+- Support managing multiple scheduling engines: AirFlow、DolphinScheduler
+- Support dirty data management and querying
+- Support querying heartbeat information based on IP
+- Limit one IP to only belong to one cluster
+- Provide an API for querying of dirty data archiving
+### Agent Module
+- Support COS data source
+- Support quick startup and shutdown
+- Support starting multiple instances
+- Support data supplementation in chronological order
+- Optimize the logic of the Installer process guardian for Agent
+- Support supplementary recording based on local data time
+### Sort Module
+- Added Elasticsearch connector based on Flink 1.18
+- Support KV separation on Kafka Sink
+- Support audit data reporting
+- Tube Connector source supports dirty data archiving
+### SDK Module
+- Transform SDK adds 7 new functions
+- Add Dirty Data Archiving SDK
+### Audit Module
+- Audit Proxy increases metric reporting
+- Audit Store adds metric reporting
+- Audit Service increases metric reporting
+- Add asynchronous flush audit data interface
+### TubeMQ Module
+- Write the consumption offset information to a local file
+- Optimize the load balancing logic of the Go version SDK
+### Others
+- Pipeline supports parallel build
+- Support Manager to configure volumes
+## 2.1.0 Feature Introduction
+### Dashboard supports batch operation of agents
+This feature is mainly used for the operation of Inlong Agent: mainly for
upgrading and restarting:
+- After finding the cluster in cluster management, select multiple nodes to
operate on and click on batch operation.
+
+ 
+
+- Select the operation type and fill in the required parameters for the
corresponding operation, then click OK.
+
+ 
+
+This feature optimizes the operation and maintenance experience of Inlong:
interface based operation eliminates the need to operate DB and increases the
cohesion of Inlong:
+- Visual Agent version upgrade, which can be upgraded in batches and at
regular intervals to control upgrade risks.
+- During agent fault recovery, this function can be used to quickly restart.
+
+Thanks to @[wohainilaodou](https://github.com/wohainilaodou) for their
contributions to this feature. For more details, please refer to
[INLONG-11187](https://github.com/apache/inlong/issues/11187)
+### Manager supports multiple scheduling engines
+Previously, for offline data synchronization, Inlong only supported Quartz
scheduling engine. This version has added two third-party engines:
DolphinScheduler and AirFlow.
+#### AirFlow engine
+- In order to facilitate the maintenance and expansion of AirFlow interface
support in the future, AirflowApi interface and BaseAirflowApi abstract class
have been designed, and subsequent extensions only need to be based on this
foundation.
+- Implement a unified request class AirflowServerClient for the interface.
+- Add two interceptors in OkHttpClient: AirflowAuthInterceptor for unified
authorization of interfaces; LoggingInterceptor is used for logging.
+
+Thanks to @[Zkplo](https://github.com/Zkplo) for their contributions to this
feature. For more details, please refer to
[INLONG-11400](https://github.com/apache/inlong/issues/11400)
+#### DolphinScheduler engine
+-Add the DolphinScheduler package to org.apache.inlong.manager.schedule
+-Add client and engine for DS, as well as util for operating open APIs for DS
+-Add pojo class for DS interaction
+
+Thanks to @[emptyOVO](https://github.com/emptyOVO) for their contributions to
this feature. For more details, please refer to
[INLONG-11401](https://github.com/apache/inlong/issues/11401)
+### Agent supports COS data source
+- Create a new COS type node and fill in the corresponding bucket name,
credential ID, credential key, and region.
+
+ 
+
+- Create a new COS type data source, select the corresponding node, IP, and
file path.
+
+ 
+
+ 
+
+This feature supports direct data collection from COS storage, and businesses
do not need to download COS files locally for data collection.
+Thanks to @[justinwwhuang](https://github.com/justinwwhuang) for their
contributions to this feature. For more details, please refer to
[INLONG-11187](https://github.com/apache/inlong/issues/11187)
+
+### Sort supports archiving dirty data through the InLong SDK.
+Added the ability to report dirty data to specified GroupId and StreamId
through the InLong SDK. Users can choose to export dirty data or consume it
independently from Pulsar.
+
+
+
+The following configuration needs to be added to the Connector:
+```
+'dirty.side-output.inlong-sdk.inlong-auth-key' = 'your auth key',
+
+'dirty.side-output.inlong-sdk.inlong-auth-id' = 'your auth id',
+
+'dirty.side-output.enable' = 'true',
+
+'dirty.side-output.inlong-sdk.inlong-group-id' = 'target_inlong_group_id',
+'dirty.side-output.inlong-sdk.inlong-stream-id' = 'target_inlong_stream_id',
+
+'dirty.side-output.labels' =
'groupId=xx&streamId=xx&serverType=tube&dataflowId=xx',
+
+'dirty.side-output.inlong-sdk.inlong-manager-addr' = 'xxx',
+
+'dirty.side-output.connector' = 'inlong-sdk',
+
+'dirty.ignore' = 'true',`
+```
+Thanks to @[vernedeng](https://github.com/vernedeng) and
@[fuweng11](https://github.com/fuweng11) for their contributions to this
feature. For more details,
+please refer to [INLONG-11481](https://github.com/apache/inlong/issues/11481)
and [INLONG-11508](https://github.com/apache/inlong/issues/11508)
+
+
+## Future Plans
+In version 2.1.0, we have enriched and improved our operational capabilities.
Welcome everyone to use it. If you have more scenarios and requirements,
+or encounter any problems during use, please feel free to raise issues and PR.
In future versions, the InLong community will continue to:
+
+- Support more data source collection capabilities
+
+- Enrich Flink 1.15, 1.18 Connector
+
+- Continuously enhance Transform capabilities.
+
+- Provide real-time synchronization support for more data sources and targets.
+
+- Optimize SDK capabilities and user experience
+
+- Optimize Dashboard experience
+
+
+We also look forward to more developers interested in InLong to contribute and
help drive the project's development!
\ No newline at end of file
diff --git a/blog/img/2.1.0/2.1.0-agent-node.png
b/blog/img/2.1.0/2.1.0-agent-node.png
new file mode 100644
index 00000000000..d7e8d968eef
Binary files /dev/null and b/blog/img/2.1.0/2.1.0-agent-node.png differ
diff --git a/blog/img/2.1.0/2.1.0-agent-param.png
b/blog/img/2.1.0/2.1.0-agent-param.png
new file mode 100644
index 00000000000..315448cdab9
Binary files /dev/null and b/blog/img/2.1.0/2.1.0-agent-param.png differ
diff --git a/blog/img/2.1.0/2.1.0-agent-type.png
b/blog/img/2.1.0/2.1.0-agent-type.png
new file mode 100644
index 00000000000..03882a1f247
Binary files /dev/null and b/blog/img/2.1.0/2.1.0-agent-type.png differ
diff --git a/blog/img/2.1.0/2.1.0-dashboard-operate.png
b/blog/img/2.1.0/2.1.0-dashboard-operate.png
new file mode 100644
index 00000000000..54b88df145c
Binary files /dev/null and b/blog/img/2.1.0/2.1.0-dashboard-operate.png differ
diff --git a/blog/img/2.1.0/2.1.0-dashboard-select.png
b/blog/img/2.1.0/2.1.0-dashboard-select.png
new file mode 100644
index 00000000000..b51cf862c03
Binary files /dev/null and b/blog/img/2.1.0/2.1.0-dashboard-select.png differ
diff --git a/blog/img/2.1.0/2.1.0-sort-dirty.png
b/blog/img/2.1.0/2.1.0-sort-dirty.png
new file mode 100644
index 00000000000..934bd3fc414
Binary files /dev/null and b/blog/img/2.1.0/2.1.0-sort-dirty.png differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-blog/2025-01-03-release-2.1.0.md
b/i18n/zh-CN/docusaurus-plugin-content-blog/2025-01-03-release-2.1.0.md
new file mode 100644
index 00000000000..bbdba1d00cf
--- /dev/null
+++ b/i18n/zh-CN/docusaurus-plugin-content-blog/2025-01-03-release-2.1.0.md
@@ -0,0 +1,151 @@
+---
+title: 2.1.0 版本发布
+author: justinwwhuang
+author_url: https://github.com/justinwwhuang
+author_image_url: https://avatars.githubusercontent.com/justinwwhuang
+tags: [Apache InLong, Version]
+---
+
+Apache InLong（应龙） 最近发布了 2.1.0 版本，该版本关闭了约 120+ 个 issue，包含 4+ 个大特性和 110+
个优化，主要完成了 Dashboard 支持批量操作节点、Manager 支持多种调度引擎、Agent 支持 COS 数据源、Sort 支持通过 InLong
SDK 进行脏数据的归档。同时优化 Apache InLong 运营运维的使用体验。Apache InLong 2.1.0 版本中，还完成了大量其它特性。
+<!--truncate-->
+
+## 关于 Apache InLong
+作为业界首个一站式、全场景海量数据集成框架，Apache InLong（应龙）
提供了自动、安全、可靠和高性能的数据传输能力，方便业务快速构建基于流式的数据分析、建模和应用。目前 InLong
正广泛应用于广告、支付、社交、游戏、人工智能等各个行业领域，服务上千个业务，其中高性能场景数据规模超百万亿条/天，高可靠场景数据规模超十万亿条/天。
+InLong
项目定位的核心关键词是“一站式”、“全场景”和“海量数据”。对于“一站式”，我们希望屏蔽技术细节、提供完整数据集成及配套服务，实现开箱即用；对于“全场景”，我们希望提供全方位的解决方案，覆盖大数据领域常见的数据集成场景；对于“海量数据”，我们希望通过架构上的数据链路分层、全组件可扩展、自带多集群管理等优势，在百万亿条/天的基础上，稳定支持更大规模的数据量。
+## 2.1.0 版本总览
+
+Apache InLong（应龙） 最近发布了 2.1.0 版本，该版本关闭了约 120+ 个 issue，包含 4+ 个大特性和 110+
个优化，主要完成了
+- Dashboard 支持批量操作节点
+- Manager 支持多种调度引擎
+- Agent 支持 COS 数据源
+- Sort 支持通过 InLong SDK 进行脏数据的归档。
+
+同时优化 Apache InLong 运营运维的使用体验。Apache InLong 2.1.0 版本中，还完成了大量其它特性。
+### Dashboard 模块
+- 支持 COS 数据源
+- 支持 Agent 批量操作：重启、升级
+- 支持审计数据导出成 CSV 文件
+- 支持审计数据排序、差值对比
+- 支持全量指标类型查询
+- 支持数据预览字段分割
+### Manager 模块
+- 支持 COS 数据源
+- 支持管理多种调度引擎：AirFlow、DolphinScheduler
+- 支持脏数据管理以及查询
+- 支持根据 IP 查询心跳信息
+- 限制一个 IP 只能归属到一个集群
+- 提供脏数据归档查询 API
+### Agent 模块
+- 支持 COS 数据源
+- 支持快速启动、关闭
+- 支持启动多实例
+- 支持按时间顺序补录数据
+- 优化 Installer 守护 Agent 逻辑
+- 支持按照数据本地时间进行补录
+### Sort 模块
+- 新增基于 Flink 1.18 的 Elasticsearch connector
+- 支持 Kafka Sink 端的 KV 分隔
+- 支持审计数据上报
+- Tube Connector source 支持脏数据归档
+### SDK 模块
+- Transform SDK 新增 7 种函数
+- 新增脏数据归档 SDK
+### Audit 模块
+- Audit Proxy 增加指标上报
+- Audit Store 增加指标上报
+- Audit Service 增加指标上报
+- 新增异步刷审计数据接口
+### TubeMQ 模块
+- 将消费位点信息写到本地文件
+- 优化 Go 版本 SDK 负载均衡逻辑
+### 其他
+- 流水线支持并行构建
+- 支持管理器配置卷
+## 2.1.0 版本主要特性
+### Dashboard 支持批量操作节点
+改特性主要是用于 Inlong Agent 的操作：主要是升级和重启：
+- 在集群管理找到集群后，选择要操作的多个节点，点击 批量操作。
+
+ 
+
+- 选择操作类型，并填写对应操作所需的参数，点击确定即可。
+
+ 
+
+该功能优化了 Inlong 的运维体验：界面化操作，运维不再需要操作 DB，增加了 Inlong 的内聚性：
+- 可视化 Agent 版本升级，可分批、定时升级，控制升级风险。
+- Agent 故障恢复时可通过该功能进行快速重启。
+
+感谢 @[wohainilaodou](https://github.com/wohainilaodou) 的贡献，详情参考
[INLONG-11187](https://github.com/apache/inlong/issues/11187)
+### Manager 支持多种调度引擎
+之前对于离线数据同步 Inlong 只支持 Quartz 调度引擎。这次的版本则增加了两个第三方引擎：DolphinScheudler 和 AirFlow。
+#### AirFlow 引擎
+- 为了便于未来 AirFlow 接口支持的维护和扩展，设计了 AirflowApi 接口和 BaseAirflowApi
抽象类，后续扩展只需要在此基础上进行。
+- 为接口实现统一的请求类 AirflowServerClient。
+- 在 OkHttpClient 中添加两个拦截器：AirflowAuthInterceptor 用于接口的统一授权；LoggingInterceptor
用于日志记录。
+
+感谢 @[Zkplo](https://github.com/Zkplo) 的贡献，详情参考
[INLONG-11400](https://github.com/apache/inlong/issues/11400)
+#### DolphinScheudler 引擎
+- 在 org.apache.inlong.manager.schedule 中添加 DolphinScheudler 包
+- 添加 DS 的客户端和引擎，以及用于操作 DS 的开放 API 的 Util
+- 为 DS 交互添加 pojo 类
+
+感谢 @[emptyOVO](https://github.com/emptyOVO) 的贡献，详情参考
[INLONG-11401](https://github.com/apache/inlong/issues/11401)
+### Agent 支持 COS 数据源
+- 新建 COS 类型节点，填写相应的 桶名、凭据 ID、凭据密钥和地区即可。
+
+ 
+
+- 新建 COS 类型数据源，选择相应的节点、IP、文件路径即可。
+
+ 
+
+ 
+
+该功能支持从 COS 存储直接采集数据，业务不需要把 COS 文件下载到本地再做数据采集。
+感谢 @[justinwwhuang](https://github.com/justinwwhuang) 的贡献，详情参考
[INLONG-11187](https://github.com/apache/inlong/issues/11187)
+
+### Sort 支持通过 InLong SDK 进行脏数据的归档
+新增通过 InLong SDK 上报脏数据至指定 GroupId 和 StreamId 的能力。用户可以选择将脏数据接出，或从 Pulsar 中自主消费。
+
+
+
+需要在 Connector 中增加如下配置
+```
+'dirty.side-output.inlong-sdk.inlong-auth-key' = 'your auth key',
+
+'dirty.side-output.inlong-sdk.inlong-auth-id' = 'your auth id',
+
+'dirty.side-output.enable' = 'true',
+
+'dirty.side-output.inlong-sdk.inlong-group-id' = 'target_inlong_group_id',
+'dirty.side-output.inlong-sdk.inlong-stream-id' = 'target_inlong_stream_id',
+
+'dirty.side-output.labels' =
'groupId=xx&streamId=xx&serverType=tube&dataflowId=xx',
+
+'dirty.side-output.inlong-sdk.inlong-manager-addr' = 'xxx',
+
+'dirty.side-output.connector' = 'inlong-sdk',
+
+'dirty.ignore' = 'true',`
+```
+感谢 @[vernedeng](https://github.com/vernedeng) 和
@[fuweng11](https://github.com/fuweng11) 的贡献，
+详情参考 [INLONG-11481](https://github.com/apache/inlong/issues/11481) 和
[INLONG-11508](https://github.com/apache/inlong/issues/11508)
+
+
+## 未来规划
+在 2.1.0 版本中，我们丰富、完善了运维能力。欢迎大家使用，如果有更多场景和需求，或者使用期间遇到的问题， 欢迎大家提 issue和
PR。在后续的版本中，InLong 社区将继续：
+
+- 支持更多数据源采集能力
+
+- 丰富 Flink 1.15、1.18 Connector
+
+- 丰富 Transform 能力，并且集成到 InLong 的各个模块
+
+- 实时同步支持更多数据源、数据目标
+
+- 优化 SDK 能力和使用体验
+
+- 优化 Dashboard 体验
+
+我们也期待更多对 InLong 感兴趣的开发者可以参与贡献。
\ No newline at end of file
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-agent-node.png
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-agent-node.png
new file mode 100644
index 00000000000..fccf516a3e7
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-agent-node.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-agent-param.png
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-agent-param.png
new file mode 100644
index 00000000000..a7cde9082eb
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-agent-param.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-agent-type.png
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-agent-type.png
new file mode 100644
index 00000000000..31946c5acca
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-agent-type.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-dashboard-operate.png
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-dashboard-operate.png
new file mode 100644
index 00000000000..c7527a5c2ca
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-dashboard-operate.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-dashboard-select.png
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-dashboard-select.png
new file mode 100644
index 00000000000..bddacc3ac91
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-dashboard-select.png
differ
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-sort-dirty.png
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-sort-dirty.png
new file mode 100644
index 00000000000..934bd3fc414
Binary files /dev/null and
b/i18n/zh-CN/docusaurus-plugin-content-blog/img/2.1.0/2.1.0-sort-dirty.png
differ
