dark-Qy opened a new pull request, #11933: URL: https://github.com/apache/inlong/pull/11933
Fixes [#11929](https://github.com/apache/inlong/issues/11929) ### Motivation As described in issue #11929, the InLong DataProxy Go SDK currently only supports TCP for data reporting. While the DataProxy server provides a public HTTP API for this purpose, it is not yet integrated into the Go SDK. This requires users to write their own HTTP client logic, which is inconvenient and leads to inconsistent implementations. This pull request addresses this gap by integrating a native HTTP reporting mechanism into the Go SDK, providing users with a more flexible choice of protocols. This is particularly useful in environments where TCP traffic is restricted or when HTTP is preferred for its simplicity and firewall-friendliness. ### Modifications This pull request introduces HTTP reporting capabilities with the following key changes: 1. **Added `Protocol` Enum and `WithProtocol` Option**: - A new `Protocol` enum (`ProtocolTCP`, `ProtocolHTTP`) has been added to allow protocol selection. - The `WithProtocol(protocol Protocol)` option enables users to easily switch to HTTP during client initialization. The default remains TCP to ensure backward compatibility. 2. **Implemented a `HTTPClient`**: - A new `http_client.go` file encapsulates the `HTTPClient`, which manages a pooled `http.Client`. - It handles the construction of HTTP requests according to the [DataProxy HTTP API specification](https://inlong.apache.org/docs/sdk/dataproxy-sdk/http/) and processes server responses. - To ensure balanced load distribution across DataProxy nodes, it uses an atomic counter for round-robin endpoint selection. 3. **Implemented a `HTTPWorker`**: - A new `http_worker.go` file contains the `HTTPWorker`, which batches messages and sends them asynchronously. - To maximize performance and resilience, the worker is fully non-blocking. When a send operation fails, it triggers an **asynchronous exponential backoff** retry mechanism, allowing it to continue processing new messages without delay. 4. **Seamless Integration into the Main `Client`**: - The main `client.go` has been updated to act as a dispatcher. Key methods like `initAll`, `Send`, `SendAsync`, and `Close` now delegate to the appropriate TCP or HTTP implementation based on the configured protocol. ### Verifying this change - [x] This change added tests and can be verified as follows: - Added `ExampleClient_Send_http` and `ExampleClient_SendAsync_http` in `dataproxy/example_test.go`. These examples demonstrate how to initialize the client with the HTTP protocol and use both synchronous (`Send`) and asynchronous (`SendAsync`) methods. - These examples can be executed via `go test`. They serve as both a verification of the implementation and as live documentation for users. ### Documentation - Does this pull request introduce a new feature? **yes** - If yes, how is the feature documented? **docs** - This PR introduces a new feature that will be documented on the official InLong website. A follow-up issue will be created to track this documentation update. The new options (`WithProtocol`, `WithHTTPTimeout`, `WithHTTPMaxConns`) and a usage example for the HTTP protocol will be added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
