davidradl commented on code in PR #28:
URL:
https://github.com/apache/flink-connector-http/pull/28#discussion_r2889038903
##########
docs/content.zh/docs/connectors/datastream/http.md:
##########
@@ -72,15 +72,15 @@ These options are specified on the builder using the
setProperty method.
| flink.connector.http.security.cert.server.allowSelfSigned | optional |
Accept untrusted certificates for TLS communication.
|
| flink.connector.http.sink.request.timeout | optional | Sets
HTTP request timeout in seconds. If not specified, the default value of 30
seconds will be used.
|
| flink.connector.http.sink.writer.thread-pool.size | optional | Sets
the size of pool thread for HTTP Sink request processing. Increasing this value
would mean that more concurrent requests can be processed in the same time. If
not specified, the default value of 1 thread will be used. |
-| flink.connector.http.sink.writer.request.mode | optional | Sets
Http Sink request submission mode. Two modes are available to select, `single`
and `batch` which is the default mode if option is not specified.
|
+| flink.connector.http.sink.writer.request.mode | optional | Sets
the Http Sink request submission mode. Two modes are available: `single` and
`batch`. Defaults to `batch` if not specified. |
| flink.connector.http.sink.request.batch.size | optional |
Applicable only for `flink.connector.http.sink.writer.request.mode = batch`.
Sets number of individual events/requests that will be submitted as one HTTP
request by HTTP sink. The default value is 500 which is same as HTTP Sink
`maxBatchSize` |
### Batch submission mode
-By default, batch size is set to 500 which is the same as Http Sink's
`maxBatchSize` property and has value of 500.
-The `maxBatchSize` property sets maximal number of events that will be
buffered by Flink runtime before passing it to Http Sink for processing.
+By default, the `flink.connector.http.sink.request.batch.size` property is set
to 500, which matches the default value of Http Sink's `maxBatchSize` property.
+The `maxBatchSize` property sets the maximum number of events that will be
buffered by Flink runtime before passing it to Http Sink for processing.
Review Comment:
I was asking myself :what do these mean. It looks like the connector
buffers records internally up to size maxBatchSize, this will be flushed when
it reaches maxBatchSize or sink.flush-buffer.timeout is met.
Example without timeout:
maxBatchSize = 500
batch.size = 100
What happens:
Flink buffers up to 500 records (maxBatchSize)
When flush is triggered due to reaching 500, those 500 records are split
into 5 HTTP requests
So this mechanism decouples the rate and size of records coming in vs how we
send them as http requests.
We should try to say this in the text.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
