morningman commented on a change in pull request #6397:
URL: https://github.com/apache/incubator-doris/pull/6397#discussion_r697794130
##########
File path: be/src/common/config.h
##########
@@ -621,6 +621,17 @@ CONF_mDouble(tablet_version_graph_orphan_vertex_ratio,
"0.1");
// else we will call sync method
CONF_mBool(runtime_filter_use_async_rpc, "true");
+// max send batch parallelism for OlapTableSink
+// The value set by the user for send_batch_parallelism is not allowed to
exceed max_send_batch_parallelism,
+// if exceed, the value of send_batch_parallelism would be
max_send_batch_parallelism
+CONF_mInt32(max_send_batch_parallelism, "1");
+CONF_Validator(max_send_batch_parallelism, [](const int config) -> bool {
return config >= 1; });
+
+// number of send batch thread pool size
+CONF_Int32(doris_send_batch_thread_pool_thread_num, "256");
Review comment:
Too large as a default value? How about -1 by default and using half of
cpu core?
##########
File path: be/src/exec/tablet_sink.h
##########
@@ -410,10 +417,11 @@ class OlapTableSink : public DataSink {
// the timeout of load channels opened by this tablet sink. in second
int64_t _load_channel_timeout_s = 0;
- // True if this sink has been closed once
- bool _is_closed = false;
- // Save the status of close() method
- Status _close_status;
+ int32_t _send_batch_parallelism = 1;
+ // True if this sink has been closed once
+ bool _is_closed = false;
Review comment:
use space instead of tab
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
