gavinchou commented on code in PR #68032:
URL: https://github.com/apache/doris/pull/68032#discussion_r4236815515
##########
be/src/common/config.cpp:
##########
@@ -1700,6 +1700,28 @@ DEFINE_String(spill_storage_limit, "20%");
// 20%
DEFINE_mInt32(spill_gc_interval_ms, "2000"); // 2s
DEFINE_mInt32(spill_gc_work_time_ms, "2000"); // 2s
DEFINE_mInt64(spill_file_part_size_bytes, "1073741824"); // 1GB
+DEFINE_String(spill_storage_type, "local");
+// s3: cloud mode only; spill is written to the S3 storage vault under
+// spill/{ip}_{port}/{query_id}/, where ip is the address of this BE and port
its
+// heartbeat_service_port.
+// The BE deletes the objects of a spill file when the query is done with it.
What a BE that
+// crashed left behind is not deleted by Doris: give the bucket a lifecycle
rule that expires
+// the keys under spill/ and aborts incomplete multipart uploads.
+DEFINE_Validator(spill_storage_type, [](const std::string& config) -> bool {
+ return config == "local" || config == "s3";
+});
+// Empty means the default vault of the instance, resolved when this BE spills
for the first
+// time; a later SET DEFAULT STORAGE VAULT applies to spill after the BE
restarts.
+DEFINE_String(spill_s3_storage_vault, "");
Review Comment:
does it mean storage vault id for spill?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
