Gabriel39 commented on code in PR #67328:
URL: https://github.com/apache/doris/pull/67328#discussion_r3911824305
##########
be/src/exec/sink/writer/vfile_result_writer.cpp:
##########
@@ -123,13 +170,21 @@ Status VFileResultWriter::_create_next_file_writer() {
Status VFileResultWriter::_create_file_writer(const std::string& file_name) {
auto file_type =
DORIS_TRY(FileFactory::convert_storage_type(_storage_type));
- _file_writer_impl = DORIS_TRY(FileFactory::create_file_writer(
- file_type, _state->exec_env(), _file_opts->broker_addresses,
- _file_opts->broker_properties, file_name,
- {
- .write_file_cache = false,
- .sync_file_data = false,
- }));
+ io::FSPropertiesRef properties(file_type);
+ properties.broker_addresses = &_file_opts->broker_addresses;
+ properties.properties = &_file_opts->broker_properties;
+ io::FileDescription file_description;
+ file_description.path = file_name;
+ _file_system = DORIS_TRY(FileFactory::create_fs(properties,
file_description));
+ // Create/open can publish a path before returning an error, so claim
deterministic ownership
+ // first. A separate filesystem preserves Broker's existing per-path
endpoint selection.
+ _created_files.emplace_back(_file_system, file_name);
Review Comment:
Fixed in d37cfa0cd7. The deferred manifest now stores filesystem IDs with
paths and retains a bounded filesystem map: one entry for a non-Broker storage
identity, or one per selected Broker endpoint. This preserves Broker path-based
endpoint selection without keeping one initialized wrapper per rotated part. A
1,000-part regression test verifies that retained wrappers remain bounded by
the two selected Broker endpoints.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
