This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new b58010c48e3 [fix](export) BufferWritable must be committed before
deconstruct (#25185)
b58010c48e3 is described below
commit b58010c48e3213dfacb10a77ba01a6b91b3a12ee
Author: Mingyu Chen <[email protected]>
AuthorDate: Mon Oct 9 22:39:45 2023 +0800
[fix](export) BufferWritable must be committed before deconstruct (#25185)
F20231009 16:03:47.659968 3342535 string_buffer.hpp:48] Check failed:
_now_offset == 0
*** Check failure stack trace: ***
@ 0x561a6f8e21e6 google::LogMessage::SendToLog()
@ 0x561a6f8de7b0 google::LogMessage::Flush()
@ 0x561a6f8e2a29 google::LogMessageFatal::~LogMessageFatal()
@ 0x561a4a409233 doris::vectorized::BufferWritable::~BufferWritable()
@ 0x561a6e202853 doris::vectorized::VCSVTransformer::write()
@ 0x561a6e1f19ba doris::vectorized::VFileResultWriter::_write_file()
@ 0x561a6e1f1522 doris::vectorized::VFileResultWriter::append_block()
@ 0x561a6e121bed
The error will occur in DEBUG mode, and doing export will invalid data.
It has been covered by baidu case.
---
be/src/vec/runtime/vcsv_transformer.cpp | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/be/src/vec/runtime/vcsv_transformer.cpp
b/be/src/vec/runtime/vcsv_transformer.cpp
index 1cbdb7ea055..d8ff48f6e46 100644
--- a/be/src/vec/runtime/vcsv_transformer.cpp
+++ b/be/src/vec/runtime/vcsv_transformer.cpp
@@ -97,8 +97,14 @@ Status VCSVTransformer::write(const Block& block) {
if (col_id != 0) {
buffer_writer.write(_column_separator.data(),
_column_separator.size());
}
- RETURN_IF_ERROR(_serdes[col_id]->serialize_one_cell_to_json(
- *(block.get_by_position(col_id).column), i, buffer_writer,
_options));
+ Status st = _serdes[col_id]->serialize_one_cell_to_json(
+ *(block.get_by_position(col_id).column), i, buffer_writer,
_options);
+ if (!st.ok()) {
+ // VectorBufferWriter must do commit before deconstruct,
+ // or it may throw DCHECK failure.
+ buffer_writer.commit();
+ return st;
+ }
}
buffer_writer.write(_line_delimiter.data(), _line_delimiter.size());
buffer_writer.commit();
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
