Baymine opened a new pull request, #64372:
URL: https://github.com/apache/doris/pull/64372
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
When a SQL query projects the same physical column multiple times with
different aliases (e.g., `SELECT col1 a, col2 b ... ORDER BY col3 DESC LIMIT
50`), the TopN lazy materialization row-id fetch path crashes or returns 0 rows
with error:
```
*** Query id: eb262350ed06459c-a8ab4307823a1e5a ***
*** is nereids: 1 ***
*** tablet id: 0 ***
*** Aborted at 1780931790 (unix time) try "date -d @1780931790" if you are
using GNU date ***
*** Current BE git commitID: 1ccfbcc4e0 ***
*** SIGSEGV invalid permissions for mapped object (@0x7fa77d496000) received
by PID 513 (TID 211603 OR 0x7f9df864d700) from PID 2101960704; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) in /etc/doris/be/lib/doris_be
1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in
/software/servers/jdk/lib/server/libjvm.so
2# JVM_handle_linux_signal in /software/servers/jdk/lib/server/libjvm.so
3# 0x00007FB56B00CC20 in /lib64/libpthread.so.0
4# memcpy in /etc/doris/be/lib/doris_be
5# doris::vectorized::ColumnStr<unsigned
int>::insert_from(doris::vectorized::IColumn const&, unsigned long) in
/etc/doris/be/lib/doris_be
6# void
doris::vectorized::IColumn::insert_from_multi_column_impl<doris::vectorized::ColumnNullable>(std::vector<doris::vectorized::IColumn
const*, std::allocator<doris::vectorized::IColumn const*> > const&,
std::vector<unsigned long, std::allocator<unsigned long> > const&) in
/etc/doris/be/lib/doris_be
7# doris::RowIdStorageReader::read_batch_external_row(unsigned long,
doris::PRequestBlockDesc const&, std::shared_ptr<doris::IdFileMap>,
std::vector<doris::SlotDescriptor, std::allocator<doris::SlotDescriptor> >&,
std::shared_ptr<doris::FileMapping>, doris::TUniqueId const&,
doris::vectorized::Block&, doris::PRuntimeProfileTree*, long*, long*, unsigned
long*) in /etc/doris/be/lib/doris_be
8# doris::RowIdStorageReader::read_by_rowids(doris::PMultiGetRequestV2
const&, doris::PMultiGetResponseV2*) in /etc/doris/be/lib/doris_be
9# std::_Function_handler<bool (),
doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*,
doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*,
google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) in
/etc/doris/be/lib/doris_be
10#
doris::vectorized::TaskExecutorSimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask,
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> const&)::{lambda()#2}::operator()() const in /etc/doris/be/lib/doris_be
11#
doris::vectorized::ScannerSplitRunner::process_for(std::chrono::duration<long,
std::ratio<1l, 1000000000l> >) in /etc/doris/be/lib/doris_be
12# doris::vectorized::PrioritizedSplitRunner::process() in
/etc/doris/be/lib/doris_be
13# doris::vectorized::TimeSharingTaskExecutor::_dispatch_thread() in
/etc/doris/be/lib/doris_be
14# doris::Thread::supervise_thread(void*) in /etc/doris/be/lib/doris_be
15# start_thread in /lib64/libpthread.so.0
16# __clone in /lib64/libc.so.6
```
Root cause: The row-id fetch code rebuilds scanner params using name-keyed
maps (`colname_to_slot_id`, `slot_name_to_schema_pos`). When a column like
`advert_name` appears twice, `emplace()` silently drops the second entry, while
`required_slots` and the result block still contain both logical output slots.
This creates an inconsistent schema between the scanner and the result block.
Fix:
- Build a deduplicated `scan_slots` list with a `source_column_to_scan_idx`
map keyed by (col_name, column_idx, col_unique_id, column_paths, access_paths),
so the scanner sees each physical column only once.
- Use a `result_column_to_scan_column` remap table so both result columns
copy from the same fetched scan column.
### Release note
Fixed a bug where TopN lazy materialization with duplicate projected columns
(same physical column aliased multiple times) caused the row-id fetch path to
return 0 rows or crash.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- [x] Manual test (add detailed scripts or steps below)
- Reproduce with `SELECT col1 a, col1 b ... ORDER BY col3 DESC LIMIT
50` on an external table. Disabling
`experimental_topn_lazy_materialization_threshold` avoids the issue; with this
fix, the query succeeds with the setting enabled.
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
