[
https://issues.apache.org/jira/browse/IMPALA-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17420634#comment-17420634
]
ASF subversion and git services commented on IMPALA-10714:
----------------------------------------------------------
Commit 80a1dc33d33538af609a0f1dd761363a06d279b8 in impala's branch
refs/heads/branch-4.0.1 from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=80a1dc3 ]
IMPALA-10714: Defer advancing read page until the buffer is attached
If a BufferedTupleStream is a read-write stream and set up with
attach_on_read = true, BufferedTupleStream::NextReadPage() expects that
the page buffer is attached to the output row batch before advancing the
read iterator. However, BufferedTupleStream::GetNextInternal() will not
attach the page buffer of a fully read page if it is a read-write page.
Consider the following scenario:
1. Only 1 page left in stream. This is a read-write page.
2. GetNext() has fully read the page, but does NOT attach the buffer to
output row batch because it is a read-write page.
3. Stream writer insert more rows, but the read-write page can not fit
any more rows. Therefore, new pages are created.
4. Stream writer call UnpinStream().
5. UnpinStream() call NextReadPage(), which in turn will fail the
assertion "read_iter->read_page_->attached_to_output_batch".
BufferedTupleStream::UnpinStream() need to defer advancing the read page
if this situation happens.
This patch adds BE test StreamStateTest.UnpinFullyExhaustedReadPage that
simulates the corner case. This patch also moves BE test
DeferAdvancingReadPage and ShortDebugString into class StreamStateTest
to reduce friend class declaration in buffered-tuple-stream.h
Testing:
- Run and pass BE test StreamStateTest.UnpinFullyExhaustedReadPage.
Change-Id: I586ed72ba01cc3f28b0dcb1e202b3ca32a6c3b83
Reviewed-on: http://gerrit.cloudera.org:8080/17853
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Intermittent DCHECK(read_iter->read_page_->attached_to_output_batch) in tests
> -----------------------------------------------------------------------------
>
> Key: IMPALA-10714
> URL: https://issues.apache.org/jira/browse/IMPALA-10714
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Zoltán Borók-Nagy
> Assignee: Riza Suminto
> Priority: Major
> Labels: broken-build, flaky
> Fix For: Impala 4.1.0
>
>
> TestSpillingDebugActionDimensions::test_spilling_large_rows hit DCHECK in
> exhaustive build.
> The Impala git hash was: e11237e29 IMPALA-10197: Add KUDU_REPLICA_SELECTION
> query option
> In impalad.FATAL:
> {noformat}
> F0525 12:45:49.307780 13122 buffered-tuple-stream.cc:531]
> 564af337ca503984:f1209fc400000000] Check failed:
> read_iter->read_page_->attached_to_output_batch
> {noformat}
> Query 564af337ca503984:f1209fc400000000 was:
> {noformat}
> I0525 12:45:48.474383 17878 impala-server.cc:1324]
> 564af337ca503984:f1209fc400000000] Registered query
> query_id=564af337ca503984:f1209fc400000000
> session_id=9e4875c17adf5e7a:eb72d33dc39b5288
> I0525 12:45:48.474486 17878 Frontend.java:1618]
> 564af337ca503984:f1209fc400000000] Analyzing query: select
> group_concat(string_col), length(bigstr) from bigstrs2
> group by bigstr db: test_spilling_large_rows_119f6bb1
> {noformat}
> I couldn't reproduce the issue locally.
>
>
> {code:java}
> be/src/runtime/buffered-tuple-stream.cc:531
> 530 DCHECK_NE(&*read_iter->read_page_, write_page_);
> 531 DCHECK(read_iter->read_page_->attached_to_output_batch);
> 532 pages_.pop_front();
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
