[
https://issues.apache.org/jira/browse/IMPALA-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254770#comment-17254770
]
ASF subversion and git services commented on IMPALA-10374:
----------------------------------------------------------
Commit a6a244099502329d9193b316ea26d5fd6451b6bd in impala's branch
refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a6a2440 ]
IMPALA-10374: Limit iteration at BufferedTupleStream::DebugString
BufferedTupleStream::DebugString() iterate std::list<Page> that can
potentially grow very large. As consequent, the returned string can grow
large as well and cause a problem as previously happen in IMPALA-9851.
With this patch, BufferedTupleStream::DebugString() only include maximum
of 100 first pages of page list.
Testing:
- Add new be test SimpleTupleStreamTest.ShortDebugString in
buffered-tuple-stream-test.cc
- Pass core tests
Change-Id: I6626c8d54f35f303c01f85be1dd9aa54c8ad9a2d
Reviewed-on: http://gerrit.cloudera.org:8080/16884
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Quanlong Huang <[email protected]>
> Limit page iteration at BufferedTupleStream::DebugString()
> ----------------------------------------------------------
>
> Key: IMPALA-10374
> URL: https://issues.apache.org/jira/browse/IMPALA-10374
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 3.4.0
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Minor
> Labels: newbie, ramp-up
>
> BufferedTupleStream::DebugString() iterate std::list<Page> that can
> potentially grow very large. While most references to this method are behind
> DCHEK logging (thus stripped in release build), one of them is used to build
> Status message through call to GroupingAggregator::Partition::DebugString()
> at this line:
> https://github.com/apache/impala/blob/5530b62/be/src/exec/grouping-aggregator-partition.cc#L155
> IMPALA-9851 already cap error message to 128kb at most, but that does not
> eliminate the cost to iterate page list in BufferedTupleStream::DebugString().
> We should limit number of page being iterated in this method to avoid
> unnecessary long string operation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
