[
https://issues.apache.org/jira/browse/IMPALA-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16915400#comment-16915400
]
Sahil Takiar edited comment on IMPALA-8890 at 8/26/19 1:15 AM:
---------------------------------------------------------------
{quote} The caller of GetNext() does the right thing by processing the returned
batch and then resetting it before any other BTS methods are called. That would
free any pages that were attached to the batch. {quote} Does this apply to
{{AddRow}} as well? In IMPALA-8819, I'm adding support for fetching less than
{{BATCH_SIZE}} rows which could mean that {{GetNext}} returns a {{RowBatch}},
the consumer does a partial read of the {{RowBatch}}, and then the producer
adds more rows to the stream. Would that cause issues?
was (Author: stakiar):
{quote} The caller of GetNext() does the right thing by processing the returned
batch and then resetting it before any other BTS methods are called. That would
free any pages that were attached to the batch. \{quote} Does this apply to
{{AddRow}} as well? In IMPALA-8819, I'm adding support for fetching less than
{{BATCH_SIZE}} rows which could mean that {{GetNext}} returns a {{RowBatch}},
the consumer does a partial read of the {{RowBatch}}, and then the producer
adds more rows to the stream. Would that cause issues?
> DCHECK(!page->attached_to_output_batch) in SpillableRowBatchQueue::AddBatch
> ----------------------------------------------------------------------------
>
> Key: IMPALA-8890
> URL: https://issues.apache.org/jira/browse/IMPALA-8890
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Affects Versions: Impala 3.4.0
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Blocker
> Attachments: impalad.INFO, resolved.txt
>
>
> Full stack:
> {code}
> F0823 13:19:42.262142 60340 buffered-tuple-stream.cc:291]
> 6a4941285b46788d:68021ec600000000] Check failed:
> !page->attached_to_output_batch
> *** Check failure stack trace: ***
> @ 0x4c987cc google::LogMessage::Fail()
> @ 0x4c9a071 google::LogMessage::SendToLog()
> @ 0x4c981a6 google::LogMessage::Flush()
> @ 0x4c9b76d google::LogMessageFatal::~LogMessageFatal()
> @ 0x2917f78 impala::BufferedTupleStream::ExpectedPinCount()
> @ 0x29181ec impala::BufferedTupleStream::UnpinPageIfNeeded()
> @ 0x291b27b impala::BufferedTupleStream::UnpinStream()
> @ 0x297d429 impala::SpillableRowBatchQueue::AddBatch()
> @ 0x25d5537 impala::BufferedPlanRootSink::Send()
> @ 0x207e94c impala::FragmentInstanceState::ExecInternal()
> @ 0x207afac impala::FragmentInstanceState::Exec()
> @ 0x208e854 impala::QueryState::ExecFInstance()
> @ 0x208cb21
> _ZZN6impala10QueryState15StartFInstancesEvENKUlvE_clEv
> @ 0x2090536
> _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala10QueryState15StartFInstancesEvEUlvE_vE6invokeERNS1_15function_bufferE
> @ 0x1e9830b boost::function0<>::operator()()
> @ 0x23e2d38 impala::Thread::SuperviseThread()
> @ 0x23eb0bc boost::_bi::list5<>::operator()<>()
> @ 0x23eafe0 boost::_bi::bind_t<>::operator()()
> @ 0x23eafa3 boost::detail::thread_data<>::run()
> @ 0x3bc1629 thread_proxy
> @ 0x7f920a3786b9 start_thread
> @ 0x7f9206b5741c clone
> {code}
> Happened once while I was running a full table scan of
> {{tpch_parquet.orders}} via JDBC (client was running on another EC2 machine).
> This was running on top of IMPALA-8819 with a fetch size of 32768.
> Attached full logs and mini-dump stack.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
