Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/11812 )
Change subject: IMPALA-5031: memcpy cannot take null arguments
......................................................................
IMPALA-5031: memcpy cannot take null arguments
This patch fixes UBSAN "null pointer passed as argument" errors in
data loading. These are undefined behavior according to "7.1.4 Use of
library functions" in the C99 standard (which is included in C++14 in
section [intro.refs]):
If an argument to a function has an invalid value (such as a value
outside the domain of the function, or a pointer outside the
address space of the program, or a null pointer, or a pointer to
non-modifiable storage when the corresponding parameter is not
const-qualified) or a type (after promotion) not expected by a
function with variable number of arguments, the behavior is
undefined.
The interesting parts of the backtraces for the errors fixed in this
patch are below:
runtime/string-buffer.h:54:12: runtime error: null pointer passed as
argument 1, which is declared to never be null
/usr/include/string.h:43:45: note: nonnull attribute specified here
StringBuffer::Append(char const*, long) runtime/string-buffer.h:54:5
ColumnStatsBase::CopyToBuffer(StringBuffer*, StringValue*)
exec/parquet-column-stats.cc:151:51
ColumnStats<StringValue>::MaterializeStringValuesToInternalBuffers()
exec/parquet-column-stats.inline.h:237:70
HdfsParquetTableWriter::BaseColumnWriter::MaterializeStatsValues()
exec/hdfs-parquet-table-writer.cc:149:63
HdfsParquetTableWriter::AppendRows(RowBatch*, vector<int> const&, bool*)
exec/hdfs-parquet-table-writer.cc:1129:53
HdfsTableSink::WriteRowsToPartition(RuntimeState*, RowBatch*,
pair<unique_ptr<OutputPartition, default_delete<OutputPartition> >, vector<int,
all> > >*) exec/hdfs-table-sink.cc:256:71
HdfsTableSink::Send(RuntimeState*, RowBatch*) exec/hdfs-table-sink.cc:591:45
util/streaming-sampler.h:111:22: runtime error: null pointer passed as
argument 2, which is declared to never be null
/usr/include/string.h:43:45: note: nonnull attribute specified here
StreamingSampler<long, 64>::SetSamples(int, vector<long> const&)
util/streaming-sampler.h:111:5
RuntimeProfile::Update(vector<TRuntimeProfileNode> const&, int*)
util/runtime-profile.cc:313:30
RuntimeProfile::Update(TRuntimeProfileTree const&)
util/runtime-profile.cc:246:3
Coordinator::BackendState::InstanceStats::Update(TFragmentInstanceExecStatus
const&, Coordinator::ExecSummary*, ProgressUpdater*)
runtime/coordinator-backend-state.cc:474:13
Coordinator::BackendState::ApplyExecStatusReport(TReportExecStatusParams
const&, Coordinator::ExecSummary*, ProgressUpdater*)
runtime/coordinator-backend-state.cc:287:21
Coordinator::UpdateBackendExecStatus(TReportExecStatusParams const&)
runtime/coordinator.cc:679:22
ClientRequestState::UpdateBackendExecStatus(TReportExecStatusParams const&)
service/client-request-state.cc:1254:18
ImpalaServer::ReportExecStatus(TReportExecStatusResult&,
TReportExecStatusParams const&) service/impala-server.cc:1343:18
ImpalaInternalService::ReportExecStatus(TReportExecStatusResult&,
TReportExecStatusParams const&) service/impala-internal-service.cc:87:19
Change-Id: Ib9acc8c32409e67253a987eb3d1fd7d921efcb51
Reviewed-on: http://gerrit.cloudera.org:8080/11812
Reviewed-by: Jim Apple <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
M be/src/runtime/string-buffer.h
M be/src/util/streaming-sampler.h
2 files changed, 3 insertions(+), 2 deletions(-)
Approvals:
Jim Apple: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/11812
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib9acc8c32409e67253a987eb3d1fd7d921efcb51
Gerrit-Change-Number: 11812
Gerrit-PatchSet: 3
Gerrit-Owner: Jim Apple <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Jim Apple <[email protected]>
Gerrit-Reviewer: Vuk Ercegovac <[email protected]>
