[
https://issues.apache.org/jira/browse/IMPALA-15422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy resolved IMPALA-15422.
----------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> WriteDecimalSlot in system-table-scanner.cc falls through its switch and
> writes 16 bytes into 8-byte DECIMAL slots
> ------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-15422
> URL: https://issues.apache.org/jira/browse/IMPALA-15422
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> The switch in WriteDecimalSlot() has no break statements:
> {code:cpp}
> // be/src/exec/system-table-scanner.cc:101
> static void WriteDecimalSlot(
> const ColumnType& type, double value, void* slot) {
> bool overflow = false;
> switch (type.GetByteSize()) {
> case 4:
> *reinterpret_cast<Decimal4Value*>(slot) =
> Decimal4Value::FromDouble(type, value, false, &overflow);
> case 8:
> *reinterpret_cast<Decimal8Value*>(slot) =
> Decimal8Value::FromDouble(type, value, false, &overflow);
> case 16:
> *reinterpret_cast<Decimal16Value*>(slot) =
> Decimal16Value::FromDouble(type, value, false, &overflow);
> }
> DCHECK(!overflow);
> }
> {code}
> All DECIMAL columns of sys.impala_query_live are DECIMAL(18,3)
> (DURATION_DECIMAL_PRECISION
> / SCALE in workload-management.h), so GetByteSize() is 8. The 8-byte value is
> written,
> then execution falls through and a Decimal16Value is written at the same
> address. The 8
> bytes after the slot are overwritten: zeros for non-negative values, 0xFF
> bytes for
> negative ones.
> Affected columns: TOTAL_TIME_MS, ROW_MATERIALIZATION_TIME_MS,
> READ_IO_WAIT_TOTAL_MS,
> READ_IO_WAIT_MEAN_MS, and all EVENT_* timeline columns (written through
> WriteEvent()).
> Depending on the tuple layout of the projected columns, the extra bytes
> overwrite one of:
> * a neighbouring slot that was already written, which then reads as 0;
> * null-indicator bytes, so a NULL value can read as non-NULL;
> * memory past the end of the tuple, if the decimal is the last slot.
> h3. Repro
> Start the cluster with --enable_workload_mgmt=true (impalad and catalogd) and
> run a few
> queries. Then:
> {code:sql}
> select query_id, backends_count from sys.impala_query_live order by query_id;
> {code}
> {noformat}
> 2c4087f2a5bfa360:b53e19f800000000 3
> d341c86929ab53f0:ad99a3f700000000 3
> {noformat}
> {code:sql}
> select query_id, backends_count, total_time_ms from sys.impala_query_live
> order by query_id;
> {code}
> {noformat}
> 2c4087f2a5bfa360:b53e19f800000000 0 466.204
> 6e4b4ced99079f15:268d851900000000 0 181.613
> d341c86929ab53f0:ad99a3f700000000 0 193.144
> {noformat}
> backends_count (INT) is written first and laid out right after the 8-byte
> decimal slot.
> The 16-byte decimal write then zeroes it. Queries on a debug build don't hit
> the DCHECK.
> h3. Proposed fix
> Add the missing break statements. Add a test that selects all columns of
> sys.impala_query_live, preferably run under ASAN, or a BE unit test for
> WriteDecimalSlot with a guard value next to the slot.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
