[
https://issues.apache.org/jira/browse/KAFKA-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax reassigned KAFKA-20769:
---------------------------------------
Assignee: sepuri sai krishna
> ListDeserializer can silently deserialize a corrupted entry when the input is
> truncated mid-entry
> -------------------------------------------------------------------------------------------------
>
> Key: KAFKA-20769
> URL: https://issues.apache.org/jira/browse/KAFKA-20769
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 4.3.1
> Reporter: sepuri sai krishna
> Assignee: sepuri sai krishna
> Priority: Minor
> Labels: serialization
>
> ListDeserializer.deserialize relies on DataInputStream.read(byte[]) to read
> each list entry's payload and only treats a -1 return value as a truncated
> stream. However, InputStream.read(byte[]) is only required to read at least
> one byte and may return a partial count without reaching EOF. When the
> serialized data is truncated in the middle of an entry, the entry buffer is
> left zero-padded and is passed to the inner deserializer with no error,
> silently producing a corrupted value instead of failing.
> Affected code in
> clients/src/main/java/org/apache/kafka/common/serialization/ListDeserializer.java
> (present since 2.7.0 / KAFKA-8326, still on trunk):
> byte[] payload = new byte[entrySize];
> if (dis.read(payload) == -1) {
> log.error("Ran out of bytes in serialized list");
> log.trace("Deserialized list so far: {}", deserializedList);
> throw new SerializationException("End of the stream was reached
> prematurely");
> }
> deserializedList.add(inner.deserialize(topic, headers, payload));
> Why the existing bounds checks don't prevent it: readEntrySize rejects
> entrySize > data.length, but data.length is the size of the whole buffer, not
> the bytes remaining after the strategy flag, null-index list, list size, and
> prior entries have been consumed. A crafted or corrupted payload can
> therefore declare an entrySize that passes the bounds check yet
> exceeds the actual remaining bytes.
> In that case:
> - 0 bytes remaining and entrySize > 0: read returns -1, so
> SerializationException is thrown (correct).
> - 1 to entrySize-1 bytes remaining (truncated mid-entry): read copies a
> partial payload and returns a positive count (not -1), the EOF check passes,
> the trailing bytes stay zero-filled, and the corrupted/zero-padded buffer is
> handed to inner.deserialize(...) with no error.
> Impact: on corrupted or truncated ListSerde data, the deserializer can return
> a silently mangled element instead of raising SerializationException. This
> defeats the method's own truncation detection (the "End of the stream was
> reached prematurely" path).
> Fix: use DataInputStream.readFully(byte[]), which throws EOFException when it
> cannot fill the buffer; map that to the existing SerializationException.
> Behavior is unchanged for
> well-formed data and for fully-empty streams.
> How it was found: identified by code inspection of
> ListDeserializer.deserialize on trunk (4.4.0-SNAPSHOT) and confirmed with a
> unit test. The defective line is unchanged since
> 2.7.0
> (KAFKA-8326), so all releases on the current line are affected; it was not
> reproduced against a packaged release binary.
>
> Reproduction (unit test): encode VARIABLE_SIZE, list length 1, entrySize ==
> 10, but supply only 5 payload bytes (total buffer 14 bytes, so the entrySize
> > data.length check passes).
> Before the fix no exception is thrown and a zero-padded entry is produced;
> after the fix a SerializationException ("End of the stream was reached
> prematurely") is raised.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
