sepuri sai krishna created KAFKA-20769:
------------------------------------------
Summary: ListDeserializer can silently deserialize a corrupted
entry when the input is truncated mid-entry
Key: KAFKA-20769
URL: https://issues.apache.org/jira/browse/KAFKA-20769
Project: Kafka
Issue Type: Bug
Components: clients
Affects Versions: 4.3.1
Reporter: sepuri sai krishna
ListDeserializer.deserialize relies on DataInputStream.read(byte[]) to read
each list entry's payload and only treats a -1 return value as a truncated
stream. However, InputStream.read(byte[]) is only required to read at least one
byte and may return a partial count without reaching EOF. When the serialized
data is truncated in the middle of an entry, the entry buffer is left
zero-padded and is passed to the inner deserializer with no error, silently
producing a corrupted value instead of failing.
Affected code in
clients/src/main/java/org/apache/kafka/common/serialization/ListDeserializer.java
(present since 2.7.0 / KAFKA-8326, still on trunk):
byte[] payload = new byte[entrySize];
if (dis.read(payload) == -1) {
log.error("Ran out of bytes in serialized list");
log.trace("Deserialized list so far: {}", deserializedList);
throw new SerializationException("End of the stream was reached
prematurely");
}
deserializedList.add(inner.deserialize(topic, headers, payload));
Why the existing bounds checks don't prevent it: readEntrySize rejects
entrySize > data.length, but data.length is the size of the whole buffer, not
the bytes remaining after the strategy flag, null-index list, list size, and
prior entries have been consumed. A crafted or corrupted payload can therefore
declare an entrySize that passes the bounds check yet
exceeds the actual remaining bytes.
In that case:
- 0 bytes remaining and entrySize > 0: read returns -1, so
SerializationException is thrown (correct).
- 1 to entrySize-1 bytes remaining (truncated mid-entry): read copies a
partial payload and returns a positive count (not -1), the EOF check passes,
the trailing bytes stay zero-filled, and the corrupted/zero-padded buffer is
handed to inner.deserialize(...) with no error.
Impact: on corrupted or truncated ListSerde data, the deserializer can return a
silently mangled element instead of raising SerializationException. This
defeats the method's own truncation detection (the "End of the stream was
reached prematurely" path).
Fix: use DataInputStream.readFully(byte[]), which throws EOFException when it
cannot fill the buffer; map that to the existing SerializationException.
Behavior is unchanged for
well-formed data and for fully-empty streams.
How it was found: identified by code inspection of ListDeserializer.deserialize
on trunk (4.4.0-SNAPSHOT) and confirmed with a unit test. The defective line is
unchanged since
2.7.0
(KAFKA-8326), so all releases on the current line are affected; it was not
reproduced against a packaged release binary.
Reproduction (unit test): encode VARIABLE_SIZE, list length 1, entrySize == 10,
but supply only 5 payload bytes (total buffer 14 bytes, so the entrySize >
data.length check passes).
Before the fix no exception is thrown and a zero-padded entry is produced;
after the fix a SerializationException ("End of the stream was reached
prematurely") is raised.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
