sepuri sai krishna created FLINK-40346:
------------------------------------------
Summary: Triton array deserialization silently replaces JSON nulls
with 0 / false / "null"
Key: FLINK-40346
URL: https://issues.apache.org/jira/browse/FLINK-40346
Project: Flink
Issue Type: Bug
Components: Table SQL / Runtime
Affects Versions: 2.3.0
Reporter: sepuri sai krishna
h3. Problem
{{TritonTypeMapper#deserializeArrayFromJson}} does not detect JSON {{null}}
elements when converting a Triton inference response into {{ArrayData}}. A null
is silently replaced by a value the model never produced:
|| Declared output type || Triton response || Deserialized result ||
| {{ARRAY<STRING>}} | {{["a", null, "b"]}} | {{["a", "null", "b"]}} - element 1
is the literal 4-character string |
| {{ARRAY<INT>}} | {{[1, null, 3]}} | {{[1, 0, 3]}} |
| {{ARRAY<DOUBLE>}} | {{[1.5, null]}} | {{[1.5, 0.0]}} |
| {{ARRAY<BOOLEAN>}} | {{[true, null]}} | {{[true, false]}} |
In every case {{ArrayData#isNullAt}} returns {{false}}, so downstream operators
cannot tell the substituted value apart from a genuine prediction. No exception
is thrown and nothing is logged.
The same substitution applies to the {{default-value}} connector option, which
is parsed through the same method via
{{TritonInferenceModelFunction#parseDefaultPayload}}: a configured fallback of
{{[1, null, 3]}} for an {{ARRAY<INT>}} output silently becomes {{[1, 0, 3]}}.
Note the asymmetry within the same class - the serialization side already
handles this correctly ({{serializeArrayToJsonArray}} emits {{addNull()}} for
null elements), and scalar deserialization handles it too
({{deserializeFromJson}} returns {{null}} for a null node). Only array-element
deserialization is missing the check, so a null does not survive a
serialize/deserialize round trip.
h3. Impact
Silent data corruption on the inference output path
({{TritonInferenceModelFunction#parseInferenceResponse}}). For
{{ARRAY<STRING>}} the corrupted element is indistinguishable from a legitimate
{{"null"}} string. For numeric arrays a missing prediction becomes a plausible
{{0}} that flows into downstream aggregations unnoticed.
h3. Affects
2.3.0 and master. The module was introduced by FLINK-38857 (fix version 2.3.0);
the file is absent from release-2.2 and earlier.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
