Jacek Wislicki created FLINK-28609:
--------------------------------------
Summary: Flink-Pulsar connector fails on larger schemas
Key: FLINK-28609
URL: https://issues.apache.org/jira/browse/FLINK-28609
Project: Flink
Issue Type: Bug
Components: Connectors / Pulsar
Affects Versions: 1.15.1, 1.14.5, 1.14.4, 1.14.3
Reporter: Jacek Wislicki
Attachments: exception.txt
When a model results in a larger schema (this seems to be related to its byte
array representation), the number of expected bytes to read is different than
the number of actually read bytes: [^exception.txt]. The "read" is such a case
is always 1018 while the expected "byteLen" gives a greater value. For smaller
schemata, the numbers are equal (less than 1018) and no issue occurs.
The problem reproduction is on
[GitHub|https://github.com/JacekWislicki/vp-test2]. There are 2 simple jobs
(SimpleJob1 and SimpleJob2) using basic models for the Pulsar source definition
(PulsarMessage1 and PulsarMessage2, respectively). Each of the corresponding
schemata is properly serialised and deserialised, unless an effective byte
array length becomes excessive (marked with "the problem begins" in model
classes). The fail condition can be achieved by a number of fields
(PulsarMessage1) or just longer field names (PulsarMessage2). The problem
occurs on either Avro or a JSON schema set in the Pulsar source definition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
