Anthippi commented on issue #10425:
URL: https://github.com/apache/seatunnel/issues/10425#issuecomment-3898791124
I've been working on the Multi-Table implementation for the RabbitMQ Source.
I have successfully implemented the discovery phase and split assignment, but I
am currently blocked by a `NullPointerException` during the data collection
phase.
When deserializing messages using `JsonDeserializationSchema`, the produced
`SeaTunnelRow` has a null `rowType`. In Multi-Table mode, the engine calls
SeaTunnelRow.getBytesSize(), which leads to this crash:
```java
java.lang.NullPointerException: Cannot invoke
"org.apache.seatunnel.api.table.type.SeaTunnelRowType.getFieldType(int)"
because "rowType" is null
at
org.apache.seatunnel.api.table.type.SeaTunnelRow.getBytesSize(SeaTunnelRow.java:122)
at
org.apache.seatunnel.engine.server.task.SeaTunnelSourceCollector.collect(SeaTunnelSourceCollector.java:103)
```
I have updated the `SourceReader` to maintain a map of
`DeserializationSchema` and `SeaTunnelRowType` per queue. I am explicitly
setting `record.setTableId(queueName)` inside the collector. Despite setting
the tableId, the engine still requires rowType for size calculation.
The main issue is that while I have the correct `SeaTunnelRowType` available
in the Reader, I haven't found a way to pass it into the `SeaTunnelRow` object.
The class lacks a public `setRowType` method, and the
`JsonDeserializationSchema` only populates the data fields, leaving the type
information null.
I would like to ask:
- What is the intended way to bind SeaTunnelRowType to the row in a
Multi-Table source before it is collected?
- Is this a known issue with the current JsonDeserializationSchema
implementation for multi-table scenarios?
I have the full implementation ready and would appreciate a tip on the
correct way to bypass this NPE so I can test my code and submit the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
