MarcusKainth commented on code in PR #4204:
URL: https://github.com/apache/iggy/pull/4204#discussion_r4040015460
##########
core/connectors/sdk/src/lib.rs:
##########
@@ -157,6 +157,26 @@ pub enum Payload {
}
impl Payload {
+ /// The `Schema` describing this payload's variant.
+ ///
+ /// Not the same thing as `StreamDecoder::schema`, which names the wire
+ /// format a decoder reads rather than the variant it hands back: the Avro
+ /// and FlatBuffer decoders return `Payload::Json` whenever
`extract_as_json`
+ /// is set, and the Proto decoder returns `Payload::Json` or `Payload::Raw`
+ /// depending on the path it takes. A transform may change the variant
again
+ /// after that. Anything tagging a payload for transport has to read the
tag
+ /// off the payload it actually holds.
+ pub const fn schema(&self) -> Schema {
+ match self {
+ Payload::Json(_) => Schema::Json,
+ Payload::Raw(_) => Schema::Raw,
+ Payload::Text(_) => Schema::Text,
+ Payload::Proto(_) => Schema::Proto,
Review Comment:
Confirmed, all three steps hold, with `Payload::Raw` being the outcome for
any payload that is not valid `Any` wire bytes.
`Schema::try_into_payload` served two callers that meant different things by
`Schema::Proto`. The sink SDK meant the payload's variant;
`runtime/src/source.rs:609` means the wire format a source plugin sent, where
the `Any` decode is right. Split in d8a8ece: the sink side now rebuilds through
`Payload::try_from_schema` (`sdk/src/sink.rs:203`), the variant-preserving
inverse of `Payload::schema()`, and the source arm is unchanged. Detail in the
thread with @rohankumardubey.
One addition to your step 1: the no-descriptor case is not the only one.
With a descriptor loaded, `encode_json_with_schema` also falls back when the
JSON is not a top-level object (`proto_convert.rs:285` against `:312`), so the
same instance can emit `Raw` for one message and `Proto` for the next.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
