timsaucer commented on PR #1678:
URL:
https://github.com/apache/datafusion-python/pull/1678#issuecomment-5496630092
@ntjohnson1 on your question about seeing which codec triggers for a given
call — partly answered now, though not in `EXPLAIN`.
`SessionContext.logical_extension_codec_ids()` and
`physical_extension_codec_ids()` list what's installed, in install order, and
those ids are exactly what a payload carries — so a decode failure names the
codec that wrote the bytes and lists what the session actually has, e.g. `a
table provider was encoded by extension codec 'lib_a.Codec', which is not
installed on this session (installed: lib_c.Codec)`.
What's still missing is your actual question: per-call attribution, "which
codec handled this node in this plan". Encoding decides that by walking the
chain until one claims the object, and nothing records the winner. Surfacing it
in `EXPLAIN` would mean threading it through plan formatting, which I'd rather
not fold into this PR. Happy to file it as a follow-up if you think it's worth
having.
Your other three comments are addressed as well: the `inner()` accessor is
replaced by `codecs()` / `codec_ids()` / `terminal()` on the Rust side and
`logical_extension_codec_ids()` / `physical_extension_codec_ids()` from Python,
and both the contributor guide and the example README now state that the
terminal fallback is `Default{Logical,Physical}ExtensionCodec` unless a Rust
caller supplies another to `Python{Logical,Physical}Codec::new`, which is not
reachable from the Python surface — so a chain of `[lib_a, lib_b]` with no
default behind it isn't something you can set up from Python.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
