GitHub user rusackas added a comment to the discussion: Switching SQL Lab / chart-data results from JSON to Apache Arrow over the wire
Update from building out the draft PR (#45363): the Arrow IPC bytes were only carrying the DataFrame, not the sidecar metadata (`cache_key`, timing, applied/rejected filters, etc.), so I had to add that back onto the wire via the Arrow schema metadata to keep things like the cache badge and "View query" accurate. That part's resolved now. Two things came up along the way that feel like real design calls rather than bugs, though. Multi-query charts (mixed time-series, for one) can't use Arrow at all right now, since the endpoint's Arrow branch is single-query-only, same constraint as the existing datasource-query Arrow branch it mirrors. And on async-query deployments, Arrow-flagged charts fall back to JSON entirely for now, since the async read-back path doesn't know how to handle Arrow bytes yet. Could probably teach the async path to carry Arrow too, but didn't want to make that call unilaterally inside a draft PR. @villebro curious if either of those change your read on whether this needs a `dev@` thread, or if they're fine as known limitations for a v1. GitHub link: https://github.com/apache/superset/discussions/45345#discussioncomment-19210226 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
