raghuvanshraj commented on issue #27089:
URL: https://github.com/apache/datafusion/issues/27089#issuecomment-6918657633

   @alamb one question for core, separate from the repro above.
   
   Today the fix for this bug lives in `VariantExprAdapterFactory` 
(datafusion-contrib/datafusion-variant#149), and it has to be installed on each 
table with `ListingTableConfig::with_expr_adapter_factory`. Any table built 
outside that path gets the default adapter and the wrong answer again. The url 
table above is one case (datafusion-contrib/datafusion-variant#177); a user's 
own `ListingTable` is another.
   
   The adapter is really about the type, not the table. Variant is a registered 
extension type (`arrow.parquet.variant`). Could core let an extension type 
supply its own `PhysicalExprAdapterFactory`, so any scan that sees a Variant 
column applies it without the table owner knowing?
   
   @adriangb you raised the other shape: fix how the table schema is merged for 
shredded Variant. I am fine with either. I can open an issue with a proposed 
shape once we pick one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to