N-Boutaib commented on PR #20646: URL: https://github.com/apache/datafusion/pull/20646#issuecomment-3997717497
> 👋 Hi! If I am understanding this PR correctly, then it sounds like it is accommodating substrait plans which are incorrect. Why are producers "widening" types in substrait plans? > > Do we have producers that are doing this in practice or is it the DF producer itself which is producing these plans? > > If it is a problem with the plan itself, then I think we should reject the plan altogether. > > Please let me know if I am misunderstanding :) 👋 The plans aren't incorrect — the widening happens because our logical Substrait producer has a simplified type system that normalizes all integer variants to 64-bit. This is intentional; it doesn't carry storage-level precision. When DataFusion resolves the table via NamedTable, the catalog reports the actual storage type (Int32), and the strict check rejects it even though the promotion is lossless. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
