tokoko commented on PR #12800: URL: https://github.com/apache/datafusion/pull/12800#issuecomment-2401465597
I refactored a fair bit to address the comments. - renamed `apply_projection` to `apply_masking` - Split `ensure_schema_compatibility` into two functions. new `ensure_schema_compatibility` takes two schemas and validates them only w/o applying any sort of transformations. Also this is now happening before we call `apply_masking` to the substrait schema, the reason being that even if some of the fields are being masked out, ReadRel's `filter` and `best_effort_filter` expressions might still depend on them as @vbarua pointed out. - The actual transform part from the original `ensure_schema_compatibility` function is now called `apply_projection` (i know... didn't have better ideas :laughing:) and is used to construct TableScan using masked substrait schema, no validations here anymore. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
