dwsmith1983 commented on PR #5868: URL: https://github.com/apache/datafusion-comet/pull/5868#issuecomment-5739783944
Hi @andygrove @sunchao @peterxcli The last three merge-queue runs for this PR fail the same test on Linux and macOS, native scan declines top-level fields that repeat a Parquet field id, with the plan still holding a CometNativeScan (for example https://github.com/apache/datafusion-comet/actions/runs/35418573600). It also failed that way in the groups that included #5874 behind this PR. The cause looks like the isSchemaSupported hunk in CometScanRule.scala: the per-field forall over requiredSchema.fields replaces the typeChecker.isSchemaSupported(scanExec.requiredSchema, ...) call, and #6004 put the root-level duplicate-field-id check in exactly that override, so it no longer runs. Keeping the schema-level call and applying the Variant admission on top of it, or checking duplicateFieldIds(requiredSchema.fields) before the loop, should bring the test back. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
