sam-1112 commented on code in PR #7196:
URL: https://github.com/apache/datafusion-comet/pull/7196#discussion_r4369441283
##########
native/core/src/parquet/field_resolver.rs:
##########
@@ -47,6 +47,43 @@ pub(crate) enum Selection<'a> {
ByName(&'a [usize]),
}
+/// The strongest claim on one file field. Root remapping and nested
conversion give a file
+/// field to this claimant only. Repeated id claims keep the last requested
field, matching
+/// Spark parquet-mr's `ParquetRowConverter` id-to-index `toMap`. Equal name
claims keep the
+/// earliest requested field, retaining the root remap's name-matching
behavior.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum FieldClaim {
+ /// The requested field selects this file field by Parquet field id.
+ IdMatch(usize),
+ /// An id-bearing requested field has this file field's folded name. It
must not reach this
+ /// field by name when its id is missing or selects a different field.
Only an id match can
+ /// override this claim. The root hides it with a placeholder; nested
fields read NULL.
+ Shadowed,
+ /// The requested field selects this file field by folded name.
+ NameMatch(usize),
+ /// No requested field claims this file field. The root keeps its original
name.
+ Unchanged,
+}
+
+impl FieldClaim {
+ fn strength(self) -> u8 {
+ match self {
+ Self::IdMatch(_) => 3,
+ Self::Shadowed => 2,
+ Self::NameMatch(_) => 1,
+ Self::Unchanged => 0,
+ }
+ }
+
+ fn claim(&mut self, claim: Self) {
+ if claim.strength() > self.strength()
+ || matches!((*self, claim), (Self::IdMatch(_), Self::IdMatch(_)))
Review Comment:
@sunchao After committing the extended fallback check, I tried a small local
PoC that limits claim arbitration to the required roots. For this example, both
`SELECT x` and `SELECT y` stay on the native path and return 7. However, I
haven't fully validated the approach yet.
Would it make sense to track this in a separate follow-up issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
