alamb commented on code in PR #8496:
URL: https://github.com/apache/arrow-rs/pull/8496#discussion_r2392799892
##########
parquet/src/arrow/schema/complex.rs:
##########
@@ -72,6 +72,52 @@ impl ParquetField {
}
}
+ /// Converts `self` into an arrow list, with its current type as the field
type
+ /// accept an optional `list_data_type` to specify the type of list to
create
+ ///
+ /// This is used to convert deprecated repeated columns (not in a list),
into their arrow representation
+ fn into_list_with_arrow_list_hint(
+ self,
+ parquet_field_type: &Type,
+ list_data_type: Option<DataType>,
+ ) -> Result<Self, ParquetError> {
+ let arrow_field = match &list_data_type {
+ Some(DataType::List(field_hint))
+ | Some(DataType::LargeList(field_hint))
+ | Some(DataType::FixedSizeList(field_hint, _)) =>
Some(field_hint.as_ref()),
+ Some(_) => unreachable!(
+ "should be validated earlier that list_data_type is only a
type of list"
Review Comment:
even though this should be impossible, I worry about panic'ing here because
if there is a bug that error is more severe than "just an error"
ALso, while this may be true at the moment, I can imagine that some future
refactor messes it up, in which case this may become reachable and the compiler
won't complain
I would prefer returning a `general_err!` with some sort of "Internal
error: should be validated..." type message
##########
parquet/src/arrow/schema/complex.rs:
##########
@@ -72,6 +72,52 @@ impl ParquetField {
}
}
+ /// Converts `self` into an arrow list, with its current type as the field
type
+ /// accept an optional `list_data_type` to specify the type of list to
create
+ ///
+ /// This is used to convert deprecated repeated columns (not in a list),
into their arrow representation
Review Comment:
Is there any reference we can add a link to (I am not familiar with the
"deprecated repeated columns" you are referencing here
##########
parquet/src/arrow/schema/complex.rs:
##########
@@ -72,6 +72,52 @@ impl ParquetField {
}
}
+ /// Converts `self` into an arrow list, with its current type as the field
type
+ /// accept an optional `list_data_type` to specify the type of list to
create
+ ///
+ /// This is used to convert deprecated repeated columns (not in a list),
into their arrow representation
Review Comment:
Maybe it is it
https://github.com/apache/parquet-format/blob/9fd57b59e0ce1a82a69237dcf8977d3e72a2965d/LogicalTypes.md?plain=1#L718-L723
?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
