paleolimbot commented on code in PR #41765:
URL: https://github.com/apache/arrow/pull/41765#discussion_r1629777943
##########
cpp/src/parquet/types.cc:
##########
@@ -464,7 +464,10 @@ std::shared_ptr<const LogicalType> LogicalType::FromThrift(
} else if (type.__isset.FLOAT16) {
return Float16LogicalType::Make();
} else {
- throw ParquetException("Metadata contains Thrift LogicalType that is not
recognized");
+ // Or provide some mechanism to optionally error here?
Review Comment:
> If we don't throw here at the very least we need to ignore statistics for
the logical type
I did some investigation and I believe statistics are already ignored for
`UndefinedLogicalType`:
https://github.com/apache/arrow/blob/93712bfc71a5013231b950b2b655d77b14f83fa7/cpp/src/parquet/types.cc#L324-L332
https://github.com/apache/arrow/blob/93712bfc71a5013231b950b2b655d77b14f83fa7/cpp/src/parquet/metadata.cc#L254-L272
(i.e., `ColumnChunkMetadata::statistics()` will return `nullptr` if
`logical_type->is_valid()` returns `false`)
I still have to figure out how to write a full-on file so that this is
tested properly! (I figured out how to test the schema conversion but not how
to check a full-on file yet).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
