wilhelmagren commented on issue #11466: URL: https://github.com/apache/arrow-rs/issues/11466#issuecomment-6148974042
How should the embedded Arrow schema in the Parquet file be considered when e.g. the user has specified another (but compatible) dtype for the field? Imagine a Parquet file with an Arrow `Utf8` field but user specifies `ArrowReaderOptions::new().with_string_type(StringType::LargeUtf8)`, should the explicit option override the embedded Arrow schema? How does the C++ reader do it? (or any reader in another language) The following precedence seems reasonable (highest to lowest): 1. Explicit `with_schema()` if that is used 2. Explicit `with_string_type()` / `with_list_type()` 3. Embedded Arrow schema in the file metadata 4. Default Parquet-to-Arrow inference -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
