[
https://issues.apache.org/jira/browse/PARQUET-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719317#comment-17719317
]
ASF GitHub Bot commented on PARQUET-2292:
-----------------------------------------
clairemcginty commented on code in PR #1078:
URL: https://github.com/apache/parquet-mr/pull/1078#discussion_r1184991179
##########
parquet-avro/pom.xml:
##########
@@ -105,6 +110,30 @@
<type>test-jar</type>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <version>2.23.0</version>
Review Comment:
It should be, but it does require an artifact migration in a few modules
(parquet-column and parquet-hadoop depend on mockito-all, which has moved to
mockito-core in 2.x). I would be happy to do it, just not sure if it's in scope
for this PR :)
> Improve default SpecificRecord model selection for Avro{Write,Read}Support
> --------------------------------------------------------------------------
>
> Key: PARQUET-2292
> URL: https://issues.apache.org/jira/browse/PARQUET-2292
> Project: Parquet
> Issue Type: Improvement
> Reporter: Claire McGinty
> Assignee: Claire McGinty
> Priority: Major
>
> AvroWriteSupport/AvroReadSupport can improve the precision of their default
> `model` selection. Currently they default to new
> SpecificDataSupplier().get()[0]. This means that SpecificRecord classes that
> contain logical types will fail out-of-the-box unless a specific
> DATA_SUPPLIER is configured that contains logical type conversions.
> I think we can improve this and make logical types work by default by
> defaulting to the value of the `MODEL$` field that every SpecificRecordBase
> implementation contains, which already contains all the logical conversions
> for that Avro type. It would require reflection, but that's what the Avro
> library is already doing to fetch models for Specific types[1].
>
> [0]
> [https://github.com/apache/parquet-mr/blob/d38044f5395494e1543581a4b763f624305d3022/parquet-avro/src/main/java/org/apache/parquet/avro/AvroWriteSupport.java#L403-L407]
> [1]
> https://github.com/apache/avro/blob/release-1.11.1/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java#L76-L86
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
