[
https://issues.apache.org/jira/browse/PARQUET-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17341768#comment-17341768
]
ASF GitHub Bot commented on PARQUET-2037:
-----------------------------------------
gszadovszky commented on pull request #901:
URL: https://github.com/apache/parquet-mr/pull/901#issuecomment-836350577
@shangxinli,
The other option was about using the `doc` field of the avro schema field.
It would be a misuse of it as anyone can set anything to that field. It is more
about documenting the related field for humans instead of taking decisions
based on it in the code. On the Parquet sync that was my only idea to resolve
this issue and even I did not like it.
I think the 2nd option (covered by this PR) is much better but only realized
its possibility after the Parquet sync meeting so we could not discuss it
personally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Write INT96 with parquet-avro
> -----------------------------
>
> Key: PARQUET-2037
> URL: https://issues.apache.org/jira/browse/PARQUET-2037
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-avro, parquet-mr
> Reporter: Gabor Szadovszky
> Assignee: Gabor Szadovszky
> Priority: Major
>
> This jira is about the write path of PARQUET-1928.
> The issue here is how to identify an Avro FIXED type that was an INT96
> before. Of course, this feature would be behind a configuration flag
> similarly to PARQUET-1928. But even with this flag it is not obvious to
> differentiate a "simple" FIXED[12] byte from one that was an INT96 before.
> Two options to solve this issue:
> * Write the doc field of the avro schema that the FIXED value was an INT96.
> * Instead of implementing a configuration flag let the user specify the names
> of the columns to be converted to INT96 via the configuration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
