[
https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=508639&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-508639
]
ASF GitHub Bot logged work on HIVE-21737:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Nov/20 22:33
Start Date: 06/Nov/20 22:33
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #1635:
URL: https://github.com/apache/hive/pull/1635#issuecomment-723329813
> so you may need to double check dependencies and perhaps add the missing
snappy one.
Can you please share with me the full logs of the CI run, just to see what
modules might require the dependency? Snappy is a optional runtime dependency
now, so it is up to the project (Hive) to know where it uses it.
> Seems this only triggers when you have "default:null" in schema? I don't
see we're doing that in Hive though.
Yes exactly, but this can happen in the context of an union too, both
[parquet_map_null.q](https://github.com/apache/hive/blob/master/ql/src/test/queries/clientpositive/parquet_map_null.q)
and
[avro_deserialize_map_null.q](https://github.com/apache/hive/blob/master/ql/src/test/queries/clientpositive/avro_deserialize_map_null.q)
refer to
[map_null_val.avro](https://github.com/apache/hive/blob/master/data/files/map_null_val.avro)
so this seem to be the case.
> Not yet, yeah feel free to do that. Have you got a chance to test this
against Spark with a upgraded Parquet version?
I have a [branch with everything on it Avro, Hive and Parquet
updates](https://github.com/iemejia/spark/tree/SPARK-27733-avro-upgrade) in
case you want to take a look, it fully compiles however I have not been able to
run the full set of Spark tests locally yet because of unrelated errors on
Kinesis (and lack of familiarity with the Spark test suite). I would like to
test it with the full CI, but to do that I would need to have the Hive
SNAPSHOTs somewhere, it is a pity Hive does not publish them for older
versions, or are they published somewhere (that way i will update things there
first)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 508639)
Time Spent: 2h 10m (was: 2h)
> Upgrade Avro to version 1.10.0
> ------------------------------
>
> Key: HIVE-21737
> URL: https://issues.apache.org/jira/browse/HIVE-21737
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Reporter: Ismaël Mejía
> Assignee: Fokko Driesprong
> Priority: Major
> Labels: pull-request-available
> Attachments:
> 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without
> Jackson in the public API and Guava as a dependency. Worth the update.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
