iemejia commented on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-723329813
> so you may need to double check dependencies and perhaps add the missing snappy one. Can you please share with me the full logs of the CI run, just to see what modules might require the dependency? Snappy is a optional runtime dependency now, so it is up to the project (Hive) to know where it uses it. > Seems this only triggers when you have "default:null" in schema? I don't see we're doing that in Hive though. Yes exactly, but this can happen in the context of an union too, both [parquet_map_null.q](https://github.com/apache/hive/blob/master/ql/src/test/queries/clientpositive/parquet_map_null.q) and [avro_deserialize_map_null.q](https://github.com/apache/hive/blob/master/ql/src/test/queries/clientpositive/avro_deserialize_map_null.q) refer to [map_null_val.avro](https://github.com/apache/hive/blob/master/data/files/map_null_val.avro) so this seem to be the case. > Not yet, yeah feel free to do that. Have you got a chance to test this against Spark with a upgraded Parquet version? I have a [branch with everything on it Avro, Hive and Parquet updates](https://github.com/iemejia/spark/tree/SPARK-27733-avro-upgrade) in case you want to take a look, it fully compiles however I have not been able to run the full set of Spark tests locally yet because of unrelated errors on Kinesis (and lack of familiarity with the Spark test suite). I would like to test it with the full CI, but to do that I would need to have the Hive SNAPSHOTs somewhere, it is a pity Hive does not publish them for older versions, or are they published somewhere (that way i will update things there first)? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
