Ramit Gupta created HIVE-30067:
----------------------------------
Summary: Include json-simple in hive-exec shaded jar for JTS
GeoJSON
Key: HIVE-30067
URL: https://issues.apache.org/jira/browse/HIVE-30067
Project: Hive
Issue Type: Bug
Reporter: Ramit Gupta
Assignee: Ramit Gupta
Geospatial UDFs that use JTS GeoJSON ( like
[GeoJsonReader.java|[https://github.com/locationtech/jts/blob/master/modules/io/common/src/main/java/org/locationtech/jts/io/geojson/GeoJsonReader.java])]
via jts-io-common e.g. in
[[GeometryUtils.java|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/esri/GeometryUtils.java]|
[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/esri/GeometryUtils.java]]
depend on {{json-simple}} for parsing
It can be reproduced locally via
{code:java}
mvn test -pl itests/qtest -Pitests -Dtest=TestMiniLlapLocalCliDriver
-Dqfile=geospatial_udfs.q -Dtest.output.overwrite{code}
running into
{code:java}
ClassNotFoundException: org.json.simple.parser.ParseException{code}
due to absance of json-simple in test classpath
Why it is not picked in CI?
CI uses
https://github.com/apache/hive/blob/4cf07f490af49d0e0e019d07c45b629ed8802136/Jenkinsfile#L408
, whose similar locally working eg. :
{code:java}
mvn -Pitests org.apache.maven.plugins:maven-surefire-plugin:test
Dtest=TestMiniLlapLocalCliDriver.java -Dqfile=geospatial_udfs.q
{code}
since it is ran from project root, due to which maven resolves the entire
dependency chain (including transitive ones) and get pulled into test
classpath due to single reactor for all modules as -pl is not specified
Similarly at runtime all the transitive deps are part of classpath so it
doesn't get picked up there as well.
{*}Note{*}: This is only for Deployments and tests that rely on a single
{{hive-exec-*.jar}} like the case I mentioned of MiniLlap driver above.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
