deniskuzZ opened a new pull request, #6902: URL: https://github.com/apache/hive/pull/6902
### What changes were proposed in this pull request? - Docker image: `-tez-snapshot <ver>` installs the Tez Maven snapshot into `TEZ_HOME` (`/opt/tez`), replacing the release's jars (now also `hadoop-shim` and `tez-protobuf-history-plugin`), instead of a separate `/opt/tez-snapshot`. Any Tez jar an older Hive release bundles leaves `$HIVE_HOME/lib`, so every process runs one Tez. - `entrypoint.sh`: the TezAM classpath starts with `TEZ_HOME`; HiveServer2 no longer symlinks snapshot jars into `lib/` as `0-*.jar` to win bin/hive's alphabetical glob. - Binary distribution: `lib/` no longer bundles Tez (`org.apache.tez:*` excluded in `bin.xml`, as `org.apache.hadoop:*` already is). ### Why are the changes needed? Two copies of Tez sat on the classpaths: the release in `TEZ_HOME` and a snapshot in `/opt/tez-snapshot` (plus the `tez-api` the dist bundles via hive-common). Which one a process ran depended on wildcard and glob order: JVM wildcard order is unspecified, and HiveServer2 needed symlink tricks to put the snapshot first. One Tez in `TEZ_HOME` removes the ordering question. The dist's `tez-api` is not needed at runtime by anything that does not run Tez: hive-common reads only `TezConfiguration.TEZ_AM_LAUNCH_ENV`, a compile-time constant, and `HiveConf` names a Tez class only as a string. Everything that runs Tez needs the full runtime (`tez-dag`, `tez-runtime-internals`, ...) from `TEZ_HOME` anyway: - HiveServer2 and the TezAM: `TEZ_HOME` on the classpath (entrypoint, bin/hive). - LLAP in the image: `LLAP_DAEMON_USER_CLASSPATH` starts with `$TEZ_HOME/*`. - LLAP on YARN: the package's `lib/tez` comes from `tez.lib.uris` (`AsyncTaskDownloadTezJars`); `AsyncTaskCopyLocalJars` copies no Tez jar. ### Does this PR introduce _any_ user-facing change? - The binary distribution's `lib/` no longer contains `tez-api-<ver>.jar` and `commons-lang-2.6.jar` (only tez-api pulled the latter in; hive-exec carries the commons-lang 2 classes unrelocated). - The Docker image has no `/opt/tez-snapshot`; with `-tez-snapshot`, the snapshot is in `/opt/tez`. ### How was this patch tested? - `mvn -Pdist package -pl packaging` before and after: `lib/` differs by exactly `tez-api-0.10.5.jar` and `commons-lang-2.6.jar` (352 -> 350 jars). - In progress: a K8s deployment of an image built with this layout and entrypoint; this description will be updated with the result. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
