[
https://issues.apache.org/jira/browse/HIVE-29483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hemanth Umashankar reassigned HIVE-29483:
-----------------------------------------
Assignee: Hemanth Umashankar
> Implement an integration testing mechanism with real Tez on Yarn including
> resource localization
> ------------------------------------------------------------------------------------------------
>
> Key: HIVE-29483
> URL: https://issues.apache.org/jira/browse/HIVE-29483
> Project: Hive
> Issue Type: Improvement
> Reporter: László Bodor
> Assignee: Hemanth Umashankar
> Priority: Major
>
> Follow-up for HIVE-29470
> {code:java}
> implement an integration testing mechanism (inspired by current qtest drivers
> maybe), that can utilize a real mini yarn cluster, that relies on the
> localized hive-exec jar (and figure out first how the current
> TestMiniTezCliDriver works and all tests pass on that driver with the above
> hack patch in place) <-- nice-to-have long-term
> {code}
> history recap, given this POC patch:
> https://github.com/abstractdog/hive/commit/06ce23fb04bb2e862d0cf511f5ac214ce1e6bee9
> on a classic yarn cluster, this patch makes all hive queries fail, given that
> the hive-exec.jar (referred to as app jar) localization is skipped, so when a
> tez job starts (in a yarn application), that jar is not present, so it
> immediately fails with some kind of ClassNotFoundException kinda problem
> the implemented testing here is definitely a heavy, integrated testing one,
> so I can imagine that it won't included into the precommit phase, but users
> must be able to start a pseudo-distributed hadoop/yarn/hdfs cluster and run
> tez jobs against that, because apparently, the mini clusters won't cover some
> cases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
