> On Sept. 18, 2017, 3:25 p.m., Attila Sasvari wrote: > > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java > > Line 69 (original), 69 (patched) > > <https://reviews.apache.org/r/61560/diff/7/?file=1828278#file1828278line69> > > > > Could you explain why it is necessary to kill child yarn jobs? If it is > > the first job, then it does not look necessary > > Peter Cseh wrote: > It's a common pattern across *Mains to call this. I don't know why it's > not inside ShellMain, but you're right, this is not the patch to put it in > there.
The purpose of this is for the case where the launcher gets restarted by Yarn (which can happen for a variety of reasons). Oozie can't resume child jobs that were in the middle other than the MR action because it has no idea what it's doing. We decided that the best thing to do would be to kill all the children and start over - that's what this does. Otherwise, you'd end up with duplicate jobs running, which could cause problems. Oozie doesn't do this in the Shell Action (and maybe also the Java Action) because Oozie doesn't manage the child jobs from these action types - that's up to the user. - Robert ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61560/#review185551 ----------------------------------------------------------- On Sept. 19, 2017, 1:59 p.m., Peter Cseh wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61560/ > ----------------------------------------------------------- > > (Updated Sept. 19, 2017, 1:59 p.m.) > > > Review request for oozie, Attila Sasvari, Peter Bacsko, Robert Kanter, and > Rohini Palaniswamy. > > > Bugs: OOZIE-2909 > https://issues.apache.org/jira/browse/OOZIE-2909 > > > Repository: oozie-git > > > Description > ------- > > Changing UGI calls and cleaning up things around requesting tokens. > > > Diffs > ----- > > core/src/main/java/org/apache/oozie/ErrorCode.java > 662e1edc9c4b23b3606c751bf5ed4b531ee7ac62 > > core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProperties.java > 20f93ce57618a529b1bd83b398add2cd1d96b36e > > core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java > 5ca8d3e80942840a5bba51a06694350929044042 > core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java > 9804c7b673a899cd9c778850c9a64731cc642fb3 > core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java > 52abbf17afa71e702d7871d10a4a190a4d21e47b > core/src/main/java/org/apache/oozie/action/hadoop/HDFSCredentials.java > PRE-CREATION > core/src/main/java/org/apache/oozie/action/hadoop/HadoopTokenHelper.java > PRE-CREATION > core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java > 4add5f14cb20ab77c755b81eff939335c0c5fddc > core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java > 0b495f75842b0033a3337ef6728a334bf5651770 > core/src/main/java/org/apache/oozie/action/hadoop/JHSCredentials.java > PRE-CREATION > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > 9d1afb51976fd8ebef4d63cda1ea2ad58de32566 > core/src/main/java/org/apache/oozie/action/hadoop/YarnRMCredentials.java > PRE-CREATION > core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java > 187cee2290ba4bbe358913d28a052d9c75f2369a > > core/src/test/java/org/apache/oozie/action/hadoop/TestHadoopTokenHelper.java > PRE-CREATION > > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java > 02e60c01db2221e7e107762caa4d0beb3fe53ca5 > core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java > a7d6c18a645b50fc35657515480da9967fa74e2e > > core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java > cf77f18c4bdbe95ea66f63aa115ce6f34c90c855 > core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java > 960c2f9750062c98fdf7a2b456d3668069e07ca5 > > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java > b4cbb4b1d9b52aaebfb562c02edc91e15a2a9a2e > > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java > 874d371664ced3ae19a7b5d615c67d4c72e7cfdd > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java > 3e6ffc90cc53debac678ac79a83864eaad596730 > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java > bde7f1daf123d663d47f5d2fb2350cfa1ac33ed8 > > sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java > 68c0f4bafe0322ff0d1007297cd6e8f38c92020a > > sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java > 37af3dd9ed43f4504ff3909484237e2d05665c6c > > > Diff: https://reviews.apache.org/r/61560/diff/10/ > > > Testing > ------- > > We're running tests on a Kerberized cluster with these changes. > > > Thanks, > > Peter Cseh > >
