----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13397/#review25092 -----------------------------------------------------------
Leave the action Main methods writing data to local files as is. In the LauncherMapper map() finally, read from the multiple files and write to one sequence file. trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java <https://reviews.apache.org/r/13397/#comment49297> Remove this method from all action executors and handle the logic only in super class. trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java <https://reviews.apache.org/r/13397/#comment49296> Remove this method altogether. Was added in OOZIE-1160, but redundant. Can be done with getActionData() itself if we need to do modify context. trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java <https://reviews.apache.org/r/13397/#comment49291> To be removed trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java <https://reviews.apache.org/r/13397/#comment49302> Load the sequence file into a map here using a readActionOutput method if running job is complete trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java <https://reviews.apache.org/r/13397/#comment49299> Check if the id is there in the loaded map trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java <https://reviews.apache.org/r/13397/#comment49301> Load the sequence file into a map here using a readActionOutput method if running job is complete and set the following based on the contents of the map<String, String>. context.setErrorInfo context.setExecutionData context.setExecutionStats context.setExternalChildIDs Sequencefile read will use the same Text object. So convert to String and store in the map trunk/core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java <https://reviews.apache.org/r/13397/#comment49303> If id was null in the sequence file, use this method to only check from the counters trunk/core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java <https://reviews.apache.org/r/13397/#comment49309> getActionOutputPath or getActionDataPath trunk/core/src/main/java/org/apache/oozie/util/IOUtils.java <https://reviews.apache.org/r/13397/#comment49306> Can be removed trunk/core/src/main/java/org/apache/oozie/util/PropertiesUtils.java <https://reviews.apache.org/r/13397/#comment49307> Can be removed or reverted. trunk/core/src/main/java/org/apache/oozie/util/StatusUtils.java <https://reviews.apache.org/r/13397/#comment49308> Change not related. Can we revert this? If not you can add more info to javadoc instead of changing the method signature. trunk/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java <https://reviews.apache.org/r/13397/#comment49304> Please use .seq extension for sequence file. Something like actionData.seq or actionOut.seq. Also the helper method names should refer to sequence file instead of properties. trunk/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java <https://reviews.apache.org/r/13397/#comment49305> Use a new or retain the action prefix, so that the variables are grouped. trunk/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java <https://reviews.apache.org/r/13397/#comment49287> Extract prepare xml and pass instead of changing the PrepareActionsDriver.doOperations api to actionXml. Leave the PrepareActionsDriver.doOperations as it is. - Rohini Palaniswamy On Aug. 13, 2013, 5:14 a.m., Mona Chitnis wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/13397/ > ----------------------------------------------------------- > > (Updated Aug. 13, 2013, 5:14 a.m.) > > > Review request for oozie. > > > Bugs: OOZIE-1486 > https://issues.apache.org/jira/browse/OOZIE-1486 > > > Repository: oozie > > > Description > ------- > > See JIRA Description > > > Diffs > ----- > > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java > 1513230 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > 1513230 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java > 1513230 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java > 1513230 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java > 1513230 > trunk/core/src/main/java/org/apache/oozie/util/IOUtils.java 1513230 > trunk/core/src/main/java/org/apache/oozie/util/PropertiesUtils.java 1513230 > trunk/core/src/main/java/org/apache/oozie/util/StatusUtils.java 1513230 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java > 1513230 > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java > 1513230 > > trunk/sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java > 1513230 > > trunk/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java > 1513230 > > trunk/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java > 1513230 > > trunk/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java > 1513230 > > trunk/sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java > 1513230 > > trunk/sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java > 1513230 > > Diff: https://reviews.apache.org/r/13397/diff/ > > > Testing > ------- > > Existing unit tests should pass, and E-2-E test pending. > > Few tests failing with a common problem of Properties object (for > output/new-id/error etc) getting stored in the file with Timestamp. Hence on > retrieval, it gives NPE on the expected keys. WIP to fix that small part > > > Thanks, > > Mona Chitnis > >
