----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38474/#review111340 -----------------------------------------------------------
client/src/main/resources/oozie-coordinator-0.5.xsd (line 19) <https://reviews.apache.org/r/38474/#comment171473> Please add some documentation on the new features. It should explain all of the options, and include some examples. It would also be good to include an example coordinator or two in the examples tarball, though this can be a followup JIRA if you want. - Robert Kanter On Dec. 15, 2015, 7:33 p.m., Purshotam Shah wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/38474/ > ----------------------------------------------------------- > > (Updated Dec. 15, 2015, 7:33 p.m.) > > > Review request for oozie. > > > Bugs: OOZIE-1976 > https://issues.apache.org/jira/browse/OOZIE-1976 > > > Repository: oozie-git > > > Description > ------- > > There are three components in this patch > > 1. User interface > A new tag is added to coordinator.xml > ex. > <input-check> > <or name="test"> > <and> > <data-in dataset="A"/>" > <data-in dataset="B"/> > </and> > <and> > <data-in dataset="C"/> > <data-in dataset="D"/> > </and> > > </or>; > <input-check> > > > input-check will have nested and/or/combine operation. It can have min and > wait at operator or at date-in. > If input-check tag is missing then it consider to be old approach where all > data dependency are needed. > > 2. Processing > input-check is converted into logical expression > (a&&B)||(c&&d) > We use jexl to parse the logical expression. > > There are three phase in parsing. > phase 1 : only resolved dataset are parsed ( only current). > phase 2 : once all current are resolved, then future/latest are parsed. > phase 3 : Doesn't do any filecheck, just return what is being parsed by > phase1 and phase2. Is used for EL functions > > > 3. Storage. > if inputcheck is enable, push_missing_dependencies and missing_dependencies > are serialized and stored in DB. > If then not then it's old approach, where they are stored in plan text. This > is backward compatible. > > > Diffs > ----- > > client/src/main/resources/oozie-coordinator-0.5.xsd > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > core/pom.xml b063dab79415447a86c1a33f5c3f5304e0dffca0 > core/src/main/java/org/apache/oozie/CoordinatorActionBean.java > 91bff4dca2ef2dece68ca7260724ba3e43b1a08e > core/src/main/java/org/apache/oozie/ErrorCode.java > 6c1e3997c9a1cb0bb0de39d687a083af0a7b5f04 > > core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java > 742d00dd47aab55392abc8fbc207f8100728b832 > > core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdatePushMissingDependency.java > 4e1c5b3392358cb6b1e98e16e469310338f27fed > core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java > 58ef483272264e0d391d4a1e6533dc1cab9940da > > core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java > 39e6ac15ce3a3ea7f2ed9178688537f7b1d7842d > > core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java > b05344d89e8df0e11fe69c1aa725d19a18eb0a2b > core/src/main/java/org/apache/oozie/coord/CoordELConstants.java > f010a817fc900821c0e429fc16e1d3902a98d8bb > core/src/main/java/org/apache/oozie/coord/CoordELEvaluator.java > 8b2f4560ae66bbcd707a446382e647663ea67be1 > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java > 5d238663aa94f0dd55a9190b60bfe621439c7b53 > core/src/main/java/org/apache/oozie/coord/CoordUtils.java > 94c69740618110ea180b188ab0c5a02db76f8b4d > core/src/main/java/org/apache/oozie/coord/SyncCoordAction.java > 44258eb5be40bf6769e32c8780117e8533d80d7e > > core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependenciesBuilder.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependenciesChecker.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependenciesUtil.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependency.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheck.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseOne.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseThree.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseTwo.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseValidate.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyFactory.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputInstance.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordOldInputDependency.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordPullInputDependency.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordPushInputDependency.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/CoordUnResolvedInputDependency.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/InputLogicParser.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/OozieJexlEngine.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/main/java/org/apache/oozie/coord/input/dependency/OozieJexlInterpreter.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > core/src/main/java/org/apache/oozie/dependency/ActionDependency.java > c280d1dc2387c9b3df96d4e83ad5563e10b1e1ef > core/src/main/java/org/apache/oozie/dependency/DependencyChecker.java > a5507575a3403103133f85a78c873c9976419857 > core/src/main/java/org/apache/oozie/util/WritableUtils.java > 76a68953541125cab05bd05c94aa73b50e5363ff > core/src/main/resources/oozie-default.xml > faf37405757719554a5b6e3671d0c87723eb9959 > > core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java > 1fe1b3adb5ceaa83985ef31278e49a3224cde0fb > > core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java > 7062e697e5162ac25688d9cb62352bd139a7013a > > core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java > 29e7ca145612fadee661fe975aa0ea2cab1cbfa3 > > core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java > 52eb9dd1fb903ac86294a55d6e2a6918390bf4e9 > > core/src/test/java/org/apache/oozie/coord/input/dependency/TestCoordInputCheckPush.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/test/java/org/apache/oozie/coord/input/dependency/TestCoordinatorInputCheck.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > > core/src/test/java/org/apache/oozie/coord/input/dependency/TestInputLogicParser.java > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > core/src/test/resources/coord-action-sla.xml > f3f1bc09c51272a35d65edd7271599e7e8ba1ba4 > core/src/test/resources/coord-dataset-offset.xml > 5dfbb1b40ee86dfc3336031eea3c5ff3144b9f5a > core/src/test/resources/coord-inputcheck-combine.xml > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > core/src/test/resources/coord-inputcheck-hcat.xml > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > core/src/test/resources/coord-inputcheck-latest.xml > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > core/src/test/resources/coord-inputcheck-range-latest.xml > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > core/src/test/resources/coord-inputcheck-range.xml > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > core/src/test/resources/coord-inputcheck.xml > e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 > pom.xml a74ffab081defd15a1c5dbbf15355aa4e0455029 > > Diff: https://reviews.apache.org/r/38474/diff/ > > > Testing > ------- > > > Thanks, > > Purshotam Shah > >
