----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52782/ -----------------------------------------------------------
(Updated July 4, 2017, noon) Review request for oozie, Attila Sasvari, Peter Cseh, Peter Bacsko, and Robert Kanter. Changes ------- Addressing new review comments. Repository: oozie-git Description ------- We get each 1000 rows into a separate JPA `EntityTransaction` to reduce heap size. Furthermore, of at least one row inside that tx fails, we retry the whole batch into separate `EntityTransaction`s each. Following error handling is implemented: 1. check if all necessary tables are present and empty 2. rows are imported till the end even if there are skipped rows in the meanwhile 3. if at least one row is skipped in the meanwhile for some `ConstraintViolationException`, we delete all rows of all necessary tables. That enables the user to have the log messages of all the erroneous rows in one run, and Oozie database is never in an inconsistent state of some rows present, some not present of an import Diffs (updated) ----- core/src/main/resources/oozie-default.xml c60a4581a84d4c67a1ac1cf3dfdc252b85ccd01c tools/src/main/java/org/apache/oozie/tools/OozieDBImportCLI.java 0e14a30693a76b8b2bdc2f7ceaf3f045d69f4155 tools/src/test/java/org/apache/oozie/tools/TestDBLoadDump.java c43223ef05aa702be49565ba2626314628e63749 tools/src/test/resources/dumpData/invalid/ooziedb_ac.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_ca.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_cj.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_sysinfo.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_wf.json PRE-CREATION tools/src/test/resources/dumpData/ooziedb_ac.json tools/src/test/resources/dumpData/ooziedb_bna.json tools/src/test/resources/dumpData/ooziedb_bnj.json tools/src/test/resources/dumpData/ooziedb_ca.json tools/src/test/resources/dumpData/ooziedb_cj.json tools/src/test/resources/dumpData/ooziedb_slareg.json tools/src/test/resources/dumpData/ooziedb_slasum.json tools/src/test/resources/dumpData/ooziedb_sysinfo.json tools/src/test/resources/dumpData/ooziedb_wf.json tools/src/test/resources/dumpData/valid/ooziedb_bna.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_bnj.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_slareg.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_slasum.json PRE-CREATION Diff: https://reviews.apache.org/r/52782/diff/5/ Changes: https://reviews.apache.org/r/52782/diff/4-5/ Testing ------- See `TestDBLoadDump` for further reference. Thanks, András Piros
