----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52782/ -----------------------------------------------------------
(Updated July 17, 2017, 4:05 p.m.) Review request for oozie, Attila Sasvari, Peter Cseh, Peter Bacsko, and Robert Kanter. Changes ------- Addressing review comments. Repository: oozie-git Description ------- We get each 1000 rows into a separate JPA `EntityTransaction` to reduce heap size. Furthermore, of at least one row inside that tx fails, we retry the whole batch into separate `EntityTransaction`s each. Following error handling is implemented: 1. check if all necessary tables are present and empty 2. rows are imported till the end even if there are skipped rows in the meanwhile 3. if at least one row is skipped in the meanwhile for some `ConstraintViolationException`, we delete all rows of all necessary tables. That enables the user to have the log messages of all the erroneous rows in one run, and Oozie database is never in an inconsistent state of some rows present, some not present of an import Diffs (updated) ----- core/src/main/resources/oozie-default.xml 832bbe14a4b027e198061527a956e2992cbec174 tools/src/main/java/org/apache/oozie/tools/OozieDBImportCLI.java 0e14a30693a76b8b2bdc2f7ceaf3f045d69f4155 tools/src/test/java/org/apache/oozie/tools/TestDBLoadDump.java c43223ef05aa702be49565ba2626314628e63749 tools/src/test/resources/dumpData/invalid/ooziedb_ac.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_ca.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_cj.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_sysinfo.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_wf.json PRE-CREATION tools/src/test/resources/dumpData/ooziedb_ac.json tools/src/test/resources/dumpData/ooziedb_bna.json tools/src/test/resources/dumpData/ooziedb_bnj.json tools/src/test/resources/dumpData/ooziedb_ca.json tools/src/test/resources/dumpData/ooziedb_cj.json tools/src/test/resources/dumpData/ooziedb_slareg.json tools/src/test/resources/dumpData/ooziedb_slasum.json tools/src/test/resources/dumpData/ooziedb_sysinfo.json tools/src/test/resources/dumpData/ooziedb_wf.json tools/src/test/resources/dumpData/valid/ooziedb_slareg.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_slasum.json PRE-CREATION Diff: https://reviews.apache.org/r/52782/diff/6/ Changes: https://reviews.apache.org/r/52782/diff/5-6/ Testing ------- See `TestDBLoadDump` for further reference. Thanks, András Piros
