----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52782/ -----------------------------------------------------------
(Updated Oct. 12, 2016, 3:01 p.m.) Review request for oozie, Attila Sasvari, Peter Cseh, Peter Bacsko, and Robert Kanter. Changes ------- Some `JPAService` modifications due to needed schema creation is not needed anymore. Repository: oozie-git Description ------- We get each 1000 rows into a separate JPA `EntityTransaction` to reduce heap size. Furthermore, of at least one row inside that tx fails, we retry the whole batch into separate `EntityTransaction`s each. Following error handling is implemented: 1. check if all necessary tables are present and empty 2. rows are imported till the end even if there are skipped rows in the meanwhile 3. if at least one row is skipped in the meanwhile for some `ConstraintViolationException`, we delete all rows of all necessary tables. That enables the user to have the log messages of all the erroneous rows in one run, and Oozie database is never in an inconsistent state of some rows present, some not present of an import Diffs (updated) ----- tools/src/main/java/org/apache/oozie/tools/OozieDBImportCLI.java 0e14a30 tools/src/test/java/org/apache/oozie/tools/TestDBLoadDump.java c43223e tools/src/test/resources/dumpData/invalid/ooziedb_ac.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_bna.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_bnj.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_ca.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_cj.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_slareg.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_slasum.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_sysinfo.json PRE-CREATION tools/src/test/resources/dumpData/invalid/ooziedb_wf.json PRE-CREATION tools/src/test/resources/dumpData/ooziedb_ac.json 22bbdc2 tools/src/test/resources/dumpData/ooziedb_bna.json e69de29 tools/src/test/resources/dumpData/ooziedb_bnj.json e69de29 tools/src/test/resources/dumpData/ooziedb_ca.json 2715b94 tools/src/test/resources/dumpData/ooziedb_cj.json 979c10e tools/src/test/resources/dumpData/ooziedb_slareg.json e69de29 tools/src/test/resources/dumpData/ooziedb_slasum.json e69de29 tools/src/test/resources/dumpData/ooziedb_sysinfo.json 15de009 tools/src/test/resources/dumpData/ooziedb_wf.json 05e7e36 tools/src/test/resources/dumpData/valid/ooziedb_ac.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_bna.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_bnj.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_ca.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_cj.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_slareg.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_slasum.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_sysinfo.json PRE-CREATION tools/src/test/resources/dumpData/valid/ooziedb_wf.json PRE-CREATION Diff: https://reviews.apache.org/r/52782/diff/ Testing ------- See `TestDBLoadDump` for further reference. Thanks, András Piros
