> On July 11, 2019, 12:56 p.m., Zoltan Haindrich wrote: > > ql/src/java/org/apache/hadoop/hive/ql/util/HiveStrictManagedMigration.java > > Lines 600 (patched) > > <https://reviews.apache.org/r/71038/diff/2/?file=2154469#file2154469line630> > > > > I think we should stop processing
The original code did not stopped processing I didn't want to change this behavior. > On July 11, 2019, 12:56 p.m., Zoltan Haindrich wrote: > > ql/src/java/org/apache/hadoop/hive/ql/util/HiveStrictManagedMigration.java > > Lines 619-620 (patched) > > <https://reviews.apache.org/r/71038/diff/2/?file=2154469#file2154469line649> > > > > I think the old behaviour was to stop processing after the first error; > > and start exiting...right now it will continue as much as it can...and at > > the end it will print out something depending on this boolean... I did not changed the original or at least I was not intend to. In the original version both databases and tables were processed in for loops. Each loop body contains a try block with a catch clause for Exception. So all exceptions were caugth logged and the variable failuresEncountered was set to true. The loop continued to the next iteration the program did not stop processing. https://github.com/apache/hive/blob/4e8b940aa2fd4ea4574d0a3ff74bfbc2f336781b/ql/src/java/org/apache/hadoop/hive/ql/util/HiveStrictManagedMigration.java#L296 https://github.com/apache/hive/blob/4e8b940aa2fd4ea4574d0a3ff74bfbc2f336781b/ql/src/java/org/apache/hadoop/hive/ql/util/HiveStrictManagedMigration.java#L416 - Krisztian ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71038/#review216522 ----------------------------------------------------------- On July 11, 2019, noon, Krisztian Kasa wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71038/ > ----------------------------------------------------------- > > (Updated July 11, 2019, noon) > > > Review request for hive, Jesús Camacho Rodríguez, Jason Dere, Zoltan > Haindrich, and Vineet Garg. > > > Bugs: HIVE-21965 > https://issues.apache.org/jira/browse/HIVE-21965 > > > Repository: hive-git > > > Description > ------- > > Implement parallel processing in HiveStrictManagedMigration > =========================================================== > > 1. Process tables paralelly using a thread pool for each. Thread pools size > can be defined by command line options. If no options are given the default > pool size is the number of cpu cores. > 2. Add option for filtering table type. > > I had to remove database parallel processing because the ForkJoinPool > implementation does not support nested parallel streams. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/util/CloseableThreadLocal.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/util/HiveStrictManagedMigration.java > 80025b7046 > > ql/src/java/org/apache/hadoop/hive/ql/util/NamedForkJoinWorkerThreadFactory.java > PRE-CREATION > ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java > 5f39fdccb5 > ql/src/test/org/apache/hadoop/hive/ql/util/CloseableThreadLocalTest.java > PRE-CREATION > > ql/src/test/org/apache/hadoop/hive/ql/util/TestHiveStrictManagedMigration.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/71038/diff/2/ > > > Testing > ------- > > New UT for testing migration. > > > Thanks, > > Krisztian Kasa > >
