> On Oct. 18, 2018, 9:13 a.m., Peter Vary wrote: > > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java > > Lines 1921 (patched) > > <https://reviews.apache.org/r/68975/diff/5/?file=2099607#file2099607line1923> > > > > nit: one extra space
done. > On Oct. 18, 2018, 9:13 a.m., Peter Vary wrote: > > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java > > Line 1876 (original) > > <https://reviews.apache.org/r/68975/diff/5/?file=2099607#file2099607line1967> > > > > This perf logging is removed... Let's think through if we need other > > places to add some instead considering the new calling structure It was not removed, but moved to another place. Now the tracking is started and finished in loadPartition(). > On Oct. 18, 2018, 9:13 a.m., Peter Vary wrote: > > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java > > Lines 2171 (patched) > > <https://reviews.apache.org/r/68975/diff/5/?file=2099607#file2099607line2197> > > > > What happens when only 1 partiton is already exists from multiple ones? > > > > Also, do we need SynchronizedMSC? Yes, we need this one in every case. The SynchronizedMetaStoreClient is just a facede for the SessionHiveMetaStoreClient, which in the other hand, does some pre-processing (eg. adding the partition to the temp table), before calling the HiveMetaStoreClient. > On Oct. 18, 2018, 9:13 a.m., Peter Vary wrote: > > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java > > Lines 2570 (patched) > > <https://reviews.apache.org/r/68975/diff/5/?file=2099607#file2099607line2607> > > > > Do we need to call this every time in the for loop? I kinda remember > > that we allow only partitions for a single table only... Or I might be > > mistaken, but still might be a good idea to not generate this every time... I think we do need. The fetching of the snapshot is done inside a callable task which is executed on thread pool in parallel. Altough we use only one table, the snapshot of it can change from time to time, depending on how many tasks were already finished. It's better to be safe, and get a new snapshot every time, than have an outdated one. - Laszlo ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68975/#review209735 ----------------------------------------------------------- On Oct. 18, 2018, 7:01 a.m., Laszlo Pinter wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68975/ > ----------------------------------------------------------- > > (Updated Oct. 18, 2018, 7:01 a.m.) > > > Review request for hive and Peter Vary. > > > Repository: hive-git > > > Description > ------- > > HIVE-20661: Dynamic partitions loading calls add partition for every > partition 1-by-1 > > > Diffs > ----- > > > ql/src/java/org/apache/hadoop/hive/metastore/SynchronizedMetaStoreClient.java > 0ab77e84c6222d35bcec9ce95ed02014911ef144 > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java > 4de038913a5c9a2c199f71702b8f70ca84d0856b > > ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java > dd23d7db3e70c9540e48c42eb7b9a33ed775cea6 > > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java > aba63f050b5b98a2aeeb0df6ff2de5e6e06761f2 > > > Diff: https://reviews.apache.org/r/68975/diff/5/ > > > Testing > ------- > > > Thanks, > > Laszlo Pinter > >
