----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66402/#review200829 -----------------------------------------------------------
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java Lines 4265 (patched) <https://reviews.apache.org/r/66402/#comment281730> I think default memory consumption of ~100MB in cache will be a good default. Assuming 1KB (?) for each entry that will be 100K entries. Lets use that. ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java Lines 4958 (patched) <https://reviews.apache.org/r/66402/#comment281729> It will be good to leave a comment here about this. ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/SessionStatsSource.java Lines 35 (patched) <https://reviews.apache.org/r/66402/#comment281767> There should be a singleton for this cache which can be called from anywhere. e.g., see MaterializationsInvalidationCache ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/StatsSources.java Lines 34 (patched) <https://reviews.apache.org/r/66402/#comment281731> Deprecated when it was first written :) Any reason for having it this way. ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/StatsSources.java Lines 64 (patched) <https://reviews.apache.org/r/66402/#comment281768> LOG.debug ql/src/java/org/apache/hadoop/hive/ql/reexec/SessionStatsPlugin.java Lines 36-40 (patched) <https://reviews.apache.org/r/66402/#comment281769> I am not sure SS stats source should be linked to driver. Its lifecycle is not tied to query or driver or SS. This should be initialized once in a HS2 lifecycle. ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java Lines 123 (patched) <https://reviews.apache.org/r/66402/#comment281766> Why is stored in SessionState. SS is per query, whereas statscache is global. - Ashutosh Chauhan On April 10, 2018, 12:31 p.m., Zoltan Haindrich wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66402/ > ----------------------------------------------------------- > > (Updated April 10, 2018, 12:31 p.m.) > > > Review request for hive and Ashutosh Chauhan. > > > Bugs: HIVE-19009 > https://issues.apache.org/jira/browse/HIVE-19009 > > > Repository: hive-git > > > Description > ------- > > * retain runtime stats at session level > * some fixes to support runtime stats during vectorized execution > * make collection less strict; there are cases when the same op tree is > evaluated multiple times in a query > * minor fixes and added some logging > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java > dd94d4db87514de9050ea5669f73027ff3dd6937 > ql/src/java/org/apache/hadoop/hive/ql/Context.java > 58fa5f2287e1221f300e16a78aa76dc6fc23cf47 > ql/src/java/org/apache/hadoop/hive/ql/Driver.java > 75f928b69d3d7b206564216d24be450848a1fe8a > ql/src/java/org/apache/hadoop/hive/ql/DriverFactory.java > 0f6a80ef0d7b768371a932e5bda75348955d0369 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java > 13a2fc478fda244eca8632d12f2d8f46ae280a63 > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/EmptyStatsSource.java > 57762edbdfbf4f77b051f42faf313db882878969 > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/SessionStatsSource.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/StatsSource.java > a4cb6e977190c4f5fa275619a8e97755ad1f55d8 > ql/src/java/org/apache/hadoop/hive/ql/plan/mapper/StatsSources.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/reexec/IReExecutionPlugin.java > 2b0d23c6f237915684d59856cb62001c01c8209c > ql/src/java/org/apache/hadoop/hive/ql/reexec/ReExecDriver.java > 93031712dc6fb60a6d618c8754e50def489a12af > ql/src/java/org/apache/hadoop/hive/ql/reexec/ReExecutionOverlayPlugin.java > 4ee3c14b3988521b3f44d02f032163734ed9e4d9 > ql/src/java/org/apache/hadoop/hive/ql/reexec/ReOptimizePlugin.java > 707858700f4d06301fb873fb47f8d9d9b101d056 > ql/src/java/org/apache/hadoop/hive/ql/reexec/SessionStatsPlugin.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java > 0071a9a4ebadd73103a6f6a5c97cc9992df7aa48 > ql/src/test/org/apache/hadoop/hive/ql/plan/mapping/TestCounterMapping.java > 9fe95e4c5684b4aa69dd7148a14c29ed9c2a0062 > > > Diff: https://reviews.apache.org/r/66402/diff/2/ > > > Testing > ------- > > > Thanks, > > Zoltan Haindrich > >
