----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69462/#review211256 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java Lines 60 (patched) <https://reviews.apache.org/r/69462/#comment296188> @Override ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java Line 80 (original), 79 (patched) <https://reviews.apache.org/r/69462/#comment296189> @Override ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/RemoteCompactorThread.java Lines 35 (patched) <https://reviews.apache.org/r/69462/#comment296199> Perhaps add that this can run inside HMS as well. ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java Line 97 (original), 94 (patched) <https://reviews.apache.org/r/69462/#comment296190> this is also intialzied in init() - should it throw here instead? It seems that the contract is that init() must be called first ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java Lines 241 (patched) <https://reviews.apache.org/r/69462/#comment296193> What is the purpose of this? Why doesn't the existing catch(Throwable) with the same log msg work? service/src/java/org/apache/hive/service/server/HiveServer2.java Lines 1016 (patched) <https://reviews.apache.org/r/69462/#comment296200> Are there any tests that actually enable HIVE_MAPREDUCE_AVAILABLE ? service/src/java/org/apache/hive/service/server/HiveServer2.java Lines 1019 (patched) <https://reviews.apache.org/r/69462/#comment296198> Why do you need reflection for this? Why not just do Worker w = new Worker();, etc? standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java Lines 1020 (patched) <https://reviews.apache.org/r/69462/#comment296194> Perhaps this should be called hive.metastore.runworker.remotely - you can run 'remote' worker even with MR or 'hive.metastore.runworker.in" and support values "metastore" and "hs2" - this is probable more extensivble standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift Lines 2448 (patched) <https://reviews.apache.org/r/69462/#comment296187> nit: could you move these to around line 2312 where the rest of TxnStore methods are standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java Lines 1476 (patched) <https://reviews.apache.org/r/69462/#comment296191> I would put both of these in CompactionInfo. If someone adds fields to CompactionInfo, they are unlikely to ever find these methods and so some info will be lost in the marshalling back and forth. Alternatively, could CompactionInfo be a subclass of CompactionInfoStruct? - Eugene Koifman On Dec. 11, 2018, 3:45 p.m., Jaume Marhuenda wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69462/ > ----------------------------------------------------------- > > (Updated Dec. 11, 2018, 3:45 p.m.) > > > Review request for hive. > > > Repository: hive-git > > > Description > ------- > > Allow the Worker thread in the metastore to run outside of it > > > Diffs > ----- > > > hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java > b290a40734 > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java > beb36d7674 > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java 18253c9bab > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java > c6cb7c5254 > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java > f5b901d6e8 > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java > cdcc0e9548 > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MetaStoreCompactorThread.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/RemoteCompactorThread.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java 4a1cac123c > ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommands2.java dc39f5ef61 > ql/src/test/org/apache/hadoop/hive/ql/TxnCommandsBaseForTests.java > 52453a2ec4 > service/src/java/org/apache/hive/service/server/HiveServer2.java 0c55654475 > > standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CompactionInfoStruct.java > PRE-CREATION > > standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java > b6a0893524 > > standalone-metastore/metastore-common/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php > 3170798663 > > standalone-metastore/metastore-common/src/gen/thrift/gen-php/metastore/Types.php > 39f8b1f05a > > standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote > d57de353c6 > > standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py > a896849989 > > standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ttypes.py > 4ef4aadfee > > standalone-metastore/metastore-common/src/gen/thrift/gen-rb/hive_metastore_types.rb > 97dc0696b7 > > standalone-metastore/metastore-common/src/gen/thrift/gen-rb/thrift_hive_metastore.rb > a5f976bc5c > > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java > 9eb1193a27 > > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java > fa19440ba2 > > standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java > e25a8cf9a1 > standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift > cb899d791f > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java > 598847df03 > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreThread.java > 6ef2e3560d > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionInfo.java > b34b7d70de > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java > 3f611d6270 > > standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClientPreCatalog.java > 9fe9a65677 > streaming/src/test/org/apache/hive/streaming/TestStreaming.java 2170178a81 > > > Diff: https://reviews.apache.org/r/69462/diff/2/ > > > Testing > ------- > > > Thanks, > > Jaume Marhuenda > >
