Daniel Dai created HIVE-18530: --------------------------------- Summary: Replication should skip MM table (for now) Key: HIVE-18530 URL: https://issues.apache.org/jira/browse/HIVE-18530 Project: Hive Issue Type: Bug Components: repl Reporter: Daniel Dai Assignee: Daniel Dai
Currently replication cannot handle transactional table (including MM table) until HIVE-18320. HIVE-17504 skips table with transactional=true explicitly. HIVE-18352 changes the logic to use AcidUtils.isAcidTable for the same purpose. However, isAcidTable returns false for mm table, thus Hive still dump mm table during replication. Here is an error message during dump mm table: {code} ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask. java.io.FileNotFoundException: Path is not a file: /apps/hive/warehouse/testrepldb5.db/test1/delta_0000261_0000261_0000 at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89) at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75) at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) INFO : Completed executing command(queryId=hive_20180119203438_293813df-7630-47fa-bc30-5ef7cbb42842); Time taken: 1.119 seconds Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask. java.io.FileNotFoundException: Path is not a file: /apps/hive/warehouse/testrepldb5.db/test1/delta_0000261_0000261_0000 at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89) at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75) at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) (state=08S01,code=1) 0: jdbc:hive2://ctr-e137-1514896590304-25219-> Closing: 0: jdbc:hive2://ctr-e137-1514896590304-25219-02-000005.hwx.site:2181,ctr-e137-1514896590304-25219-02-000012.hwx.site:2181,ctr-e137-1514896590304-25219-02-000009.hwx.site:2181,ctr-e137-1514896590304-25219-02-000004.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com {code} We shall switch to use AcidUtils.isTransactionalTable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)