Thanks Zhichao, I tried that and am now getting this:
[hadoop@ip-172-31-81-255 apache-kylin-3.1.0-bin-hbase1x]$ [hadoop@ip-172-31-81-255 apache-kylin-3.1.0-bin-hbase1x]$ ./bin/kylin.sh org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc ADMIN:KYLIN@ip-172-31-81-255.ec2.internal:7070 -kylinUriDst ADMIN:KYLIN@ip-172-31-91-198.ec2.internal:7070 -cube kylin_sales_cube -execute true -overwrite true -schemaOnly false Using cached dependency... OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x/tool/kylin-tool-3.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/spark/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Running org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc ADMIN:KYLIN@ip-172-31-81-255.ec2.internal:7070 -kylinUriDst ADMIN:KYLIN@ip-172-31-91-198.ec2.internal:7070 -cube kylin_sales_cube -execute true -overwrite true -schemaOnly false 2021-02-26 10:06:54,000 INFO [main] common.KylinConfig:118 : Loading kylin-defaults.properties from file:/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x/tool/kylin-tool-3.1.0.jar!/kylin-defaults.properties 2021-02-26 10:06:54,028 DEBUG [main] common.KylinConfig:346 : KYLIN_CONF property was not set, will seek KYLIN_HOME env variable 2021-02-26 10:06:54,029 INFO [main] common.KylinConfig:352 : Use KYLIN_HOME=/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x 2021-02-26 10:06:54,031 INFO [main] common.KylinConfig:153 : Initialized a new KylinConfig from getInstanceFromEnv : 245765246 2021-02-26 10:06:56,326 INFO [main] zookeeper.RecoverableZooKeeper:120 : Process identifier=hconnection-0x545607f2 connecting to ZooKeeper ensemble=ip-172-31-81-255.ec2.internal:2181 2021-02-26 10:06:56,331 INFO [main] zookeeper.ZooKeeper:100 : Client environment:zookeeper.version=3.4.14-5f788d5e8f90539ee331702c753fa250727128f4, built on 12/14/2019 00:35 GMT 2021-02-26 10:06:56,331 INFO [main] zookeeper.ZooKeeper:100 : Client environment:host.name=ip-172-31-81-255.ec2.internal 2021-02-26 10:06:56,332 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.version=1.8.0_272 2021-02-26 10:06:56,332 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.vendor=Red Hat, Inc. 2021-02-26 10:06:56,332 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.272.b10-1.56.amzn1.x86_64/jre 2021-02-26 10:06:56,332 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.class.path= /*removed*/ 2021-02-26 10:06:56,333 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.library.path=:/usr/lib/hadoop-lzo/lib/native:/usr/lib/hadoop/lib/native 2021-02-26 10:06:56,333 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.io.tmpdir=/tmp 2021-02-26 10:06:56,334 INFO [main] zookeeper.ZooKeeper:100 : Client environment:java.compiler=<NA> 2021-02-26 10:06:56,334 INFO [main] zookeeper.ZooKeeper:100 : Client environment:os.name=Linux 2021-02-26 10:06:56,334 INFO [main] zookeeper.ZooKeeper:100 : Client environment:os.arch=amd64 2021-02-26 10:06:56,335 INFO [main] zookeeper.ZooKeeper:100 : Client environment:os.version=4.14.154-99.181.amzn1.x86_64 2021-02-26 10:06:56,335 INFO [main] zookeeper.ZooKeeper:100 : Client environment:user.name=hadoop 2021-02-26 10:06:56,335 INFO [main] zookeeper.ZooKeeper:100 : Client environment:user.home=/home/hadoop 2021-02-26 10:06:56,335 INFO [main] zookeeper.ZooKeeper:100 : Client environment:user.dir=/usr/local/kylin/apache-kylin-3.1.0-bin-hbase1x 2021-02-26 10:06:56,336 INFO [main] zookeeper.ZooKeeper:442 : Initiating client connection, connectString=ip-172-31-81-255.ec2.internal:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@7207cb51 2021-02-26 10:06:56,357 INFO [main-SendThread(ip-172-31-81-255.ec2.internal:2181)] zookeeper.ClientCnxn:1025 : Opening socket connection to server ip-172-31-81-255.ec2.internal/172.31.81.255:2181. Will not attempt to authenticate using SASL (unknown error) 2021-02-26 10:06:56,363 INFO [main-SendThread(ip-172-31-81-255.ec2.internal:2181)] zookeeper.ClientCnxn:879 : Socket connection established to ip-172-31-81-255.ec2.internal/172.31.81.255:2181, initiating session 2021-02-26 10:06:56,369 INFO [main-SendThread(ip-172-31-81-255.ec2.internal:2181)] zookeeper.ClientCnxn:1299 : Session establishment complete on server ip-172-31-81-255.ec2.internal/172.31.81.255:2181, sessionid = 0x1000002a32a0020, negotiated timeout = 40000 2021-02-26 10:06:56,878 DEBUG [main] hbase.HBaseConnection:316 : HTable 'kylin_metadata' already exists 2021-02-26 10:06:57,222 INFO [main] common.KylinConfig:476 : Creating new manager instance of class org.apache.kylin.metadata.TableMetadataManager 2021-02-26 10:06:57,229 INFO [main] persistence.ResourceStore:90 : Using metadata url kylin_metadata@hbase for resource store 2021-02-26 10:06:57,386 DEBUG [main] hbase.HBaseConnection:181 : Using the working dir FS for HBase: hdfs://ip-172-31-81-255.ec2.internal:8020 2021-02-26 10:06:57,386 INFO [main] hbase.HBaseConnection:267 : connection is null or closed, creating a new one 2021-02-26 10:06:57,387 INFO [main] zookeeper.RecoverableZooKeeper:120 : Process identifier=hconnection-0x10cd6753 connecting to ZooKeeper ensemble=ip-172-31-81-255.ec2.internal:2181 2021-02-26 10:06:57,387 INFO [main] zookeeper.ZooKeeper:442 : Initiating client connection, connectString=ip-172-31-81-255.ec2.internal:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@71ad3d8a 2021-02-26 10:06:57,388 INFO [main-SendThread(ip-172-31-81-255.ec2.internal:2181)] zookeeper.ClientCnxn:1025 : Opening socket connection to server ip-172-31-81-255.ec2.internal/172.31.81.255:2181. Will not attempt to authenticate using SASL (unknown error) 2021-02-26 10:06:57,389 INFO [main-SendThread(ip-172-31-81-255.ec2.internal:2181)] zookeeper.ClientCnxn:879 : Socket connection established to ip-172-31-81-255.ec2.internal/172.31.81.255:2181, initiating session 2021-02-26 10:06:57,392 INFO [main-SendThread(ip-172-31-81-255.ec2.internal:2181)] zookeeper.ClientCnxn:1299 : Session establishment complete on server ip-172-31-81-255.ec2.internal/172.31.81.255:2181, sessionid = 0x1000002a32a0021, negotiated timeout = 40000 2021-02-26 10:06:57,407 DEBUG [main] hbase.HBaseConnection:316 : HTable 'kylin_metadata' already exists 2021-02-26 10:06:57,450 DEBUG [main] cachesync.CachedCrudAssist:122 : Reloading TableDesc from kylin_metadata(key='/table')@kylin_metadata@hbase 2021-02-26 10:06:57,528 INFO [main] measure.MeasureTypeFactory:121 : Checking custom measure types from kylin config 2021-02-26 10:06:57,529 INFO [main] measure.MeasureTypeFactory:150 : registering COUNT_DISTINCT(hllc), class org.apache.kylin.measure.hllc.HLLCMeasureType$Factory 2021-02-26 10:06:57,534 INFO [main] measure.MeasureTypeFactory:150 : registering COUNT_DISTINCT(bitmap), class org.apache.kylin.measure.bitmap.BitmapMeasureType$Factory 2021-02-26 10:06:57,536 INFO [main] measure.MeasureTypeFactory:150 : registering TOP_N(topn), class org.apache.kylin.measure.topn.TopNMeasureType$Factory 2021-02-26 10:06:57,538 INFO [main] measure.MeasureTypeFactory:150 : registering RAW(raw), class org.apache.kylin.measure.raw.RawMeasureType$Factory 2021-02-26 10:06:57,539 INFO [main] measure.MeasureTypeFactory:150 : registering EXTENDED_COLUMN(extendedcolumn), class org.apache.kylin.measure.extendedcolumn.ExtendedColumnMeasureType$Factory 2021-02-26 10:06:57,540 INFO [main] measure.MeasureTypeFactory:150 : registering PERCENTILE_APPROX(percentile), class org.apache.kylin.measure.percentile.PercentileMeasureType$Factory 2021-02-26 10:06:57,541 INFO [main] measure.MeasureTypeFactory:150 : registering COUNT_DISTINCT(dim_dc), class org.apache.kylin.measure.dim.DimCountDistinctMeasureType$Factory 2021-02-26 10:06:57,542 INFO [main] measure.MeasureTypeFactory:150 : registering STDDEV_SUM(stddev_sum), class org.apache.kylin.measure.stddev.StdDevSumMeasureType$Factory 2021-02-26 10:06:57,546 INFO [main] measure.MeasureTypeFactory:150 : registering COUNT_DISTINCT(bitmap_map), class org.apache.kylin.measure.map.bitmap.BitmapMapMeasureType$Factory 2021-02-26 10:06:57,548 DEBUG [main] cachesync.CachedCrudAssist:155 : Loaded 6 TableDesc(s) out of 6 resource with 0 errors 2021-02-26 10:06:57,550 INFO [main] common.KylinConfig:476 : Creating new manager instance of class org.apache.kylin.metadata.cachesync.Broadcaster 2021-02-26 10:06:57,551 DEBUG [main] cachesync.Broadcaster:102 : 1 nodes in the cluster: [localhost:7070] 2021-02-26 10:06:57,553 DEBUG [main] cachesync.CachedCrudAssist:122 : Reloading TableExtDesc from kylin_metadata(key='/table_exd')@kylin_metadata@hbase 2021-02-26 10:06:57,556 DEBUG [main] cachesync.CachedCrudAssist:155 : Loaded 0 TableExtDesc(s) out of 0 resource with 0 errors 2021-02-26 10:06:57,557 DEBUG [main] cachesync.CachedCrudAssist:122 : Reloading ExternalFilterDesc from kylin_metadata(key='/ext_filter')@kylin_metadata@hbase 2021-02-26 10:06:57,558 DEBUG [main] cachesync.CachedCrudAssist:155 : Loaded 0 ExternalFilterDesc(s) out of 0 resource with 0 errors 2021-02-26 10:06:57,561 INFO [main] common.KylinConfig:476 : Creating new manager instance of class org.apache.kylin.metadata.model.DataModelManager 2021-02-26 10:06:57,562 DEBUG [main] cachesync.CachedCrudAssist:122 : Reloading DataModelDesc from kylin_metadata(key='/model_desc')@kylin_metadata@hbase 2021-02-26 10:06:57,588 INFO [main] common.KylinConfig:476 : Creating new manager instance of class org.apache.kylin.metadata.project.ProjectManager 2021-02-26 10:06:57,588 INFO [main] project.ProjectManager:85 : Initializing ProjectManager with metadata url kylin_metadata@hbase 2021-02-26 10:06:57,589 DEBUG [main] cachesync.CachedCrudAssist:122 : Reloading ProjectInstance from kylin_metadata(key='/project')@kylin_metadata@hbase 2021-02-26 10:06:57,602 DEBUG [main] cachesync.CachedCrudAssist:155 : Loaded 1 ProjectInstance(s) out of 1 resource with 0 errors 2021-02-26 10:06:57,608 DEBUG [main] cachesync.CachedCrudAssist:155 : Loaded 2 DataModelDesc(s) out of 2 resource with 0 errors 2021-02-26 10:06:57,610 INFO [main] common.KylinConfig:476 : Creating new manager instance of class org.apache.kylin.storage.hybrid.HybridManager 2021-02-26 10:06:57,610 INFO [main] hybrid.HybridManager:71 : Initializing HybridManager with config kylin_metadata@hbase 2021-02-26 10:06:57,610 DEBUG [main] cachesync.CachedCrudAssist:122 : Reloading HybridInstance from kylin_metadata(key='/hybrid')@kylin_metadata@hbase 2021-02-26 10:06:57,612 DEBUG [main] cachesync.CachedCrudAssist:155 : Loaded 0 HybridInstance(s) out of 0 resource with 0 errors 2021-02-26 10:06:57,616 INFO [main] common.KylinConfig:476 : Creating new manager instance of class org.apache.kylin.cube.CubeManager 2021-02-26 10:06:57,622 INFO [main] cube.CubeManager:136 : Initializing CubeManager with config kylin_metadata@hbase 2021-02-26 10:06:57,623 DEBUG [main] cachesync.CachedCrudAssist:122 : Reloading CubeInstance from kylin_metadata(key='/cube')@kylin_metadata@hbase 2021-02-26 10:06:57,659 INFO [main] common.KylinConfig:476 : Creating new manager instance of class org.apache.kylin.cube.CubeDescManager 2021-02-26 10:06:57,659 INFO [main] cube.CubeDescManager:91 : Initializing CubeDescManager with config kylin_metadata@hbase 2021-02-26 10:06:57,660 DEBUG [main] cachesync.CachedCrudAssist:122 : Reloading CubeDesc from kylin_metadata(key='/cube_desc')@kylin_metadata@hbase 2021-02-26 10:06:57,707 INFO [main] model.RowKeyColDesc:99 : Implicitly convert encoding to time 2021-02-26 10:06:57,707 INFO [main] model.RowKeyColDesc:99 : Implicitly convert encoding to time 2021-02-26 10:06:57,707 INFO [main] model.RowKeyColDesc:95 : Implicitly convert encoding to date 2021-02-26 10:06:57,708 DEBUG [main] cachesync.CachedCrudAssist:155 : Loaded 2 CubeDesc(s) out of 2 resource with 0 errors 2021-02-26 10:06:57,708 DEBUG [main] cachesync.CachedCrudAssist:155 : Loaded 2 CubeInstance(s) out of 2 resource with 0 errors 2021-02-26 10:06:57,709 INFO [main] common.KylinConfig:476 : Creating new manager instance of class org.apache.kylin.metadata.realization.RealizationRegistry 2021-02-26 10:06:57,709 INFO [main] realization.RealizationRegistry:54 : Initializing RealizationRegistry with metadata url kylin_metadata@hbase 2021-02-26 10:06:57,709 INFO [main] realization.RealizationRegistry:81 : RealizationRegistry is {HYBRID=org.apache.kylin.storage.hybrid.HybridManager@4c58255, CUBE=org.apache.kylin.cube.CubeManager@eac3a26} 2021-02-26 10:06:57,710 INFO [main] common.KylinConfig:476 : Creating new manager instance of class org.apache.kylin.dict.DictionaryManager 2021-02-26 10:06:57,711 INFO [main] common.KylinConfig:476 : Creating new manager instance of class org.apache.kylin.dict.lookup.SnapshotManager 2021-02-26 10:06:58,164 INFO [main] zookeeper.RecoverableZooKeeper:120 : Process identifier=hconnection-0x1e1e9ef3 connecting to ZooKeeper ensemble=ip-172-31-91-198.ec2.internal:2181 2021-02-26 10:06:58,165 INFO [main] zookeeper.ZooKeeper:442 : Initiating client connection, connectString=ip-172-31-91-198.ec2.internal:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@3dd31157 2021-02-26 10:06:58,166 INFO [main-SendThread(ip-172-31-91-198.ec2.internal:2181)] zookeeper.ClientCnxn:1025 : Opening socket connection to server ip-172-31-91-198.ec2.internal/172.31.91.198:2181. Will not attempt to authenticate using SASL (unknown error) 2021-02-26 10:06:58,167 INFO [main-SendThread(ip-172-31-91-198.ec2.internal:2181)] zookeeper.ClientCnxn:879 : Socket connection established to ip-172-31-91-198.ec2.internal/172.31.91.198:2181, initiating session 2021-02-26 10:06:58,169 INFO [main-SendThread(ip-172-31-91-198.ec2.internal:2181)] zookeeper.ClientCnxn:1299 : Session establishment complete on server ip-172-31-91-198.ec2.internal/172.31.91.198:2181, sessionid = 0x10000026cd90019, negotiated timeout = 40000 2021-02-26 10:06:58,202 DEBUG [main] hbase.HBaseConnection:316 : HTable 'kylin_metadata' already exists 2021-02-26 10:06:58,209 INFO [main] migration.CubeMigrationCrossClusterCLI:327 : start to migrate cube CUBE[name=kylin_sales_cube] 2021-02-26 10:07:01,164 INFO [main] migration.DstClusterUtil:249 : saved resource /table/DEFAULT.KYLIN_CAL_DT.json 2021-02-26 10:07:01,165 INFO [main] migration.DstClusterUtil:162 : saved table TableDesc{name='KYLIN_CAL_DT', columns=/*removed*/ 2021-02-26 10:07:01,169 INFO [main] migration.DstClusterUtil:249 : saved resource /table/DEFAULT.KYLIN_CATEGORY_GROUPINGS.json 2021-02-26 10:07:01,169 INFO [main] migration.DstClusterUtil:162 : saved table TableDesc{name='KYLIN_CATEGORY_GROUPINGS', columns=/*removed*/ 2021-02-26 10:07:01,172 INFO [main] migration.DstClusterUtil:249 : saved resource /table/DEFAULT.KYLIN_ACCOUNT.json 2021-02-26 10:07:01,172 INFO [main] migration.DstClusterUtil:162 : saved table TableDesc{name='KYLIN_ACCOUNT', columns=/*removed*/ 2021-02-26 10:07:01,174 INFO [main] migration.DstClusterUtil:249 : saved resource /table/DEFAULT.KYLIN_COUNTRY.json 2021-02-26 10:07:01,174 INFO [main] migration.DstClusterUtil:162 : saved table TableDesc{name='KYLIN_COUNTRY', columns=/*removed*/ 2021-02-26 10:07:01,177 INFO [main] migration.DstClusterUtil:249 : saved resource /table/DEFAULT.KYLIN_SALES.json 2021-02-26 10:07:01,177 INFO [main] migration.DstClusterUtil:162 : saved table TableDesc{name='KYLIN_SALES', columns=/*removed*/ 2021-02-26 10:07:01,180 INFO [main] migration.DstClusterUtil:249 : saved resource /model_desc/kylin_sales_model.json 2021-02-26 10:07:01,180 INFO [main] migration.DstClusterUtil:170 : saved model DataModelDesc [name=kylin_sales_model] 2021-02-26 10:07:01,198 INFO [main] migration.DstClusterUtil:249 : saved resource /cube_desc/kylin_sales_cube.json 2021-02-26 10:07:01,198 INFO [main] migration.DstClusterUtil:178 : saved cube desc CubeDesc [name=kylin_sales_cube] 2021-02-26 10:07:01,200 INFO [main] migration.CubeMigrationCrossClusterCLI:485 : start to migrate segment: CUBE[name=kylin_sales_cube] 20120101000000_20120701000000 2021-02-26 10:07:01,214 INFO [main] migration.DstClusterUtil:256 : saved resource /cube_statistics/kylin_sales_cube/f836913c-4bb7-e8ab-907d-70efc78f9f08.seq 2021-02-26 10:07:01,216 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_ACCOUNT/ACCOUNT_COUNTRY/e5af18b4-3e78-8acd-0549-c75d868bcedd.dict has already existed in destination cluster 2021-02-26 10:07:01,217 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_CATEGORY_GROUPINGS/CATEG_LVL3_NAME/8668e511-36d6-24cf-4c3f-ec0051f1fd03.dict has already existed in destination cluster 2021-02-26 10:07:01,218 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_SALES/SELLER_ID/08a91346-5d12-ad4f-3ed7-0fd2330bedb9.dict has already existed in destination cluster 2021-02-26 10:07:01,219 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_SALES/LSTG_SITE_ID/1d0b222c-8600-6874-8c43-9c730500061c.dict has already existed in destination cluster 2021-02-26 10:07:01,220 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_SALES/OPS_REGION/1837f2a0-dc07-9094-01e4-69140c9c602a.dict has already existed in destination cluster 2021-02-26 10:07:01,221 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_SALES/LEAF_CATEG_ID/c9184a84-7a75-4136-8b59-459c977c67e0.dict has already existed in destination cluster 2021-02-26 10:07:01,222 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_COUNTRY/NAME/14682a87-1854-d657-6609-44a1c06fbf50.dict has already existed in destination cluster 2021-02-26 10:07:01,223 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_ACCOUNT/ACCOUNT_COUNTRY/e5af18b4-3e78-8acd-0549-c75d868bcedd.dict has already existed in destination cluster 2021-02-26 10:07:01,224 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_SALES/LSTG_FORMAT_NAME/ffedd2e1-9145-d8a3-07f1-a0f925a23627.dict has already existed in destination cluster 2021-02-26 10:07:01,225 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_COUNTRY/NAME/14682a87-1854-d657-6609-44a1c06fbf50.dict has already existed in destination cluster 2021-02-26 10:07:01,226 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_ACCOUNT/ACCOUNT_BUYER_LEVEL/7dd3e83e-5061-aa49-c2f1-5f405afa7e2b.dict has already existed in destination cluster 2021-02-26 10:07:01,227 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_ACCOUNT/ACCOUNT_SELLER_LEVEL/32a2dfd4-6f87-c00c-ef28-4272f9bc017c.dict has already existed in destination cluster 2021-02-26 10:07:01,228 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_SALES/OPS_USER_ID/1280658b-9adb-f8eb-83a2-60dfb729a6d5.dict has already existed in destination cluster 2021-02-26 10:07:01,229 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_CATEGORY_GROUPINGS/META_CATEG_NAME/57228867-3388-54d4-c560-ddce4ccc456f.dict has already existed in destination cluster 2021-02-26 10:07:01,230 INFO [main] migration.CubeMigrationCrossClusterCLI:571 : Item /dict/DEFAULT.KYLIN_CATEGORY_GROUPINGS/CATEG_LVL2_NAME/7a8f9706-ecf2-eca0-07c7-1d757a261062.dict has already existed in destination cluster 2021-02-26 10:07:01,230 INFO [main] migration.CubeMigrationCrossClusterCLI:590 : Item /table_snapshot/DEFAULT.KYLIN_CAL_DT/c2b99839-f1f7-0fa3-6d0a-92312e36d993.snapshot has already existed in destination cluster 2021-02-26 10:07:01,231 INFO [main] migration.CubeMigrationCrossClusterCLI:590 : Item /table_snapshot/DEFAULT.KYLIN_CATEGORY_GROUPINGS/f1859dd6-f508-9cae-8796-1e98a64ac743.snapshot has already existed in destination cluster 2021-02-26 10:07:01,232 INFO [main] migration.CubeMigrationCrossClusterCLI:590 : Item /table_snapshot/DEFAULT.KYLIN_ACCOUNT/f95df070-7036-8173-9f72-e255e0174a2f.snapshot has already existed in destination cluster 2021-02-26 10:07:01,233 INFO [main] migration.CubeMigrationCrossClusterCLI:590 : Item /table_snapshot/DEFAULT.KYLIN_COUNTRY/e8d8fe45-06a6-b9cc-354f-b76e4e8007a2.snapshot has already existed in destination cluster 2021-02-26 10:07:01,234 INFO [main] migration.CubeMigrationCrossClusterCLI:510 : add segment kylin_sales_cube[20120101000000_20120701000000] to migration list 2021-02-26 10:07:01,307 INFO [Cube-kylin_sales_cube-data-migration-pool-1] migration.CubeMigrationCrossClusterCLI:695 : start to copy hdfs directory from hdfs://ip-172-31-81-255.ec2.internal:8020/user/hbase/data/default/KYLIN_5CKFDB9L12 to hdfs://ip-172-31-91-198.ec2.internal:8020/kylin/kylin_metadata/migration/hbase/data/default/KYLIN_5CKFDB9L12 2021-02-26 10:07:01,307 INFO [Cube-kylin_sales_cube-data-migration-pool-0] migration.CubeMigrationCrossClusterCLI:695 : start to copy hdfs directory from hdfs://ip-172-31-81-255.ec2.internal:8020/kylin/kylin_metadata/kylin-4f8a3d06-8bd0-ec10-81ef-55c4a06ac66a to hdfs://ip-172-31-91-198.ec2.internal:8020/kylin/kylin_metadata/kylin-4f8a3d06-8bd0-ec10-81ef-55c4a06ac66a 2021-02-26 10:07:01,495 INFO [Cube-kylin_sales_cube-data-migration-pool-1] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-81-255.ec2.internal/172.31.81.255:8032 2021-02-26 10:07:01,495 INFO [Cube-kylin_sales_cube-data-migration-pool-0] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-81-255.ec2.internal/172.31.81.255:8032 2021-02-26 10:07:01,689 INFO [Cube-kylin_sales_cube-data-migration-pool-0] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-81-255.ec2.internal/172.31.81.255:8032 2021-02-26 10:07:01,695 INFO [Cube-kylin_sales_cube-data-migration-pool-1] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-81-255.ec2.internal/172.31.81.255:8032 2021-02-26 10:07:01,915 INFO [Cube-kylin_sales_cube-data-migration-pool-0] tools.SimpleCopyListing:589 : Paths (files+dirs) cnt = 71; dirCnt = 15 2021-02-26 10:07:01,916 INFO [Cube-kylin_sales_cube-data-migration-pool-0] tools.SimpleCopyListing:384 : Build file listing completed. 2021-02-26 10:07:01,915 INFO [Cube-kylin_sales_cube-data-migration-pool-1] tools.SimpleCopyListing:589 : Paths (files+dirs) cnt = 12; dirCnt = 6 2021-02-26 10:07:01,916 INFO [Cube-kylin_sales_cube-data-migration-pool-1] tools.SimpleCopyListing:384 : Build file listing completed. 2021-02-26 10:07:01,917 INFO [Cube-kylin_sales_cube-data-migration-pool-0] Configuration.deprecation:1285 : io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb 2021-02-26 10:07:01,917 INFO [Cube-kylin_sales_cube-data-migration-pool-1] Configuration.deprecation:1285 : io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor 2021-02-26 10:07:02,397 INFO [Cube-kylin_sales_cube-data-migration-pool-1] tools.DistCp:94 : Number of paths in the copy list: 12 2021-02-26 10:07:02,402 INFO [Cube-kylin_sales_cube-data-migration-pool-0] tools.DistCp:94 : Number of paths in the copy list: 71 2021-02-26 10:07:02,427 INFO [Cube-kylin_sales_cube-data-migration-pool-1] tools.DistCp:94 : Number of paths in the copy list: 12 2021-02-26 10:07:02,432 INFO [Cube-kylin_sales_cube-data-migration-pool-0] tools.DistCp:94 : Number of paths in the copy list: 71 2021-02-26 10:07:02,466 INFO [Cube-kylin_sales_cube-data-migration-pool-1] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-81-255.ec2.internal/172.31.81.255:8032 2021-02-26 10:07:02,466 INFO [Cube-kylin_sales_cube-data-migration-pool-0] client.RMProxy:123 : Connecting to ResourceManager at ip-172-31-81-255.ec2.internal/172.31.81.255:8032 2021-02-26 10:07:02,514 WARN [Cube-kylin_sales_cube-data-migration-pool-0] mapreduce.JobResourceUploader:64 : Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 2021-02-26 10:07:02,514 WARN [Cube-kylin_sales_cube-data-migration-pool-1] mapreduce.JobResourceUploader:64 : Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 2021-02-26 10:07:02,637 INFO [Cube-kylin_sales_cube-data-migration-pool-1] mapreduce.JobSubmitter:200 : number of splits:4 2021-02-26 10:07:02,644 INFO [Cube-kylin_sales_cube-data-migration-pool-0] mapreduce.JobSubmitter:200 : number of splits:26 2021-02-26 10:07:02,772 INFO [Cube-kylin_sales_cube-data-migration-pool-1] mapreduce.JobSubmitter:289 : Submitting tokens for job: job_1614300651018_0027 2021-02-26 10:07:02,789 INFO [Cube-kylin_sales_cube-data-migration-pool-0] mapreduce.JobSubmitter:289 : Submitting tokens for job: job_1614300651018_0026 2021-02-26 10:07:02,906 INFO [Cube-kylin_sales_cube-data-migration-pool-1] mapreduce.JobSubmitter:251 : Cleaning up the staging area /tmp/hadoop-yarn/staging/hadoop/.staging/job_1614300651018_0027 2021-02-26 10:07:02,906 INFO [Cube-kylin_sales_cube-data-migration-pool-0] mapreduce.JobSubmitter:251 : Cleaning up the staging area /tmp/hadoop-yarn/staging/hadoop/.staging/job_1614300651018_0026 2021-02-26 10:07:02,912 ERROR [Cube-kylin_sales_cube-data-migration-pool-1] migration.CubeMigrationCrossClusterCLI:680 : fail to migrate htable KYLIN_5CKFDB9L12 due to {} java.lang.NullPointerException at org.apache.hadoop.mapreduce.v2.util.MRApps.setMRFrameworkClasspath(MRApps.java:210) at org.apache.hadoop.mapreduce.v2.util.MRApps.setClasspath(MRApps.java:259) at org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:467) at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:296) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:242) at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1341) at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1338) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1338) at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:193) at org.apache.hadoop.tools.DistCp.execute(DistCp.java:155) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.copyHDFSPath(CubeMigrationCrossClusterCLI.java:701) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.copyHFileByDistCp(CubeMigrationCrossClusterCLI.java:690) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.copyHTable(CubeMigrationCrossClusterCLI.java:662) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.copyHTable(CubeMigrationCrossClusterCLI.java:614) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.access$200(CubeMigrationCrossClusterCLI.java:88) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI$3.doRun(CubeMigrationCrossClusterCLI.java:505) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI$MyRunnable.run(CubeMigrationCrossClusterCLI.java:742) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2021-02-26 10:07:02,914 ERROR [main] migration.CubeMigrationCrossClusterCLI:431 : java.lang.NullPointerException 2021-02-26 10:07:02,915 ERROR [main] migration.CubeMigrationCrossClusterCLI:332 : fail to migrate cube CUBE[name=kylin_sales_cube] due to java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NullPointerException at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.migrateCube(CubeMigrationCrossClusterCLI.java:432) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.execute(CubeMigrationCrossClusterCLI.java:329) at org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.main(CubeMigrationCrossClusterCLI.java:755) Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.migrateCube(CubeMigrationCrossClusterCLI.java:426) ... 3 more Caused by: java.lang.NullPointerException at org.apache.hadoop.mapreduce.v2.util.MRApps.setMRFrameworkClasspath(MRApps.java:210) at org.apache.hadoop.mapreduce.v2.util.MRApps.setClasspath(MRApps.java:259) at org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:467) at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:296) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:242) at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1341) at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1338) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1338) at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:193) at org.apache.hadoop.tools.DistCp.execute(DistCp.java:155) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.copyHDFSPath(CubeMigrationCrossClusterCLI.java:701) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.copyHDFSJobInfo(CubeMigrationCrossClusterCLI.java:602) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI.access$100(CubeMigrationCrossClusterCLI.java:88) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI$2.doRun(CubeMigrationCrossClusterCLI.java:497) at org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI$MyRunnable.run(CubeMigrationCrossClusterCLI.java:742) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2021-02-26 10:07:02,915 INFO [main] migration.DstClusterUtil:363 : update meta cache for org.apache.kylin.common.restclient.RestClient@7bab5898 2021-02-26 10:07:02,920 WARN [main] migration.CubeMigrationCrossClusterCLI:361 : Failed to migrate cubes [CUBE[name=kylin_sales_cube]] and need to check the detailed reason and retry again!!! 2021-02-26 10:07:02,924 INFO [close-hbase-conn] hbase.HBaseConnection:137 : Closing HBase connections... 2021-02-26 10:07:02,924 INFO [close-hbase-conn] client.ConnectionManager$HConnectionImplementation:2259 : Closing master protocol: MasterService 2021-02-26 10:07:02,929 INFO [close-hbase-conn] client.ConnectionManager$HConnectionImplementation:1776 : Closing zookeeper sessionid=0x1000002a32a0021 2021-02-26 10:07:02,930 INFO [close-hbase-conn] zookeeper.ZooKeeper:693 : Session: 0x1000002a32a0021 closed 2021-02-26 10:07:02,930 INFO [main-EventThread] zookeeper.ClientCnxn:522 : EventThread shut down for session: 0x1000002a32a0021 [hadoop@ip-172-31-81-255 apache-kylin-3.1.0-bin-hbase1x]$ Please let me know if I am missing something -- Sent from: http://apache-kylin.74782.x6.nabble.com/
