foreverneverer opened a new pull request, #1006: URL: https://github.com/apache/incubator-pegasus/pull/1006
# Related-Issue https://github.com/apache/incubator-pegasus/issues/962 # Change This pr support migrate table online via duplication. the tool will auto-execute some command and check current progress to copy table to another cluster online. ## Note - you must add config such as follow to open the duplication: ```diff [replication] + duplication_enabled = true [pegasus.clusters] + tjsrv-account = 127.0.0.1:51601,127.0.0.1:51601 # the target cluster info [duplication-group] + tjsrv-account = 1 # current cluster + bjsrv-account = 2 # target cluster ``` - In some stage, current server will keep reject-write status for some minute to consume the remaining wal - If want to auto-switch the table meta addr, user need deploy `meta-proxy` and client connect table via the `meta-proxy`, otherwise, user need update client config to connect the new cluster - Only `replica data version = 1` support the feature, otherwise the tool will retuern error # Manual-Test When you use the tool, the info as follow(`-h` to get the args usage): ```bash Pegasus-AdminCli-1.2.0 ยป table-migrator -t duptable -c c4tst-function2 -m 127.0.0.1:22601,127.0.0.1:22601 -r /pegasus/c4srv-metaproxy INFO: set pendingMutationThreshold = 100000 means that server will reject all write and ready to switch cluster if the pending less the value INFO: check data version INFO: create table duplication successfully add duplication [dupid: 0] INFO: check pending mutation count if less 100000 WARN: node1 has pending_mutations_count 201501 WARN: node2 has pending_mutations_count 279044 INFO: all the node pending_mutations_count has less 100000 INFO: set env config deny write request INFO: check duplicate qps if equal 0 WARN: node1[139.0] still sending pending mutation 970 WARN:node2[139.0] still sending pending mutation 435 INFO: all the node has stop duplicate the pending wal and wait 30s to switch cluster INFO: switch table addrs in metaproxy 2022/06/16 15:31:22 connected to zk:11000 2022/06/16 15:31:22 authenticated: id=178669580305488139, timeout=4000 2022/06/16 15:31:22 re-submitting `0` credentials after reconnect duptable has updated metaproxy addr from &{c4tst-function1 127.0.0.1:12601,127.0.0.1:12601} to {"cluster_name": "c4tst-function2", "meta_addrs": "127.0.0.1:22601,127.0.1:22601"}, current table env is map[replica.deny_client_request:reconfig*all] 2022/06/16 15:31:22 recv loop terminated: EOF 2022/06/16 15:31:22 send loop terminated: <nil> ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
