On Thu, Oct 4, 2018 at 10:10 AM Vikas Rana <[email protected]> wrote: > > Thanks Jason for great suggestions. > > but somehow rbd mirror status not working from secondary to primary. Here;s > the status from both sides. cluster name is ceph on primary side and cephdr > on remote site. mirrordr is the user on DR side and mirrorprod is on primary > prod side. > > # rbd mirror pool info nfs > Mode: image > Peers: > UUID NAME CLIENT > 3ccd7a67-2343-44bf-960b-02d9b1258371 cephdr client.mirrordr. > > rbd --cluster cephdr mirror pool info nfs > Mode: image > Peers: > UUID NAME CLIENT > e6b9ba05-48de-462c-ad5f-0b51d0ee733f ceph client.mirrorprod > > > From primary site, when i query the remote site, its looks good. > # rbd --cluster cephdr --id mirrordr mirror pool status nfs > health: OK > images: 0 total > > but when i query from secondary site to primary side, I'm getting this error > # rbd --cluster ceph --id mirrorprod mirror pool status nfs > 2018-10-03 10:21:06.645903 7f27a44ed700 0 -- 165.x.x.202:0/1310074448 >> > 192.168.3.21:6804/3835 pipe(0x55ed47daf480 sd=4 :0 s=1 pgs=0 cs=0 l=1 > c=0x55ed47db0740).fault > > > We were using 192.168.3.x for MON network before we renamed it to use 165 > address since its routeable. why its trying to connect to 192.x address > instead of 165.x.y address?
Are your OSDs on that 192.168.3.x subnet? What daemons are running on 192.168.3.21? > I could do ceph -s from both side and they can see each other. Only rbd > command is having issue. > > Thanks, > -Vikas > > > > > On Tue, Oct 2, 2018 at 5:14 PM Jason Dillaman <[email protected]> wrote: >> >> On Tue, Oct 2, 2018 at 4:47 PM Vikas Rana <[email protected]> wrote: >> > >> > Hi, >> > >> > We have a CEPH 3 node cluster at primary site. We created a RBD image and >> > the image has about 100TB of data. >> > >> > Now we installed another 3 node cluster on secondary site. We want to >> > replicate the image at primary site to this new cluster on secondary site. >> > >> > As per documentation, we enabled journaling on primary site. We followed >> > all the procedure and peering looks good but the image is not copying. >> > The status is always showing down. >> >> Do you have an "rbd-mirror" daemon running on the secondary site? Are >> you running "rbd mirror pool status" against the primary site or the >> secondary site? The mirroring status is only available on the sites >> running "rbd-mirror" daemon (the "down" means that the cluster you are >> connected to doesn't have the daemon running). >> >> > So my question is, is it possible to replicate a image which already have >> > some data before enabling journalling? >> >> Indeed -- it will perform a full image sync to the secondary site. >> >> > We are using the image mirroring instead of pool mirroring. Do we need to >> > create the RBD image on secondary site? As per documentation, its not >> > required. >> >> The only difference between the two modes is whether or not you need >> to run "rbd mirror image enable" or not. >> >> > Is there any other option to copy the image to the remote site? >> >> No other procedure should be required. >> >> > Thanks, >> > -Vikas >> > _______________________________________________ >> > ceph-users mailing list >> > [email protected] >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >> >> >> -- >> Jason -- Jason _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
