Hi all, I'm currently in process of upgrading a replicated cluster (1 x 4) from 3.6.3 to 3.10.5. The nodes run CentOS 6. However after upgrading the first node, the said node fails to connect to other peers (as seen via 'gluster peer status'), but somehow other non-upgraded peers can still see the upgraded peer as connected.
Writes to the Gluster volume via local mounts of non-upgraded peers are replicated to the upgraded peer, but I can't write via the upgraded peer as the local mount seems forced to read-only. Launching heal operations from non-upgraded peers will output 'Commit failed on <upgraded peer IP>. Please check log for details'. In addition, during upgrade process there were warning messages about my old vol files renamed with .rpmsave extension. I tried starting Gluster with my old vol files but the problem persisted. I tried generating new vol files with 'glusterd --xlator-option "*.upgrade=on" -N', still no avail. Also I checked the brick log it had several messages about "failed to get client opversion". I don't know if this is pertinent. Could it be that the upgraded node cannot connect to older nodes but still can receive instructions from them? Below are command outputs; some data are masked. I'd provide more information if required. Thanks in advance. ===> 'gluster volume status' ran on non-upgraded peers Status of volume: gsnfs Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick gs-nfs01:/ftpdata 49154 Y 2931 Brick gs-nfs02:/ftpdata 49152 Y 29875 Brick gs-nfs03:/ftpdata 49153 Y 6987 Brick gs-nfs04:/ftpdata 49153 Y 24768 Self-heal Daemon on localhost N/A Y 2938 Self-heal Daemon on gs-nfs04 N/A Y 24788 Self-heal Daemon on gs-nfs03 N/A Y 7007 Self-heal Daemon on <IP> N/A Y 29866 Task Status of Volume gsnfs ------------------------------------------------------------------------------ There are no active volume tasks ===> 'gluster volume status' on upgraded peer Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick gs-nfs02:/ftpdata 49152 0 Y 29875 Self-heal Daemon on localhost N/A N/A Y 29866 Task Status of Volume gsnfs ------------------------------------------------------------------------------ There are no active volume tasks ===> 'gluster peer status' on non-upgraded peer Number of Peers: 3 Hostname: gs-nfs03 Uuid: 4c1544e6-550d-481a-95af-2a1da32d10ad State: Peer in Cluster (Connected) Hostname: <IP> Uuid: 17d554fd-9181-4b53-9521-55acf69ac35f State: Peer in Cluster (Connected) Other names: gs-nfs02 Hostname: gs-nfs04 Uuid: c6d165e6-d222-414c-b57a-97c64f06c5e9 State: Peer in Cluster (Connected) ===> 'gluster peer status' on upgraded peer Number of Peers: 3 Hostname: gs-nfs03 Uuid: 4c1544e6-550d-481a-95af-2a1da32d10ad State: Peer in Cluster (Disconnected) Hostname: gs-nfs01 Uuid: 90d3ed27-61ac-4ad3-93a9-3c2b68f41ecf State: Peer in Cluster (Disconnected) Other names: <IP> Hostname: gs-nfs04 Uuid: c6d165e6-d222-414c-b57a-97c64f06c5e9 State: Peer in Cluster (Disconnected) -- - Yong
_______________________________________________ Gluster-users mailing list [email protected] http://lists.gluster.org/mailman/listinfo/gluster-users
