On 12/06/2014, at 6:47 PM, Pranith Kumar Karampuri wrote: > On 06/12/2014 11:16 PM, Anand Avati wrote: <snip> >> The client can actually be fixed to be compatible with both old and new >> servers. We can change the errno from ESTALE to ENOENT before doing the GFID >> mismatch check in client_lookup_cbk. > But this will require a two hop upgrade. Is that normal/acceptable?
Thoughts on this from Corvid Tech: > From: "David F. Robinson" <david.robin...@corvidtec.com> > Subject: Re: Rolling upgrades discussion on mailing list > Date: 12 June 2014 11:01:27 PM GMT+01:00 > > Read through the discussion. Not sure I fully understood it, but here are my > thoughts regardless... > > - We have roughly 1000-nodes (HPC setup) and 100 workstations that hopefully > will be using gluster as the primary storage system (if we can resolve the > few outstanding issues)... If these all had gluster-3.4.x rpm's and I wanted > to upgrade to 3.5.x rpm's, I don't particular care if I had to incrementally > update the packages (3.4.5 --> 3.4.6 --> 3.5.0 --> 3.5.1). Requiring a > minimum version seems reasonable and workable to me as long as there is a > documented process to update. > - What I wouldn't want to do if it was avoidable was to have to stop all i/o > on all of the hpc-nodes and workstations for the update. If I could do > incremental 'rpm -Uvh' for the various versions "without" killing the > currently ongoing i/o, that is what would be most important. It wasn't clear > to me if this was not possible at all, or not possible unless you upgraded > all of the clients before upgrading the server. > - If the problem is that you have to update all of the clients BEFORE > updating the server, that doesn't sound unworkable as long as there is a > mechanism to list all clients that are connected and display the gluster > version each client is using. > - Also, a clearly documented process to update would be great. We have lots > of questions and taking the entire storage device offline to do an upgrade > isn't a really feasible. My assumption after reading the link you sent is > that we would have to upgrade all of the client software (assumes that a > newer client can still write to an older server version... i.e. client 3.5.2 > can write to a 3.4.5 server). This isn't a big deal and seems reasonable, if > there is some method to find all of the connected clients. > > - From the server side, I would like to be able to take one of the > nodes/bricks offline nicely (i.e. have the current i/o finish, accept no more > i/o requests, and then take the node offline for writes) and do the update. > Then bring that node back up, let the heal finish, and move on to the next > node/brick. We are told that simply taking a node offline will not interrupt > the i/o if you are using a replica. We have experienced mixed results with > this. I believe that the active i/o to the brick fails but additional i/o > fails over to the other brick. A nicer process to take a brick offline would > be desirable. > - The other suggestion I have seen for server software updates is to > migrate all of the data from a node/brick to the other bricks of the system > (gluster remove-brick xxx), remove this brick from the pool, do the software > update, and then add it back to the pool. The issue here is that each of our > bricks is roughly 100TB. Moving all of that data to the other bricks in the > system will take a very long time. Not a practical solution for us... > > > I don't mind having to use a process (incrementally upgrade all clients, take > bricks offline, update bricks), as long as I didn't have to turn off all i/o > to the primary storage system for the entire company in order to execute an > upgrade... The number one requirement for us would be to have a process to do > an upgrade on a "live" system... Who cares if the process is painful... That > is the IT guys problem :)... + Justin -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift _______________________________________________ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailman/listinfo/gluster-devel
