----- Original Message ----- > I think a reasonable alternative to this is to just sit in an infinite retry > loop until the i/o succeeds. > > Dave > [Mark Syms] I would hope that this code would only trigger after some effort > has been put into retrying as panicing the host on the first I/O failure > seems like a sure fire way to get unhappy users (and in our case paying > customers). As Edvin points out there may be other filesystems that may be > able to cleanly unmount and thus avoid having to check everything on > restart.
Hi Mark, Perhaps. I'm not block layer or iscsi expert, but afaik, it's not the file system's job to retry IO, and never has been, right? There are already iscsi tuning parameters, vfs tuning parameters, etc. So if an IO error is sent to GFS2 for a write operation, it means the retry algorithms and operation timeout algorithms built into the layers below us (the iscsi layer, scsi layer, block layer, tcp/ip layer etc.) have all failed and given up on the IO operation. We can't really justify adding yet another layer of retries on top of all that, can we? I see your point, and perhaps the system should stay up to continue other mission-critical operations that may not require the faulted hardware. But what's a viable alternative? As Dave T. suggested, we can keep resubmitting the IO until it completes, but then the journal never gets replayed and nobody can have those locks ever again, and that would cause a potential hang of the entire cluster, especially in cases where there's only one device that's failed and the whole cluster is using it. In GFS2, we've got a concept of marking a resource group "in error" so the other nodes won't try to use it, but the same corruption that affects resource groups could be extrapolated to "hot" dinodes as well. For example, suppose the root (mount-point) dinode was in the journal. Now the whole cluster is hung rather than just the one node. Regards, Bob Peterson Red Hat File Systems
