Hi, Dear Expert,

We have use glusterfs as a network filesystem, and syslog store in there, some 
clients on different host may write the syslog file via "glusterfs" mount point.

Now we encounter an issue when we "tail" the syslog file, it will occasional 
failed with error "file truncated"
As we study and trace with the "tail" source code, it failed with the following 
code:
if (S_ISREG (mode) && stats.st_size < f[i].size)
                {
                  error (0, 0, _("%s: file truncated"), quotef (name));
                  /* Assume the file was truncated to 0,
                     and therefore output all "new" data.  */
                  xlseek (fd, 0, SEEK_SET, name);
                  f[i].size = 0;
                }

When stats.st_size < f[i].size, what mean the size report by fstat is less than 
"tail" had read, it lead to "file truncated", we also use "strace" tools to 
trace the tail application, the related tail strace log as the below:

nanosleep({1, 0}, NULL)                 = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=192543105, ...}) = 0
nanosleep({1, 0}, NULL)                 = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=192543105, ...}) = 0
nanosleep({1, 0}, NULL)                 = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=192543105, ...}) = 0
nanosleep({1, 0}, NULL)                 = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=192544549, ...}) = 0
read(3, "Data ...-"..., 8192) = 1444
read(3, "Data.."..., 8192) = 720
read(3, "", 8192)                       = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=192544789, ...}) = 0
write(1, "DATA.....") = 2164
write(2, "tail: ", 6tail: )                   = 6
write(2, "/mnt/log/master/syslog: file tru"..., 38/mnt/log/master/syslog: file 
truncated) = 38

as the above strace log, tail has read 1444+720=2164 bytes,
but fstat tell "tail" 192544789 - 192543105 = 1664 which less than 2164, so it 
lead to "tail" application "file truncated".

And if we turn off "write-behind" feature, the issue will not be reproduced any 
more.
So we think it may be related to cache consistence issue due to performance 
consider, but we still have concern that:

The syslog file is used only with "Append" mode, so the size of file shouldn't 
be reduced, when a client read the file, why "fstat" can't return the really 
size match to the cache?
>From current investigation, we doubt that the current implement of "glusterfs" 
>has a bug on "fstat" when cache is on.

Your comments is our highly appreciated!

Thanks & Best Regards

George





_______________________________________________
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Reply via email to