Hi, >> Add one more suggestion, if we have not a perfect solution to consider all >> the case of truncate, could we add an option to tail, such like tail >> -no-truncate >> If tail run with this option, than application not consider any truncate >> case. >> >> For example, I suppose syslog output file will not have any truncate case in >> our environment, then the tail could use the option to avoid the >> mis-truncated case?
>Note for case 2) above, we only update fspec->size _after_ the read, >so I'm not sure how practical the race with reading a _smaller_ st_size after >that is? >I.E. the heuristic is fairly good I think, >so an option may be overkill. >We'd have to see a demonstratable issue to consider such an option. We have an issue now for tail a syslog file which stored in a network-based file system. A automated cased need tail the syslog about one hour to get the syslog of that period, in that period of one hour , happen 6 times of un-expected file truncated issue, so the output of tail has 6 times full syslog file, so the output file is so huge and eat all of the disks. The network-based file system maybe not so easy to change to meet the current implement of "tail" application. So I need helps from yours :) And which your mean for demonstratable? The issue we encounter could be easy to reproduce, maybe the file-system is not so strict like ext4 file system, but I still suggest "tail" application could do some change to adapt this kinds network-based file system? Thanks & Best Regards, George -----Original Message----- From: Pádraig Brady [mailto:[email protected]] Sent: Monday, November 07, 2016 11:06 PM To: Lian, George (Nokia - CN/Hangzhou) <[email protected]>; [email protected] Cc: Zhang, Bingxuan (Nokia - CN/Hangzhou) <[email protected]>; Li, Deqian (Nokia - CN/Hangzhou) <[email protected]>; Zizka, Jan (Nokia - CZ/Prague) <[email protected]>; Bao, Xiaohui (Nokia - CN/Hangzhou) <[email protected]> Subject: Re: some concern about the fix of " tail: consistently output all data for truncated files" On 07/11/16 07:18, Lian, George (Nokia - CN/Hangzhou) wrote: > From: Lian, George (Nokia - CN/Hangzhou) > Sent: Monday, November 07, 2016 1:17 PM > To: '[email protected]' <[email protected]> > Cc: Zizka, Jan (Nokia - CZ/Prague) <[email protected]>; Zhang, Bingxuan > (Nokia - CN/Hangzhou) <[email protected]>; Li, Deqian (Nokia - > CN/Hangzhou) <[email protected]>; Bao, Xiaohui (Nokia - CN/Hangzhou) > <[email protected]> > Subject: some concern about the fix of " tail: consistently output all data > for truncated files" > > > Hi, > > There are a fix about truncated for tail application of coreutils, > > The change messages is "tail: consistently output all data for truncated > files", > > Now I have some concern for this case > > 1) I suppose Truncate maybe not always truncate to ZERO, for example, > sometimes a file maybe only has been truncate the bottom 100 lines of 1M > lines , in this case , SEEK from head will be confused by the users. > 2) Sometimes truncated maybe not really happen even "stats.st_size < > fspec->size" is meet, for example, a network based filesystems, for it's > current implementation, it will active the read data-cache first and then > update the meta-data cache, it is not an atomic operation, I suppose it is > reasonable that the "tail application" will read the data-cache with new > updated data, but get stat size with old value, in this case, the tail assume > it is truncated and dump the data from the begin, it also make more confuse > to users. > 3) As the comments in your changes, the stat size is bigger thand the > length has read, it also maybe happ truncate. > 4) Why can't get size of fstat first, and read not more than the size of > fstat during the tail_forever loop? > > Could you please share your comments for the above concerns? > Hi, > > Add one more suggestion, if we have not a perfect solution to consider all > the case of truncate, could we add an option to tail, such like tail > -no-truncate > If tail run with this option, than application not consider any truncate case. > > For example, I suppose syslog output file will not have any truncate case in > our environment, then the tail could use the option to avoid the > mis-truncated case? Note for case 2) above, we only update fspec->size _after_ the read, so I'm not sure how practical the race with reading a _smaller_ st_size after that is? I.E. the heuristic is fairly good I think, so an option may be overkill. We'd have to see a demonstratable issue to consider such an option. thanks, Pádraig
