>On 12/11/18 06:03, Wolaver, David wrote: >> On 08/11/18 09:02, Wolaver, David wrote: >>> Hi all, >>> >>> I have developed a command line program that I would like to have >>> considered >>> for inclusion in a future revision of coreutils. The name >>> of the program is "curtail". It reads stdin and appends to an output file >>> while ensuring that the file will never exceed a specified maximum size. >>> The main intent is to place a bound on the output of a program such that it >>> cannot create a runaway log file. The output file is truncated (in >>> multiple of block size) at the beginning such that the output file always >>> contains the latest output of the program. >>> >>> user_program | curtail -s 4M output.txt >> >> It's a perfect name for such a command anyway :) >> >> Does this depend on FALLOC_FL_COLLAPSE_RANGE support. >> Would this command fail on file systems not supporting this? >> Being block based seems a little problematic, especially considering >> multi-byte characters, but maybe that's OK. >> >> cheers, >> Pádraig >> >> Hi Pádraig, >> >> Thanks for spending some time to think about curtail. I purposely omitted >> implementation details from the first post because they are irrelevant >> unless the following can be established... >> >> 1. There exists problem(s) that can be addressed by such a program (I >> have only presented one scenario) 2. Consensus that existing >> coreutils/scripts cannot produce the same result - at least not in an >> efficient manner 3. A solution to the problem belongs in coreutils >> >> To answer your question.. yes FALLOC_FL_COLLAPSE_RANGE is the mechanism by >> which files are truncated in the Linux implementation. Any OS/filesystem >> can be supported if it provides a mechanism to query block size and remove >> blocks from the beginning of a file. > >Implementation details do consolidate things in my mind at least. >They're important as coreutils tend to be general in that they'll fallback to >more generic solutions where system support for certain features is lacking. > >As for the need for a separate command, playing devil's advocate, given that >this is not just a view of the data (like tail(1)), and controls the eviction >of older data, perhaps this functionality is less generally useful, and more >suited as an operation mode of logrotate etc. > >cheers, >Pádraig
It would be great to have this generally supported, but that will requires more OS/filesystem support of primitives that make it possible. Why hold back a useful feature from the users that could benefit from it today? As for logrotate, this is typically run as a cron job instead of being used in a pipeline. With that being said, I'm not adverse to including it in other programs - it already creates a lib specifically for this purpose. I really just want to make the feature available for others to benefit from. -Dave
