On Tue, Feb 18, 2020 at 01:18:54PM +0200, Nikolay Ivanets wrote: > вт, 18 лют. 2020 о 10:48 Richard W.M. Jones <[email protected]> пише: > > > > When calculating the initial size of the disk we must estimate > > how much space is taken by the input. This is quite difficult. > > > > For directories we used ‘du --apparent-size -bs DIR’. This is wrong > > because ’-b’ implies ‘--apparent-size --block-size=1’. But also > > ‘--apparent-size’ causes du to count the file size rather than number > > of blocks used by files. > > > > If you have a directory containing many small files this usually > > underestimates resulting in disk sizes which are far too small to > > actually contain the files. > > > > There's no really good answer here because du can't exactly do what we > > want, but we can at least remove this flag. This causes much larger > > estimates and therefore much larger virtual disks. > > > > Thanks: Nikolay Ivanets > > --- > > make-fs/make-fs.c | 11 +++++------ > > 1 file changed, 5 insertions(+), 6 deletions(-) > > > > diff --git a/make-fs/make-fs.c b/make-fs/make-fs.c > > index 5d8c3a385..386142280 100644 > > --- a/make-fs/make-fs.c > > +++ b/make-fs/make-fs.c > > @@ -393,7 +393,7 @@ static int > > estimate_input (const char *input, uint64_t *estimate_rtn, char **ifmt_rtn) > > { > > struct stat statbuf; > > - const char *argv[6]; > > + const char *argv[5]; > > CLEANUP_UNLINK_FREE char *tmpfile = NULL; > > CLEANUP_FCLOSE FILE *fp = NULL; > > char line[256]; > > @@ -424,11 +424,10 @@ estimate_input (const char *input, uint64_t > > *estimate_rtn, char **ifmt_rtn) > > } > > > > argv[0] = "du"; > > - argv[1] = "--apparent-size"; > > - argv[2] = "-b"; > > - argv[3] = "-s"; > > - argv[4] = input; > > - argv[5] = NULL; > > + argv[1] = "--block-size=1"; > > + argv[2] = "-s"; > > + argv[3] = input; > > + argv[4] = NULL; > > > > if (exec_command ((char **) argv, tmpfile) == -1) > > return -1; > > -- > > 2.24.1 > > > > That didn't help either. And with sparse files even made things worse: > holes are not taken into account now at all. > > I repeat my thoughts here again: it is all like "cats and rats game". > We cannot properly handle all the possible cases. > I think we can agree that virt-make-fs MAY fail and it is expected > situation. Just let the user know that he need to adjust size > manually.
I think it's a difficult problem and one which won't be solved using du. For comparison we have some nbdkit plugins for creating filesystems and they are far more sophisticated, especially this one: https://github.com/libguestfs/nbdkit/tree/master/plugins/floppy I would note that the reason the test fails was nothing to do with free space. It was actually running out of inodes. With ext2/3/4 the number of inodes in the metadata is fixed at creation and based on the size of the disk (see /etc/mke2fs.conf inode_ratio and the mke2fs -i and -N options). Because of the small size of the disk, mke2fs was choosing < 100 inodes, but because the test had 100+ files it failed when creating the later files even though there was enough disk space. There's no easy way to fix this. I think it could even be considered as a bug in mke2fs. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top _______________________________________________ Libguestfs mailing list [email protected] https://www.redhat.com/mailman/listinfo/libguestfs
