Oh forgot to mention... It's also more system calls to write to temporary files in RAM, as opposed to just copying structs to pointers.
Also, with a small buffer, there'd be many more calls to fillbuf, which does involve talking to the original input file on disk, and the seek times of the many smaller reads will kill you, as opposed to with a bigger buffer you'd do fewer seeks to read more of the file in at once.
