I was explaining this loop to my brother who remarked that reading from
the database every time is a good idea if you'd otherwise need to malloc()
a huge chunk of memory for the message... but if it's just a few hundred
K, then this is a huge waste of time and database.
To address this, I'm going to add a few lines of logic that will allocate
a bit of memory to hold the full message if it's below a certain cut off.
What might be a fair value for this? 1MB? 5MB? Configurable?
Aaron
On Sun, 27 Apr 2003, Aaron Stone wrote:
> Here's forward.c, much simpler and very straightforward:
>
> while( more targets to go )
> {
> parse to see what command to run
> open a pipe to the command
> send the header to the pipe
> send remaining lines from the database to the pipe
> close the pipe
>
> next target
> }
>
> It might look like I only picked a different bottleneck than before (here
> it's the database, before it was the number of open pipes) but actually
> this algorithm is just the database version as before, but with the pipe
> openings happening once for each loop rather than all at once and stored
> to a list, then going through the list and sending on all the pipes. My
> goal was the eliminate the "start everything all at once" resource hog.
>
> The only consumer of forward.c was pipe.c, so there was no trouble paring
> it to the bone to match up with the pared down pipe.c. Note that the
> function definition has changed. The new pipe.c I posted does not reflect
> this change yet, but it's all alpha 1 anyways... I haven't tested
> anything, although it looks correct and compiles cleanly ;-)
>
> Aaron
>
