The 'really_finish_p' is a local variable and hence, lost on each function return, what you want is that when you set really_finish_p = TRUE it keeps the value.
I see. It happens that the ahexenc filter return PDF_ENINPUT when it is called with an empty input buffer, so the apply function of the next filter in the chain returns EEOF and thus I got the desired functionality :) Ok, it seems that we can fix it storing really_finish_p in the filter structure, so the value will be persistent through several calls to apply. I already did that little change and it seems to work, but I will test it tomorrow at the morning and apply it if it really fix the problem. Thanks for the report and the diagnosis, Gerel.
