8-Nov-98 12:03 you wrote:
> On Sun, Nov 08, 1998, Khimenko Victor wrote:
>> EAPI = bloatware ? Of I'm just confused ...
> <grin> Not intentionally bloatware, of course. More the result of trying to
> combine three major requirements: portability (full ANSI C compliant),
> functionality (works with the ugly Apache API restrictions and pitfalls) and
> minimum API (to reduce the source patching to a minimum and provide a
> intuitive ap_hook_xx interface).
> So the current design is more the result of trying to integrate the
> requirements. But perhaps I've integrated it too complex. This can be the
> case, of course. Here you can help me to simplify it now...
This is my try to simlify things :-) Take a look on attachment. This is not
full solution, but rather "working demo". This is patch for mod_ssl 2.1b8 to
use my approach (still horrible slow on startup but lighting fast on actuall
calls to hooks). Just "bare bones" -- no context handling and such, but
working...
>> After look on EAPI I'm could not understood why it's designed to be so bloat
>> and slow. Why other (MUCH more simpler) design is unacceptable:
>>
>> #define hook_define(hook_name,hook_signature,hook_params) \
>> extern struct hook_struct_##hook_name { \
>> int (hook_addr)hook_signature; \
>> struct hook_struct##hook_name* next; \
>> } *hook_start_##hook_name; \
>> static __inline__ int hook_call_##hook_name(hook_signature) { \
>> hook_struct_##hook_name *p=&hook_start_##hook_name; \
>> while (p) { \
>> if (p->hook_addr hook_params ) return 1; \
>> p=p->next; \
>> } \
>> }
>>
>> Then you'll use this all like
>> hook_define(Great_hook,(char x,void ** y,int(z)(char,int)),(x,y,z))
>> in .h file (to be included in both "client" module and "server" module) and
>> hook_register(Great_hook) in the initialization of client_module plus
>> hook_enable(Great_hook) in the initialization of server_module. Then all
>> hook calls (hook_call_Great_hook('a',&p,f) :-) will be just few comparisions
>> instead of lookup in string table. 10-15 times faster when simple hook is used
>> (almost the same speed as simple function call!) and 100+ times faster when
>> hooks are not used. Of course current approach is acceptable for mod_ssl
>> (encryption is slow by itself) but it's not acceptable as generic hook
>> mechanism (IMO, anyway). And even for mod_ssl it's not so good since this will
>> slow down non-SSL server as well. Or I'm misunderstood something ?
>>
>> P.S. Of course this is only schematic description -- you'll need pool to keep
>> track of pools, etc. But with current approach when there are will be a lot of
>> hooks each and every rputc will lead to lookup in big string table ! Clearly
>> unappropriate IMO :-((
> Correct, Martin Kraemer and I already recognized that the string comparisons
> can be too slow (although regarding Deans performance statements on the
> non-optimized ap_table_xxx stuff the performance penalty for
> string-comparisons is not such noticeable as one might expect. The real
> performance problems is I/O in Apache).
In fact I'm not sure that string comparision ON SERVER STARTUP are too slow.
Of course if apache used via inetd this could be a case but AFAIK most
installations are standalone and time for startup is not so big problem...
Of course current approach has speed n*n (where n -- number of hooks) and
it's slow (better to have n*log n :-) but it's only "working demo".
> But we can for instance optimize the EAPI stuff to use numeric IDs through a
> hash-table instead of strings as the unique identifiers. So, you're right
> that we can optimize EAPI a lot. But this doesn't mean we also can replace it
> directly with your non-bloated inline-approach, IMHO. These are two totally
> different things.
<grin> Not so different after all :-)) I'm want to inline (where inline is
supported :-) only actual hook calls (including setup for already setuped hooks
since this is required for each call in mod_proxy). This is done (see patch).
It works. At least here (KSI-Linux 2.0 beta, SSLeay 0.9.0b shared, mod_ssl DSO,
etc).
> Let me explain the design behind the EAPI stuff a little bit: I've thought
> about a pre-processing based approach first, of course. Because this way I
> could avoid the encoding and dispatching over the various function signatures
> (the only thing which restricts the ap_hook stuff).
Exactly! Take a look: no signatures at all!
> But as you already noticed your approach is still only schematic and when
> you think deeper you recognize that pool handling, various signature handling,
> the context variable handling etc. gets such complex that you cannot do it
> via the pre-processor approach.
Hm. May be I'm oversimplified things there ? Hm. May be.
> I've though really very long three weeks ago about this stuff but was never
> able to incorporate _all_ needed functionality under the hat of a
> pre-processor based approach.
Oh. Gosh. Why need you _all_ needed functionality under the hat of a
pre-processor ??? IMO ap_hook_call & ap_hook_configure for already-configured
hook (see attachment for details) is more the enough...
> Because the current design directly follows the requirements. I've first added
> the hooks to buff.c, mod_log_config.c and proxy_http.c, etc. and then tried
> to implement them. But it was really horrible to get such a generic mechanism
> working. Because there are very subtle problems which crazy call-variants like
> ap_hook_use() solve.
Gm. After some thinking I'm decided just use two separate lines:
ap_hook_configure followed with ap_hook_call ...
> For instance we cannot make an assumption whether a hook is really configured
> before functions are registered because of the Apache API control flow and
> module ordering.
<grin> Hook is configured when both ap_hook_configure and ap_hook_register are
called. In ANY order. What other style it's possible to do ?
> Then there are other problems: For instance
> I wanted to max reduce the patches and make the ap_hook_xxx() API very clean
> and intuitive.
Hm. Yes. My version is more hard to use. But my version has also advantage:
more compile-time types checks! There are NEVER used (...) ! And it's good
thing (IMO anyway) from portability viewpoint as well: "is mod_ssl 2.1b8
compatible with Win32" ? I'm not sure. At least in Win16 API arguments for
function with non-limited amount of arguments was pushed in stack in different
order then functions with limited amount of arguments ! May be this is changed
in Win32 API (I'm not know for sure) but anyway...
> Here the only solution to overcome the various argument type and compiler
> restrictions is to use var-args (unions doesn't work here).
Hm. Afaik my solution is 100% ANSI C compatible. Am I wrong ?
> But in order to use them you need to know the function signatures, etc.
> This leaded to the idea to encode them into a bit-field, etc. Then there is
> the problem that the return value of a hook function can have totally
> different semantics. That's for what AP_HOOK_DECLINE() exists, etc.
In my version there are no AP_HOOK_DECLINE() :-(( But if it's REALLY important
I'll try to do something...
> In one sentence: I've tried really a lot of variants and the current approach
> is a first (although still non-optimized) cut for a _working_ solution which
> is really portable and ANSI C compliant but still provides both the required
> functionality and the wished ap_hook_xxx() API.
API is changed in my version of course but still usable IMO...
> When you really find a pre-processor based approach which provides the same,
> I'm more than happy to give it a try. But sorry to say this, I think you will
> hate to work on it and perhaps finally fail.
See attachment. It's WORKING solution. (More like "working demo", but
anyway).
> Because portability and functionality is really hard to combine here.
AFAIK my texts are [almost] 100% pure ANSI C (if compiler does not support
inline functions there are will be a lot of duplicated functions so functions
are REALLY small: bigges one is (ap_hook_call) is only 6 lines)... Almost
since inline is used but this is already used (with defines) in Apache...
> So, I would appreciate to concentrate now on only _optimizing_ the existing
> ap_hook_call() and ap_hook_use() functions instead of finding a totally
> different alternative approach.
Take at least look on "working demo"!
> Because I'm 95% sure the current approach is a very portable one (and
> portability is more important than speed here, of course).
I'm sure that my approach is MORE portable then yours due problems with (...).
And IMO portability is not a fetish. Of course it's great to have portable code
but not via "portability by any cost" approch...
I really want to avoid finding again an alternative implementation -
> it's a horrible complex and very time-consuming task.
It was done in 10 hours. May be 5-10 hours more will be needed to clean up
things. It's not horrible complex task. More like task for pupils in our
school (I'm teacher is "The Moscow State 57th School" as you could guess from
my e-mail :-) but since now it's autumn vacations time I was forced to do this
myself. Pupil from the last form (16-17 years old pupils) and one of the most
advanced programmers there will be needed of course.
> According to my working-hour sheet for the mod_ssl project I already spent
> around 60-80 working hours to find the current solution, so please understand
> that I'm not very keen to again find one myself.
Undertendable :-)) If you'll try to see and make suggestions then I'll clean up
things (but not document them -- my English is way to horrible for such task).
> When you contribute one which also works but is faster, great.
It works. It's faster where it's important. It's could be optimized in other
places as well.
> But please don't expect euphoria on this horrible task from my side ;-)
I'm could not see why it's so horrible task.
> So, I would appreciate that you have a very careful look at the ap_hook_call()
> cand ap_hook_use() functions and help me to only _optimize_ them - now that
> they proved to be portable and working. One idea is to change the string
> comparisons to an ID lookup, etc.
> Perhaps you have more optimizing ideas?
Not for now... I'm really want to have MINIMAL overhead for ap_hook_call.
My solution has tiny overhead for GCC but still should works somehow on all
ANSI C compatible compilers... It's more hard to use, perhaps, but still very
friendly IMO.
P.S. For now the whole hook subsystem is cleared when ANY module is unloaded :-((
May be there are more clean solutions exists ?
P.P.S. Is it possible to include EAPI is some form in future version of Apache?
Without crypto-hooks, but this way it's possible to have call to mod_perl in
mod_include activated only when mod_perl is available. IMO it's generally
usefull interface (especially if performance problems will be resolved one
way or another)...
EAPI-2-EAPIk.patch.bz2
