Re: mod_ssl NPN API rejig (was Re: Intent to revert commit r1332643)

[Joe Orton]

On Wed, May 29, 2013 at 03:04:30PM -0400, Matthew Steele wrote:
 Looks good to me.  Thanks!

Thanks a lot for reviewing.

http://svn.apache.org/viewvc?view=revisionrevision=1487772

Gregg, thanks for confirming and sorry again about leaving the builds 
broken.

Regards, Joe

Re: mod_ssl NPN API rejig (was Re: Intent to revert commit r1332643)

[William A. Rowe Jr.]

On Wed, 29 May 2013 17:06:14 +0100
Joe Orton jor...@redhat.com wrote:

 On Wed, May 29, 2013 at 11:37:14AM -0400, Matthew Steele wrote:
  Oops, yes, RUN_ALL semantics are desired; the misleading API
  description is my fault, sorry.  (I confess I never really
  understood why RUN_ALL hooks accept both OK and DECLINED values,
  but then don't actually treat them any differently.)  So probably
  we should update the doc comments.  I'd be happy to draft new
  versions for those if that would be helpful, or not, as you prefer.
 
 (Really attached this time)

This all looks great, thanks Joe!  Pull my comments along with my
the old backport proposal, because I'm certainly ready to support
this change in 2.2 and 2.4.  Addressed all of my concerns.

mod_ssl NPN API rejig (was Re: Intent to revert commit r1332643)

[Joe Orton]

Guenter, can you test if the attached compiles on Windows?  It is 
nothing special so it should be OK.

This redesigns the NPN API with a cheap and crappy callback interface 
which doesn't rely on the actual hooks API; it is not pretty but it 
avoids the inter-module hard linkage issue (which is even more crap and 
I should have noticed when committing, sorry about that).  

(Tested to compile and not segfault on Unix but no further, I'll mock up 
something to test properly.)

Regards, Joe
Index: modules/ssl/mod_ssl.c
===
--- modules/ssl/mod_ssl.c   (revision 1487480)
+++ modules/ssl/mod_ssl.c   (working copy)
@@ -285,18 +285,6 @@
 AP_END_CMD
 };
 
-/* Implement 'modssl_run_npn_advertise_protos_hook'. */
-APR_IMPLEMENT_OPTIONAL_HOOK_RUN_ALL(
-modssl, AP, int, npn_advertise_protos_hook,
-(conn_rec *connection, apr_array_header_t *protos),
-(connection, protos), OK, DECLINED)
-
-/* Implement 'modssl_run_npn_proto_negotiated_hook'. */
-APR_IMPLEMENT_OPTIONAL_HOOK_RUN_ALL(
-modssl, AP, int, npn_proto_negotiated_hook,
-(conn_rec *connection, const char *proto_name, apr_size_t proto_name_len),
-(connection, proto_name, proto_name_len), OK, DECLINED)
-
 /*
  *  the various processing hooks
  */
@@ -444,6 +432,35 @@
 return 1;
 }
 
+static int modssl_register_npn(conn_rec *c, 
+   ssl_npn_advertise_protos advertisefn,
+   ssl_npn_proto_negotiated negotiatedfn)
+{
+#ifdef HAVE_TLS_NPN
+SSLConnRec *sslconn = myConnConfig(c);
+
+if (!sslconn) {
+return DECLINED;
+}
+
+sslconn-npn_advertfns = 
+apr_array_make(c-pool, 0, sizeof(ssl_npn_advertise_protos));
+sslconn-npn_negofns = 
+apr_array_make(c-pool, 0, sizeof(ssl_npn_proto_negotiated));
+
+if (advertisefn)
+APR_ARRAY_PUSH(sslconn-npn_advertfns, ssl_npn_advertise_protos) =
+advertisefn;
+if (negotiatedfn)
+APR_ARRAY_PUSH(sslconn-npn_negofns, ssl_npn_proto_negotiated) =
+negotiatedfn;
+
+return OK;
+#else
+return DECLINED;
+#endif
+}
+
 int ssl_init_ssl_connection(conn_rec *c, request_rec *r)
 {
 SSLSrvConfigRec *sc;
@@ -615,6 +632,7 @@
 
 APR_REGISTER_OPTIONAL_FN(ssl_proxy_enable);
 APR_REGISTER_OPTIONAL_FN(ssl_engine_disable);
+APR_REGISTER_OPTIONAL_FN(modssl_register_npn);
 
 ap_register_auth_provider(p, AUTHZ_PROVIDER_GROUP, ssl,
   AUTHZ_PROVIDER_VERSION,
Index: modules/ssl/mod_ssl.h
===
--- modules/ssl/mod_ssl.h   (revision 1487480)
+++ modules/ssl/mod_ssl.h   (working copy)
@@ -63,26 +63,37 @@
 
 APR_DECLARE_OPTIONAL_FN(int, ssl_engine_disable, (conn_rec *));
 
-/** The npn_advertise_protos optional hook allows other modules to add entries
- * to the list of protocol names advertised by the server during the Next
- * Protocol Negotiation (NPN) portion of the SSL handshake.  The hook callee is
- * given the connection and an APR array; it should push one or more char*'s
- * pointing to null-terminated strings (such as http/1.1 or spdy/2) onto
- * the array and return OK, or do nothing and return DECLINED. */
-APR_DECLARE_EXTERNAL_HOOK(modssl, AP, int, npn_advertise_protos_hook,
-  (conn_rec *connection, apr_array_header_t *protos))
+/** The npn_advertise_protos callback allows another modules to add
+ * entries to the list of protocol names advertised by the server
+ * during the Next Protocol Negotiation (NPN) portion of the SSL
+ * handshake.  The callback is given the connection and an APR array;
+ * it should push one or more char*'s pointing to NUL-terminated
+ * strings (such as http/1.1 or spdy/2) onto the array and return
+ * OK, or do nothing and return DECLINED. */
+typedef int (*ssl_npn_advertise_protos)(conn_rec *connection, 
+apr_array_header_t *protos);
 
-/** The npn_proto_negotiated optional hook allows other modules to discover the
- * name of the protocol that was chosen during the Next Protocol Negotiation
- * (NPN) portion of the SSL handshake.  Note that this may be the empty string
- * (in which case modules should probably assume HTTP), or it may be a protocol
- * that was never even advertised by the server.  The hook callee is given the
- * connection, a non-null-terminated string containing the protocol name, and
- * the length of the string; it should do something appropriate (i.e. insert or
- * remove filters) and return OK, or do nothing and return DECLINED. */
-APR_DECLARE_EXTERNAL_HOOK(modssl, AP, int, npn_proto_negotiated_hook,
-  (conn_rec *connection, const char *proto_name,
-   apr_size_t proto_name_len))
+/** The npn_proto_negotiated callback allows other modules to discover
+ * the name of the protocol that was chosen during the Next 

Re: mod_ssl NPN API rejig (was Re: Intent to revert commit r1332643)

[Matthew Steele]

Hi Joe,

Two questions about this change:

- In modssl_register_npn, it appears that the code creates
new npn_advertfns and npn_negofns arrays on every call, even if they
already exist.  This would seem to prevent multiple modules from
registering callbacks.  Presumably this is not intended?  Am I misreading
the code?

- The old optional hooks were both declared as RUN_ALL, but the new
implementation seems to stop calling callbacks as soon as one returns OK,
which is different behavior than before.  Is there a reason for this change?

Best,
-Matthew



On Wed, May 29, 2013 at 10:36 AM, Joe Orton jor...@redhat.com wrote:

 Guenter, can you test if the attached compiles on Windows?  It is
 nothing special so it should be OK.

 This redesigns the NPN API with a cheap and crappy callback interface
 which doesn't rely on the actual hooks API; it is not pretty but it
 avoids the inter-module hard linkage issue (which is even more crap and
 I should have noticed when committing, sorry about that).

 (Tested to compile and not segfault on Unix but no further, I'll mock up
 something to test properly.)

 Regards, Joe

Re: mod_ssl NPN API rejig (was Re: Intent to revert commit r1332643)

[Joe Orton]

Hi Matthew - thanks for taking a look at the patch so quickly.

On Wed, May 29, 2013 at 10:52:10AM -0400, Matthew Steele wrote:
 Two questions about this change:
 
 - In modssl_register_npn, it appears that the code creates
 new npn_advertfns and npn_negofns arrays on every call, even if they
 already exist.  This would seem to prevent multiple modules from
 registering callbacks.  Presumably this is not intended?  Am I misreading
 the code?

No, just a braino.  Updated patch attached.

 - The old optional hooks were both declared as RUN_ALL, but the new
 implementation seems to stop calling callbacks as soon as one returns OK,
 which is different behavior than before.  Is there a reason for this change?

I was wondering about this.  The API description weakly implies that OK 
does something different to DECLINED in both cases, which is not true 
for RUN_ALL; so I presumed RUN_FIRST was desired.  Are RUN_ALL semantics 
desired?

Regards, Joe

Re: mod_ssl NPN API rejig (was Re: Intent to revert commit r1332643)

[Matthew Steele]

On Wed, May 29, 2013 at 11:14 AM, Joe Orton jor...@redhat.com wrote:

 On Wed, May 29, 2013 at 10:52:10AM -0400, Matthew Steele wrote:
  - In modssl_register_npn, it appears that the code creates
  new npn_advertfns and npn_negofns arrays on every call, even if they
  already exist.  This would seem to prevent multiple modules from
  registering callbacks.  Presumably this is not intended?  Am I misreading
  the code?

 No, just a braino.  Updated patch attached.


Gotcha.  I'm not seeing a new attachment, though?


  - The old optional hooks were both declared as RUN_ALL, but the new
  implementation seems to stop calling callbacks as soon as one returns OK,
  which is different behavior than before.  Is there a reason for this
 change?

 I was wondering about this.  The API description weakly implies that OK
 does something different to DECLINED in both cases, which is not true
 for RUN_ALL; so I presumed RUN_FIRST was desired.  Are RUN_ALL semantics
 desired?


Oops, yes, RUN_ALL semantics are desired; the misleading API description is
my fault, sorry.  (I confess I never really understood why RUN_ALL hooks
accept both OK and DECLINED values, but then don't actually treat them any
differently.)  So probably we should update the doc comments.  I'd be happy
to draft new versions for those if that would be helpful, or not, as you
prefer.

Thanks,
-Matthew

Re: mod_ssl NPN API rejig (was Re: Intent to revert commit r1332643)

[Joe Orton]

On Wed, May 29, 2013 at 11:37:14AM -0400, Matthew Steele wrote:
 Oops, yes, RUN_ALL semantics are desired; the misleading API description is
 my fault, sorry.  (I confess I never really understood why RUN_ALL hooks
 accept both OK and DECLINED values, but then don't actually treat them any
 differently.)  So probably we should update the doc comments.  I'd be happy
 to draft new versions for those if that would be helpful, or not, as you
 prefer.

OK vs DECLINED has always confused me too ;)

How does this look?  I've specified the behaviour for OK and DONE as 
return codes for both the callbacks; since the caller doesn't pay 
attention to errors I've left behaviour undefined for any other values.

(Really attached this time)

Regards, Joe
Index: modules/ssl/mod_ssl.c
===
--- modules/ssl/mod_ssl.c   (revision 1487480)
+++ modules/ssl/mod_ssl.c   (working copy)
@@ -285,18 +285,6 @@
 AP_END_CMD
 };
 
-/* Implement 'modssl_run_npn_advertise_protos_hook'. */
-APR_IMPLEMENT_OPTIONAL_HOOK_RUN_ALL(
-modssl, AP, int, npn_advertise_protos_hook,
-(conn_rec *connection, apr_array_header_t *protos),
-(connection, protos), OK, DECLINED)
-
-/* Implement 'modssl_run_npn_proto_negotiated_hook'. */
-APR_IMPLEMENT_OPTIONAL_HOOK_RUN_ALL(
-modssl, AP, int, npn_proto_negotiated_hook,
-(conn_rec *connection, const char *proto_name, apr_size_t proto_name_len),
-(connection, proto_name, proto_name_len), OK, DECLINED)
-
 /*
  *  the various processing hooks
  */
@@ -444,6 +432,37 @@
 return 1;
 }
 
+static int modssl_register_npn(conn_rec *c, 
+   ssl_npn_advertise_protos advertisefn,
+   ssl_npn_proto_negotiated negotiatedfn)
+{
+#ifdef HAVE_TLS_NPN
+SSLConnRec *sslconn = myConnConfig(c);
+
+if (!sslconn) {
+return DECLINED;
+}
+
+if (!sslconn-npn_advertfns) {
+sslconn-npn_advertfns = 
+apr_array_make(c-pool, 5, sizeof(ssl_npn_advertise_protos));
+sslconn-npn_negofns = 
+apr_array_make(c-pool, 5, sizeof(ssl_npn_proto_negotiated));
+}
+
+if (advertisefn)
+APR_ARRAY_PUSH(sslconn-npn_advertfns, ssl_npn_advertise_protos) =
+advertisefn;
+if (negotiatedfn)
+APR_ARRAY_PUSH(sslconn-npn_negofns, ssl_npn_proto_negotiated) =
+negotiatedfn;
+
+return OK;
+#else
+return DECLINED;
+#endif
+}
+
 int ssl_init_ssl_connection(conn_rec *c, request_rec *r)
 {
 SSLSrvConfigRec *sc;
@@ -615,6 +634,7 @@
 
 APR_REGISTER_OPTIONAL_FN(ssl_proxy_enable);
 APR_REGISTER_OPTIONAL_FN(ssl_engine_disable);
+APR_REGISTER_OPTIONAL_FN(modssl_register_npn);
 
 ap_register_auth_provider(p, AUTHZ_PROVIDER_GROUP, ssl,
   AUTHZ_PROVIDER_VERSION,
Index: modules/ssl/mod_ssl.h
===
--- modules/ssl/mod_ssl.h   (revision 1487480)
+++ modules/ssl/mod_ssl.h   (working copy)
@@ -63,26 +63,40 @@
 
 APR_DECLARE_OPTIONAL_FN(int, ssl_engine_disable, (conn_rec *));
 
-/** The npn_advertise_protos optional hook allows other modules to add entries
- * to the list of protocol names advertised by the server during the Next
- * Protocol Negotiation (NPN) portion of the SSL handshake.  The hook callee is
- * given the connection and an APR array; it should push one or more char*'s
- * pointing to null-terminated strings (such as http/1.1 or spdy/2) onto
- * the array and return OK, or do nothing and return DECLINED. */
-APR_DECLARE_EXTERNAL_HOOK(modssl, AP, int, npn_advertise_protos_hook,
-  (conn_rec *connection, apr_array_header_t *protos))
+/** The npn_advertise_protos callback allows another modules to add
+ * entries to the list of protocol names advertised by the server
+ * during the Next Protocol Negotiation (NPN) portion of the SSL
+ * handshake.  The callback is given the connection and an APR array;
+ * it should push one or more char*'s pointing to NUL-terminated
+ * strings (such as http/1.1 or spdy/2) onto the array and return
+ * OK.  To prevent further processing of (other modules') callbacks,
+ * return DONE. */
+typedef int (*ssl_npn_advertise_protos)(conn_rec *connection, 
+apr_array_header_t *protos);
 
-/** The npn_proto_negotiated optional hook allows other modules to discover the
- * name of the protocol that was chosen during the Next Protocol Negotiation
- * (NPN) portion of the SSL handshake.  Note that this may be the empty string
- * (in which case modules should probably assume HTTP), or it may be a protocol
- * that was never even advertised by the server.  The hook callee is given the
- * connection, a non-null-terminated string containing the protocol name, and
- * the length of the string; it should do something appropriate (i.e. insert or
- * remove filters) and return OK, or do nothing and 

Re: mod_ssl NPN API rejig (was Re: Intent to revert commit r1332643)

[Guenter Knauf]

Hi Joe,
On 29.05.2013 18:06, Joe Orton wrote:

On Wed, May 29, 2013 at 11:37:14AM -0400, Matthew Steele wrote:

Oops, yes, RUN_ALL semantics are desired; the misleading API description is
my fault, sorry.  (I confess I never really understood why RUN_ALL hooks
accept both OK and DECLINED values, but then don't actually treat them any
differently.)  So probably we should update the doc comments.  I'd be happy
to draft new versions for those if that would be helpful, or not, as you
prefer.


OK vs DECLINED has always confused me too ;)

How does this look?  I've specified the behaviour for OK and DONE as
return codes for both the callbacks; since the caller doesn't pay
attention to errors I've left behaviour undefined for any other values.

(Really attached this time)

thanks for looking into it!
I will test tomorrow; need a rest just now since was the whole day busy 
at customers; but perhaps Gregg beats me soon ...


Gün.

Re: mod_ssl NPN API rejig (was Re: Intent to revert commit r1332643)

[Matthew Steele]

On Wed, May 29, 2013 at 12:06 PM, Joe Orton jor...@redhat.com wrote:

 On Wed, May 29, 2013 at 11:37:14AM -0400, Matthew Steele wrote:
  Oops, yes, RUN_ALL semantics are desired; the misleading API description
 is
  my fault, sorry.  (I confess I never really understood why RUN_ALL hooks
  accept both OK and DECLINED values, but then don't actually treat them
 any
  differently.)  So probably we should update the doc comments.  I'd be
 happy
  to draft new versions for those if that would be helpful, or not, as you
  prefer.

 OK vs DECLINED has always confused me too ;)

 How does this look?  I've specified the behaviour for OK and DONE as
 return codes for both the callbacks; since the caller doesn't pay
 attention to errors I've left behaviour undefined for any other values.

 (Really attached this time)


Looks good to me.  Thanks!

Cheers,
-Matthew

Re: mod_ssl NPN API rejig (was Re: Intent to revert commit r1332643)

[Gregg Smith]

On 5/29/2013 10:52 AM, Guenter Knauf wrote:

Hi Joe,
On 29.05.2013 18:06, Joe Orton wrote:

On Wed, May 29, 2013 at 11:37:14AM -0400, Matthew Steele wrote:
Oops, yes, RUN_ALL semantics are desired; the misleading API 
description is
my fault, sorry.  (I confess I never really understood why RUN_ALL 
hooks
accept both OK and DECLINED values, but then don't actually treat 
them any
differently.)  So probably we should update the doc comments.  I'd 
be happy
to draft new versions for those if that would be helpful, or not, as 
you

prefer.


OK vs DECLINED has always confused me too ;)

How does this look?  I've specified the behaviour for OK and DONE as
return codes for both the callbacks; since the caller doesn't pay
attention to errors I've left behaviour undefined for any other values.

(Really attached this time)

thanks for looking into it!
I will test tomorrow; need a rest just now since was the whole day 
busy at customers; but perhaps Gregg beats me soon ...


Compiles fine now. Didn't do much testing other than access a simple 
page via https


Gregg

Re: Intent to revert commit r1332643

[Mario Brandt]

Any news on this?

Greetz

Am Samstag, 18. Mai 2013 schrieb Jim Jagielski :

 Please don't submit what could be controversial reverts
 over a weekend.

 On May 17, 2013, at 10:36 AM, Guenter Knauf fua...@apache.orgjavascript:;
 wrote:

  Hi all,
  I will revert the changes done with:
  http://svn.apache.org/viewvc?view=revisionrevision=1332643
  after 72 hours if nobody is going to fix the stuff properly for Windows
 since I'm tired of always copying mod_ssl over from 2.4.x branch in order
 to get a working mod_ssl with trunk.
 
  Reasons:
  1) within last 12 months there was no attempt made to fix the issues
 which wrowe mentioned in this thread [1] - instead discussion died
  2) a suggestion to fix the issue [2] was not applied due to the concerns
 wrowe brought up, and to which I agree.
  3) the same issue also causes a stalled backport in 2.2.x STATUS [3] for
 the last 12 months
 
  [1]
 http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3C20130205115224.33547872@hub%3E
  [2]
 http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3c510d8293.8010...@gknw.net%3E
  [3] http://svn.apache.org/viewvc?view=revisionrevision=1333501
 
  I believe that one year in trunk without further review is long enough,
 and if someone wants to continue working on it its easy enough to checkout
 the last revision before removal.
 
  Gün.
 
 

Re: Intent to revert commit r1332643

[Graham Leggett]

On 24 May 2013, at 2:44 PM, Mario Brandt jbl...@gmail.com wrote:

 Any news on this?

Not yet, there is however a big push to get 2.4.5 out the door, which is where 
the focus is lying at the moment.

Regards,
Graham
--



smime.p7s
Description: S/MIME cryptographic signature

Whither Windows (Was: Re: Intent to revert commit r1332643)

[Jim Jagielski]

Maybe the real question is where exactly do we stand with
Windows right now...

We haven't had (complimentary) binary builds for Windows in
quite awhile and, afaict, there are really no people focusing
on Windows compatibility anymore.

For me, I wouldn't want to stunt httpd development for every
other platform we care about simply because it breaks
Windows. But it's not just my decision, 'natch.

On May 17, 2013, at 10:36 AM, Guenter Knauf fua...@apache.org wrote:

 Hi all,
 I will revert the changes done with:
 http://svn.apache.org/viewvc?view=revisionrevision=1332643
 after 72 hours if nobody is going to fix the stuff properly for Windows since 
 I'm tired of always copying mod_ssl over from 2.4.x branch in order to get a 
 working mod_ssl with trunk.
 
 Reasons:
 1) within last 12 months there was no attempt made to fix the issues which 
 wrowe mentioned in this thread [1] - instead discussion died
 2) a suggestion to fix the issue [2] was not applied due to the concerns 
 wrowe brought up, and to which I agree.
 3) the same issue also causes a stalled backport in 2.2.x STATUS [3] for the 
 last 12 months
 
 [1] 
 http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3C20130205115224.33547872@hub%3E
 [2] 
 http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3c510d8293.8010...@gknw.net%3E
 [3] http://svn.apache.org/viewvc?view=revisionrevision=1333501
 
 I believe that one year in trunk without further review is long enough, and 
 if someone wants to continue working on it its easy enough to checkout the 
 last revision before removal.
 
 Gün.
 
 

Re: Intent to revert commit r1332643

[Jeff Trawick]

On Fri, May 17, 2013 at 10:36 AM, Guenter Knauf fua...@apache.org wrote:

 Hi all,
 I will revert the changes done with:
 http://svn.apache.org/viewvc?**view=revisionrevision=1332643http://svn.apache.org/viewvc?view=revisionrevision=1332643
 after 72 hours if nobody is going to fix the stuff properly for Windows
 since I'm tired of always copying mod_ssl over from 2.4.x branch in order
 to get a working mod_ssl with trunk.

 Reasons:
 1) within last 12 months there was no attempt made to fix the issues which
 wrowe mentioned in this thread [1] - instead discussion died
 2) a suggestion to fix the issue [2] was not applied due to the concerns
 wrowe brought up, and to which I agree.
 3) the same issue also causes a stalled backport in 2.2.x STATUS [3] for
 the last 12 months

 [1] http://mail-archives.apache.**org/mod_mbox/httpd-dev/201302.**
 mbox/%3C20130205115224.**33547872@hub%3Ehttp://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3C20130205115224.33547872@hub%3E
 [2] http://mail-archives.apache.**org/mod_mbox/httpd-dev/201302.**
 mbox/%3C510D8293.8010103@gknw.**net%3Ehttp://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3c510d8293.8010...@gknw.net%3E
 [3] 
 http://svn.apache.org/viewvc?**view=revisionrevision=1333501http://svn.apache.org/viewvc?view=revisionrevision=1333501

 I believe that one year in trunk without further review is long enough,
 and if someone wants to continue working on it its easy enough to checkout
 the last revision before removal.

 Gün.




NPN is pretty important, but I understand the frustration at having to deal
with this in its incomplete state.

I promise to post a patch (or just commit if it is as trivial an issue as
it sounds) in the next week to fix the hard link between core and ssl.
 Maybe I'll mess with the AP-SSL hook issue too.

(Yeah, I know this is a rather uninspiring promise but I'm busy ;) )

How close does that get you to goodness on Windows, and may I presume
there's no desire to revert as long as progress is being made?

-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

[Jim Jagielski]

On May 24, 2013, at 9:08 AM, Jeff Trawick traw...@gmail.com wrote:
 
 Lots of us are employees of or otherwise manage to siphon money from these 
 companies.  Make a pitch...  (And some of us are happy to freelance ;) )
 

I'll be honest: I don't even know to to *build* for Windows,
at least with any Visual Studio release from this century.

Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))

[William A. Rowe Jr.]

On Fri, 24 May 2013 09:26:34 -0400
Jim Jagielski j...@jagunet.com wrote:

 
 On May 24, 2013, at 9:08 AM, Jeff Trawick traw...@gmail.com wrote:
  
  Lots of us are employees of or otherwise manage to siphon money
  from these companies.  Make a pitch...  (And some of us are happy
  to freelance ;) )
 
 I'll be honest: I don't even know to to *build* for Windows,
 at least with any Visual Studio release from this century.

That fortunately is documented, with some pretty good notes in
the wiki as well that aught to percolate into the docs.  That
said, documenting every Microsoft-version-quirk seems out of
scope for a general purpose 'compiling' doc.

Symbol resolution on Linux is rather loosey-goosey, and very
relaxed.  Resolution in OS/X 2-level namespace mode is quite
strict and a very good mirror of the world of Windows, z/OS and
many others[1].  It should be relatively easy to enforce explicit
exports on some other platforms which more of the developers
use in day-to-day practice.

The patch to 'fix' this is trivial.  The well-documented issues
with that 'fix' is the reason this patch should be reverted.

httpd has a reasonably straightforward pattern to register
and consume optional functions, hook functions etc.  This has
existed since close to the dawn of 2.0 and /had/ resolved almost
all module load-ordering issues in the server software.  It was
close to world-class in behavior and quite foolproof when the
return codes were observed.  To see how simple these are in
practice, see my backport of the Mutex behavior for 2.4-based 
modules to borrow when compiled under 2.2;
  http://people.apache.org/~wrowe/httpd-2.2-ports/util_mutex.h
  http://people.apache.org/~wrowe/httpd-2.2-ports/mod_mutex.c

Optional hooks were introduced to allow for the registration of 
a sometimes-present, sometimes absent hook provider because not 
every hook was relevant to the core httpd server and not every
hook should always exist.  Both proxy and dav are good examples
of modules which must introduce new hooks themselves.

This patch ignored that, and introduced a hook provider that now
exists in a sometimes-loaded module which the core is looking
to register in.  It breaks the pattern because that hook is not
an optional hook.  The implementation is a mess and it has been
documented for a year what would fix it to follow httpd convention.

The much greater problem is that the current proxy provider stack
is riddled with load-time linkage between proxy modules, rather
than run-time optional functions and hooks.  This means that the
entire LoadModule mod_proxy schema is broken and fails to follow
the design principals of httpd 2.0.

How can developers without Windows possibly cope with identifying
these problems?  Magically, all it takes is to re-sort your
LoadModule list in inverse order and see what breaks.  There are
very few Windows-only behaviors in the server that can't be
documented with some trivial tests even on Linux.

So unless we really no longer care about this 'enhancement' to
the Apache 2.0 family, it seems past time to start re-factoring
such newly-reintroduced fragile behaviors and come back to that
design principal.  Or perhaps chuck it and go back to hard module
linkages and document the required load orders?  That would save
a few ms at startup.

It seems that we would all benefit from working out the libtool
and exports.c logic so that we can have some explicit-export model 
on Linux that will approximate windows and demonstrate this logic 
to those who don't (and shouldn't have to) build on Windows, WDYT?

p.s. I think your prior thread is rather dismissive of Gregg,
and folks like Steffan who go out of their way to follow these
docs and regularly report problems on Windows.  It's dismissive
of folks like Jeff and the rest who untangled the thorny change
in mod_ssl that broke the windows MPM preloaded socket data
approach so badly.  The fact that reports and objections are 
ignored does not mean the platform is ignored.  This is the only 
major issue I am aware of, nobody seems to have problems generating 
builds outside of the ASF, and we haven't seemed to have a large 
issue of following the Subversion model of binaries.

p.p.s. We have delivered 2.0 and 2.2 binaries and aught to update
those, now that my VM's are healed with the necessary crazy MSVCRT
based compilation environment the 'final' 2.0 binary can be built
and a refresh of 2.2 can be provided.  (Mladen's hybrid solution 
rocks, but there are some insane to workaround quirks for httpd 
with his solution that weren't present in building apr, -util etc).

p.p.p.s. I'll build a demonstration 2.4 package over the holiday
weekend, but don't consider it 'prime time' just yet.  We have a 
ways to go before all the Windows' players concerns can be resolved,
I'll put up a scratchpad in the wiki for us to discuss.  Lua et al
introduce more 3rd-party build questions than they answer.

p.p.p.p.s. Note that the Lua project makes life difficult for ALL

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

[William A. Rowe Jr.]

On Fri, 24 May 2013 08:52:05 -0400
Jim Jagielski j...@jagunet.com wrote:

 Maybe the real question is where exactly do we stand with
 Windows right now...
 
 We haven't had (complimentary) binary builds for Windows in
 quite awhile and, afaict, there are really no people focusing
 on Windows compatibility anymore.

Thanks you just reminded me...

Another question is where exactly do we stand with OS/X right now?

Apple HFS+ is still not supported, there exists a forced lower-case
canonicalization hack authored by Apple, but AFAICT still no progress
on retrieving the true name of a file on a case-insensitive OS/X
volume which I suspect are still in common use on most OS/X boxen.
Of course, all the BSD-based file systems are strict case sensitive
and don't have security bypass issues when running 'vanilla' httpd.

Unfortunately I don't run an OS/X box anymore so it's awfully hard
for me to complete research on such a fix (and isn't so personally
interesting since my ppc laptop was retired for lack of OS updates).

Also wondering where the OS/X download lives?  It will build on any
OS/X box with a deployed toolchain, but I imagine many OS/X users
don't install that toolchain and live with the Apple provided
flavors, and would guess that 2.4.x is not part of that Apple OS
distributions so far.

Re: Intent to revert commit r1332643

[Guenter Knauf]

Hi,
On 24.05.2013 14:57, Jeff Trawick wrote:

NPN is pretty important,

granted.


I promise to post a patch (or just commit if it is as trivial an issue
as it sounds) in the next week to fix the hard link between core and
ssl.  Maybe I'll mess with the AP-SSL hook issue too.

cool!


How close does that get you to goodness on Windows, and may I presume
there's no desire to revert as long as progress is being made?
it was more an attempt by me to kick on discussion again; I feel 
sometimes that must be done in a drastical way when over one year 
nothing happend, and discussion just died ;-)
so that was kinda last resort - of course if com0ilation breaks no 
longer all is fine again!


Gün.

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

[Guenter Knauf]

Hi Jim,
On 24.05.2013 14:52, Jim Jagielski wrote:

For me, I wouldn't want to stunt httpd development for every
other platform we care about simply because it breaks
Windows. But it's not just my decision, 'natch.
well, for me its no reason to just accept every code as long as it 
compiles on *nix platforms regardless of design flaws; also we could 
easily fix the stuff so that Windows would compile again even with the 
design flaw, but that's not what Bill would accept IIUC, and not what I 
would like to do ...


finally what I was mainly after was kicking on discussion again about 
how to fix it correctly, and thats now in progress ... ;-)


Gün.

Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))

[Ben Reser]

On Fri, May 24, 2013 at 8:13 AM, William A. Rowe Jr.
wr...@rowe-clan.net wrote:
 That fortunately is documented, with some pretty good notes in
 the wiki as well that aught to percolate into the docs.  That
 said, documenting every Microsoft-version-quirk seems out of
 scope for a general purpose 'compiling' doc.

The build system should be able to compile with the major tool chains,
nobody expects to know how to work around weird autoconf, make, gcc,
etc quirks on Linux.  I don't say this to be dismissive of anyones
contributions but just to point out that producing Windows builds with
a modern toolchain is not simple.

I did a bunch of work on scripting building the dependencies for
Subversion on Windows that's located here:
http://svn.apache.org/repos/asf/subversion/trunk/tools/dev/build-svn-deps-win.pl

By far httpd was the biggest pain of any of the dependencies to get to work.

If your only interest is building httpd on Windows with Visual Studio
2012, taking a look at build_httpd() in that file should give a good
starting point.

Sometime when I find the time I want to fix the problems that I had to
work around the right way and not the hackish way I did in that script
and submit them back.  But I haven't gotten to it.

I'll admit that I haven't tried to build httpd-trunk on Windows so
maybe improvements have been made that haven't made their way over to
the 2.4 releases, Subversion certainly has similar issues with our 1.7
release branch.

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

[Ben Reser]

On Fri, May 24, 2013 at 8:23 AM, William A. Rowe Jr.
wr...@rowe-clan.net wrote:
 Another question is where exactly do we stand with OS/X right now?

 Apple HFS+ is still not supported, there exists a forced lower-case
 canonicalization hack authored by Apple, but AFAICT still no progress
 on retrieving the true name of a file on a case-insensitive OS/X
 volume which I suspect are still in common use on most OS/X boxen.
 Of course, all the BSD-based file systems are strict case sensitive
 and don't have security bypass issues when running 'vanilla' httpd.

Can't speak to this, it's not important to my use since I avoid the
case sensitivity issues on OS X.

I've had no problems building httpd on OS/X on 10.7 (I haven't
bothered to upgrade to 10.8).  I do a fair amount of work on OS X
directly and often build httpd-trunk and releases with debugging.
There may be computability issues like you pointed out above, but I
quite frankly have never noticed them.  Granted my use of httpd on OS
X is purely developmental and I likely wouldn't run into the types of
issues that someone using httpd for production servers on OS X would.

 Also wondering where the OS/X download lives?  It will build on any
 OS/X box with a deployed toolchain, but I imagine many OS/X users
 don't install that toolchain and live with the Apple provided
 flavors, and would guess that 2.4.x is not part of that Apple OS
 distributions so far.

I've never bothered to try to download a httpd binary build, it's easy
enough to build that I don't feel the need.

10.7 still had httpd 2.2, not sure what 2.4 has.

Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))

[Guenter Knauf]

On 24.05.2013 21:37, Ben Reser wrote:

The build system should be able to compile with the major tool chains,
nobody expects to know how to work around weird autoconf, make, gcc,
etc quirks on Linux.  I don't say this to be dismissive of anyones
contributions but just to point out that producing Windows builds with
a modern toolchain is not simple.
yeah ...; and from what I see our project files are already broken even 
when not converted and used directly with MSVC6, f.e. when doing a 
release build a bunch of files land in the debug folder, and finally at 
linking stage it breaks ...
probably we should think about moving either to plain Nmakefiles (as 
Pierre Joy also suggested), or add a Cmake build system; for me SCon is 
no alternative since I saw it too often already fail on modern Linux 
boxes (with other projects), so I have no hope that it works any better 
on Windows ...


and regarding trunk: AFAICT there are no improvements (what I mentioned 
above was with trunk) ...


Gün.

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

[William A. Rowe Jr.]

On Fri, 24 May 2013 12:43:23 -0700
Ben Reser b...@reser.org wrote:

 On Fri, May 24, 2013 at 8:23 AM, William A. Rowe Jr.
 wr...@rowe-clan.net wrote:
 
  Also wondering where the OS/X download lives?  It will build on any
  OS/X box with a deployed toolchain, but I imagine many OS/X users
  don't install that toolchain and live with the Apple provided
  flavors, and would guess that 2.4.x is not part of that Apple OS
  distributions so far.
 
 I've never bothered to try to download a httpd binary build, it's easy
 enough to build that I don't feel the need.
 
 10.7 still had httpd 2.2, not sure what 2.4 has.

As far as we are aware, no commercial distributions and so far,
no released free stable distributions incorporate 2.4.  A couple
have started integrating it, but the single biggest obstacle that
is reported by the packagers community is that mod_perl had not yet
supported 2.4 and that is an important package dependency to them.

Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))

[William A. Rowe Jr.]

On Fri, 24 May 2013 21:53:50 +0200
Guenter Knauf fua...@apache.org wrote:

 On 24.05.2013 21:37, Ben Reser wrote:
  The build system should be able to compile with the major tool
  chains, nobody expects to know how to work around weird autoconf,
  make, gcc, etc quirks on Linux.  I don't say this to be dismissive
  of anyones contributions but just to point out that producing
  Windows builds with a modern toolchain is not simple.

 yeah ...; and from what I see our project files are already broken
 even when not converted and used directly with MSVC6, f.e. when doing
 a release build a bunch of files land in the debug folder, and
 finally at linking stage it breaks ...
 probably we should think about moving either to plain Nmakefiles (as 
 Pierre Joy also suggested), or add a Cmake build system; for me SCon
 is no alternative since I saw it too often already fail on modern
 Linux boxes (with other projects), so I have no hope that it works
 any better on Windows ...

The concensus seems to be forming around cmake, and it is certainly
my preference from working with pcre and other libraries and projects.

But in the walk-before-you-run department, it's probably best to swap
the scons for cmake over at the apr project, and that will take our
most significant obstacle out of the way.  The httpd cmake will still
be quite complex, but not obnoxiously so, and our apr experience would
be beneficial to setting out a well thought out plan.

Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))

[Gregg Smith]

On 5/24/2013 12:53 PM, Guenter Knauf wrote:
yeah ...; and from what I see our project files are already broken 
even when not converted and used directly with MSVC6, f.e. when doing 
a release build a bunch of files land in the debug folder, and finally 
at linking stage it breaks ... 
Vice-versa, I never have a problem with release builds which probably 
explains why I do so few debug builds.


It ends up being all objects land in /Release but then the linker looks 
in /Debug for them.


libhttpd is the culpret here for me, I do not think I have had the 
problem with anything else. It's now been two or three weeks since I 
fought the dreaded debug build so who knows as I have done too many 
release builds since :)  But libhttpd stands out above all and I have no 
idea why it's happening on this project, I'm just not seeing it.


Gregg

Intent to revert commit r1332643

[Guenter Knauf]

Hi all,
I will revert the changes done with:
http://svn.apache.org/viewvc?view=revisionrevision=1332643
after 72 hours if nobody is going to fix the stuff properly for Windows 
since I'm tired of always copying mod_ssl over from 2.4.x branch in 
order to get a working mod_ssl with trunk.


Reasons:
1) within last 12 months there was no attempt made to fix the issues 
which wrowe mentioned in this thread [1] - instead discussion died
2) a suggestion to fix the issue [2] was not applied due to the concerns 
wrowe brought up, and to which I agree.
3) the same issue also causes a stalled backport in 2.2.x STATUS [3] for 
the last 12 months


[1] 
http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3C20130205115224.33547872@hub%3E
[2] 
http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3c510d8293.8010...@gknw.net%3E

[3] http://svn.apache.org/viewvc?view=revisionrevision=1333501

I believe that one year in trunk without further review is long enough, 
and if someone wants to continue working on it its easy enough to 
checkout the last revision before removal.


Gün.

Re: Intent to revert commit r1332643

[Jim Jagielski]

Please don't submit what could be controversial reverts
over a weekend.

On May 17, 2013, at 10:36 AM, Guenter Knauf fua...@apache.org wrote:

 Hi all,
 I will revert the changes done with:
 http://svn.apache.org/viewvc?view=revisionrevision=1332643
 after 72 hours if nobody is going to fix the stuff properly for Windows since 
 I'm tired of always copying mod_ssl over from 2.4.x branch in order to get a 
 working mod_ssl with trunk.
 
 Reasons:
 1) within last 12 months there was no attempt made to fix the issues which 
 wrowe mentioned in this thread [1] - instead discussion died
 2) a suggestion to fix the issue [2] was not applied due to the concerns 
 wrowe brought up, and to which I agree.
 3) the same issue also causes a stalled backport in 2.2.x STATUS [3] for the 
 last 12 months
 
 [1] 
 http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3C20130205115224.33547872@hub%3E
 [2] 
 http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3c510d8293.8010...@gknw.net%3E
 [3] http://svn.apache.org/viewvc?view=revisionrevision=1333501
 
 I believe that one year in trunk without further review is long enough, and 
 if someone wants to continue working on it its easy enough to checkout the 
 last revision before removal.
 
 Gün.