Re: A new release process?
On Thu, Dec 29, 2016 at 11:16 PM, David Zuelkewrote: > As a, more or less, "outside observer", I happen to think that the current > method of voting on finals, instead of a practice of rolling out RCs (that > are then left up for testing for at least a week), is fundamentally broken. > The 2.4 changelog in particular is littered with releases that were never > officially published. For users, that's really confusing. For maintainers, > it's painful to start over the process each time, and it sometimes leads to > months and months without a release that contains certain fixes. Then a > backport goes wrong (still using SVN, in my opinion, does not help there, but > that's a whole different discussion :)), and a regression is in the latest > release until someone eventually picks up a fix. It seems to me that most "not released" versions are immediately superseded. The practice of throwing away the number instead of calling the initial contents an RC shouldn't contribute meaningfully to a fix being delayed. New blood for release managers would be nice, and smaller/more frequent /more predictable releases would probably help that. -- Eric Covener cove...@gmail.com
Re: [users@httpd] Copyright notices in httpd source files
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Nick, On 12/29/16 3:56 AM, Nick Kew wrote: > Cc: dev list. Looks like a catch? (my reply might be bounced from the dev@ list... I don't think I'm subscribed) > On Wed, 2016-12-28 at 17:44 -0500, Christopher Schultz wrote: >> All, >> >> Is it common to have a copyright notice in httpd C source files? > > Not common, but neither is it unusual. I'd expect it to mean > someone else copyrighted it before contributing it to apache. > >> [ mod_proxy_protocol.c ] Copyright 2014 Cloudzilla Inc. > > If that's in our svn, it should probably have another line > asserting Apache copyright alongside that one. As in, for > instance, mod_proxy_html. Yep, this is ASF svn. That's why I included the link in my original post : https://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters/mod_prox y_protocol.c?view=markup=1776076 - -chris -BEGIN PGP SIGNATURE- Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJYZeM4AAoJEBzwKT+lPKRYpN4P/30FjhTLL1iMSVRETFzQkBsC T+7e8+0NejwaqSoAqWpX/VbIawgyGFtsqgFjqOzGkp8Oy3FSnWpCFfG3IVR/4DCW dEe+/LWU5f+B0T1dJyDL2DOCxnHQPiyPc3AYGF+NUtIvKEys8m7zO8hTQ0Dz5TFB k4GFt69aP+MJnKAQAx8T1Gilwbu5vUZWmxu80rbl85FixYiW3iTSPHzApQ53ZSdx mUUHYoV9wv0fwm1+Udj6SHQRkVDBmzwddeTdqEa9mukdf6zc+8R4pcx8K0YLxuMP iduKLbXCxpy2oi28UpqqD8/5JD9adNBc3aYUpm0b/7ceJpr05wkFtZfh7OerULwr nYqxaZtogrdMPzV3fgOvz2S136uqDd0qSFOyvV7Qy8i/87Yr+kkk1GDaVcenhM1m z6+0iV9ftm5F4QbivheaKsJJEut4urfiVBnUha1l+BveWO+IOR047U/gdP7gfvcP RrCqaYLSclCP0UtlBAa0RWlZSsa4HS3/m+YyJ/dm4pDbjY6kyWAkA8w4+GOJhQLf MJ//TKbZ9LGNo4ueye+pRN1H4OFuJqTI1c4LdOsXqOW62IhATUIU8/DNCxKrlkTL pGp2C+5qUDQGj/yotEdyI9oDshm3ZDgPAplMQcl2itVyKgKYVrjmHfgLectRNAKf m+W8ul5tp4Dp0N0vJSU1 =Uymx -END PGP SIGNATURE-
A new release process?
Hi everyone, Given the several current threads where there's arguing about what and how and when to release features, backported or not, I'd like to offer a tale of a project that was, more or less, in the same dire spot, and pulled itself out of that misery with great success and universal acclaim. That project is PHP. Back in 2010, it was stuck at version 5.3.something, there was no standards around what feature goes into a minor release, how a feature is even accepted for inclusion, how security updates were treated, and so forth. Enter: https://wiki.php.net/rfc/releaseprocess The tl;dr of this approach is that - any x.y.z release only introduces bugfixes. These releases are done every four weeks, like clockwork. If a fix doesn't make the cut for a release, it'll end up in the next one; - x.y.0 releases, on the other hand, may introduce new features, fixes, and deprecations, but no breaking changes; - x.0.0 releases are the big ones (think PHP 7.0.0 in late 2015). where backward compatibility may be changed, etc. This makes for very predictable update cycles, and for pretty easy updating, as it's very very unusual that a x.y.z release breaks existing behavior. At the same time, since x.y.0 releases are made roughly once a year, bigger changes do not exist in a limbo branch forever, but will see the light of day eventually (and, again, after a predictable amount of time). There is a defined roadmap with active/security maintenance status and EOL dates for all versions; see http://php.net/supported-versions.php From a user perspective, this change has been fantastic. Once virtually stagnant, development of the language itself has been much more active; users have a much better idea up front around when fixes or features will land; upgrades have become easy as there are clear definitions of what kind of changes a version may contain. But most importantly, the language itself has seen a steady flow of new features as a result from this discipline. For features and bigger changes, an RFC is written with rationale, analysis, suggestion of implementation, and then it can be voted on. Once accepted, it can be committed, reviewed, and merged. Even for "outside" contributors, this process is transparent, and the mental threshold towards contribution has been lowered greatly, as acceptance of an RFC basically means that the feature will go into a new release anywhere between within a few months and a year and a bit (depending on when in the release cycle it happens). There are a bunch of technicalities that would need adjusting to fit HTTPD, such as release intervals, release management (for PHP, every x.y.* series has two managers who jointly coordinate releases), etc, but overall the idea is, IMO, worth considering. As a, more or less, "outside observer", I happen to think that the current method of voting on finals, instead of a practice of rolling out RCs (that are then left up for testing for at least a week), is fundamentally broken. The 2.4 changelog in particular is littered with releases that were never officially published. For users, that's really confusing. For maintainers, it's painful to start over the process each time, and it sometimes leads to months and months without a release that contains certain fixes. Then a backport goes wrong (still using SVN, in my opinion, does not help there, but that's a whole different discussion :)), and a regression is in the latest release until someone eventually picks up a fix. Much of this, and many of the "what do we backport from trunk" and "I'd like to squeeze in a change I've had sitting around locally, please wait with the new release, because who knows when the next one after that will be" are, from what I can tell, a significant source of discord on this mailing list. All these unnecessary distractions that deteriorate personal relationships, while at the same time slowing down the pace of the project (several people have already pointed out Nginx's rate of innovation in comparison) and raising the threshold for contributions, can be fixed. PHP is the perfect example, and I think HTTPD would be wise to at least consider following this example. Happy New Year! David
Welcome Daniel Ruggeri and Jacob Champion to the HTTP Server PMC
HTTP Server committers Daniel Ruggeri and Jacob Champion were recently elected to the HTTP Server Project Management Committee (PMC). A project management committee (PMC) is a committee of the Apache Software Foundation charged with responsibility and governance for their top level project. The PMC is the vehicle through which decision making power and responsibility for oversight is devolved to developers. Congratulations, thanks, and welcome! -- Eric Covener cove...@gmail.com
Re: Post 2.4.25
On Thu, Dec 29, 2016 at 8:23 AM, Jim Jagielskiwrote: > >> On Dec 28, 2016, at 6:28 PM, William A Rowe Jr wrote: >> >> Because fixing r->uri is such a priority, trust that I'll be voting every >> 2.6 candidate a -1 until it exists. I don't know why the original httpd >> founders are so hung up on version number conservation, they are cheap, but >> we are breaking a key field of a core request structure and no other OSS >> project would be stupid enough to call that n.m+1. > > Who is digging in their heels and blocking new development > now? > > So you are admitting that you will "veto" (although you > can't veto a release) any 2.5.* "releases" unless and > until r->uri is "fixed". Wow, Jim, how did you misread my assertion that I'd oppose 2.6 GA or 3.0 GA release until feature "X", where "X" represents the heavy-lift of not using filesystem syntax as the uri path except for files, honoring the URI and HTTP RFC, and therefore requiring some module authors to re-think how they consumed or changed/assigned r->uri. Modules such as proxy would actually pass on the *presented* uri (if valid) to the back end http server - just imagine that. That change I'm expecting we all want to call out as 3.0 for our developers, even though there are no directives changed for our users so administration doesn't change. How did you jump to the conclusion that I'd block an -alpha or -beta release on the 2.5.x trunk? Usually takes some number of incremental -alpha/-beta tags to get to GA. And how did you translate 'vote -1' to veto?
Re: The Version Bump fallacy [Was Re: Post 2.4.25]
On Thu, Dec 29, 2016 at 8:25 AM, Jim Jagielskiwrote: > It wasn't the paste that was the problem, but the inability > of other email clients to determine from your email what > parts/sections are quoted from *previous* emails. Yann pointed me in the right direction, I believe it is fixed now. Thanks for the heads-up! >> On Dec 28, 2016, at 5:49 PM, William A Rowe Jr wrote: >> >> Hi Jim, >> >> Talk to Google and the OpenOffice Team, that was a paste from OpenOffice >> Calc. >> >> I'll be happy to start summarizing as a shared Google sheet. Google sheet might still be useful, so I'll maintain that as a general purpose collection of shared-with-httpd-dev tabs.
Re: [users@httpd] Copyright notices in httpd source files
I didn't know you guys were working on it. Cool. I had been working the donation angle for awhile and finally got approval so wanted to get it in quick! :) > On Dec 29, 2016, at 3:56 AM, Nick Kewwrote: > > Cc: dev list. Looks like a catch? > > On Wed, 2016-12-28 at 17:44 -0500, Christopher Schultz wrote: >> All, >> >> Is it common to have a copyright notice in httpd C source files? > > Not common, but neither is it unusual. I'd expect it to mean > someone else copyrighted it before contributing it to apache. > >> [ mod_proxy_protocol.c ] >> Copyright 2014 Cloudzilla Inc. > > If that's in our svn, it should probably have another line > asserting Apache copyright alongside that one. As in, > for instance, mod_proxy_html. > > -- > Nick Kew >
Re: The Version Bump fallacy [Was Re: Post 2.4.25]
> On Dec 28, 2016, at 7:40 PM, Yehuda Katzwrote: > > On Wed, Dec 28, 2016 at 12:35 AM, William A Rowe Jr > wrote: > Our adoption is *broadly* based on the OS distributions > from vendors, not from people picking up our sources. > Yes - some integrate directly from source, and others > use a non-OS distribution. > > I think a significant number of users of nginx add the official nginx yum/apt > sources and keep up to date that way > (http://nginx.org/en/linux_packages.html#mainline). > This is particularly true because the vendor-supplied version are so old. You > can see this in the w3techs data: nginx 1.10.2 came out in October and > already makes up 75% of all nginx 1.10 users. nginx 1.11.8 usage has similar > trends. > > A possible solution to this would be to start publishing binaries in a > package-manager-accessible format. > I am confident it would see a much higher rate of adoption. > Good point. +1
Re: The Version Bump fallacy [Was Re: Post 2.4.25]
It wasn't the paste that was the problem, but the inability of other email clients to determine from your email what parts/sections are quoted from *previous* emails. > On Dec 28, 2016, at 5:49 PM, William A Rowe Jrwrote: > > Hi Jim, > > Talk to Google and the OpenOffice Team, that was a paste from OpenOffice Calc. > > I'll be happy to start summarizing as a shared Google sheet. > > Cheers, > > Bill > > > On Dec 28, 2016 14:22, "Jim Jagielski" wrote: > Bill, I don't know if it's just my Email client or not (doesn't > look like it) but could you fix your Email client? It's impossible to > reply and have the quoted parts parsed out correctly. I think > it's to do w/ your messages being RTF or something. > > Thx! > > Included is an example of how a Reply misses quote levels... > > > On Dec 28, 2016, at 1:34 PM, William A Rowe Jr wrote: > > > > On Wed, Dec 28, 2016 at 9:13 AM, Jim Jagielski wrote: > > cPanel too... They are moving to EA4 which is Apache 2.4. > > > > If not moved yet, that example wouldn't be helpful, it reinforces my point > > four years later. But EA itself seems to track pretty closely to the most > > contemperanious versions, looks like within a month. > > > > > > So the idea that supplemental (ie: 2.4.x->2.4.y) patches don't > > have the reach or range of larger ones (2.4.x->2.6/3.0) isn't > > quite accurate. > > > > It's entirely accurate. It isn't all-encompassing. We have that data too, > > let's tear down SecuritySpace's Nov '16 dataset; > > http://www.securityspace.com/s_survey/data/201611/servers.html > > >
Re: Post 2.4.25
> On Dec 28, 2016, at 6:28 PM, William A Rowe Jrwrote: > > > Because fixing r->uri is such a priority, trust that I'll be voting every 2.6 > candidate a -1 until it exists. I don't know why the original httpd founders > are so hung up on version number conservation, they are cheap, but we are > breaking a key field of a core request structure and no other OSS project > would be stupid enough to call that n.m+1. > Who is digging in their heels and blocking new development now? So you are admitting that you will "veto" (although you can't veto a release) any 2.5.* "releases" unless and until r->uri is "fixed". Which implies, obviously, a very substantial refactoring. Which implies time. Which implies that if you also say "no new enhancements in 2.4" that it will be a long time until anything new and useful will be added to, or available to, our end-users until some unknown future time. And that is acceptable to you? And no one I know of in any way is "hung up on version number conservation", and that is moot and strawman anyway. As fair warning, I fully expect that we will release 2.4.26 within the next 3 months. I also fully expect that some "new enhancements" from trunk to be backported and be in that release. I simply care about continuing to keep Apache httpd relevant and a continued viable offering for our community. That means us working on next-gen, of course, but also maintaining and fostering a community until next-gen exists.
Re: HTTP/2 frame prioritization not honored
> > That means the images should get a minim of ~30% of the available > bandwidth as long as they have data. My reading. Right. Makes sense.
Re: HTTP/2 frame prioritization not honored
> Am 29.12.2016 um 12:10 schrieb Kyriakos Zarifis: > > > 1. priority dependencies: streams with lower prio, but depending on a high > level stream, inherit this priority. This would make all preloads *together* > have the same prio as page B. Last I knew, chrome did no dependencies. Were > these resources PUSHed by the server? > > Not PUSHed; they were requested by the browser upon A's onLoad using the ' > link rel=”prefetch” ' directive in HTML A's . > > Right, I forgot to add this in the description: Chrome's log shows the > priorities/dependencies as follows: > HTML A : stream_id=1, parent_stream=0, weight=256 > 1st Prefetched image : stream_id=3, parent_stream=0, weight=110 > 2nd Prefetched image : stream_id=5, parent_stream=3, weight=110 > 3nd Prefetched image : stream_id=7, parent_stream=5, weight=110 > ... > 8th Prefetched image : stream_id=17, parent_stream=15, weight=110 > HTML B : stream_id=19, parent_stream=0, weight=256 > > So my understanding was that the dependencies looked reasonable, i.e. the > prefetched objects form a separate lower priority chain starting at the root > and as such should not block anything with higher prio with the same parent > (0). Am I reading this correctly? That is different than Firefox is doing things, but it should work. Notice though, that the images are chained with weight 110 on stream 0 and HTML B with max weight 255. That means the images should get a minim of ~30% of the available bandwidth as long as they have data. My reading. > > 2. mod_http2 is, in its current implementation, very eager to fill the core > network buffers. That means a lot of frames have already been pre-packaged > for sending before page B becomes ready to sent. These are then not > pre-empted by page B, but will be sent first. > > If you are willing to test a github beta of the module, I might find the time > next week to tweak the behaviour in 2) and you could verify the benefits in > your setup. I will not promise anything, though. > > This was my (as well as a few Chrome devs') guess, especially considering > that HTML B is not blocked behind the prefetched images (in fact not even > blocked by completing any single one of them: the images are sent serially > and HTML B is correctly interleaved between frames of one of those images - > it just doesn't seem to happen as quickly as it could have (~500ms earlier, > in this particular case) ). > > I'd be really interested in repeating the experiment with a tweaked version > of the module. Please feel free to let me know if/how I can help. I see what I can come up with. Thanks! > > Thanks, > Kyriakos > > > -Stefan > > > > > Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: HTTP/2 frame prioritization not honored
> 1. priority dependencies: streams with lower prio, but depending on a high > level stream, inherit this priority. This would make all preloads > *together* have the same prio as page B. Last I knew, chrome did no > dependencies. Were these resources PUSHed by the server? > Not PUSHed; they were requested by the browser upon A's onLoad using the ' link rel=”prefetch” ' directive in HTML A's . Right, I forgot to add this in the description: Chrome's log shows the priorities/dependencies as follows: HTML A : stream_id=1, parent_stream=0, weight=256 1st Prefetched image : stream_id=3, parent_stream=0, weight=110 2nd Prefetched image : stream_id=5, parent_stream=3, weight=110 3nd Prefetched image : stream_id=7, parent_stream=5, weight=110 ... 8th Prefetched image : stream_id=17, parent_stream=15, weight=110 HTML B : stream_id=19, parent_stream=0, weight=256 So my understanding was that the dependencies looked reasonable, i.e. the prefetched objects form a separate lower priority chain starting at the root and as such should not block anything with higher prio with the same parent (0). Am I reading this correctly? > 2. mod_http2 is, in its current implementation, very eager to fill the > core network buffers. That means a lot of frames have already been > pre-packaged for sending before page B becomes ready to sent. These are > then not pre-empted by page B, but will be sent first. > > If you are willing to test a github beta of the module, I might find the > time next week to tweak the behaviour in 2) and you could verify the > benefits in your setup. I will not promise anything, though. > This was my (as well as a few Chrome devs') guess, especially considering that HTML B is not blocked behind the prefetched images (in fact not even blocked by completing any single one of them: the images are sent serially and HTML B is correctly interleaved between frames of one of those images - it just doesn't seem to happen as quickly as it could have (~500ms earlier, in this particular case) ). I'd be really interested in repeating the experiment with a tweaked version of the module. Please feel free to let me know if/how I can help. Thanks, Kyriakos > > -Stefan > > > > >
Re: HTTP/2 frame prioritization not honored
> Am 29.12.2016 um 02:07 schrieb Kyriakos Zarifis: > > I am experimenting with contention between lower/higher priority HTTP/2 > streams, and I think I am noticing that high-priority frames are not given > high priority (quickly enough) > > The process is: Download dummy page A, which, after onLoad, Prefetches 8 > objects (5MB each) at low priority. While the first few of those are > downloaded, click on link to page B (high priority stream), and measure how > long it takes to receive. > > The client-server RTT is 20ms, and ideally because of higher prioritization > we would want the second HTML to be received at a timeframe as close to that > RTT as possible. However, I am noticing that the second HTML is delayed > (although not completely HoL-blocked) by objects that are being Prefetched at > lower priority. > > This plots the client-side trace (Chrome log): HTML A is stream_id 1, the 8 > prefetched objects are streams 3-17, and HTML B (the object that is delayed) > is stream_id 19. Dots below the x axis are the GET requests for the > respectively colored streams. Dots above the x axis are received DATA frames. > GETs for the prefetched objects are sent after onLoad, and that of stream_id > 19 is sent when I click on the link, around t=1900. As this shows, the reply > for the high-priority stream (19) arrives ~1.5s later. > > Apache's log shows that it sent the reply to stream_id 19 (HTML B) ~500ms > after it got its request. During those 500ms, it wrote another ~350 frames > (~5MB) from other streams. So it honors the higher priority in the sense that > it interleaves the HTML between the currently written object, but it > continues to write many lower-priority frames before it does so, delaying > that high-prio HTML. In contrast, when I wait until all prefetched objects > are served and then click on the link, the logs shows that the reply is > written within a few ms from the request arrival, as expected, and the second > page loads observably faster. > > This seems to break any default prioritization or prioritization imposed by > the client, and leads to unexpected performance in different > prioritization-dependent scenarios that I have tried. > > Has anyone experience this before, and do you know what causes it and what > the right way to fix it would be? Reading your excellent description, there are two main things that can affect this: 1. priority dependencies: streams with lower prio, but depending on a high level stream, inherit this priority. This would make all preloads *together* have the same prio as page B. Last I knew, chrome did no dependencies. Were these resources PUSHed by the server? 2. mod_http2 is, in its current implementation, very eager to fill the core network buffers. That means a lot of frames have already been pre-packaged for sending before page B becomes ready to sent. These are then not pre-empted by page B, but will be sent first. If you are willing to test a github beta of the module, I might find the time next week to tweak the behaviour in 2) and you could verify the benefits in your setup. I will not promise anything, though. -Stefan
Re: Post 2.4.25
Am 29.12.2016 um 07:08 schrieb William A Rowe Jr: (Again, it's gmail, /shrug. I can attempt to undecorate but doubt I'm moving to a local client/mail store again. If anyone has good gmail formatting tips for their default settings, I'd love a pointer.) yes, setup thunderbird and gmail with IMAP for mailing-lists so that your sent and received mail are as now on the server or setup roundcube to access gmail via IMAP/SMTP and configure it to prefer plaintext or complain loud enough to google that they are fools when they convert a plaintext message to html while press reply
Re: The Version Bump fallacy [Was Re: Post 2.4.25]
> Am 29.12.2016 um 01:40 schrieb Yehuda Katz: > > On Wed, Dec 28, 2016 at 12:35 AM, William A Rowe Jr > wrote: > Our adoption is *broadly* based on the OS distributions > from vendors, not from people picking up our sources. > Yes - some integrate directly from source, and others > use a non-OS distribution. > > I think a significant number of users of nginx add the official nginx yum/apt > sources and keep up to date that way > (http://nginx.org/en/linux_packages.html#mainline). > This is particularly true because the vendor-supplied version are so old. You > can see this in the w3techs data: nginx 1.10.2 came out in October and > already makes up 75% of all nginx 1.10 users. nginx 1.11.8 usage has similar > trends. > > A possible solution to this would be to start publishing binaries in a > package-manager-accessible format. > I am confident it would see a much higher rate of adoption. Very good point. Myself using a ppa for my ubuntu server via deb http://ppa.launchpad.net/ondrej/apache2/ubuntu trusty main that updates very quickly. It already has 2.4.25. There are other people doing this for various distros. The least we could do is document the ones we know, talk to people how they see it continue. Offer a https place and visibility on Apache servers maybe? Does that make sense? > - Y Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: [users@httpd] Copyright notices in httpd source files
Cc: dev list. Looks like a catch? On Wed, 2016-12-28 at 17:44 -0500, Christopher Schultz wrote: > All, > > Is it common to have a copyright notice in httpd C source files? Not common, but neither is it unusual. I'd expect it to mean someone else copyrighted it before contributing it to apache. > [ mod_proxy_protocol.c ] > Copyright 2014 Cloudzilla Inc. If that's in our svn, it should probably have another line asserting Apache copyright alongside that one. As in, for instance, mod_proxy_html. -- Nick Kew