Re: STATUS and Backport Review efficiency
On 6/8/15 10:17 AM, William A Rowe Jr wrote: In this example, the patch was enhanced and the original reviewers' efforts were thrown away. It's a shame to waste the limited review cycles. Moving forwards, can we please do two things. 1) retain the original patch and vote in the STATUS, adding a revised patch below the vote that can be reviewed, and 2) show the changelog of patch 1 vs. patch 2 so that the original reviewer doesn't have to bisect two patches to determine whether the change had any measurable effect on the original patch review? We might start an http://svn.apache.org/repos/asf/httpd/httpd/patches/backports/2.[24].x/ tree, so that viewvc can be helpful in this respect, instead of multiple patches residing in our individual public_html/ trees. WDYAT? All of this is to say that we should pay extra close attention to correctness when we first submit a backport patch proposal, and request 3) is to please not use STATUS as a discussion tool. If the comment/observation doesn't fit in 240 characters (3 lines), it probably belongs on the list instead, or can be a short note in STATUS with a long explanation on dev. This would be a lot less painful if the httpd project would actually use SVN for managing these patches. The the SVN project does this. Here's an example of a SVN STATUS entry (from the 1.8.x release branch): [[[ * r1654932, r1654933, r1654934, r1654937 Fix issue #4554, 0 file length reported in FSFS. Justification: This causes 'svnadmin dump' to create corrupted output that fails to load and we provide no way to detect that problem other than loading the respective dump. We also want to prevent further instances of that issue to be added to the repository. Notes: r1561419 has been removed from this change set and proposed for a separate backport because it is not a necessary part of the #4554 fix. Branch: ^/subversion/branches/1.8.x-issue4554-v2 Votes: +1: rhuijben +1: stefan2 (before -v2 branch) +0: danielsh (hard to review all potential causes of expanded_size==0 in 7*3*2 cases (1.1…1.8) × (file/dir/prop rep) × (plain/delta)) ]]] The first line is the trunk revisions being backported. However, these changes do not cleanly merge from trunk onto 1.8.x. So a backport branch was made. The change was merged and any conflicts resolved (or possibly other changes). This particular change is a tad unusual in that a second backport branch was made. Usually any modifications just mean additional changes on the branch and the vote split would notice which branch changes weren't included in the review. The workflow for the httpd project would look something like this (presuming you start with the 2.4.x branch checked out in your $PWD): svn cp ^/httpd/httpd/branches/2.4.x ^/httpd/httpd/branches/2.4.x-ap-listeners-buckets svn switch ^/httpd/httpd/branches/2.4.x-ap-listeners-buckets svn merge -c 'r1640763, r1643179, r1656368' ^/httpd/httpd/trunk . # handle conflicts or anything needed. svn ci # Further changes repeat the merge/commit. svn switch ^/httpd/httpd/branches/2.4.x Then when approved you merge the branch onto the release branch like so onto the 2.4.x branch (presuming you start with 2.4.x branch checked out in your $PWD): svn merge ^/httpd/httpd/branches/2.4.x-ap-listeners-buckets . svn rm ^/httpd/httpd/branches/2.4.x-ap-listeners-buckets Using this workflow resolves almost all of your requests. 1) You can preserve votes easily by noting when someone didn't vote for new revisions on the branch and moving their vote to a separate line. 2) You have history, no need to try and track the history in the STATUS file. 3) This doesn't resolve this issue and the SVN project somewhat does the same as you can see above. But I think it helps because it's far less confusing about what everyone reviewed and what their comments are about.
Re: Looking ahead to 2.4.13 / 2.2.30
On 5/4/15 7:40 AM, Brian J. France wrote: While you are in mod_dav, could you review these patches and see if it makes sense to add them? httpd-2.2.x : http://www.brianfrance.com/software/apache/dav/mod_dav_fs.diff.22 httpd-2.4.x : http://www.brianfrance.com/software/apache/dav/mod_dav_fs.diff.24 We have been running these for a while at work, just haven't had free time to send them. Can you sure what issue you're trying to work around with those? In and of themselves they look alright but I'd like to understand what motivated the change.
Re: Looking ahead to 2.4.13 / 2.2.30
On 5/3/15 8:05 AM, Jim Jagielski wrote: Thx! On May 1, 2015, at 3:29 PM, Ben Reser b...@reser.org wrote: On 4/30/15 2:52 PM, William A Rowe Jr wrote: It seems that we have 2 groups of good things to come out of ApacheCon, some immediate fixes for things like BSD project efforts, some pretty straightforward defects that have been resolved... and then there's a bunch of energy about enhancements and the h2 universe. In the short term, WDYT about giving the trees a week to settle, grab the low hanging fruit and move forward for 2.4.13 and 2.2.30 end of this coming week? Guessing Jim's up for RM'ing 2.4.13, and I'm happy to TR 2.2.30 in tandem. Concerns / observations / thoughts? I have a mod_dav fix that really ought to be in the next 2.4 release. I'll get it committed and nominated sometime this weekend. My fix is committed and backports nominated. If someone can look at them it'd be appreciated.
Re: Looking ahead to 2.4.13 / 2.2.30
On 4/30/15 2:52 PM, William A Rowe Jr wrote: It seems that we have 2 groups of good things to come out of ApacheCon, some immediate fixes for things like BSD project efforts, some pretty straightforward defects that have been resolved... and then there's a bunch of energy about enhancements and the h2 universe. In the short term, WDYT about giving the trees a week to settle, grab the low hanging fruit and move forward for 2.4.13 and 2.2.30 end of this coming week? Guessing Jim's up for RM'ing 2.4.13, and I'm happy to TR 2.2.30 in tandem. Concerns / observations / thoughts? I have a mod_dav fix that really ought to be in the next 2.4 release. I'll get it committed and nominated sometime this weekend.
Re: Apache 2.2.28 release timing.
On 8/21/14 6:26 PM, William A. Rowe Jr. wrote: That about sums it up. Sorry, I am still drowning in my late father's affairs for another 3-4 weeks, but will make time to do this in 2 hours from now, sum up votes and move files Sun a.m. for a Mon a.m. release. That saves anyone else from creating an older toolchain (even I never use this one on the development branches). Thanks Bill, if you don't get to it I'll try to do it next week. I'm traveling right now or I'd do it now.
Re: Apache 2.2.28 release timing.
On 8/5/14 2:21 PM, Mark Blackman wrote: This might be more of user than dev question, but as the discussions about timing were here, I’ll go with here. http://mail-archives.apache.org/mod_mbox/httpd-dev/201407.mbox/20140721075315.ec908e91c20de17e6e448089a4bc3ed2.f963b4ea46.wbe%40email11.secureserver.net suggested the 2.2.28 tagging and presumably release is imminent, however, http://svn.apache.org/repos/asf/httpd/httpd/tags/2.2.28 is still a 404. I understand the mechanics of open source projects, so this is not a “hurry-up”, it’s just a can I get Apache 2.2.28 into my next hosting platform release or not”, the contents of which will be frozen on Aug. 15. I’m mostly interested in the CVE updates, so I can tell users we’re clear of them. If the 2.2.28 release is not likely before Aug. 15, that’s fine, I just wanted to be sure. Beyond just the CVE changes there are other changes such as the fix for PR 56480 that people want in a 2.2.x release. Some Subversion users are still needing to use 2.2.x for reasons that aren't terribly important to go into here. I've been patiently waiting for this release to happen, but it's been nearly a month now since the original intent to roll email came out. I'd do the rolling myself but I'm not 100% clear on what needs to happen. So if someone can do a little hand holding I'll be happy to do the release myself. I'm generally familiar with how the ASF does releases since I do the Subversion release regularly. So this would be entirely about the specifics of rolling the release.
Re: Time for httpd 2.2.28??
On 7/15/14 10:45 AM, Jim Jagielski wrote: If so, I can RM. Yes please, need the mod_dav fix that's already approved for 2.2.x.
Re: [VOTE] Release Apache httpd 2.4.10 as GA
On 7/15/14 10:20 AM, Jim Jagielski wrote: The pre-release test tarballs for Apache httpd 2.4.10 can be found at the usual place: http://httpd.apache.org/dev/dist/ I'm calling a VOTE on releasing these as Apache httpd 2.4.10 GA. [ ] +1: Good to go [ ] +0: meh [ ] -1: Danger Will Robinson. And why. Vote will last the normal 72 hrs. NOTE: The *-deps are only there for convenience. +1 Unbutu 12.04 and OS X 10.9.4. Passes the Subversion test suite with the XPASS for the mod_dav PROPFIND encoding fix as expected.
Re: FYI: Looking for a release of 2.4.x soonish
On 7/9/14 7:57 AM, Yann Ylavic wrote: Maybe one more vote for the latest mod_deflate fix (PR 56196) so it has no know issue in 2.4.10? I see several more mod_deflate fixes. I'll try to take a look at these tomorrow because I've run into some issues with 2.2.x and mod_deflate and I'm hoping these resolve them and if so I'd like to see them included.
Re: Looking for a release of 2.4.x soonish
On 6/24/14 12:35 PM, Bert Huijben wrote: I would really like to see the mod_dav escaping fixes (where breser is looking at) to be backported for this next release. Without those patches Subversion doesn't properly support some special characters inside repository paths. (Regression against 2.0.x, 2.2.x and earlier 2.4.x versions). As soon as these are backported: +2 :-) Nominated if people can vote for them.
Re: CI for httpd 2.4 and trunk?
On 6/14/14 2:34 AM, Takashi Sato wrote: +1 How about automated test? http://svn.apache.org/repos/asf/httpd/test/ Yes there should be automated testing that runs. But sadly that test suite is pretty limited. It needs a lot of work and a commitment to adding regression tests for bugs as we fix them. That's not to say that running that test suite is worthless. One problem in particular is that it provides a wrapper around the LWP::UserAgent wrapper. That's well and fine for testing well formed requests. But our test suite should be generating invalid requests and verifying that we're responding with the correct error messages. This design drives tests being written towards only valid input testing.
Re: Change of web site layout
On 6/17/14 3:24 PM, Rich Bowen wrote: There is no requirement that a project site look like the main foundation site. Pick any project. Say, http://flume.apache.org/ or http://cloudstack.apache.org/ or http://etch.apache.org/ - each has their own unique feel. And, frankly, at this point in time, I think that basing our design after the http://apache.org/ design is not at all desirable, as that site has a strong feeling of nostalgia too. The best TLP site I've ever seen at Apache is CouchDB's http://couchdb.apache.org/ I think it meets all the functional requirements that people have been mentioning in this thread while at the same not looking like something someone made in 1999. I believe the only thing the Javascript does on this page is make it so when you click on an anchor in the same page it smoothly scrolls to the anchor. Without CSS and in Lynx it seems perfectly reasonable. So if you're looking for something to emulate. It seems like a good one to emulate.
Re: ApacheCon 2014 question
On 1/16/14, 9:57 AM, Jeff Trawick wrote: The conference announcement has a very vague reference to hackathons and barcamp; we'll need to find out when space is set aside for these activities. AFAIK this has not yet been communicated internally. (Hackathons usually take place on the same day(s) as training, and training is scheduled for April 10.) If it's anything like last year the hackathon is just a big room where people gather. If a project has enough people saying they'll be going they reserve a table for that project with a card on the table saying the project name so everyone knows where to gather. So if we'd like to get together it's typically up to us to plan this. BTW, who is hoping/planning to be in Denver for a hackathon? I'm anticipating being there.
Re: ApacheCon 2014 question
On 1/16/14, 10:07 AM, Eric Covener wrote: Me. I modified this existing topic in the wiki if anyone wants to track there or propose anything. https://wiki.apache.org/httpd/Face2Face If someone can give me (BenReser) edit rights I'll add myself.
Re: [VOTE] obscuring (or not) commit logs/CHANGES for fixes to vulnerabilities
On 1/14/14, 7:35 AM, Jeff Trawick wrote: The simple answer to all of this is look how httpd releases with security fixes have been handled in the past. The RM commits the fixes just before Tag Roll and, depending on the impact of the vulnerabilities, may call for an abbreviated testing schedule. That really doesn't answer any of the questions I just asked. If I thought the existing process handled all of this I wouldn't have asked the questions. This is back to choice 1 or choice 2, and whether or not you think that showing the code gives a would-be attacker useful information. Committing the code always gives a would-be attacker useful information, I don't dispute that. The distinction is that committing a CVE number with a security fix gives them MORE information. It creates the opportunity to automate detection of fixes that can be analyzed and exploited before we can get fixes in the hands of our users. If you don't narrow the gap between that commit and putting the fix in a usable state to end users then you've harmed our users rather than helped them in my opinion. There will always be some sort of gap, that's just the nature of an open source project. And less critical issues can afford a larger gap than more critical issues. However, that gap should be no more than a few days in my opinion. Certainly not weeks or months. The vote is about reaffirming support for and documenting a long-standing practice around commit/disclosure which has been followed in the vast majority of cases and which most of us feel should always be followed. It is not about inventing completely new procedures to deal with vulnerabilities. My observation has been that we haven't been doing that. I decided to go back and look at what we've been doing. I only bothered to look at 2.4.x (which I realize is not a huge sample). Here's how we've done with 2.4.x so far: CVE-2013-1896 trunk revision: 1485668 (2013-05-23) CVE in commit: no CVE (even now) comments: mentions segfault release with fix: 2013-07-22 CVE-2013-2249 trunk revision: 1488158 (2013-05-31) CVE in commit: no CVE (even now) comments: doesn't mention security impact at all release with fix: 2013-07-22 CVE-2012-3499 trunk revision: 1413732 (2012-11-26) 1418752 (2012-12-08) CVE in commit: no CVE (even now) comments: only says missing html escaping release with fix: 2013-02-25 CVE-2012-4558 trunk revision: 1413732 (2012-11-26) CVE in commit: no CVE (even now) comments: only says missing html escaping release with fix: 2013-02-25 CVE-2012-3502 trunk revision: 1373955 (2012-08-16) CVE in commit: no CVE original version: http://mail-archives.apache.org/mod_mbox/httpd-cvs/201208.mbox/%3C20120816175451.51AC32388900%40eris.apache.org%3E has been updated since to include CVE comments: no mention of security impact in initial commit, subsequent update is good release with fix: 2012-08-21 CVE-2012-2687 trunk revision: 1349905 (2012-06-13) CVE in commit: CVE in initial commit comments: Good explanation of security issue release with fix: 2012-08-21 CVE-2012-0883 trunk revision: 1296428 (2012-03-02) CVE in commit: CVE in initial commit comments: Explanation done in initial commit release with fix: 2012-04-17 Now the last couple actually do commit with a CVE number and an exmplation, so maybe I'm wrong and this has been a long standing practice. From the limited data of 2.4.x it looks like something changed (or maybe it's just some people do things one way and another group does it another). To that end I applaud your effort to standardize what's being done. Maybe the people committing obscured logs were following this page: http://www.apache.org/security/committers.html But up till now I've been approaching this as a change in policy since that's how it appeared to me based on the above. For this small aspect of the overall policy that is being voted on, implementation is as simple as having the security team determine whether or not a vulnerability can be fully disclosed prior to the time a release is prepared. If it can, commit away. If not, wait for TagRoll. I acknowledged that my questions went beyond the simple question in your vote. The goal as I understand it is to avoid security by obscurity and to put our users on a equal footing to potential attackers reading our source commits. I don't think ever committing with some sort of security issue explained in the commit message and without an advisory and/or release coming out very soon is ever really helpful towards that goal. Our users do not read our commits, probably do not understand what our commit messages mean with respect to impact and may not be able to apply the fix committed (especially if it doesn't easily
Re: How to run the BIND and UPDATE functions in mod_dav.so module of Apache HTTPD server?
On 1/14/14, 12:10 AM, vancaho wrote: How to write these DAV providers? I'm not sure how much more info I could give short of writing it myself. The closest we have to documentation is what's in mod_dav.h. There's a nice HTMLized version of what's in mod_dav.h here: http://ci.apache.org/projects/httpd/trunk/doxygen/group__MOD__DAV.html For BIND you need to implement this (vtable): http://ci.apache.org/projects/httpd/trunk/doxygen/structdav__hooks__binding.html For UPDATE you need to implement the update function on the dav_hooks_vsn struct (vtable): http://ci.apache.org/projects/httpd/trunk/doxygen/structdav__hooks__vsn.html#a58f63b737fe4264e04066cb59f0813bb In the end you're probably going to have to read the source of mod_dav and figure out how to do it as you go. Are there any existing DAV providers or examples? There are only two publicly available DAV providers. mod_dav_svn (Subversion) and mod_dav_fs (included with httpd). Neither of them implement these features. Someone probably could extend mod_dav_fs to support BIND by using hard or symbolic links on the local file system. BIND might be implementable by Subversion by converting it into a svn:special file. Though I don't really see a good reason to do this unless there's a DAV client out there that supports BIND and it's useful to someone when using autoversioning mode. UPDATE can't be implemented by mod_dav_fs without a lot of work, since it's part of the DeltaV protocol which mod_dav_fs doesn't support. Specifically, mod_dav_fs would have to grow a whole versioning backend. UPDATE isn't important to Subversion because we don't allow HEAD to point to anything other than the last checked-in version. I'd assume whoever wrote the hooks to support BIND and UPDATE wrote some sort of code to test it, but I have no idea where that code would even be. To the best of my knowledge it's never been published. Given that it hasn't been used much, it's likely there are bugs that would need fixing if someone did try to use it.
Re: How to run the BIND and UPDATE functions in mod_dav.so module of Apache HTTPD server?
On 1/13/14, 12:29 AM, vancaho wrote: Hi everyone, I'm learning WebDAV protocol with apache httpd server and subversion. Atfer reading the source code of Mod_Dav.so(which is responsible for interpreting the WebDAV protocol), I find that there are methods dav_method_bind(http://www.webdav.org/bind/draft-ietf-webdav-bind-27.html#rfc.section.4.1) and dav_method_update(http://www.webdav.org/specs/rfc3253.html#METHOD_UPDATE) which are used to process the BIND and UPDATE methods. However I find that the subversion and tortoiseSVN do not support BIND or UPDATE methods. Can anybody tell me how to use the BIND and UPDATE medhods in the mod_dav.so module of Apache HTTPD server? Thanks very much! You'll have to write your own DAV provider that implements these methods. BIND requires that you setup the binding entry on the dav_provider structure which is a dav_hooks_binding. UPDATE requires that you setup the update function on the vsn entry (which is a dav_hooks_vsn) on the dav_provider structure. As you've noticed Subversion doesn't implement these since we don't need them. To the best of my knowledge no open source DAV provider actually utilizes these features.
Re: [VOTE] obscuring (or not) commit logs/CHANGES for fixes to vulnerabilities
On 1/11/14, 5:02 AM, Jeff Trawick wrote: I think a lot of your concerns revolve around assessment of when a vulnerability can be disclosed, and that has to be determined on a case by case basis. The vote is just about whether there will be an in-between situation where we share some information we have (the code change) without sharing the rest, vs. deciding that, separate from the timing of how we release information about a vulnerability, we either release all we have (code + impact) or none of it. I guess my point is that there is a lot of process involved in not having that in-between situation. The vast majority of the situations do not present a critical threat. It's a serious effort to take on that task. How do you see the release process working if you delay committing code until we're ready to announce the vulnerability? Are security advisories going to be separated from releases? Will the advisories include patches for released versions? If so does that constitute a release and require a vote? If so how does that vote take place? If we're not putting out advisories when the code is committed, how do we expect users to know about the fixes? If we're not putting out releases with the advisories: For users that are dependent on only using released version (binary packages that don't patch, have policies against patching, etc...) are we committed to limiting the time between that code commit/advisory and a release version? Are we committed to releasing all supported versions at roughly the same time? Are we not concerned that we're increasing the time frame that these users receive the fixes since there is now a gap between us describing the vulnerability and the release being available to package. I know there are fixes that impact security that slip through the gaps. Compare this change (which was handled as a security issue): *) SECURITY: CVE-2013-1896 (cve.mitre.org) mod_dav: Sending a MERGE request against a URI handled by mod_dav_svn with the source href (sent as part of the request body as XML) pointing to a URI that is not configured for DAV will trigger a segfault. [Ben Reser ben reser.org] vs this change (which was not): *) mod_dav: When a PROPPATCH attempts to remove a non-existent dead property on a resource for which there is no dead property in the same namespace httpd segfaults. PR 52559 [Diego Santa Cruz diego.santaCruz spinetix.com] What happens when someone commits a fix for something like this without considering the possible security implications? Is someone going to immediately update the log message to point out the security impact? Given that the issues are usually relatively minor and that they've usually existed for a long time I'm not sure that the level of effort such a change would constitute a positive change. I'm all for keeping things as confidential as can be done, but I don't think that you can make a policy decision like this without considering all the support for the decision. If we can't resolve some of these issues, the policy may have a negative effect on the security of our users. I suspect there are very few people interested in monitoring all our commits for security issues. The signal to noise ratio is too low to justify it. However, once you start including CVE numbers in every issue immediately upon commit then it will be very easy to monitor for these issues. That creates the opportunity for exploitation that may not have existed before. As such I don't think given what discussion has gone on so far that I'm comfortable voting for the mandatory option. I could be convinced to vote for it with the effort to make the decisions necessary to support it. But based on the votes cast so far I guess my vote really won't matter. So consider this email as asking how you plan to implement your policy.
Re: [VOTE] obscuring (or not) commit logs/CHANGES for fixes to vulnerabilities
On 1/10/14, 5:38 AM, Jeff Trawick wrote: [ ] It is an accepted practice (but not required) to obscure or omit the vulnerability impact in CHANGES or commit log information when committing fixes for vulnerabilities to any branch. [ ] It is mandatory to provide best available description and any available tracking information when committing fixes for vulnerabilities to any branch, delaying committing of the fix if the information shouldn't be provided yet. [ ] ___ (fill in the blank) The Subversion project has struggled with this same issue. To the degree that there has actually been a fair amount of thought into how to avoid doing security by obscurity. The processes we've discussed have varied from executing the release entirely hidden from anyone but the PMC to simply publishing an advisory with a patch right before committing to trunk (treating that advisory with patch as a release with appropriate voting handled by PMC members privately). You're always dealing with a certain amount of security by obscurity. The bugs we find often have existed for a long time. If one person can find it someone else certainly could. For all we know the issues may have already been found and only exploited in limited ways such that the issue was never reported. Even with advance notification I've found that the binary packagers can take their sweet time getting security fixes included. Some binary packagers don't really have a process that supports patching (they release one package for each version without a method of identify versions that have been patched). Administrators may not always know what to do with patches. So frankly all of the processes stink. Yet, I think that the best process is to reveal security issues when you can put your best foot forward and have things positioned to get the fixes in the hands of as many users as possible as quickly as possible. I think that's best served by withholding details (even if you're doing so imperfectly) until release or the issue is widely disclosed to the public. It should be noted that not all security issues are equal. For the most part highly critical fixes are rare, when they do come up we could use a release process that hides everything from non-PMC members until release time frame. With other less severe issues possibly just disclosing immediately when we apply the fix. There doesn't need to be a one size fits all answer to this. But I certainly would like to see us have a consistent policy for determining which process to follow. So my vote wouldn't really fit into the options presented above. I'd suggest coming up with a process for varying levels of issues and criteria to determine which process to follow.
Security Advisories
So I've received at least two people asking me for more details about CVE-2013-1896. I thinking it might be better to provide more than a couple sentences on the issues. It can be hard to understand the impact of an issue from what we're providing now.
mod_rewrite and mod_dav_svn
We've recently made a change to mod_dav_svn to start implementing translate_name and map_to_storage hooks in order to prevent r-filename from being set to a bogus path since Subversion is servering content that isn't actually accessible via the standard file I/O APIs... You can see the reasoning for that here: https://subversion.apache.org/docs/release-notes/1.8.html#mod_dav_svn-fsmap However, this has had the side effect of creating problems for people who use mod_rewrite. See this email to the d...@subversion.apache.org list for an example: https://mail-archives.apache.org/mod_mbox/subversion-dev/201312.mbox/%3C76026FDD-FF7C-4C71-84EC-3A73F12B6F34%40simonsoft.se%3E Ultimately, what's happening here is that mod_rewrite is changing the r-uri and that results in mod_dav_svn's configuration being inaccurate for what would actually be served, mod_dav_svn continues to believe it should rewrite the path and hook the map_to_storage to prevent the ap_core_translate() from running and thus producing an r-filename based off the configured DocumentRoot and r-uri. This results in the mod_rewrite modification of r-uri to something mod_dav_svn doesn't serve to result in a 404. We could take and ignore translate_name calls that have been messed with by mod_rewrite like the patch in this email: https://mail-archives.apache.org/mod_mbox/subversion-dev/201312.mbox/%3c52a8b2bf.2000...@reser.org%3E The problem here is that mod_rewrite could change the URI to anything, including a path that mod_dav_svn is responsible for or a path that it isn't. With that patch in place then rewritten paths that mod_dav_svn does serve are back to original behavior that we made the change to avoid. I can work around this by triggering a ap_location_walk() and ap_if_walk() in mod_dav_svn's translate_name hook before retrieving my configuration as I did in the patch on this email: https://mail-archives.apache.org/mod_mbox/subversion-dev/201312.mbox/%3c52a8d7ae.5040...@reser.org%3E However, I'm messing with httpd internals that I don't think I should be and it also seems like this fix really belongs in mod_rewrite. Probably by way of an API call to update the per_dir_config so the module doesn't have to be kept in sync with whatever is in ap_process_request_internal(). Before anyone says but you shouldn't be using the per dir config from a translate_name hook. I'll point out that mod_rewrite itself is accessing its per directory configuration. Several other modules are accessing per directory configuration as well (mod_proxy, mod_proxy_express, and mod_log_debug). I'm not sure if this presents an issue for them since I haven't spent a lot of time looking into their behavior. I can say that they are all hooking translate_name with APR_HOOK_FIRST and only mod_proxy is configured to always run after mod_rewrite. I don't seem to have any better place to put this since I need to prevent the directory_walk that happens in core_map_to_storage. The new post_perdir_config might seem like a good place but it can't prevent a request from being subject to directory access control. Thoughts?
Re: Do pools lead to bad programming?
On 12/11/13 4:00 PM, Kean Johnston wrote: Am I being too obsessive? If not, would you like patches to correct these as I find them, and if so, should I open a bug about this or just post patches here (they are all likely to be a simple move of 1 or 2 lines)? There are two ways this sort of thing can happen. The person who wrote the original code didn't feel like breaking the declaration of the variable and the allocation into two lines. Which is a mistake. Alternatively, the allocation could have always been necessary and the only difference was the ordering when it happened. I.E. there may have not been any returns before the use of the variable. This would be an error on the part of the person changing the code. Looking at this specific case, the variable was added in r357444 as part of the original template of mod_proxy_fcgi. So it seems like an error on the original developers part. It certainly seems worthwhile to clean these up when they are found in my opinion. I wouldn't say that pools lead to bad programming. It's mostly that pools limit the consequences of these sorts of mistakes. The mistakes are going to happen regardless.
Re: [VOTE] Release Apache httpd 2.4.7 as GA
On 11/19/13 9:45 AM, Jim Jagielski wrote: I'm calling a VOTE on releasing these as Apache httpd 2.4.7 GA. [ ] +1: Good to go [ ] +0: meh [ ] -1: Danger Will Robinson. And why. +1: OS X 10.7.5 and Ubuntu 12.04: Subversion test suite passes with trunk, 1.8 and 1.7 using Subversion's HTTPv1 and HTTPv2 protocols.
Re: [VOTE] Release Apache httpd 2.2.26 as GA
On 11/13/13 9:03 AM, Jim Jagielski wrote: The pre-release test tarballs for Apache httpd 2.2.26 can be found at the usual place: http://httpd.apache.org/dev/dist/ I'm calling a VOTE on releasing these as Apache httpd 2.2.26 GA. [ ] +1: Good to go [ ] +0: meh [ ] -1: Danger Will Robinson. And why. +1: OS X 10.7.5 (x86_64), Ubuntu 12.04 (x86_64) Passes Subversion test suite with 1.7, 1.8 and trunk using both our HTTPv1 and HTTPv2 protocols.
Re: NOTE: Intent to TR 2.2.6 tomorrow
On Tue Nov 12 11:25:57 2013, Jim Jagielski wrote: Oh yeah... I recall you had an issue with me building because of potential issues with using a later, but still 100% valid autoconf/libtool setup. I am not going to downgrade just to build 2.2 so if that is *really* a concern, backed-up by the PMC, then someone else will need to RM. You don't need to downgrade at all. Just build autoconf/libtool versions and install them in another dir. Add that to the front of your path and off you go. Subversion includes the ability to download and build the autoconf/libtool versions we use as part of our build-env command to release.py which ensures we have consistent versions no matter who does the release. Maybe we should look at doing something like this: https://subversion.apache.org/docs/community-guide/releasing.html#before-release
Timetable for 2.2.26 release?
Does anyone have a timetable for a 2.2.26 release? Subversion users have been running into the issues introduced in 2.2.25 and we've been pointing them at patches. Some but not all of the binaries floating around have been patched. It would be really nice to be able to have a httpd release to point them at. For instance this guy popped up yesterday: https://mail-archives.apache.org/mod_mbox/subversion-users/201311.mbox/%3C4A2A8BDDBE6EFE4199FB22121DB971B1A1B07E6B%40AAPLExchange.aapl.com.au%3E In the meantime I've posted this thread on the us...@subversion.apache.org list in an effort to help users: https://mail-archives.apache.org/mod_mbox/subversion-users/201311.mbox/%3C527C7AC3.5050906%40reser.org%3E The PR 55306 and PR 55397 changes that are accepted in the 2.2.x branch STATUS file are the fixes for these issues. I know that 2.4.7 is held up by waiting on APR 1.5 but the vast majority of Subversion users are still using 2.2.x. So in the meantime it'd be really nice if we could get the 2.2.x release done. I'd be happy to take on some of the work myself if someone can point me in the right directions.
Re: svn commit: r1533810 - in /httpd/httpd/trunk: docs/man/rotatelogs.8 docs/manual/programs/rotatelogs.html.en support/rotatelogs.c
On 10/20/13 5:31 AM, Ruediger Pluem wrote: Can't we use apr_psprintf with %pm instead of the constant length buffer char [120]? Done in r1534895, r1534896 and r1534914.
Re: svn commit: r1533810 - in /httpd/httpd/trunk: docs/man/rotatelogs.8 docs/manual/programs/rotatelogs.html.en support/rotatelogs.c
On 10/20/13 5:31 AM, Ruediger Pluem wrote:
apr_pool_create(newlog.pool, status-pool);
+if (config-create_path) {
+char *ptr = strrchr(newlog.name, '/');
+if (ptr ptr newlog.name) {
+char *path = apr_pstrmemdup(newlog.pool, newlog.name, ptr -
newlog.name);
+if (config-verbose) {
+fprintf(stderr, Creating directory tree %s\n, path);
+}
+rv = apr_dir_make_recursive(path, APR_FPROT_OS_DEFAULT,
newlog.pool);
+if (rv != APR_SUCCESS) {
+char error[120];
+
+apr_strerror(rv, error, sizeof error);
+fprintf(stderr, Could not create directory '%s' (%s)\n,
path, error);
Can't we use apr_psprintf with %pm instead of the constant length buffer
char [120]?
Only if we require APR 1.3.x or newer since %pm wasn't added till 1.3.0:
https://apr.apache.org/docs/apr/1.4/group__apr__lib.html#gad2cd3594aeaafd45931d1034965f48c1
The implementation I used here is similar to out error handling already in the
file.
Re: svn commit: r1533810 - in /httpd/httpd/trunk: docs/man/rotatelogs.8 docs/manual/programs/rotatelogs.html.en support/rotatelogs.c
On 10/20/13 5:37 AM, André Malo wrote: * bre...@apache.org wrote: Author: breser Date: Sat Oct 19 19:10:33 2013 New Revision: 1533810 * docs/man/rotatelogs.8, docs/manual/programs/rotatelogs.html.en: Update for -d option. Huh. These files are both generated (or should be). Please update rotatelogs.xml instead. Oops thanks fixed in r1533935. Why are the generated files committed though?
Re: svn commit: r1533810 - in /httpd/httpd/trunk: docs/man/rotatelogs.8 docs/manual/programs/rotatelogs.html.en support/rotatelogs.c
On 10/20/13 11:23 AM, Jeff Trawick wrote: trunk and 2.4.x branch: checking for APR... configure: WARNING: APR version 1.4.0 or later is required, found 1.3.13 configure: WARNING: Found APR in /home/trawick/inst/apr13-64/bin/apr-1-config, but we think it is considered unacceptable configure: error: the --with-apr parameter is incorrect. It must specify an install prefix, a build directory, or an apr-config file. configure failed (I always forget about %pm :( ) Thanks I went look for that a bit and didn't find it. So yes I guess we could switch to %pm.
Re: svn commit: r1531670 - /httpd/httpd/branches/2.2.x/STATUS
On 10/13/13 4:51 AM, minf...@apache.org wrote: Author: minfrin Date: Sun Oct 13 11:51:03 2013 New Revision: 1531670 URL: http://svn.apache.org/r1531670 Log: Vote, comment. Modified: httpd/httpd/branches/2.2.x/STATUS Modified: httpd/httpd/branches/2.2.x/STATUS URL: http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/STATUS?rev=1531670r1=1531669r2=1531670view=diff == --- httpd/httpd/branches/2.2.x/STATUS (original) +++ httpd/httpd/branches/2.2.x/STATUS Sun Oct 13 11:51:03 2013 @@ -126,7 +126,7 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK: * mod_dav: Fix 55306. Don't require lock tokens for COPY source. trunk patches: https://svn.apache.org/r1528718 2.2.x: trunk works, CHANGES needs to be written when merging -+1: breser ++1: breser, minfrin * mod_dav: Fix 55397. dav_resource-uri treated as unencoded. This was an unnecessary ABI changed introduced in 2.2.25 @@ -135,6 +135,7 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK: 2.4.x: trunk works, CHANGES needs to be written when merging 2.2.x: https://people.apache.org/~breser/httpd/2.2.x/patches/pr55397.patch +1: breser +minfrin: r1529559 puts modules/dav/fs/repos.c into conflict for me That's why there's a patch file for 2.2.x. There are some minor conflicts from some code cleanup that happened between 2.2.x and 2.4.x.
Re: svn commit: r1531559 - /httpd/httpd/branches/2.4.x/STATUS
On 10/12/13 2:19 PM, Graham Leggett wrote: A quick reminder, these fixes had been back ported to v2.2 as well, would it be possible to propose them there too? Yes, there are some minor conflicts. I'm finishing up testing them with 2.2.x, though I'm about to catch a plane to London, so I might not nominate them today.
Re: will anyone build httpd/apr with cmake on Windows?
On 8/30/13 5:25 AM, Jeff Trawick wrote: I will be throwing a bit more time at the cmake effort in the short term, starting with comparing the installed artifacts with those of existing Windows builds and adding missing pieces to the todo lists. I might not do much else proactively until my own use of the builds catches up and starts to exceed what is implemented. I'd be interested in using it, but haven't gotten around to looking at it yet. I have a perl script that builds httpd along with a bunch of other things that Subversion needs on Windows. Without a doubt the biggest pain in doing so has always been httpd and its dependencies. The existing project file method has the following issues for me: * Not all versions of VC have good support for converting projects from the command line, some of them fail with cryptic errors. Fortunately VC2012 finally does this fairly well... But, the Express version won't convert at all from the command line. This means it's almost impossible to fully script the build. * There are a lot of things that have to be done for fixing the existing project files to work properly with a current tool chain on Windows. Fixing them in the GUI isn't too hard, but it's a manual process. I'm not 100% sure the hacks I've written to modify the files in place are really correct. The build falls apart all over the place due to shared intermediary directories and I've played whack-a-mole fixing it because what fails is random. Beyond time the biggest barrier for me messing with this is that I primarily care about release builds on Windows. I only really build Subversion on Windows (even that is pretty sporadic) and I don't ever hack on things over there, just simply test releases.
Re: mod_autoindex string pluggability
On Mon, Aug 5, 2013 at 5:51 AM, Tim Bannister is...@jellybaby.net wrote: How about implementing XHTML → JSON as a filter? Either with existing modules or with something dedicated to autoindex. That sounds really ugly if you ask me. For one thing he's trying to avoid parsing XHMTL so now you're suggesting that he write an XHTML parser in C. If it was possible to generalize it so that it was useful to other things than just mod_autoindex then that might be different. But I suspect it'll have to be fairly complex to achieve this.
Re: Resolved (sort of): Struggling with AuthMerging
On Fri, Aug 2, 2013 at 8:24 PM, Mikhail T. mi+t...@aldan.algebra.com wrote: The modules in your examples deliberately use the authz mechanism to generate different output based on the results. But what is doing it in the case I describe -- where the generated content is exactly the same? Obviously nothing is doing a subrequest to validate the auth, they're doing sub-requests to fulfill the original request. httpd is doing the auth because it's the right thing to do from a security perspective. The generated content is not the same. The main request has no generated content it's essentially a redirect, it just happens to be handled internally. Why would anyone want to provide distinct authorization rules to /foo/ and /foo/index.html, for example? What is a possible use-case? They are different paths they may have different auth rules. httpd can't assume that it'll be okay. Keep reading and I'll explain why. Likewise, why would anyone configure mod_fastcgi to hand off processing of PHP-files to FPM: FastCGIExternalServer /usr/sbin/php-fpm -socket /var/run/fpm/main.sock AddHandler php-fastcgi .php Action php-fastcgi /php-fpm ScriptAlias /php-fpm/usr/sbin/php-fpm but need the /php-fpm/foo/ to have authorization rules different from those of /foo/? You don't seriously expect the auth system to know all of those intricacies? It has no idea why the subrequest is being made it just knows there's a request to process. Even if it did know why the subrequest is being made the variety of authz systems doesn't allow you to make assumptions like the above. In 2.4.x there's the new provider architecture that at least lets you optimize away subrequests that have the same conf. But the modules have to say it's ok because if the modules are handling path based matching on their own then even httpd has no idea if /foo and /foo/index.html have the same configuration. mod_authz_svn is an example of such a module that could not use the CONF flag (but at this time it isn't using the new provider system and is still using the old hooks system). given that there are authz modules like this you can't even make the argument that we can ever assume that authz isn't necessary If all of your authz/authn providers are using the CONF flag and you're getting duplicated authz processing for subrequests that have the same conf applied to them then it's possible there's a bug here. I haven't ever specifically looked at this code so I can't tell you how it works, all I can do is tell you what the documentation says it does. Unfortunately I have other priorities right now so I don't really have the time to go down that rabbit hole and find out if there is a bug that's causing extra authz calls. Any attempt to optimize when authn/authz happens on subrequests is going to have to be done very carefully since you really can't make any assumptions about what should be allowed. I realize that in your specific case that seems absurd but the httpd core code supports use cases that are very different from yours.
Re: Resolved (sort of): Struggling with AuthMerging
On Sat, Aug 3, 2013 at 11:34 AM, Mikhail T. mi+t...@aldan.algebra.com wrote: Point is, it is erring. I asked Ben for possible use-cases and his two examples were modules, which use the authorization rules to generate different content depending on the result. Rather than to decide, whether to authorize the request at all. I think you're jumping to massive conclusions about my ability to answer questions. While I've been an httpd user for a relatively long time I've only had httpd commit access for all of a week. I wasn't involved in writing the auth code that exists and I may not have provided you the best examples. Just because I haven't provided better examples doesn't mean they exist. I gave you the examples that came to mind. First and foremost the primary concern of security code is always correctly implementing the security model, not running as fast as possible. If you can make it run faster without breaking the model, then I'm sure your patch would be accepted. For what it's worth I can't replicate your issue. I tried replicating just the duplicate authz check against the directory and then against the DirectoryIndex. I end up getting exactly one authz process: [Sat Aug 03 11:48:43.170081 2013] [authz_core:debug] [pid 48038:tid 4316540928] mod_authz_core.c(845): [client 127.0.0.1:63874] AH01626: authorization result of Require all granted: granted [Sat Aug 03 11:48:43.170096 2013] [authz_core:debug] [pid 48038:tid 4316540928] mod_authz_core.c(845): [client 127.0.0.1:63874] AH01626: authorization result of RequireAny: granted [Sat Aug 03 11:48:43.170102 2013] [core:trace3] [pid 48038:tid 4316540928] request.c(238): [client 127.0.0.1:63874] request authorized without authentication by access_checker_ex hook: / Can you produce a minimal example configuration that reproduces the issue. Let's avoid the PHP scripts for now. The situation would've made some sense, if I could configure things separately. For example: # Lock-out attempts to invoke php-fpm directly: Location /php-fpm Require none granted /Location # Allow any PHP script under DocumentRoot to be executed otherwise: LocationMatch \.php$ Require all granted /LocationMatch But I can't -- all requests for foo.php would go through both of the above... Have you actually tried this? Because it does work. The only problem you'll run into is that the main request for the /php-fpm/ URI will fail before the DirectoryIndex applies. For instance I have something that looks like this: Directory / Require all denied /Directory Directory /Users/breser/httpd-2.4.6-root/htdocs DirectoryIndex index.html Require all granted /Directory Location / Require all denied /Location LocationMatch \.html$ Require all granted /LocationMatch I can hit /index.html but not index.txt. I can't get / and the directory index doesn't work, but I can easily fix that by doing changing my LocationMatch to: (^/$)|(\.html$) If I change DirectoryIndex to index.txt it'll break because I've disallowed that. Note that you've actually just shown an example of why sub-requests should check security. Side note: It'd be Require all denied to block all access.
Re: Resolved (sort of): Struggling with AuthMerging
On Thu, Aug 1, 2013 at 7:54 PM, Mikhail T. mi+t...@aldan.algebra.com wrote: 01.08.2013 22:47, Ben Reser написав(ла): That's not a bug at all. In some cases it may be necessary for authorization to run for sub-requests. Could you give an example or two? Thanks, Sure. mod_autoindex automatically provides a directory listing of files under a path. However, by default it doesn't display any paths that you don't have access to, e.g. .htaccess. It does this by issuing subrequests for those other paths so that authz can run on them. (This behavior could be changed with IndexOptions ShowForbidden). mod_dav_svn. Numerous commands in SVN impact other paths than the URI (e.g. `svn list` which is similar to the autoindex case above, commiting a copy or move which touch two paths one of which is on in the URI but rather in the headers). mod_dav_svn issues sub requests for these other paths to find out if the actions are allowed on them. Because of the performance impact of this mod_dav_svn has the SVNPathAuthz directive to change this behavior. One of the possible values is short_circuit which causes mod_dav_svn to talk directly to mod_authz_svn rather than doing a subrequest. You might ask why don't we just do this by default? The reason being is if there are other authorization limits placed from other modules then they would no longer respected.
Re: Struggling with AuthMerging
On Wed, Jul 31, 2013 at 8:02 AM, Mikhail T. mi+t...@aldan.algebra.com wrote: As a minimum, testing the subsequent children of RequireAll after one of them already responded with denied seems like a bug... I'm not sure about the AuthMerging but I can say that trying the tiv expiration is not a bug. First of all you should understand that authz providers are called TWICE. The general process for authz handling is as such: 1) authz providers are called before authentication is processed to check for anonymous access (i.e. r-user == NULL). If the resulting response is AUTHZ_GRANTED then access is provided and no other authn/authz processing occurs. If the resulting response is AUTHZ_DENIED or AUTHZ_NEUTRAL then the request is provided with HTTP_FORBIDDEN. If the resulting response is AUTHZ_DENIED_NO_USER then processing continues. 2) authentication is processed. r-user gets set. Errors out here if the user can't authenticate. 3) authz providers are called this time with r-user set. If the result is AUTHZ_GRANTED then access is granted and http starts the work to server the request. If AUTHZ_DENIED_USER HTTP_UNAUTHORIZED is returned. If AUTHZ_DENIED_ or AUTHZ_NEUTRAL then the response is HTTP_FORBIDDEN. Your log shows: authorization result of Require tiv ipaddress: denied (no authenticated user yet) This only happens when your module returns AUTHZ_DENIED_NO_USER. If you return this value httpd must call into each module since one of your RequireAll directives might be able to return AUTHZ_DENIED and then we would need to deny the request without going through the authn processing or the second authz pass. A good way of thinking of this is that with RequireAll and RequireNone the priority of response codes are: AUTHZ_GENERAL_ERROR (immediate return) AUTHZ_DENIED (short-circuits other siblings and continues with processing) AUTHZ_DENIED_NO_USER AUTHZ_GRANTED AUTHZ_NEUTRAL RequireAny: AUTHZ_GENERAL_ERROR (immediate return) AUTHZ_GRANTED (short-circuits other siblings and continues with processing) AUTHZ_DENIED_NO_USER AUTHZ_DENIED AUTHZ_NEUTRAL
Re: Resolved (sort of): Struggling with AuthMerging
On Thu, Aug 1, 2013 at 6:55 PM, Mikhail T. mi+t...@aldan.algebra.com wrote: The problem -- and I do think, it is a bug -- is that the entire authorization is invoked not just for the request, but for the internal subrequests too. So, in my scenario, when I asked for /tiv/, the authz core made the following checks (color-coded to match the above-quoted log-entries): Check location /tiv/ -- granted, no problem. Check location /tiv/index.php -- granted as well, so far so good. We use mod_actions to hand-off processing of php-files to php-fpm, so Apache also checks location /php-fpm/tiv/index.php... Because there is no explicit sublocation defined for /php-fpm/, the rules for the Location / are invoked. Which leads to our tiv being queried -- denying the request... That's not a bug at all. In some cases it may be necessary for authorization to run for sub-requests. If you're using the ap_register_auth_provider() system then you have some control over that by using AP_AUTH_INTERNAL_PER_CONF. I got things to work by changing the sub-location configuration to read simply: LocationMatch ^(/php-fpm)?/tiv/ Require all granted DirectoryIndex index.php /LocationMatch I'm guessing you're using AP_AUTH_INTERNAL_PER_CONF which should avoid the 3rd call with the above configuration. There is no need for AuthMerging even, after all. But I struggle to understand, why the same HTTP-hit results in multiple processing of the same authorization rules (some of which may, actually, be heavy...). Is this really normal and expected behavior? Yes. In fact there are cases where subrequests are explicitly used to find out if a path is authorized. e.g. the -F and -U tests for RewriteCond: http://httpd.apache.org/docs/current/mod/mod_rewrite.html#rewritecond In fact it used to be worse in 2.2.x because AP_AUTH_INTERNAL_PER_URI was the old behavior. See this following text from http://httpd.apache.org/docs/2.4/developer/new_api_2_4.html [[[ When possible, registering all access control hooks (including authentication and authorization hooks) using AP_AUTH_INTERNAL_PER_CONF is recommended. If all modules' access control hooks are registered with this flag, then whenever the server handles an internal sub-request that matches the same set of access control configuration directives as the initial request (which is the common case), it can avoid invoking the access control hooks another time. If your module requires the old behavior and must perform access control checks on every sub-request with a different URI from the initial request, even if that URI matches the same set of access control configuration directives, then use AP_AUTH_INTERNAL_PER_URI. ]]] 01.08.2013 21:05, Ben Reser wrote: If the resulting response is AUTHZ_DENIED_NO_USER then processing continues. Is that so that if any of the subsequent children of the same RequireAll say AUTHZ_DENIED, the server will not even bother figuring out the user? Ok, makes sense, thank you. Turns out, this is not related to the main problem, after all. Right that is how it would behave.
[PATCH PR55304] mod_dav: COPY should not validate the parent of request.
This patch fixes a regression created by the PR54610. COPY does not modify the parent of the source, so it should not be validating the parent. This issue actually disallows the ability to COPY the root of a DAV repository since a properly implemented DAV provider will return NULL and dav_method_copymove() will error on that. We ran into this with Subversion, which actually revealed a security issue with our implementation of get_parent_resource() since it failed on the root. But beyond that we realized we were not properly returning NULL for some resources when the resource is the root and thus has no parent. If we fix this without this patch being made to mod_dav then HTTP 2.2.25 and 2.4.6 will lose the ability to COPY the root. If someone can apply it that would be appreciated. It's certainly been looked at by several eyes over on the Subversion side.
Re: apache process ps -aux
On Wed, Jul 10, 2013 at 8:25 AM, peter_bateman jrweisb...@gmail.com wrote: I just haven't seen the apache processes listing with the -k start option on any of my other servers, and wasn't sure why it was being displayed here... If you've been using a platform where the ps command doesn't list the command arguments you might not have even though it's still there.
Re: [VOTE] The 'RM' Baton
On Wed, Jul 10, 2013 at 3:30 PM, Guenter Knauf fua...@apache.org wrote: I was also thinking about learning how to release - but the lack of proper documentation for the whole process holds me back; I remember how Graham fell from one trap into another when he did his 1st APR release, and I dont want to get same ... so, if we want to have more folks doing more frequently releases the startpoint would be to 1st document how a release is done: - required auto* tools versions - step by step description how to prepare for a release You guys may want to look at our release process over at the Subversion project: http://subversion.apache.org/docs/community-guide/releasing.html#release-creating We also have it somewhat automated. Including getting the right auto* tools versions. Of particular interest might be our release.py tool: http://svn.apache.org/repos/asf/subversion/trunk/tools/dist/release.py For actually rolling that python script ends up driving this shell script that I wrote long before the python script was developed: http://svn.apache.org/repos/asf/subversion/trunk/tools/dist/dist.sh All of this is obviously under ASLv2 so feel free to borrow and adapt. :)
Re: apr_atomic functions usage
On Mon, May 27, 2013 at 8:42 PM, kalyan sita kalyansit...@gmail.com wrote: I see that the below functions have specific assembly implementations for os32,ia32 architectures: apr_atomic_add32 apr_atomic_sub32 apr_atomic_inc32 apr_atomic_dec32 apr_atomic_set32 apr_atomic_cas32 apr_atomic_casptr apr_atomic_xchg32 apr_atomic_xchgptr How frequently are these functions used. I am planning to write arm specific code for the above functions in arm.c file. Can anyone help me where to start ? You really should direct this at the d...@apr.apache.org list since it's not part of httpd (granted there's a lot of overlap between the projects). These are only used if the compiler doesn't provide the atomic builtins: http://gcc.gnu.org/onlinedocs/gcc-4.8.0/gcc/_005f_005fsync-Builtins.html#_005f_005fsync-Builtins If HAVE_ATOMIC_BUILTINS is true in include/arch/unix/apr_private.h then you don't need the ASM versions. One of the Linux Kernel hackers (Jon Masters) has a blog post up about ARM atomic operations: http://www.jonmasters.org/blog/2012/11/13/arm-atomic-operations/
Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))
On Fri, May 24, 2013 at 8:13 AM, William A. Rowe Jr. wr...@rowe-clan.net wrote: That fortunately is documented, with some pretty good notes in the wiki as well that aught to percolate into the docs. That said, documenting every Microsoft-version-quirk seems out of scope for a general purpose 'compiling' doc. The build system should be able to compile with the major tool chains, nobody expects to know how to work around weird autoconf, make, gcc, etc quirks on Linux. I don't say this to be dismissive of anyones contributions but just to point out that producing Windows builds with a modern toolchain is not simple. I did a bunch of work on scripting building the dependencies for Subversion on Windows that's located here: http://svn.apache.org/repos/asf/subversion/trunk/tools/dev/build-svn-deps-win.pl By far httpd was the biggest pain of any of the dependencies to get to work. If your only interest is building httpd on Windows with Visual Studio 2012, taking a look at build_httpd() in that file should give a good starting point. Sometime when I find the time I want to fix the problems that I had to work around the right way and not the hackish way I did in that script and submit them back. But I haven't gotten to it. I'll admit that I haven't tried to build httpd-trunk on Windows so maybe improvements have been made that haven't made their way over to the 2.4 releases, Subversion certainly has similar issues with our 1.7 release branch.
Re: Whither Windows (Was: Re: Intent to revert commit r1332643)
On Fri, May 24, 2013 at 8:23 AM, William A. Rowe Jr. wr...@rowe-clan.net wrote: Another question is where exactly do we stand with OS/X right now? Apple HFS+ is still not supported, there exists a forced lower-case canonicalization hack authored by Apple, but AFAICT still no progress on retrieving the true name of a file on a case-insensitive OS/X volume which I suspect are still in common use on most OS/X boxen. Of course, all the BSD-based file systems are strict case sensitive and don't have security bypass issues when running 'vanilla' httpd. Can't speak to this, it's not important to my use since I avoid the case sensitivity issues on OS X. I've had no problems building httpd on OS/X on 10.7 (I haven't bothered to upgrade to 10.8). I do a fair amount of work on OS X directly and often build httpd-trunk and releases with debugging. There may be computability issues like you pointed out above, but I quite frankly have never noticed them. Granted my use of httpd on OS X is purely developmental and I likely wouldn't run into the types of issues that someone using httpd for production servers on OS X would. Also wondering where the OS/X download lives? It will build on any OS/X box with a deployed toolchain, but I imagine many OS/X users don't install that toolchain and live with the Apple provided flavors, and would guess that 2.4.x is not part of that Apple OS distributions so far. I've never bothered to try to download a httpd binary build, it's easy enough to build that I don't feel the need. 10.7 still had httpd 2.2, not sure what 2.4 has.
Re: URL scanning by bots
On Wed, May 1, 2013 at 7:16 AM, André Warnier a...@ice-sa.com wrote: If it tries just one URL per server, and walks off if the response takes longer than some pre-determined value, then it all depends on what this value is. If the value is very small, then it will miss a larger proportion of the potential candidates. If the value is larger, then it miss less candidate servers, but it will be able to scan comparatively less servers within the same period of time. The question becomes can they still achieve the number of hacked servers they require in the timeframe they require while ignoring some proportion of the servers on the Internet. I think the answer to that question is clearly yes. The reason for that is the number of poorly updated and secured servers is much higher than the number of secure servers. If you want the scanning to stop a far more productive effort would be to try and get the people running vulnerable systems to secure them. Until this happens there will always be incentive to scan, even if you're tar pitting them on some systems.
Re: URL scanning by bots
On Tue, Apr 30, 2013 at 5:23 PM, André Warnier a...@ice-sa.com wrote: Alternatives : 1) if you were running such a site (which I would still suppose is a minority of the 600 Million websites which exist), you could easily disable the feature. 2) you could instead return a redirect response, to a page saying that one was sold, but look at these. That may be even more friendly to search engines, and to customers. My point isn't that there aren't alternatives, but that 404's are legitimate responses that legitimate users can be expected to receive. As such you'll find it nearly impossible in my opinion to convince people to degrade performance for them as a default. If it isn't a default you're hardly any better off than you are today since it will not be widely deployed. If you want to see a case where server behavior has been tweaked in order to combat miscreants go take a look at SMTP. SMTP is no longer simple, largely because of the various schemes people have undertaken to stop spam. Despite all these schemes, spam still exists and the only effective counters has been: 1) Securing open-relays. 2) Removing the bot-nets that are sending the spam. 3) Ultimately improving the security of the vulnerable systems that are sending the spam. All the effort towards black lists, SPF, domainkeys, etc... has been IMHO a waste of time. At best it has been a temporarily road block.
Re: URL scanning by bots
On Thu, May 2, 2013 at 4:53 PM, Guenter Knauf fua...@apache.org wrote: isnt that one of the core issues - that folks who dont know what they do run a webserver? And then, shouldnt these get punished with being hacked so that they try to learn and finally *know* what they do, and do it right next time? ;-) I have to say this is a horrible attitude. Nobody should be advocating that a lack of knowledge should result in getting hacked. Yes, it's the obvious result of the situation that exists today. But we should be striving to make things as easy for the inexperienced user as possible. Just the other day I saw someone saying pretty much that unless you're an Apache httpd expert you should just use nginx. That if you aren't an expert then you won't have Apache configured properly and when you get a heavy load your website will fall over. It's just disappointing to see this attitude coming from a httpd PMC member.
Re: URL scanning by bots
On Tue, Apr 30, 2013 at 3:03 AM, André Warnier a...@ice-sa.com wrote: Let us imagine for a moment that this suggestion is implemented in the Apache webservers, and is enabled in the default configuration. And let's imagine that after a while, 20% of the Apache webservers deployed on the Internet have this feature enabled, and are now delaying any 404 response by an average of 1000 ms. And let's re-use the numbers above, and redo the calculation. The same botnet of 10,000 bots is thus still scanning 300 Million webservers, each bot scanning 10 servers at a time for 20 URLs per server. Previously, this took about 6000 seconds. However now, instead of an average delay of 10 ms to obtain a 404 response, in 20% of the cases (60 Million webservers) they will experience an average 1000 ms additional delay per URL scanned. This adds (60,000,000 / 10 * 20 URLs * 1000 ms) 120,000,000 seconds to the scan. Divided by 10,000 bots, this is 12,000 additional seconds per bot (roughly 3 1/2 hours). Let's assume that such a feature gets added, however it's not likely going to be the default feature. There are quite a few places that serve a lot of legitimate soft 404s for reasons that I'm not going to bother to get into here. Any site that goes to the trouble of enabling such a feature is probably not going to be a site that is vulnerable to what these scanners are looking for. So if I was a bot writer I'd wait for some amount of time and if I didn't have a response I'd move on. I'd also not just move along with the next scan on your web server, I'd probably just move on to a different host. If nothing else a sever that responds to request slowly is not likely to be interesting to me. As a result I'd say your suggestion if wildly practiced actually helps the scanners rather than hurting them, because they can identify hosts that are unlikely to worth their time scanning with a single request.
Re: URL scanning by bots
On Tue, Apr 30, 2013 at 4:09 PM, André Warnier a...@ice-sa.com wrote: But I have been trying to figure out a real use case, where expecting 404 responses in the course of legitimate applications or website access would be a normal thing to do, and I admit that I haven't been able to think of any. Can you come up with an example where this would really be a user case and where delying 404 responses would really break something ? Imagine you're a real estate agent. You have listings for properties, each one gets a unique URL. You want search engines to index your properties that are for sale. When they sell you want those properties to stop being shown in search engines. So you start returning a 404 for those pages. For a while you might still show something about the property with something saying that it's been sold and some links to other similar properties that aren't sold. Eventually you purge that property entirely and it's a generic 404 page. Clearly in such a scenario you don't want to delay 404 pages. There are of course other examples in other industries. But basically any situation where you were you have pages for things that are temporary you're probably going to want to do something like this.
[PATCH 54145] Improve some error messages in mod_dav
I submitted this patch quite a while ago. Another situation has come up where better error logging would have been nice in this case. Which reminded me that I hadn't see any action on this patch. I know that everyone is busy, but was hoping someone could take some time to look at it. Thanks.
Re: [PATCH 54145] Improve some error messages in mod_dav
Thanks. On Wed, Apr 3, 2013 at 5:23 PM, Jeff Trawick traw...@gmail.com wrote: On Wed, Apr 3, 2013 at 7:56 PM, Ben Reser b...@reser.org wrote: I submitted this patch quite a while ago. Another situation has come up where better error logging would have been nice in this case. Which reminded me that I hadn't see any action on this patch. I know that everyone is busy, but was hoping someone could take some time to look at it. Thanks. SendingCHANGES Sendinginclude/ap_mmn.h Sendingmodules/dav/main/mod_dav.c Sendingmodules/dav/main/mod_dav.h Sendingmodules/dav/main/util.c Transmitting file data . Committed revision 1464241 -- Born in Roswell... married an alien... http://emptyhammock.com/
