Re: Licensing claims (fnmatch)
I still monitor the list, I just don’t post. If I remember correctly, this code came from httpd 1.3. I might have made some small changes, but I am pretty sure this code’s lineage could be traced all the way back to when it was BSD licensed. I say go ahead for my part. Ryan On Thu, Feb 22, 2018 at 3:36 AM Nick Kew <n...@apache.org> wrote: > On Wed, 2018-02-21 at 11:30 -0600, William A Rowe Jr wrote: > > then all further patches to apr_fnmatch.c would still be licensed in > > BSD terms and consumable upstream, provided the PMC is agreeable; > > > > 5. Major modifications/additions to third-party should be dealt with > > on a case-by-case basis by the PMC. > > > > This would make the synchronization of POSIX [:class:] and other bug > > fixes and extensions to apr_fnmatch much simpler. > > > > Would this be acceptable to APR PMC? > > As far as I'm concerned, that's fine provided we have no objections > from any significant contributors. A quick look at svn suggests > that's basically yourself, with some early commits from Ryan. > I don't think Ryan is active here any more, so perhaps he should > be pinged as a matter of courtesy? > > Or is his name there a mere technicality, perhaps a legacy > of an initial code drop into svn? > > -- > Nick Kew > > -- Ryan Bloom rbl...@gmail.com
Re: APR: Portable across Operating Systems, or Libraries?
Why do you want to jettison edge platforms? The original goal was to keep HTTPd as portable as 1.3 was, which meant APR had to support mainframes, OS/2, etc. All of those edge platforms are what made APR challenging to create and maintain, but they also provide a lot of value for the people who want their code to work on mainframes, but don't want to write their own portability library. Removing this support takes away a web server (at the very least) from openBeOS, OS400, OS/2, etc. While these platforms may not be mainstream these days, dropping support for them from HTTPd (the natural result of dropping support from APR) seems like a decision that can only be made after discussion with APR's users, not the developers of APR itself. Just a few thoughts from the gallery. Ryan Ryan Bloom r...@apache.org r...@rkbloom.net rbl...@gmail.com On Fri, Jan 23, 2009 at 6:26 AM, Graham Leggett minf...@sharp.fm wrote: Greg Stein wrote: When thinking about 2.0, I'm having a hard time with the idea of pulling apr-util into regular apr. We've got a lot of stuff in apr-util that has nothing to do with Portability. Basically, I see apr-util doing one of two types of things: * common API to access functionality (dbd, ldap, crypto) * useful functionality built on APR I think it would be great if we could concentrate on just a core APR that offers OS portability, and that we also jettison edge platforms (keep posix and windows only). And that we trim out functionality (i.e. apr_tables) that have nothing to do with portability (tho we keep pools as a lifetime mgmt capability for OS objects). Thoughts? I think both apr and apr-util are still both based on the idea of portability. In apr, the focus is on making individual or small sets of functions available in a portable way, while the focus of apr-util is to have large or complex sets of functionality (access a database, access an LDAP server, encrypt a string) available in a portable way. That said you're right that some parts of it, like tables, fall into the category of useful stuff rather than portable stuff. Perhaps an idea could be to move the useful stuff into (a want for a better name) apr-useful, which would be the useful stuff library built on top of the portability provided by apr. Regards, Graham --
Re: tabs
On Fri, Jun 13, 2008 at 3:15 PM, Jim Jagielski [EMAIL PROTECTED] wrote: Topic for a fun conversation: Should we detab the various APR source? We've never really instituted a specific coding style, ala httpd, but 1.3.x might not be a bad place to start. I'm proposing this for post-1.3.1 release, btw :) Way to stir the pot Jim. :-) I would say go for it. Ryan
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
by an allocator. But the allocator node shouldn't be destroyed (or reused) while the pool isn't, so I don't see why a copy of the pool couldn't do the job as well. The copy of the pointers and the like still reference the allocator node, and it should work. However all this becomes cheat and I'd better look at apr_reslist. Or should I consider using my_p*() functions strictly with my_pools ? Thanks for your advices, answers. Cheers, Sander Cheers, and thanks for your answers, Yann. -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
On Fri, Jun 6, 2008 at 9:18 AM, Yann [EMAIL PROTECTED] wrote: Erik Huelsmann wrote: On 6/6/08, Ryan Bloom [EMAIL PROTECTED] wrote: I don't think that there is any reason to not have a sizeof() function, other than any code that does play with the pointers will be non-portable code. The reason that I originally went with opaque data structures (I did it before giving the code to the ASF), was that most of the structures are defined totally differently on each platform. By making the structures opaque, it became much harder for a developer to write code with APR that worked on some APR platforms, but not others. If you play with the pointers, your code is very likely to work only on the platforms that you code on. But, I would like to hear from some of the active developers about this as well. Well, as soon as you provide its size, it's not completely opaque anymore, now is it :-) Yes of course, but as soon you provide an accessor it isn't neither ... I think the entire issue is centered around the fact that Yann doesn't really want to play by the pool-rules... I would love to play with the APR pools if they could become subpools of others after their creation. That is, a pool being the whole object (destroyed with its pool), and objects living or dying with their changing owners. Why couldn't that be compatible with the pool rules ? If this is what you really want/need, then I would suggest focusing on a patch that implements this functionality. Ryan -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
I agree with Sander, this should be relatively easy to implement using parent relationships. And, I didn't intend sarcasm. I really meant that if you have a feature that APR is missing, it is better to implement it inside APR, rather than implement a wrapper around the existing concept. Ryan On Fri, Jun 6, 2008 at 12:16 PM, Sander Striker [EMAIL PROTECTED] wrote: On Fri, Jun 6, 2008 at 5:10 PM, Yann [EMAIL PROTECTED] wrote: Christopher Key wrote: [...] Ryan Bloom wrote: If this is what you really want/need, then I would suggest focusing on a patch that implements this functionality. Ryan OK, I can try to do that although I feel some sarcasm in your suggestion, don't you think it is possible ? I'm not sure if I really understand what you need. It sounds like you'd want to have something like /** * Detach @a p from it's parent, making it an orphan. Destruction or cleanup of * the original parent pool will not destruct @a p. * * @warning Use this function wisely, you're changing original lifetimes. */ apr_status_t apr_pool_orphan(apr_pool_t *p); /** * Attach @a p to @a parent, making it a child pool. Destruction or cleanup of * @a parent will also destroy @a p. * * @warning Use this function wisely, you're changing original lifetimes. */ apr_status_t apr_pool_adopt(apr_pool_t *p, apr_pool_t *parent); If that's what you want it's not particularly hard to do. Moreover, my solution looks quite simple and I'm afraid it has already been discussed in this list, but I give a try ... If I had to do a patch, I would try to use the apr_pool_cleanup_register()ing / _kill()ing mechanism to bind the pools each others. I would just manipulate the child/parent relationships. Including the subpools that would just be pools registered in the cleanups list of their parent/owner. Do you see another constraint, for a pool to be lately attached to another parent/owner, than the two pools have to use the same allocator and the to-attach pool *not* to be an ancestor of the attached owner/parent pool (that is, not having its own cleanup registered in the owner) ? I'm surely missing something, and I surely need some advices, regards, Yann. Cook up a patch and we'll give it a look :) Cheers, Sander -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: custom memory allocators ?
We did at one time (years ago) look at adding the flexibility that you are talking about here. It has a serious impact on performance of the Apache web server, which was the reason the code was removed from the system. Also, the reality is that you can't really swap memory allocation schemes easily in the middle of development, because the code is usually written to assume a specific memory allocation scheme. Consider going from pool-based memory to malloc/free. None of the pool-based code has calls to free, so the switch just doesn't work. Large portions of APR allocate memory based on the pool model, so changing allocation schemes would require quite a bit of rework to the APR internals. Having said all of that, having APR depend so closely on pools, is one of my biggest regrets with APR in general. I really wish that I had listened to Manoj when I was originally prototyping the APIs, and had provided an API to allocate the memory, and then passed the pre-allocated structure into the method instead of passing around pools. But, I also remember the reasons for not implementing APR that way, and they were valid, so I can't complain too much. Ryan On Wed, Mar 12, 2008 at 3:07 PM, Graham Leggett [EMAIL PROTECTED] wrote: Adrian Stan wrote: Pool based allocation is good for some class of applications indeed, but the library should be flexible enough to allow the user to specify his own memory allocation scheme (or not to use one at all and to directly allocate memory on the heap using malloc) I've never tried this, but would it not be enough to create a pool, use the pool, and then destroy the pool directly afterwards if necessary? The existence of pools significantly simplifies a lot of APR, because you no longer need to keep track of whether a buffer you have been given needs to be freed or not. As a result, if a function needs to return a static string (for argument's sake), it will just return the static string, instead of returning a malloc'ed copy of the static string so that when the caller calls free it will always work. Regards, Graham -- -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Small trip on the way way back machine
I left it with very little done, but I never lost interest in the idea. I just ran out of time. Work and all. I am more than happy to pick it back up or give my current base to the ASF for the effort. I need to install a mail script on my svn machine this weekend, but then I'll add your program to the repo if you would like me to. Ryan On 4/12/07, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: Some of you remember Ryan's idea of collecting a toolchain of the good old standby shell commands written in apr, which he started here; https://rkbloom.dyndns.org/viewsvn/porttools/trunk/ I'm not sure where he left that (I pinged him) and am unsure if anyone else took up this challenge of replacing the old favorites. But if anyone knows, I'd appreciate a pointer. If there is none active, and Ryan has little interest, does anyone else share an interest in hacking some of these most-commonly-needed tools? My biggie of this week is fighting with overloaded solaris and hpux boxes with too many copies of gcc/cc, and needed to pin down which-is-which. Of course your typical, stock 'which' doesn't support -a. But I'd always wanted 'which' to do more, such as tell me where the heck some specific lib or include lives on an INCLUDE or LIB path list. So my solution is at http://people.apache.org/~wrowe/witch.c - and I'm considering different possible homes for it - either here at the ASF, or elsewhere if there really isn't much interest. Thoughts or interest? Bill -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: [PROPOSAL/PATCH] add ssl sockets
Another lurker popping up in this thread. BUFF was removed for many of the reasons that Jeff stated. People were doing all kinds of strange and unusual things with BUFF to provide filtering. Krow (of Slashdot fame) was getting a basic level of filtering with BUFF, and mod_gzip was another example of where BUFF was getting in the way of solving real-world problems. Note, it also solved some problems. mpm_perchild was markedly easier when we had the BUFF concept. (I can go into great detail if you all want me to). As for an iol system, it really should be added to apr-util, assuming that the clean APIs remain available in APR proper. This gives you great flexibility and good performance. Also, since we are talking a lot about performance and APR 2.0, I want to mention again an improvement that I think should be made in APR 2.0. The incomplete structures were a great idea for APR 1.0, but I overused them. We should have used incomplete structures for the non-portable portions of each APR structure, but the portable stuff should have been accessible directly. A great example of this is apr_file_t-name. There is no reason to require a function call for this stuff. The current functions can and should be left for source compatibility, but for people wanting to directly access the common fields, they should be able to IMHO. Ryan On 6/24/06, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: david reid wrote: James Mansion wrote: What's wrong with just using OpenSSL BIOs anyway? Nothing, but why layer upon layer when there is no real need? Also keep in mind that we *should* be able to link against netscape's SSL API invisibly; but building in sophisticated BIO related features would be fine for RSA SSLC or OpenSSL, but would stink when it comes to using other SSL implementations. I see every modern crypto provider supporting the API David's proposed, and the SHA/MD5 hashing I'm about to propose. The more 'special features' of one provider we define, the more code we will have to write when those API's are not present. -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: apr_proc_create
Garrett is right in theory, but doing what he suggests will break Apache. :-) Basically, Apache relies on child processes to ensure that the server keeps running. If something happens to one child, the parent will create another. So, if you set the socket to be not inherited in subsequent child processes, then everything will apear to work for a while, and then it will just stop working when the child process dies for some reason. On unix, I would tell you to use apr_fork(), and close the socket yourself, but that isn't an option on Windows. Probably your best option is to figure out how the Apache child process on Windows finds the listening socket, re-attach to that in your child process, and then close that socket. I used to know how this all works, but I haven't played in Apache for a number of years at this point. Also, while this is an apr question, it is more of an Apache module question, and belongs on [EMAIL PROTECTED] There are developers there that can tell you exactly what you need to know about how to close the socket in your child process. Ryan On 4/11/06, Garrett Rooney [EMAIL PROTECTED] wrote: On 4/11/06, Mike Behne [EMAIL PROTECTED] wrote: I have a question about behavior of a process launched by apr_proc_create from within an Apache plugin module on Windows. The created process is intended to remain running regardless of whether the Apache server is running. I'm noticing that after the process is created, and the Apache server is stopped and I attempt to restart it, I get an indication that the socket (port 80) is in use and Apache won't start. Once I stop the created process, Apache can successfully start. I tried a few things to change this behavior--apr_procattr_cmdtype_set, apr_procattr_detach_set, apr_proc_detach--but no change. Now I've implemented some techniques to take the created proc down when Apache stops to avoid the failed restart. Is there anything I could do differently that would allow the created proc to stay up without preventing Apache from restarting? I suspect the problem is that the socket httpd is listening on is set to be inherited (as you'd expect, since it's created in a parent process and inherited by worker processes who do the accept), but what you want is for it to not be inheritted by subsequent child processes. There are functions to control this inheritance, apr_socket_inherit_unset and apr_socket_inherit_set, but I'm not sure if there's an easy way to get at the appropriate socket to call them from your module. It might need to be done inside the httpd core. On the other hand, I could be wrong, I'm just theorizing at this point. -garrett -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: apr_proc_create
You could on Unix, but on Windows there is no fork command. It is done with CreateProc (or something like that), which doesn't have flexibility that fork has. Ryan On 4/11/06, Garrett Rooney [EMAIL PROTECTED] wrote: On 4/11/06, Ryan Bloom [EMAIL PROTECTED] wrote: Garrett is right in theory, but doing what he suggests will break Apache. :-) Basically, Apache relies on child processes to ensure that the server keeps running. If something happens to one child, the parent will create another. So, if you set the socket to be not inherited in subsequent child processes, then everything will apear to work for a while, and then it will just stop working when the child process dies for some reason. Couldn't you only set the socket to not inherit in the child processes, after the fork? The parent will still be set up so its next child inherits, right? -garrett -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Opinions on refactored read_with_timeout, please?
I'm with Garrett, this looks reasonable enough, but I just don't know enough about win32 to say that on a visual inspection. I think this needs to bake for a while. Ryan On 3/25/06, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: Garrett Rooney wrote: I've refactored the read_with_timeout() code to account for observations about both WAIT_ABANDONED and the short race between ReadFile, WaitFor... CancelIo and GetCompletion. Considering the lack of regression tests to catch problems with this code, I think it's rather premature to stick it in 0.9.11. Thank you for taking the time to review these changes Garrett. Although we have a few reporters keenly interested in such things, they apparently aren't watching svn commits closely enough to give us confidence that this code is golden. I'd really like some feedback from original contributors to file_io/win32/readwrite.c but know that people are busy. We did identify the issues with m4 caches and xml/expat (wish those had slipped into 1.2.x sigh) and the aprutil cache, so those fixes will go in with 0.9.11 and we'll call it a day. Another couple weeks and we can push all these changes to readwrite etc into a release, maybe get some folks to try out snapshots if it causes them grief. Bill -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: A quick and dirty howto write an ABTS unit test
Ok, it is committed to APR's README file now, with just a few changes. I took your name out of the patch, because it is in the commit message. Ryan On 10/15/05, Ryan Bloom [EMAIL PROTECTED] wrote: This is still in my inbox. I am in the middle of a big release at work, but that is heading to staging this weekend, so my evenings are about to free up. Expect the commit tonight sometime. Sorry this is taking so long, Ryan On 10/15/05, Maxime Petazzoni [EMAIL PROTECTED] wrote: * Maxime Petazzoni [EMAIL PROTECTED] [2005-09-19 09:02:10]: * Ryan Bloom [EMAIL PROTECTED] [2005-09-18 19:42:53]: If you don't mind, I'd like to include this with the ABTS package. I'll apply this patch over the next few days unless somebody beats me to it. I'm fine with it. Thanks you, too. Ping ? - Sam -- Maxime Petazzoni (http://www.bulix.org) -- gone crazy, back soon. leave message. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFDUNXl7KsOe4Pmrg0RAozgAJ9Il5dov5xIcXrjNZMAhyM4hj4etQCeN7k9 cIiTrVzFfY4EbHgq2/gSsl4= =ayST -END PGP SIGNATURE- -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: A quick and dirty howto write an ABTS unit test
This is still in my inbox. I am in the middle of a big release at work, but that is heading to staging this weekend, so my evenings are about to free up. Expect the commit tonight sometime. Sorry this is taking so long, Ryan On 10/15/05, Maxime Petazzoni [EMAIL PROTECTED] wrote: * Maxime Petazzoni [EMAIL PROTECTED] [2005-09-19 09:02:10]: * Ryan Bloom [EMAIL PROTECTED] [2005-09-18 19:42:53]: If you don't mind, I'd like to include this with the ABTS package. I'll apply this patch over the next few days unless somebody beats me to it. I'm fine with it. Thanks you, too. Ping ? - Sam -- Maxime Petazzoni (http://www.bulix.org) -- gone crazy, back soon. leave message. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFDUNXl7KsOe4Pmrg0RAozgAJ9Il5dov5xIcXrjNZMAhyM4hj4etQCeN7k9 cIiTrVzFfY4EbHgq2/gSsl4= =ayST -END PGP SIGNATURE- -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: A quick and dirty howto write an ABTS unit test
Maxime, Thank you, If you don't mind, I'd like to include this with the ABTS package. I'll apply this patch over the next few days unless somebody beats me to it. Ryan On 9/18/05, Maxime Petazzoni [EMAIL PROTECTED] wrote: Hi, Attached, a patch to the apr/test/README file adding a section on writing an ABTS unit test. My writing style may not be very good or relevant to this kind of documentation. This is also my first piece of documentation to the APR project, so feel free to comment ! For on-screen readers, a PDF version is also available at http://people.apache.org/~maxime/abts-test.pdf Regards, - Sam -- Maxime Petazzoni (http://www.bulix.org) -- gone crazy, back soon. leave message. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDLfNF7KsOe4Pmrg0RAnMAAKDHay9V+rfWvDj/oQgBp+ZDGoEl/wCfVNyg M24dLnpGVpVv10iQbL/9zFI= =icGO -END PGP SIGNATURE- -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Returning partial data in apr_bucket_read
Why don't you want to create a new bucket? The reason for this model
is that very often the httpd code looks like:
APR_BRIGADE_FOREACH() {
apr_bucket_read()
}
So, with this model, you need each bucket to read the whole data or
add a new bucket with the rest of the data after the current bucket,
so that on the next iteration of the loop, you continue to the next
bucket's worth of data.
What I am basically saying is that not following this convention is
likely to break a lot of apps that use buckets and expect this
behavior.
Ryan
On 9/7/05, Nick Kew [EMAIL PROTECTED] wrote:
It seems that standard practice when apr_bucket_read returns less than
the entire contents of a bucket is to make an additional bucket of the
remaining data - or data source, in a case like socket.
For a bucket type I've created, I need read to return partial data, but
I'd prefer not to have to keep creating new buckets. In principle it
would be great to return a chunk of data with an EAGAIN status,
or to expect apps to read until EOS, but neither of those is compatible
with existing practice.
What I'm thinking of is having the bucket maintain its own state, and then
reinsert _itself_ (as opposed to a new bucket) until the data is finished.
Should that work, or is it asking for trouble?
--
Nick Kew
--
Ryan Bloom
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Re: Creating .so with static APR
Couple of things. 1) This is an httpd thread at this point, and should be migrated to that list. 2) You can't support loading multiple modules with APR statically until httpd can be configured to load modules with local symbols. You can't do this by default because of the mod_proxy case. Ryan On 8/16/05, Mladen Turk [EMAIL PROTECTED] wrote: Nick Kew wrote: William A. Rowe, Jr. wrote: The *right* thing to do in httpd2.2 is probably quit loading global, since we getsym the module structure, and launch everything from there. Perhaps, if a user needs all the symbols, they can use LoadFile to obtain those. LoadModule foo_modules modules/mod_foo.so [local|global] If we do that, might we want to support LAZY|NOW at the same time? What would be required to at least support safe loading of multiple modules with APR statically bundled. I presume the 'local' would be a way to go as default as Bill proposed. Regards, Mladen -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Hash table problems
You haven't really given enough information to let us help you. What
is hej, and how is it allocated? The spot that the memory is getting
overwritten is in the Apache core, so this really should be happening.
But, if that memory is allocated from a pool that doesn't live as
long as you think it does, then this could happen. You can either
send me a copy of the module so that I can look at what is happening,
or reply here with more information about what you are doing with the
key variable.
Ryan
On 8/7/05, Ante [EMAIL PROTECTED] wrote:
Hi, I'm writing a apache module and uses apr hash tables but my hash key
keeps getting overwritten, here's the code I use
if(!(torrent = apr_hash_get(torrents,hej-info_hash, 20)))
{
torrent = apr_pcalloc(pool, sizeof(struct torrent));
apr_hash_set(torrents, hej-info_hash, 20, torrent); }
I set a watchpoint with gdb and the key got overwritten in make_array_core.
Here is a backtrace.
(gdb) print (char *) 0x820ea20
$183 = 0x820ea20 wl^\004Uú¥f\027µO2HP~\/\220
(gdb) watch *0x820ea20
Hardware watchpoint 17: *136374816
(gdb) c
Continuing.
Hardware watchpoint 17: *136374816
Old value = 73297015
New value = 0
0x403d621b in memset () from /lib/libc.so.6
(gdb) backtrace
#0 0x403d621b in memset () from /lib/libc.so.6
#1 0x4027ecdc in make_array_core (res=0x820ea08, p=0x820d330, nelts=4,
elt_size=8, clear=1) at apr_tables.c:63
#2 0x4027ed8f in apr_array_make (p=0x820d330, nelts=4, elt_size=8)
at apr_tables.c:86
#3 0x0808352d in prep_walk_cache (t=0, r=0x820d368) at request.c:306
#4 0x080838a6 in ap_directory_walk (r=0x820d368) at request.c:511
#5 0x080802af in core_map_to_storage (r=0x820d368) at core.c:3420
#6 0x0808263b in ap_run_map_to_storage (r=0x820d368) at request.c:67
#7 0x08083052 in ap_process_request_internal (r=0x820d368) at request.c:149
#8 0x080666ba in ap_process_request (r=0x820d368) at http_request.c:247
#9 0x08060d63 in ap_process_http_connection (c=0x8207330) at
http_core.c:251 #10 0x0807496e in ap_run_process_connection (c=0x8207330) at
connection.c:43
#11 0x08074d5c in ap_process_connection (c=0x8207330, csd=0x8207278)
at connection.c:176
#12 0x08068023 in child_main (child_num_arg=0) at prefork.c:610
#13 0x080680fa in make_child (s=0x80a9ea0, slot=0) at prefork.c:650
#14 0x08068222 in startup_children (number_to_start=10) at prefork.c:722
#15 0x0806862b in ap_mpm_run (_pconf=0x80a80f0, plog=0x80d2198, s=0x80a9ea0)
at prefork.c:941
#16 0x0806f239 in main (argc=2, argv=0xbb24) at main.c:618
Any suggestions how I should fix this?
//Andreas
--
Ryan Bloom
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Re: Hash table problems
I haven't really dug into the code yet, but one of the problems could
be the global variables. If your Apache is using a multithreaded MPM,
then you are accessing a pool with multiple threads, which can be very
bad.
However, I believe the real problem is pool lifetime. Your hej
structure is being allocated out of r-pool, which will only live for
a single request. This happens in req2struct. The torrents hashtable
that you are accessing is allocated out of the pool that you created
in your child_init function. When a key/value pair are inserted into
a hash table, we don't copy the key or value, a pointer to the passed
in values are used. This means that the pool used to allocate the
key/value must have at least the same lifetime as the pool used to
allocate the hash table. In this case, you haven't met that
condition. So, when your first request finishes, the request's pool
is cleared, and your key is overwritten.
Ryan
On 8/7/05, Ante [EMAIL PROTECTED] wrote:
I attached the whole source, the pool is a global variable and is created in
p_child_init. I think that should work anyways. The hej structure is
allocated on line 229 and added to the hash list if the key isn't found.
-Original Message-
From: Ryan Bloom [mailto:[EMAIL PROTECTED]
Sent: den 7 augusti 2005 20:33
To: Ante
Cc: dev@apr.apache.org
Subject: Re: Hash table problems
You haven't really given enough information to let us help you. What is
hej, and how is it allocated? The spot that the memory is getting
overwritten is in the Apache core, so this really should be happening.
But, if that memory is allocated from a pool that doesn't live as long as
you think it does, then this could happen. You can either send me a copy of
the module so that I can look at what is happening, or reply here with more
information about what you are doing with the key variable.
Ryan
On 8/7/05, Ante [EMAIL PROTECTED] wrote:
Hi, I'm writing a apache module and uses apr hash tables but my hash
key keeps getting overwritten, here's the code I use
if(!(torrent = apr_hash_get(torrents,hej-info_hash, 20))) {
torrent = apr_pcalloc(pool, sizeof(struct torrent));
apr_hash_set(torrents, hej-info_hash, 20, torrent); }
I set a watchpoint with gdb and the key got overwritten in
make_array_core.
Here is a backtrace.
(gdb) print (char *) 0x820ea20
$183 = 0x820ea20 wl^\004Uú¥f\027µO2HP~\/\220
(gdb) watch *0x820ea20
Hardware watchpoint 17: *136374816
(gdb) c
Continuing.
Hardware watchpoint 17: *136374816
Old value = 73297015
New value = 0
0x403d621b in memset () from /lib/libc.so.6
(gdb) backtrace
#0 0x403d621b in memset () from /lib/libc.so.6
#1 0x4027ecdc in make_array_core (res=0x820ea08, p=0x820d330,
nelts=4,
elt_size=8, clear=1) at apr_tables.c:63
#2 0x4027ed8f in apr_array_make (p=0x820d330, nelts=4, elt_size=8)
at apr_tables.c:86
#3 0x0808352d in prep_walk_cache (t=0, r=0x820d368) at request.c:306
#4 0x080838a6 in ap_directory_walk (r=0x820d368) at request.c:511
#5 0x080802af in core_map_to_storage (r=0x820d368) at core.c:3420
#6 0x0808263b in ap_run_map_to_storage (r=0x820d368) at request.c:67
#7 0x08083052 in ap_process_request_internal (r=0x820d368) at
request.c:149
#8 0x080666ba in ap_process_request (r=0x820d368) at
http_request.c:247
#9 0x08060d63 in ap_process_http_connection (c=0x8207330) at
http_core.c:251 #10 0x0807496e in ap_run_process_connection
(c=0x8207330) at
connection.c:43
#11 0x08074d5c in ap_process_connection (c=0x8207330, csd=0x8207278)
at connection.c:176
#12 0x08068023 in child_main (child_num_arg=0) at prefork.c:610
#13 0x080680fa in make_child (s=0x80a9ea0, slot=0) at prefork.c:650
#14 0x08068222 in startup_children (number_to_start=10) at
prefork.c:722
#15 0x0806862b in ap_mpm_run (_pconf=0x80a80f0, plog=0x80d2198,
s=0x80a9ea0)
at prefork.c:941
#16 0x0806f239 in main (argc=2, argv=0xbb24) at main.c:618
Any suggestions how I should fix this?
//Andreas
--
Ryan Bloom
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
--
Ryan Bloom
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Re: example programs for the use of apr
I have written at least two articles about APR development for C/C++ Users Journal and Linux Journal recently. In all of my articles, I include simple examples. Ryan On 6/14/05, Mads Toftum [EMAIL PROTECTED] wrote: On Tue, Jun 14, 2005 at 06:44:28PM +0200, Joerg Beyer wrote: since this is not an development question about apr, I have to apologize first for asking here. Nevertheless: I look for simple examples that use apr (simpler than samba, svn or apache). It might be worth taking a look in the test dir - http://svn.apache.org/viewcvs.cgi/apr/apr/trunk/test/ vh Mads Toftum -- `Darn it, who spiked my coffee with water?!' - lwall -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: apr_pcalloc resulting in segfault
I really don't think we should take a function like apr_pcalloc and convert it to a macro. The only reason to go to a macro is for performance reasons. First, the performance boost should be relatively minimal, and for anybody who needs it, they can use apr_palloc and memset themselves. As for documentation mentioning if it is a function or macro, I don't think we should do that. macros should be written so that arguments aren't evaluated multiple times. If the documentation says what is macro, then that is almost guaranteed to be out of date at some point in the future. And, what happens when sometimes it is a function and sometimes it is a macro, depending on the compiler used or what #defines are set? Ryan On Thu, 31 Mar 2005 15:08:04 +0200, Uwe Zeisberger [EMAIL PROTECTED] wrote: Hello, in apr_pools.h[1] the following is defined: #define apr_pcalloc(p, size) memset(apr_palloc(p, size), 0, size) If I don't set an abort function and an allocation with palloc fails, this results in a segfault. There is still a function with this name, so removing the macro would fix this and should be as compatible as required. I see no way to implement this in a save manner as macro. One would need something like: #define apr_pcalloc(p, size) \ (void *__t = apr_palloc(p, size), __t ? memset(__t, 0, size), __t) but I think this doesn't work, or at least is not portable. Additionally it would be nice, if in the documentation would appear, which functions are realized as macros, above all because sometimes the arguments are evaluated more than once. Regards Uwe [1] This is true for HEAD, apr-0.9.x and apr-1.x -- Uwe Zeisberger http://www.google.com/search?q=i+squared -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: apr_pcalloc resulting in segfault
On Thu, 31 Mar 2005 15:37:21 +0200, Sander Striker [EMAIL PROTECTED] wrote: Ryan Bloom wrote: If the documentation says what is macro, then that is almost guaranteed to be out of date at some point in the future. And, what happens when sometimes it is a function and sometimes it is a macro, depending on the compiler used or what #defines are set? It's a function when pool debugging is enabled. Both the function and the macro will be present, so linking is always possible. This was one of the requirements since we didn't want to inconvenience developers switching between debug/release builds. The argument that the code segfaults when apr_palloc isn't able to allocate any memory is moot. Dean and others have indicated more than once that memory allocators usually do lazy allocation, which would result in segfaults in the code anyway because malloc never returns NULL. Given that, we could even opt for removing the abort callback function altogether. No, we couldn't opt for removing the abort callback. The reason that was added, is that it isn't valid for a library to seg fault or just abort in most situations. The abort callback was added because HTTPd wanted to just kill the child process ASAP. Other applications may want to try to print an error message in this case, so that the user or developer can try to figure out what went wrong. We can't gaurantee that the app will survive the call to abort, but we need to at least attempt to give other applications the chance to behave differently from HTTPd. Ryan -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: do we still want sendfile enabled with our default conf files?
funny, I took the list of exceptions to be so large and hard to maintain that it made more sense to go with Jeff's original idea of just disabling sendfile by default unless a user specifically decided to enable it. I just had to debug a problem for a friend with sendfile on Linux. I don't know what caused the problem, but disabling sendfile solved it immediately. Seems to me that until our sendfile support is better, we should err on the side of always sending the data correctly instead of absolutely as fast as possible. I would much rather have APR default to not using the native sendfile, and only enable native sendfile when we have a lot of evidence that it does work correctly. Ryan On Fri, 18 Mar 2005 08:07:01 -0800, Justin Erenkrantz [EMAIL PROTECTED] wrote: --On Friday, March 18, 2005 5:59 AM -0500 Jeff Trawick [EMAIL PROTECTED] wrote: ...snip, snip... AIX: Doesn't really fail in the normal sense of not putting the right data on the wire, but can trigger a kernel memory issue if some kernel tuning is incorrect. So always fail if the APR_SENDFILE_AUTODETECT is on. (This kernel tuning is irrelevant unless sendfile or more obscure TCP usage is actually occuring, so the tuning issue has typically been there all along without hurting anything.) Is the kernel turning incorrect on AIX by default? Will this be fixed in some future releases? You could do lots of things to corrupt your kernel by tuning in other ways - so unless this is by default, I can't see why we should block this. ...snip, snip... +1 to this list of exceptions and adding a new flag called APR_SENDFILE_CHECK (or APR_SENDFILE_AUTODETECT) to apr_socket_sendfile. -- justin -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: do we still want sendfile enabled with our default conf files?
That's fine. Pay attention to what I suggested. Default to
non-native sendfile, until we have know that it works. If you have an
OS that you know for a fact does sendfile correctly, then that would
be a case where we know that it works.
Instead of
#ifndef LINUX|WIN32|AIX|HPUX
sendfile()
#else
fake_sendfile()
#endif
do:
#ifdef SomeOSThatWorks
sendfile(.)
#else
fake_sendfile()
#endif
Since the beginning of APR, we thought that we knew how to get
sendfile working on most of our supported OS'. That's why we included
sendfile support. It turns out though, that for many of those
platforms, sendfile just doesn't work in some situations. When we
have a platform without the bugs, then yes we should use sendfile, but
for all other cases I care infinately more about correctness than
speed.
Ryan
On Fri, 18 Mar 2005 08:23:18 -0800, Justin Erenkrantz
[EMAIL PROTECTED] wrote:
--On Friday, March 18, 2005 11:12 AM -0500 Ryan Bloom [EMAIL PROTECTED]
wrote:
funny, I took the list of exceptions to be so large and hard to
maintain that it made more sense to go with Jeff's original idea of
just disabling sendfile by default unless a user specifically decided
to enable it. I just had to debug a problem for a friend with
sendfile on Linux. I don't know what caused the problem, but
disabling sendfile solved it immediately. Seems to me that until our
sendfile support is better, we should err on the side of always
sending the data correctly instead of absolutely as fast as possible.
I absolutely refuse to punish users who are using good OSes because some OSes
are brain-dead. This is exactly the role that APR is meant to fill: if we
know of conditions where it is unsafe to use sendfile, we won't use it unless
explicitly told so by the user.
The minimal check can be:
if (flags APR_SENDFILE_CHECK) {
#ifdef LINUX || WIN32 || AIX || HPUX
return APR_ENOTIMPL;
#endif
}
As people determine what conditions sendfile is safe (or causes problems),
then we can add those.
Feel free to advocate Linux always returning APR_ENOTIMPL for sendfile - I
don't care. However, blocking sendfile on non-buggy OSes is not a solution
that I am willing to sign off on. -- justin
--
Ryan Bloom
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Re: do we still want sendfile enabled with our default conf files?
I didn't dig into the problems that I was seeing on Linux, but I know
that it was IPv4. I don't know the filesystem, but I can find out. I
assumed it was the mutli-threaded aspect that was causing the trouble,
but that was really just a guess.
Ryan
On Fri, 18 Mar 2005 09:00:26 -0800, Justin Erenkrantz
[EMAIL PROTECTED] wrote:
--On Friday, March 18, 2005 11:27 AM -0500 Ryan Bloom [EMAIL PROTECTED]
wrote:
That's fine. Pay attention to what I suggested. Default to
non-native sendfile, until we have know that it works. If you have an
OS that you know for a fact does sendfile correctly, then that would
be a case where we know that it works.
I tend to prefer Jeff's solution of having APR return APR_ENOTIMPL when the
APR_SENDFILE_AUTODETECT flag is set and we'd fail. I'm ambivalent if we
decide to have apr_socket_sendfile() internally call emulate_sendfile because
apr_socket_sendfile() has always been an optional function (APR_HAS_SENDFILE).
If we go this route of having it mask the choice, then apr_socket_sendfile()
should always be present and we can clean up the code in httpd accordingly.
I also think that we likely already know the cases when sendfile is going to
succeed on a particular platform. I haven't heard any claims that sendfile()
fails on Linux when using only IPv4 and ext{23}. So, yes, I think we can do
better than a straight APR_ENOTIMPL - but if people don't want to write the
checks, then we'll just live with writev() on that platform. -- justin
--
Ryan Bloom
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Re: use apr for enterprise application?
On Fri, 25 Feb 2005 00:16:17 +0100, Branko ibej [EMAIL PROTECTED] wrote: Garrett Rooney wrote: Branko ibej wrote: Edward Rudd wrote: The biggest reasons to use APR, is pool method of memory management, As a matter of fact, pools can be a huge PITA, as we've found to our detriment in Subversion. The problem isn't that pools are good or pools are bad, it's that they don't fit every problem. In many cases pools simplify things a great deal, in some they are a huge pain, it depends what you're trying to do. I still think that they're easier than standard malloc/free C style coding. Oh, I never meant to imply that pools were bad in general. They're certainly well suited for server applications such as httpd. Exactly. Pools are designed for processes that do the same operation over and over and over again. And each iteration has to be completely separate from every other iteration. Every other application ends up jumping through hoops to use pools. Pools have their uses, but they aren't meant to be used by every application, and I really wish I had listened to the other developers who said I was making a mistake. Oh well, live and learn. Ryan -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: use apr for enterprise application?
Since somebody else said it first, I will admit that APR's reliance on pools were my absolute biggest mistake in APR. I wrote an article for Linux Magazine last month where I made it very clear that pools were my biggest mistake. My personal goal for APR 2.0 is to divorce APR from pools completely, so that you can easily use pools if you want to, but you absolutely aren't forced to do so. And, it should be on a per allocation basis, not per application. Ryan On Thu, 24 Feb 2005 22:27:29 +0100, Branko ibej [EMAIL PROTECTED] wrote: Edward Rudd wrote: The biggest reasons to use APR, is pool method of memory management, As a matter of fact, pools can be a huge PITA, as we've found to our detriment in Subversion. -- Brane -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: Licensing for apr_dbd and MySQL
Mail sent to Brian Aker at MySQL. If he responds directly to me, I will let the list know. Ryan On Sun, 06 Feb 2005 07:30:04 +, David Reid [EMAIL PROTECTED] wrote: Jim Jagielski wrote: Nick Kew wrote: On Saturday 05 February 2005 21:15, David Reid wrote: They do seem, as an organisation, to be quite well aware that restricting their reach by licensing isn't helpful and therefore seem qilling to talk about exceptions and different clauses, so maybe such a clause can be crafted for APR. Without a dialogue we won't know. I'm not convinced we need or want such a clause - unless you're definitely unhappy with keeping my proposed solution longer-term. Bear in mind that opensource distributors can and will have no trouble bundling the components and generating a combined platform - and that's what most non-techie end-users will see. Well, the more obstacles we have in the way to people using apr + dbd the less it will be used. Sad but true. Personally I see the lack of MySQL support within APR as a big minus. Let's be honest, MySQL is the most popular and used OS DB out there. Having an incredibly useful APR capability which cannot be fully utilized and still be within the spirit of being commercial-friendly I think will limit the reach and potential audience of dbd. I think it's worth our effort and time to look into what can be done/negotiated with MySQL. Agreed. -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re-architecture for 2.0 tree
gr, should have sent this to the list.
Ryan
-- Forwarded message --
From: Ryan Bloom [EMAIL PROTECTED]
Date: Wed, 19 Jan 2005 15:05:10 -0500
Subject: Re: Re-architecture for 2.0 tree
To: William A. Rowe, Jr. [EMAIL PROTECTED]
So, I took a few shortcuts in my original message, explained below.
On Wed, 19 Jan 2005 13:16:10 -0600, William A. Rowe, Jr.
[EMAIL PROTECTED] wrote:
At 12:33 PM 1/19/2005, Ryan Bloom wrote:
One of my biggest mistakes when initially designing APR was that I
forced each platform to have completely distinct implementations for
simple functions if the structures were distinct. This has led to a
great amount of duplicate code in the library, and makes it harder to
see where the differences between the platforms truly are. I would
love to see APR 2.0 solve this problem.
+1
In another header file:
#ifdef WIN32
struct arch_file {
HANDLE filedes;
}
#else
struct arch_file {
int filedes;
}
#endif
I still like the concept of doing something like
struct apr_foo_t {
apr_pool_t *pool;
void * foodat;
struct apr_foo_arch_t arch;
}
and defining struct apr_foo_arch_t within an include/arch/xxx/foo.h
file. This gains you the transparency of common elements (pool,
foodat, etc) while leaving certain elements opaque. Because the
arch element above is undefined, the structure remains incomplete.
(We can't anticipate the maximum size for the object.)
Right now, in 1.2.0, we could add an element onto the end of the
apr_foo_t opaque structure, and not break abi. Entirely transparent
structures don't share that attribute, folks can determine the size
of the complete, transparent structure in their code, and force
minor version dependencies that don't exist today.
The bit that disturbed me was long #ifdef litanies which we
know (from apache-1.3/src/server/main.c) get illegible quickly.
Yeah, I know, this was a shortcut. I honestly do expect those
arch_file_t type of structures to be hidden inside private headers and
to have them as incomplete structures, but I was too lazy to make that
clear, you caught me.
This can allow simple functions, like apr_file_get_name(),
to read the name of the file quickly (and only have one
implementation of that function), while still allowing native
functions to be used for the more complex work.
Again +1. I'd even suggest the structure
apr/foo/common/*.c
apr/foo/netware/*.c
apr/foo/unix/*.c
apr/foo/win32/*.c
where we drop the concept of '/unix/' as the 'common' impl,
and build apr/*/common/ on all platforms.
++1. Using Unix as the common base was a hack, which I only got away
with because POSIX was renamed to UNIX98 at some point. :-D
If we don't want the apr_file_t structure to be incomplete, we can
still provide the getters/setters for the internal structures, but
also open up our structures, so that people can get the file's name by
accessing fp-name directly.
Huge risks, garbage collection and modifying associated fields.
The more we let people diddle in the transparent elements, the
less they recognize side effects that are only possible when
they invoke the set'ter function. This will vary based on the
apr_xxx_t object we are talking about of course.
Yeah, but I am a firm believer that some stuff is private, and some
stuff isn't private. The filename doesn't necessarily need to be a
private field. We could change my original proposal to be:
typedef struct apr_file_t {
char *name;
apr_common_file_t *common;
apr_arch_file_t *arch_specific;
}
Then the common stuff is hidden in an incomplete type, but some stuff
can be easily exposed. Obviously not all structures would necessarily
need all three types of data.
This may also help us remove the depenance on pools throughout the
code. My least favorite part of APR is that all allocation _must_ be
done through pools, even if pools don't make sense for your
application (I take full blame for that).
Unfortuantely -we- perform allocations within our own operations
on these objects. When that happens, we have to look at the pool
allocator. I believe we can and should revisit allocator/free
indirection. Yes, a small performance hit will result, but it
should not be substantial enough to invalidate the entire concept.
I think we should take a closer look at why _we_ allocate structures.
Or, we could make the allocator stuff a compile-time choice, which
would mean no performance hit. Or, we could have functions to
allocate from a pool, and other functions that allocate using
malloc/free. Lots of options.
With this model, end-users can allocate the space for apr_file_t
themselves, and all we need to do is figure out how to allocate
the arch_file structure correctly, a much smaller allocation,
and I believe we will find that we allocate less within APR and
allow developers to pass us pre-allocated structures.
Again, garbage collection is the primary issue
Re-architecture for 2.0 tree
Those of you who have been here for a long time have heard me talking
about this idea, but I wanted to get it out there for everybody and
on list. Obviously, I probably won't implement much of it, but I
would like people to think about it. This is a 2.0 change, because it
completely changes the source and binary compatibility of APR.
One of my biggest mistakes when initially designing APR was that I
forced each platform to have completely distinct implementations for
simple functions if the structures were distinct. This has led to a
great amount of duplicate code in the library, and makes it harder to
see where the differences between the platforms truly are. I would
love to see APR 2.0 solve this problem.
My idea is simple enough, share as much of the structures as possible
between platforms. This allows the functions that act on those
structures to be common code. To do this, the basic format of our
structures have to change, so that they look more like the apr_poll_t
structure. I'll use files as an example (forgive the syntax):
typedef struct arch_file arch_file;
typedef struct apr_file_t {
apr_pool_t *p;
char *name;
int eof_hit;
union
apr_arch_file_t *arch_file;
} apr_file_t;
In another header file:
#ifdef WIN32
struct arch_file {
HANDLE filedes;
}
#else
struct arch_file {
int filedes;
}
#endif
This should be enough that you can see what I am thinking about.
Basically, anything that is common to all files is in a shared
structure which can be incomplete, but doesn't have to be. Anything
that isn't common is stuck in an internal structure, which must be
incomplete. This can allow simple functions, like
apr_file_get_name(), to read the name of the file quickly (and only
have one implementation of that function), while still allowing native
functions to be used for the more complex work.
If we don't want the apr_file_t structure to be incomplete, we can
still provide the getters/setters for the internal structures, but
also open up our structures, so that people can get the file's name by
accessing fp-name directly.
This may also help us remove the depenance on pools throughout the
code. My least favorite part of APR is that all allocation _must_ be
done through pools, even if pools don't make sense for your
application (I take full blame for that). With this model, end-users
can allocate the space for apr_file_t themselves, and all we need to
do is figure out how to allocate the arch_file structure correctly, a
much smaller allocation, and I believe we will find that we allocate
less within APR and allow developers to pass us pre-allocated
structures.
Thoughts, comments, critisms, questions?
Ryan
--
Ryan Bloom
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Article reviewers needed
You may remember that Linux Magazine asked for authors a while ago to write a feature about APR. Well, I took them up on that, and I need to deliver an article on December 15th. I expect to have the first draft done by Sunday night, and I need people to review the article before I submit it. Any takers? I am only looking for one or two people. Ryan -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: [PATCH] Fix testing of BEOS symbol
I'm not saying who is correct here, but in the _VERY_ early days of APR (back before we had our own mailing list), we had a discussion about this very topic. The goal back then was to determine if #if could be used instead of #ifdef or #if defined(). We decided back then to always use #if instead of #ifdef. Obviously, this hasn't been maintained, but IIRC, that was the original code style that we decided on. Ryan On Tue, 16 Nov 2004 01:41:43 +, Julian Foad [EMAIL PROTECTED] wrote: Brane, you have caught me in zero-tolerance week. Please bear with me while I rant again. :-) Branko ibej wrote: Julian Foad wrote: This patch only fixes one instance, but one which is in a header file and so is encountered frequently. There are other BEOS-related symbols being tested badly in C files which I am not fixing here. I found these with gcc -Wundef. I'm not sure this is a good change. As has been said before, it is perfectly valid in C to test an undefined symbol with #if and expect the test to behave as if the symbol's value was 0. That said, I don't think -Wundef should be a standard option, even in maintainer mode. The fact that it is perfectly valid C syntax doesn't mean it was the intended meaning, and judging by other occurrences of it within APR, I don't think that was the intended meaning. Programmers adopt practices (such as usually avoiding testing undefined symbols with #if SYMBOL) that reflect their higher-level intentions better than if they just obeyed the letter of the C syntax law and disregarded all other style considerations. Because certain such practices are well known and respected, tools like compilers can warn (optionally - if the programmer reckons that such practices are generally being followed in his project) about usage which seems to contradict those practices. I don't see why you wouldn't enable these warnings if you are working on a project that generally follows this practice. Now, APR does not currently fully follow that practice, but it mostly does, and the benefit comes in as follows: by enabling that warning, I very quickly discovered a place where APR was testing a symbol that it thought was defined (APR_HAS_SIGACTION - see my fourth patch in this mini-series of six) which was not in fact defined. By investigating this, I think I found that that is a bug, and it meant to test APR_HAVE_SIGACTION instead. Now, it is possible that I am wrong about that particular SIGACTION test (I hope somebody will review it) but in general the ability to detect such bugs is a useful benefit of always testing definedness with #ifdef or #if defined(). Imagine what happens if some system header does #define BEOS 0 The code will suddenly be incorrect. Nearly all other tests for that symbol in APR test it with #ifdef BEOS or #if defined(BEOS). I don't know the intended meaning of that symbol, but I strongly suspect it is intended to be either defined or undefined. Please correct me if I am wrong. Note that I am not saying we should disobey the C syntax - of course we can't - but rather that it is not a good idea to exploit every crevice of it. There are significant benefits to being conservative and consistent in usage of C syntax, and employing tools (e.g. warnings) which are not checking for conformance with the C standard but are checking for particular styles of C usage that we choose to uphold in order to better avoid bugs. I happen to advocate such practices to a greater extent than it seems you would choose to. We have to work to a common consensus of style within a given project. If you, and the project as a whole, choose generally to test undefined symbols with plain #if, then I can only turn off or ignore the warnings and accept that. But it seems that the practice of explicitly testing for definedness is generally followed in Subversion and in APR so I enable the warnings and investigate any anomalies. I hope this makes sense. - Julian -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: 1.0
The chances that I have time to look at this are slim to none currently. My time is currently being swallowed by my job and my real life. Ryan On Mon, 09 Aug 2004 14:52:27 -0500, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: At 02:46 PM 8/9/2004, malc wrote: No, but i would guess taking some tests from GLIBC and/or Win32 Pthreads would do, as APIs are quite similar. As for patches, current condition variable code is broken beyond repair, the whole aproach can't possibly work. There is at least one algorithm that does work and thats the one in Win32 Pthreads. I do not have enough expertise in this field to contribute any patches tests, i'm not even a Win32 coder. Unfortunately, since everything you just mentioned is GPL, afaik, it's worthless to us. I'll review from a logical perspective. Threads and the proposed threading patch I grok (and accept :), while condition variables I'm less familiar with. Perhaps Bannert, Bloom and some other condition variable gurus will pipe up. Bill -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: using APR_STATUS_IS_SUCCESS
Basically, the macro is wrong and needs to be removed. The contract that _all_ APR API's live up to is that on a successful result, they must return APR_SUCCESS. The reason we chose to use 0 as success is simple: 1) Most platforms can check for equality with 0 faster than they can check for any other integer equality. 2) It makes checking for success _really_ easy, because all APIs use the same value for success, there is no guessing or research needed, if the result wasn't 0, then the function didn't succeed. It didn't necessarily fail, because there are status codes that aren't full success and aren't failures, but more research is needed. 3) It provides you an opportunity to have a lot of different values for errors and statuses without having to use a separate variable. I assumed that the original addition of the macro was so that success was handled like any other result code, ie you always use a macro. If the reason was so that some functions could return non-zero success codes, then the macro definately needs to go, because that is a really bad idea. Ryan On Wed, 28 Jul 2004 13:47:20 -0700, Geoffrey Young [EMAIL PROTECTED] wrote: cross-posted to [EMAIL PROTECTED] Garrett Rooney wrote: Geoffrey Young wrote: hi all I was just in garrett's APR talk here at oscon and he was mentioning the APR_STATUS_IS_SUCCESS macro, which I found interesting since httpd only uses it in a few places, opting for a direct comparison to APR_SUCCESS instead. should we move to APR_STATUS_IS_SUCCESS in all places? can someone who groks the nuances of the macro add some insight here? This is actually something I was wondering about as I wrote the presentation. Neither Apache or Subversion use APR_STATUS_IS_SUCCESS everywhere, but I wonder if we should, since if you look at the definition of the macro there are cases when it's more than just (s) == APR_SUCCESS. just another note, I grep'd the code for rc == APR_SUCCESS and it looks like not even APR is using the macro everywhere... --Geoff -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: testall.c is in the attic?
testall.c was replaced when the testsuite moved to abts. ABTS does the same thing as testall, only much cleaner. Ryan On Fri, 2 Jul 2004, Ed Holyat wrote: I am trying to build testall.dsw on Windows XP. testall.dsw fails to buil= d because testall.obj isn't available. Is the test suite available in the HEAD of the branch or APR_1_0_RC2? testall.c is in the Attic. Configuration: testall - Win32 Debug-= --- Microsoft (R) Program Maintenance Utility Version 6.00.9782.0 Copyright (C) Microsoft Corp 1988-1998. All rights reserved. cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include globalmutexchild.c globalmutexchild.c link /nologo globalmutexchild.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include sendfile.c sendfile.c link /nologo sendfile.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include proc_child.c proc_child.c link /nologo /debug /subsystem:console /machine:I386 proc_child.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.= lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include tryread.c tryread.c link /nologo tryread.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include occhild.c occhild.c link /nologo occhild.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include sockchild.c sockchild.c link /nologo sockchild.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include testlockperf.c testlockperf.c link /nologo testlockperf.obj ..\LibD\apr-1.lib kernel32.lib advapi32.li= b ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include testshmproducer.c testshmproducer.c link /nologo testshmproducer.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include testshmconsumer.c testshmconsumer.c link /nologo testshmconsumer.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include testmutexscope.c testmutexscope.c link /nologo testmutexscope.obj ..\LibD\apr-1.lib kernel32.lib advapi32.= lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib NMAKE : fatal error U1073: don't know how to make 'testall.obj' Stop. Error executing NMAKE. testall.exe - 1 error(s), 0 warning(s) =A9 2004 OpenLink Financial Copyright in this message and any attachments remains with us. It is confidential and may be legally privileged. If this message is not intended for you it must not be read, copied or used by you or disclosed to anyone else. Please advise the sender immediately if you have received this message in error. Although this message and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by Open Link Financial, Inc. for any loss or damage in any way arising from its use.
Re: testall.c is in the attic?
I don't build using the dsw on Windows, so I don't know. I always use Makefile.win. So, I just type: nmake /f Makefile.win and it works. We should collapse the build system to have just a single way to build the tests on Windows. Ryan On Fri, 2 Jul 2004 11:14:06 -0400 , Ed Holyat [EMAIL PROTECTED] wrote: How Do I build it on Windows? -Original Message- From: Ryan Bloom [mailto:[EMAIL PROTECTED] Sent: Friday, July 02, 2004 11:10 AM To: dev@apr.apache.org Subject: Re: testall.c is in the attic? testall.c was replaced when the testsuite moved to abts. ABTS does the same thing as testall, only much cleaner. Ryan On Fri, 2 Jul 2004, Ed Holyat wrote: I am trying to build testall.dsw on Windows XP. testall.dsw fails to buil= d because testall.obj isn't available. Is the test suite available in the HEAD of the branch or APR_1_0_RC2? testall.c is in the Attic. Configuration: testall - Win32 Debug-= --- Microsoft (R) Program Maintenance Utility Version 6.00.9782.0 Copyright (C) Microsoft Corp 1988-1998. All rights reserved. cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include globalmutexchild.c globalmutexchild.c link /nologo globalmutexchild.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include sendfile.c sendfile.c link /nologo sendfile.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include proc_child.c proc_child.c link /nologo /debug /subsystem:console /machine:I386 proc_child.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.= lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include tryread.c tryread.c link /nologo tryread.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include occhild.c occhild.c link /nologo occhild.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include sockchild.c sockchild.c link /nologo sockchild.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include testlockperf.c testlockperf.c link /nologo testlockperf.obj ..\LibD\apr-1.lib kernel32.lib advapi32.li= b ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include testshmproducer.c testshmproducer.c link /nologo testshmproducer.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include testshmconsumer.c testshmconsumer.c link /nologo testshmconsumer.obj ..\LibD\apr-1.lib kernel32.lib advapi32.lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib cl /nologo /c /MDd /W3 /GX /Zi /Od /DWIN32 /D_DEBUG /D_WINDOWS /DAPR_DECLARE_STATIC /I ../include testmutexscope.c testmutexscope.c link /nologo testmutexscope.obj ..\LibD\apr-1.lib kernel32.lib advapi32.= lib ws2_32.lib mswsock.lib ole32.lib shell32.lib rpcrt4.lib NMAKE : fatal error U1073: don't know how to make 'testall.obj' Stop. Error executing NMAKE. testall.exe - 1 error(s), 0 warning(s) =A9 2004 OpenLink Financial Copyright in this message and any attachments remains with us. It is confidential and may be legally privileged. If this message is not intended for you it must not be read, copied or used by you or disclosed to anyone else. Please advise the sender immediately if you have received this message in error. Although this message and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by Open Link Financial, Inc. for any loss or damage in any way arising from its use. © 2004 OpenLink Financial Copyright in this message and any attachments remains
Re: What if a system already has expat?
Similarily, if the system has OpenSSH with its crypto library, wouldn't it be nice if APU's crypto stuff acted as a simple wrapper? We've got md4, md5, and sha hashes in our crypto support. IOW, it is only cryptographic hashing algorithms -- not full cryptography. I don't really see much need to reuse those functions from an installed library -- it could cause more problems that its worth. Is there a specific problem with apu including those hashing algorithms? I am pretty sure that our MD5 algorithm is slightly modified. I remember when we put MD5 into Apache 1.3 we found that the BSD platforms had used a modified MD5 algorithm. I can't remember if we took theirs, or somebody elses implementation. I do remember that one of the reasons we don't just wrap the platform MD5 algorithm, is so that we know the resoluts from our MD5 algorithm are portable to other platforms. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: error handling in apr_rmm_foo busted
On 19 Sep 2002, Jeff Trawick wrote: Looking at the problem behind PR 12616 exposes some brokenness with error handling in apr_rmm_*(). Example 1: apr_rmm_malloc() can return apr_status_t if a lock operation fails or an apr_rmm_off_t otherwise. How can the caller know which it is? The error codes will have to be clear enough so that the caller can know. This is the limitation of the design we have for error checking. However, it is the best design I have ever seen. Example 2: apr_rmm_realloc() checks for a negative return code from apr_rmm_malloc(). The return code can't be negative because of the data type (not to mention the logic of apr_rmm_malloc()). Yeah, that just sounds like somebody wrote code for malloc/realloc, and tried convert it quickly. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: [PROPOSAL] Create apr-build repository
We got into this mess by using autoconf. Unfortunately, there is nothing better than Autoconf...yet... :-) Ryan On Fri, 13 Sep 2002, David Reid wrote: Huh? Oh, OK I see now. How in the hell did we get ourselves in this mess? If these are projects they should be self supporting. I'm really not in favour of having a sub directory. Nor do I really want so many copies of these things on my machine! Sigh... david On Fri, Sep 13, 2002 at 07:19:25PM +0100, David Reid wrote: OK, but let's assume that apr is needed for these. I mean they are based on apr (apr as in the original plain old boring apr library) right? Then I guess what I don't understand is why we don't simply have the packages use a script that tells them the location and then just use them... No, they're not built from APR - they are simply m4 macros or shell scripts that autoconf requires. We could expand it to include other functionality, but the key here is to support autoconf. The script is built by apr and installed, then either used from an install or from the apr dir if we're in a cvs type environment... Nope. Can't do that as autoconf needs these files at both generation and invocation of the configure scripts. That precludes any 'oh, hey, look they are here' magic. They really do have to be in the local repository or hardcoded paths. -- justin -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: El-Kabong -- HTML Parser
I would prefer that this became it's own project either under the httpd project or the APR project. I don't believe that it should be a part of the APR-util library however. Ryan On Mon, 26 Aug 2002, Jon Travis wrote: On Mon, Aug 26, 2002 at 08:49:38PM -0700, Justin Erenkrantz wrote: On Mon, Aug 26, 2002 at 08:32:16PM -0700, Jon Travis wrote: Hi all... Jon Travis here... Covalent has written a pretty keen HTML parser (called el-kabong) which we'd like to offer to the ASF for inclusion in APR-util (or whichever other umbrella it fits under.) It's faster than anything I can find, provides a SAX stylee interface, uses APR for most of its operations (hash tables, etc.), and has a pretty nice testsuite. We use it in our code to re-write HTML on the fly. I would be the initial maintainer of the code. Please voice any interest, thanks. Interested, but would like to see the code first. =) -- justin Yeah, thought about that, however the problem is that if the ASF doesn't accept it, then it is unlikely that it will be under an ASF style license. So it's kinda a chicken egg problemo.. :-( -- Jon -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: APR benefits beyond portability for www.asterisk.org
On Mon, 26 Aug 2002, TC wrote: Hi Over on the asterisk mailing list ([EMAIL PROTECTED]) there is a little discussion on what would be the best match for a portability layer for the core phone functions in asterisk... custom brew cygwin GTK ACS Netscape etc etc Beyond the portability are there other benefits you have noticed using APR in the last few years on the Apache Web server I would be happy to answer this. But first, let me comment on a few of the options above, and some of what we have learned. I want to be clear that I am not bad-mouthing ANY of those projects, they are all good, but we have some knowledge in this area, so I am trying to help. 1) Custom brew: Please don't. There are hundreds of custom portability projects in the world today (most of them proprietary), the portability problem will only really be solved when we all work together on it. APR (and the other projects above) have learned a lot from creating portable libraries, use an existing library, and just make it better. 2) cygwin: I like the idea of cygwin, but in practice, I have found that it doesn't work. The problem is that cygwin binaries are not native Windows binaries, so you will never get the performance you want when using cygwin. As for other advantages to APR. -maintenace -learning curve maintenance and learning curve generally go together in my mind. If the learning curve is too high, the maintenance is impossible. If the learning curve is low, the maintenance is generally easy. APR excels in both areas. People new to APR generally pick it up quickly, because the functions are easy to understand, and they mirror POSIX as closely as possible. The hardest thing to grasp is the memory pooloing. -how about performance?? Apache has not suffered from performance problems related to APR, although any portable run-time does incur some performance penalty. We have a couple of people who work very hard to ensure that our performance is the best that it can be. -stability ?? APR is stable. Code written on top of APR is stable, assuming the program is written correctly. As for the library itself, we do still make small tweaks to the API occaisionally, but we are finishing all of those up now. You are asking about APR at the best possible time, because we are about to release 1.0, and we have stated that the API will not change during major releases. Hope all of this helps you make your decision. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: El-Kabong -- HTML Parser
On Tue, 27 Aug 2002, Aaron Bannert wrote: Flood is there to test the server. Serf is a (nonexistant) client library, initially slated for HTTP but not necessarily protocol specific. They are both in the correct places. Couple of things. APR-serf is only ever going to support HTTP. That was in fact a requirement when the project was accepted. We specifically stated that there were other projects with acceptable licesnses that do multiple protocols, we were only interested in HTTP. To me, APR is only about raw system-level portability - not about producing portable libraries. I'm confused how that got distorted the way it has. -- justin APR is whatever we want it to be. If we start building things on top of APR that are functionally distinct from other projects under the ASF, then I believe it makes sense to keep them as subprojects of APR. Either we extend the meaning of APR to mean any portable libraries or we take away the server in HTTP Server Project. APR is NOT what we make it. The APR project has a VERY well defined mission and goal. Take a look at the web site: The mission of the Apache Portable Runtime (APR) is to provide a free library of C data structures and routines, forming a system portability layer to as many operating systems as possible, including Unices, MS Win32, BeOS and OS/2. This wasn't arbitrary, the mission was decided upon at the start of the project, and it was approved by the board. Changing that mission requires acceptance by the entire PMC, and the board must approve it. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: El-Kabong -- HTML Parser
On Tue, 27 Aug 2002, Aaron Bannert wrote: On Tue, Aug 27, 2002 at 08:02:51PM +0200, Dirk-Willem van Gulik wrote: Actually; no - the Board gave the APR crowd a well defined charter (or rather, the APR folks came with a very well defined set of things they wanted to work on - and hence got their PMC in the ASF umbrella). Changing that charter will require this community to do some very deep thinking :-) and possibly you'll have to put things in perspective or reconsider to prevent bloat, feature creep, version # aligement problems with your dependencies and general quality. Right now your charter keeps you out of such troubled waters: The mission of the Apache Portable Runtime (APR) is to provide a free library of C data structures and routines, forming a system portability layer to as many operating systems as possible, including Unices, MSWin32, BeOS and OS/2. rather nicely. Hmm.. how do we justify apr-util, and even moreso, how did serf get approved (especially if it is HTTP-centric)? apr-util was created to move non-portability code out of APR, but since the logic was all portable, and the features were considered useful, and they are found in other portability run-times, we decided that they could live in the APR project. As for apr-serf, it was controversial when it was created, and it is in the wrong project. I said as much when it was created, and the fact that nobody commits code to it just proves the point. To me it seems that el-kabong is a natural sibling of apr-util -- both are a set of portable utilities which are dependent on our portability layer, and applicable to a wide variety of projects. look in other portable run-times, and you will find logic for tables, hashes, queues, etc. I challenge you to find one that has HTML parsing logic. I won't stop it from being part of HTTPD, but I do want to start thinking about our projects in terms of functionality rather than language or protocol. But the functionality for APR is a portable run-time, not every library that we can think to write. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: El-Kabong -- HTML Parser
On Tue, 27 Aug 2002, Aaron Bannert wrote: On Tue, Aug 27, 2002 at 01:56:39PM -0400, Ryan Bloom wrote: APR is whatever we want it to be... APR is NOT what we make it. The APR project has a VERY well defined mission and goal. Take a look at the web site: Just to clarify, by we I meant the voting majority of the APR PMC (aparently only with the approval of the board), not the whims of the casual contributor. If this is indeed a paradigm shift, then I'm suggesting that we reexamine the charter. I'll tell you right now that I will strenuosly object to making APR the kitchen sink of Apache libraries. We are a portability project, which is all the library was when I wrote it, and all I wish the project to be. If you want a library project, then create a new PMC please. I don't believe that you can create such a charter that is well defined enough to be useful. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: El-Kabong -- HTML Parser
On Tue, 27 Aug 2002, Aaron Bannert wrote: On Tue, Aug 27, 2002 at 02:15:43PM -0400, Ryan Bloom wrote: Just to clarify, by we I meant the voting majority of the APR PMC (aparently only with the approval of the board), not the whims of the casual contributor. If this is indeed a paradigm shift, then I'm suggesting that we reexamine the charter. I'll tell you right now that I will strenuosly object to making APR the kitchen sink of Apache libraries. We are a portability project, which is all the library was when I wrote it, and all I wish the project to be. If you want a library project, then create a new PMC please. I don't believe that you can create such a charter that is well defined enough to be useful. Woah woah... We should not add any new functionality into APR that does not have an immediate use. I think we all agree on that. Serf may have slipped through the cracks, but El-kabong is not immune to that scrutiny. I'm also saying that apr-util seems to have broken the charter, in a very subtle way. On the one hand, the types of things we have in there are Incredibly Useful. However, it seems that the pieces are only useful for very specific applications. El-Kabong is useful (although not immediately necessary) in the same sense as apr-util. I'm just saying we can't have it both ways. The argument for apr-util is that we are providing the same benefits as other portability libraries. That argument can't be made for either apr-serf or el-kabong. Therefore those two libraries should not be under the APR project. If you can find an actual portability run-time that includes the functionality in apr-serf or el-kabong, then cool we can disucss adding those to the APR project. But, I don't believe you will be able to find one. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: APR charter (was: El-Kabong -- HTML Parser)
We are a portability project, which is all the library was when I wrote it, and all I wish the project to be. If That's what *you* wanted Ryan. Tough luck, but there are more of us. And we have wanted it to be more, and it *is* more. That's right, I opened the project up to other people, which doesn't mean that I no longer get a say. I am saying that I believe we have violated our charter, and I wish to correct that problem. We created a mission statement as a PMC a long time ago (it was removed from the STATUS file some time ago, but here are the three options that were originally in the STATUS file: The Apache Portable Run-time mission is to provide a library of routines that allows programmers to write a program once and be able to compile it anywhere. Votes: +0: Jeff, Ryan -0: Greg, Sascha, Fred The Apache Portable Run-time mission is to provide a free library of C data structures and routines, forming a system portability layer to most Unices and the major commercial operating systems. Votes: +1: Karl, Sascha, Ryan, Jeff, +0: OtherBill, Fred The Apache Portable Run-time mission is to provide a free library of C data structures and routines, forming a system portability layer to as many operating systems as possible, including Unices, MS Win32, BeOS and OS/2. Votes: +1: OtherBill, Karl, Fred, Greg Notice the final one is word for word what is on the web site. As a TEAM we agreed to that as our mission statement. You can't just change the mission statement because you want to. If you want to change that statement, then bring it to the PMC, and let us discuss it. As things stand right now, however, apr-serf and apr-html are in violation of the mission statement, as decided by the PMC. Therefore, I am suggesting that all discussion of these projects be tabled until we either uphold our current mission, or create a new one. Incredibly Useful. However, it seems that the pieces are only useful for very specific applications. El-Kabong is useful (although not immediately necessary) in the same sense as apr-util. I'm just saying we can't have it both ways. Of course we can. Our charter allows us to define our scope. I see that as focusing on portable libraries. Which, from the votes above, I would consider to be a change from your previous position. Which is fine, but you don't get to decide on the mission statement for the whole project. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: El-Kabong -- HTML Parser
On Tue, 27 Aug 2002, Greg Stein wrote: On Tue, Aug 27, 2002 at 10:15:38AM -0700, Justin Erenkrantz wrote: ... To me, APR is only about raw system-level portability - not about producing portable libraries. I'm confused how that got distorted the way it has. -- justin It didn't get distorted as it never has the limitation that you ascribe to it. It did so. Please go back and review the original discussions about the scope of the project. The text on the web site was the mission statement that the PMC decided upon. If we want to change that fine, but let's have that discussion please. Right now, people are changing the mission without ever discussing it on-list. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: APR benefits beyond portability for www.asterisk.org
IMHO, if you are going to be creating a GUI, then use a PR that is specifically designed to give good X-platform GUI's. APR is not that project. If you had a non-GUI project, then APR would be a much better fit for you. Ryan On Tue, 27 Aug 2002, TC wrote: Beyond the portability are there other benefits you have noticed using APR in the last few years on the Apache Web server I would be happy to answer this. But first, let me comment on a few of the options above, and some of what we have learned. I want to be clear that I am not bad-mouthing ANY of those projects, they are all good, but we have some knowledge in this area, so I am trying to help. So what types of projects are more suited to APR. right now with asterisk for example they are trying to get the functions for a basic phone up on win32, these are running in linux OS now it includes these types of functions the IAX VoIP protocol layer... phonecore which actually implements a phone, with controls for driving various aspects of its operation (i.e. adding new calls, switching between active calls, changing audio devices, conferencing, etc) so these are not like a server based process like the core asterisk PBX but more like a personal productivity app, most of this port once the phonecore is up will be about the GUI portion... getting this UI over gnophone.com/ so does it still make sense to use APR as the portability of the core then yet another portability layer for the GUI say like wxWindows or is there a better strategy .. -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: apr_snprintf not compliant
On Fri, 23 Aug 2002, Jim Jagielski wrote: We are already not compliant, since we overload %p. Not sure if I understand #1: If len is 0, we return 0 and don't check buff at all. Or do you mean a length of 0 (or 1) should set *buffer to NULL? right now, if length is 0, we return 0, and that is it. However, the spec says that if length is zero, then *buffer is allowed to be NULL, but whether it is or isn't, *buffer isn't modified, and snprintf returns the number of characters that _would_ be returned if length was infinite. An example: apr_snprintf(p, NULL, 0, %s BAR, FOO); currently returns 0, after the changes, it would return 7. Ryan Assuming the later then +1 on both, not so much to be compliant but because it's the Right Thing :) [EMAIL PROTECTED] wrote: apr_snprintf and apr_vsnprintf are not compliant with the POSIX snprint standard currently. There are two problems. 1) a length of 0 should allow the buffer to be NULL, and it should return the computed length of the requested string. This doesn't currently happen. 2) Specifying a string precision doesn't work correctly. The common reason for specifying a string precision, is that you don't ahve a NULL-terminated string, but we always call strlen on the string that is passed in. Problem 2 is in GNATS as bug 8554, and it has a fix. Problem 1 is not in any bug DB, but I have a fix already. My question, is whether we really want to fix these problems, because we don't _have_ to be POSIX compliant. My own opinion is that they should be fixed. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 --- -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: apr_snprintf not compliant
On Sun, 25 Aug 2002, Ben Laurie wrote: Ryan Bloom wrote: On Fri, 23 Aug 2002, Jim Jagielski wrote: We are already not compliant, since we overload %p. Not sure if I understand #1: If len is 0, we return 0 and don't check buff at all. Or do you mean a length of 0 (or 1) should set *buffer to NULL? right now, if length is 0, we return 0, and that is it. However, the spec says that if length is zero, then *buffer is allowed to be NULL, but whether it is or isn't, *buffer isn't modified, and snprintf returns the number of characters that _would_ be returned if length was infinite. An example: apr_snprintf(p, NULL, 0, %s BAR, FOO); currently returns 0, after the changes, it would return 7. 8, surely? Nope, 7, unless I counted wrong. 3 for FOO, 3 for BAR, and 1 for the space. snprintf doesn't include the \0. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: apr_snprintf.c patch - len == 0
+1. I thought about this case this morning after re-reading one of your
messages. I was going to look at it tonight, but this looks like the
correct fix to me.
Ryan
On Sun, 25 Aug 2002, Jim Jagielski wrote:
Hmm. The patch doesn't seem to do the right thing if buff is non-NULL
and len == 0 (we simple want to return the length that would be written
but *never* actually touch buff if len == 0).
The below patch should address that, but:
1. This is rushed, so I'm not sure if my assumption about
apr_snprintf() not working right is valid
2. If the below will fix it.
I wanted to get this out quick though...
Index: strings/apr_snprintf.c
===
RCS file: /home/cvs/apr/strings/apr_snprintf.c,v
retrieving revision 1.27
diff -u -r1.27 apr_snprintf.c
--- strings/apr_snprintf.c25 Aug 2002 04:22:35 - 1.27
+++ strings/apr_snprintf.c25 Aug 2002 18:19:18 -
@@ -1249,9 +1249,18 @@
va_list ap;
apr_vformatter_buff_t vbuff;
-/* save one byte for nul terminator */
-vbuff.curpos = buf;
-vbuff.endpos = buf + len - 1;
+if (len == 0) {
+/* In this special case, we don't care if buff is NULL or not
+ * we just want to return the number of chars that would be written.
+ * So we leverage the fact that INS_CHAR just does the inserts
+ * iff the pointer is non-NULL */
+vbuff.curpos = NULL;
+vbuff.endpos = NULL;
+} else {
+ /* save one byte for nul terminator */
+ vbuff.curpos = buf;
+vbuff.endpos = buf + len - 1;
+}
va_start(ap, format);
cc = apr_vformatter(snprintf_flush, vbuff, format, ap);
va_end(ap);
@@ -1268,9 +1277,18 @@
int cc;
apr_vformatter_buff_t vbuff;
-/* save one byte for nul terminator */
-vbuff.curpos = buf;
-vbuff.endpos = buf + len - 1;
+if (len == 0) {
+/* In this special case, we don't care if buff is NULL or not
+ * we just want to return the number of chars that would be written.
+ * So we leverage the fact that INS_CHAR just does the inserts
+ * iff the pointer is non-NULL */
+vbuff.curpos = NULL;
+vbuff.endpos = NULL;
+} else {
+ /* save one byte for nul terminator */
+ vbuff.curpos = buf;
+vbuff.endpos = buf + len - 1;
+}
cc = apr_vformatter(snprintf_flush, vbuff, format, ap);
if (len != 0) {
*vbuff.curpos = '\0';
--
___
Ryan Bloom [EMAIL PROTECTED]
550 Jean St
Oakland CA 94610
---
Re: Question about committing to APR.
According to the avail list, you do not have commit access to APR. Please just submit the patch. Ryan On Wed, 21 Aug 2002, Paul J. Reder wrote: I would swear I have committed things to APR in the past, but seem to be having trouble now. I get the following error messages... cvs server: failed to create lock directory for `/home/cvs/apr/strings' (/home/cvs/apr/strings/#cvs.lock): Permission denied cvs server: lock failed - giving up cvs [server aborted]: lock failed - giving up cvs commit: saving log message in /tmp/cvsQHwqsq Which I believe means I don't have commit access. Could I be added to the APR project, or should I just submit my patch? Thanks, -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: An open source project using APR
Both Subversion and Apache have been listed on that page since I first committed that change to the web site. Thank you for your contribution. Ryan On Mon, 19 Aug 2002, David Mankin wrote: On this page http://apr.apache.org/projects.html#open_source there's no mention of Subversion as an open source project using APR. (There are not actually any users listed of any type.) I think subversion (http://subversion.tigris.org) should definitely be listed now that it has reached Alpha and is quite dependant on APR. Probably Apache httpd-2.0 should be listed too. :-) (http://apr.apache.org/ says If you are using APR, and would like your project recognized, please send e-mail to the developer's mailing list.) -David Mankin -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: An open source project using APR
Okay, I had the time to update the site. Ryan On Mon, 19 Aug 2002, Ryan Bloom wrote: I'll try to do this today. Fair warning, I just started a new job, so my time will be sparse for a few days. Ryan On Mon, 19 Aug 2002, B. W. Fitzpatrick wrote: David Mankin [EMAIL PROTECTED] writes: Wow, turns out to be an HTML rendering problem not a real omission. Sorry for the bogus report. I'm using IE 5.1.4 for Mac OS X. The page source contains: I've just verified this and checked in the fix. Ryan, can you update the live site? -Fitz -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: cvs commit: apr/include apr_strings.h
On Thu, 15 Aug 2002, [ISO-8859-1] Wilfredo Sánchez wrote: I'm going to go ahead and rename the function, so the API is correct. How do I test for the size of a type with autoconf? I believe that Will Rowe already changed the function name. Also, you don't need to check the size, we already have the logic in APR's configure script to do that. :-) Ryan
Re: cvs commit: apr-site versioning.html
On Tue, 13 Aug 2002, Aaron Bannert wrote: On Tue, Aug 13, 2002 at 11:15:48AM -0700, Brian Pane wrote: I think APR is close to being ready for 1.0. We still need to fix the apr_time_t performance problems (the old 64-bit division issue), though. Can we plan on doing this for 1.1 once we have the versioning in place and a solid release? I have a better question. Has anybody come up with a design that we can agree on for this? If there is no design, then we really can't solve the problem quickly. The last I saw about this issue, nobody agreed on how to fix it, or even that there was a real problem. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: cvs commit: apr-site versioning.html
On Tue, 13 Aug 2002, Jim Jagielski wrote: Aaron Bannert wrote: My main concern is that if we go down that path now then it may be a long time before we become stable again, and it will take time for our dependent projects to sync up too. It seems that we are at a point that we could call stable now, and if we want to redo the time interfaces at some later date we can do so on our own schedule, and call it 1.1 or 2.0 depending on what it takes to change it and how it fits with our versioning rules. I'm OK with that. I'd like to propose that we wait at least until Bill Rowe comes back from vacation until we make any full/final decision. Why are we waiting for Bill to come back? I thought the whole point of OSS was that no one person was so important that we couldn't continue without them? Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
RE: cvs commit: apr-site versioning.html
On Tue, 13 Aug 2002, Bill Stoddard wrote: I have a better question. Has anybody come up with a design that we can agree on for this? If there is no design, then we really can't solve the problem quickly. The last I saw about this issue, nobody agreed on how to fix it, or even that there was a real problem. There were a few promising solutions, but all of them had one technical problem or another. Basically, each of the proposals performed well for some set of operations but poorly for at least one other operation that might be important to some application. Toward the end of that last round of discussions, I thus proposed that APR get out of the business of managing microseconds: http://marc.theaimsgroup.com/?l=apr-devm=102678180413988 I'm interested in hearing people's comments on that proposal. Doesn't that completely ignore Cliff's message early in this thread that specifically stated that he needed microsecond resolution? Go read the post in detail. microsecond resolution is available. It is just stored in a different field. I am +0 on the proposal pending further mastication :-) That is in Apache, which has nothing to do with APR. The proposal specifically states that APR will stop dealing with microseconds, which makes it useless for an app outside of httpd that wants microsecond resolution. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: cvs commit: apr-site versioning.html
On Tue, 13 Aug 2002, Jim Jagielski wrote: Ryan Bloom wrote: Why are we waiting for Bill to come back? I thought the whole point of OSS was that no one person was so important that we couldn't continue without them? I'm not saying that Bill is so important just that I know that he's done quite a bit of investigating into this issue and might have some final ideas. That's all :) Ok. I don't honestly think we will get a release out the door this week anyway, but I also don't believe that Will has done a lot with the licensing stuff. Most of that was Greg Stein, and AFAIK, we do have a run-time versioing scheme already. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: cvs commit: apr-site versioning.html
On Tue, 13 Aug 2002, Jim Jagielski wrote: Ryan Bloom wrote: Ok. I don't honestly think we will get a release out the door this week anyway, but I also don't believe that Will has done a lot with the licensing stuff. Most of that was Greg Stein, and AFAIK, we do have a run-time versioing scheme already. My point was that if the only think holding up a 1.0 was the time issue, then it would make sense to ensure that it's closed/addressed before releasing 1.0. :) Sorry, I was looking at the versioning side, not the time side. Agreed, Bill should be involved in the time discussion. My bad. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: cvs commit: apr-site versioning.html
On Tue, 13 Aug 2002, Aaron Bannert wrote: On Tue, Aug 13, 2002 at 03:10:14PM -0400, Jim Jagielski wrote: Uh oh. Looks like we'll be hashing this out again :) Uh oh is right. Can we pleeease table this until we have the versioning code settled? (I'd like to make a release once we get the versioning into code, but you already knew that. :) Dude, settled how? We have run-time versioning, Greg committed it a long time ago, and it is being used by APR and Subversion. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: cvs commit: apr-site versioning.html
Take a look at apr/misc/unix/version.c. Ryan On Tue, 13 Aug 2002, Ryan Bloom wrote: On Tue, 13 Aug 2002, Aaron Bannert wrote: On Tue, Aug 13, 2002 at 03:10:14PM -0400, Jim Jagielski wrote: Uh oh. Looks like we'll be hashing this out again :) Uh oh is right. Can we pleeease table this until we have the versioning code settled? (I'd like to make a release once we get the versioning into code, but you already knew that. :) Dude, settled how? We have run-time versioning, Greg committed it a long time ago, and it is being used by APR and Subversion. Ryan ___ Ryan Bloom[EMAIL PROTECTED] 550 Jean St Oakland CA 94610 --- -- ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
RE: [PATCH] poll.c pollacc.c ab.c
If you're polling on a single file/socket descriptor, both APIs do about the same processing on the first poll (mostly just copying from the apr_pollfd_t to the OS's pollfd struct). If you expect to have to call poll on that same descriptor repeatedly, the apr_pollset API is more efficient than the apr_poll API, because the pollset API doesn't have to repeat this setup work on subsequent requests. On the other hand, to take advantage of this property of the apr_pollset API, you need to be able to keep track of the pollset object between calls. In situations where it's prohibitively complex for the app to maintain a pollset object, the apr_poll API is simpler to use. For larger numbers of descriptors, the apr_pollset API scales better than apr_poll, as the pollset code doesn't need to do an O(n) copy on each poll call. I should point out for completeness that apr_poll is O(2n), and apr_pollset is O(n). Both functions do a traversal through the list after calling poll to set the return events correctly. Ryan
RE: cvs commit: apr/poll/unix poll.c
Modified:poll/unix poll.c Log: We safely ignore palloc failures [we can segv in the allocator]. We cannot ignore alloca/malloc failures. We generally ignore memory allocation errors of all kinds in the server and APR. The general thought has always been that if you are actually running out of memory or stack space, then your computer is hosed anyway, and you are going to seg fault. Why can't we follow the same rules here? Ryan
RE: New poll code
Where can I get access to a machine that is displaying this behavior? Ryan -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] -Original Message- From: David Reid [mailto:[EMAIL PROTECTED] Sent: Friday, August 02, 2002 12:12 PM To: dev@apr.apache.org Subject: New poll code New poll test works fine on beos and apache builds OK but ab -n1000 -c2 crashes with a segfault in apr_poll. Off on hols so can't diagnose further - sorry. david - Original Message - From: Ryan Bloom [EMAIL PROTECTED] To: dev@apr.apache.org; [EMAIL PROTECTED] Sent: Friday, August 02, 2002 7:59 PM Subject: RE: cvs commit: apr/poll/unix poll.c Modified:poll/unix poll.c Log: We safely ignore palloc failures [we can segv in the allocator]. We cannot ignore alloca/malloc failures. We generally ignore memory allocation errors of all kinds in the server and APR. The general thought has always been that if you are actually running out of memory or stack space, then your computer is hosed anyway, and you are going to seg fault. Why can't we follow the same rules here? Ryan
RE: cvs commit: apr/poll/unix poll.c
From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED] At 01:59 PM 8/2/2002, Ryan Bloom wrote: Modified:poll/unix poll.c Log: We safely ignore palloc failures [we can segv in the allocator]. We cannot ignore alloca/malloc failures. We generally ignore memory allocation errors of all kinds in the server and APR. The general thought has always been that if you are actually running out of memory or stack space, then your computer is hosed anyway, and you are going to seg fault. Why can't we follow the same rules here? apr_palloc can be told exactly how to die, the platform malloc and alloca are different beasts altogether. Huh? We don't have any logic to tell apr_palloc how to die. The assumption is that if the server runs out of memory, then apr_palloc will return NULL, and the code will seg fault. The exact same assumption can be made for malloc and alloca. There is no logic in apr_palloc which should actually cause the seg fault AFAIK, the seg fault is expected in the code that calls apr_palloc. Ryan
RE: cvs commit: httpd-2.0/build httpd_roll_release
From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED] At 11:42 AM 8/1/2002, you wrote: ianh2002/08/01 09:42:33 Modified:buildhttpd_roll_release Log: we need apr-iconv now Even if we don't build it, this is extremely good practice that the folks rolling and releasing the tarball TAG the apr-iconv tree in sync with the current apr and apr-util trees.. I completely disagree. The problem is that the httpd_roll_release script is for rolling httpd releases, not APR releases. This change doesn't help people realize that they have to tag APR-iconv before they can release httpd. I really agree with Cliff, the change to pull apr-iconv out of APR is annoying, and it is going to cause problems. I understand that it is the best solution we have right now, it is still a bad solution. Ryan
RE: cvs commit: httpd-2.0/build httpd_roll_release
Even if we don't build it, this is extremely good practice that the folks rolling and releasing the tarball TAG the apr-iconv tree in sync with the current apr and apr-util trees.. I completely disagree. The problem is that the httpd_roll_release script is for rolling httpd releases, not APR releases. This change doesn't help people realize that they have to tag APR-iconv before they can release httpd. Amazing that we tag APR at all, no? That APR gets tagged with Apache, is a side-effect of not having released APR yet, nothing more. In time, we won't tag APR with an Apache tag. I really agree with Cliff, the change to pull apr-iconv out of APR is annoying, and it is going to cause problems. I understand that it is the best solution we have right now, it is still a bad solution. Of course it is bad. That's why I suggest a separate tarball for iconv. But it doesn't matter, we need trees in-sync, so apr-iconv must be tagged with apr's tags, from here forwards. If you want to do that as an rtag, that would be fine too. The other thing, is that httpd_roll_release doesn't do the tag, it simply checks out the code that has already been tagged. Ryan
RE: AF_UNIX sockets in APR...
While the code is small, to the best of my knowledge, it is also not portable. If it is portable, +1. If it is Unix only, -1. Well, it is not a globally portable feature... As fork(), for instance, which doesn't exist on Windows, but still APR supports it. Okay, I can accept that, except the while AF_UNIX isn't portable, the concept can be written portably. Aaron, Will, and I had this conversation a couple of months ago, and you can write domain socket-like logic for Windows. I do consider AF_UNIX different from fork(), because fork() is required for any complex programs on Unix, and Windows just can't support it. However, AF_UNIX can be written for Windows, albeit, it is a hack, and requires a combination of sockets and named pipes. If APR provided me AF_UNIX, I would just use that... Please, don't make the same stupid mistake that Java did. I'm bashing my head on the wall everytime I see the java.net classes because they are not extensible, and because every time that someone wants to use AF_UNIX, they have to write it on their own... Ok, it doesn't work on Windows, but it works on another 2 bazillions of UNIX implementations... I have no problem implementing this feature, but do it right (which may mean not using apr_socket_t), so that it is portable. We have enough people who have asked for this feature, that not implementing it is kind of stupid, but please, please, please, don't write the API such that it absolutely can't work on Windows. That completely removes the goal of APR. We have tried hard not to create functions that can't be written on one of our platforms. Please don't add a type of communication that isn't portable, that isn't useful in a portable library. Ryan
RE: AF_UNIX sockets in APR...
From: Pier Fumagalli [mailto:[EMAIL PROTECTED] Ryan Bloom [EMAIL PROTECTED] wrote: I have no problem implementing this feature, but do it right (which may mean not using apr_socket_t), so that it is portable. We have enough people who have asked for this feature, that not implementing it is kind of stupid, but please, please, please, don't write the API such that it absolutely can't work on Windows. That completely removes the goal of APR. We have tried hard not to create functions that can't be written on one of our platforms. Please don't add a type of communication that isn't portable, that isn't useful in a portable library. I agree with your sentiment, and frankly, I don't care whether we use apr_socket_t or not, as long as I can use the same functions to read and write to AF_UNIX and AF_INET sockets, because that's what I really need to do... :) Okay, the read/write stuff is going to be hard to do, but maybe this is the reason to finally implement the IO Layers. Justin, as for the API, it should mirror the apr_socket_t API almost exactly. I say almost, because there are a lot of functions in apr_socket_t that aren't needed in AF_UNIX sockets. The only reason not to use apr_socket_t, is that the Windows implementation is 100% separate from the AF_INET implementation. While a separate type will suck on Unix, it will speed up Windows and make the implementation workable and readable on that platform. Ryan
RE: [Request for comments] new poll API
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Brian Pane [EMAIL PROTECTED] writes: To continue the recent discussions on the problems in the current apr_poll API, here's a patch for apr_poll.h that illustrates my proposed fix. What I'm proposing here is to split the API into two parts: - A lightweight, single-function poll API for use (only!) with very small sets of descriptors. Do we really need this API? What is the sort of APR application for which the heavy-duty API is harmful? I am biting my tongue here, but Jeff, you are the person who specifically stated that the heavy-duty API was too slow for us to use. Ryan
RE: [Request for comments] new poll API
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Ryan Bloom [EMAIL PROTECTED] writes: From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Brian Pane [EMAIL PROTECTED] writes: To continue the recent discussions on the problems in the current apr_poll API, here's a patch for apr_poll.h that illustrates my proposed fix. What I'm proposing here is to split the API into two parts: - A lightweight, single-function poll API for use (only!) with very small sets of descriptors. Do we really need this API? What is the sort of APR application for which the heavy-duty API is harmful? I am biting my tongue here, but Jeff, you are the person who specifically stated that the heavy-duty API was too slow for us to use. I said it was too slow and/or cumbersome to use in a particular situation that does not correspond to something an APR app would do, so I don't consider that a valid use-case for justifying the simpler API. (An APR app should be using an APR timeout socket option for that situation.) Let me see if I understand. Apr_poll() is too slow for APR to use, but because you don't expect APR apps to use it too often, that is okay. Sorry, that doesn't hold water. Like I said above, I'd first like to see a description of an APR app that is harmed by doing the setup and using the accessor functions. This should be helpful in determining how important it is to support the simpler API flavor. Well, Greg Ames used to say all the time that apr_poll was seriously hurting the performance of Apache. Ryan
RE: [Request for comments] new poll API
I am biting my tongue here, but Jeff, you are the person who specifically stated that the heavy-duty API was too slow for us to use. I said it was too slow and/or cumbersome to use in a particular situation that does not correspond to something an APR app would do, so I don't consider that a valid use-case for justifying the simpler API. (An APR app should be using an APR timeout socket option for that situation.) Let me see if I understand. Apr_poll() is too slow for APR to use, but because you don't expect APR apps to use it too often, that is okay. Sorry, that doesn't hold water. You are grasping for generalizations and taking a lot of liberties with the facts along the way. Sorry, but I don't think so. The facts are simple. You said that apr_poll was both too slow and too complex for use inside of APR. To fix that, I re-implemented apr_poll() to resolve both of those issues. Now, (because there is an infinitely fixable memory leak), we are throwing out the re-write, and going back to the abstract type, which was too slow and too complex for use inside of APR. So, where am I mis-remembering? Like I said above, I'd first like to see a description of an APR app that is harmed by doing the setup and using the accessor functions. This should be helpful in determining how important it is to support the simpler API flavor. Well, Greg Ames used to say all the time that apr_poll was seriously hurting the performance of Apache. Are you suggesting that Apache is a use-case to support the simpler flavor? I'm not sure how. I suspect that rebuilding the native pollset (e.g., struct pollfd array) every time apr_poll() is called is harmful to Apache. It should actually perform better than the previous version of the code, for small pollsets, which is what most people use with Apache. I suspect that rebuilding the abstract pollset (apr_pollfd_t array) in its entirety after calling poll() is harmful to Apache, which is only going to process the first match. I am unable to parse this at all. If you are talking about the current implementation, then one of the advantages is that you don't need to do that anymore. If you are talking about the previous implementation, then I really don't see which side of the discussion you are on. Ryan
RE: [Request for comments] new poll API
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Ryan Bloom [EMAIL PROTECTED] writes:
I suspect that rebuilding the native pollset (e.g., struct pollfd
array) every time apr_poll() is called is harmful to Apache.
It should actually perform better than the previous version of the
code,
for small pollsets, which is what most people use with Apache.
I started off agreeing that the new implementation is faster for small
pollsets, but I'm not sure that is the case when you consider
steady-state operations. We save the overhead of the function call to
look up the returned events after the poll call but we pick up the
overhead of the internal call to get_event() just before poll() is
called.*
That get_event() call can easily be removed in 99% of the cases (see
below).
If APR had a small-pollset API and a big-pollset API, I suspect we'd
be better off in Apache just using the big-pollset API rather than
deciding at run-time which API to pick since implementing a choice
would likely introduce an extra function call which would erase any
small benefit of being able to use the small-pollset API.
In how many situations would we actually need to use the big-pollset
API? I would much rather just write the code to use the small-pollset
API, and possible #ifdef the big-pollset API. It would be faster to use
the small-pollset API, and if you _must_ have the large-pollset, then
you configure for it.
I suspect that rebuilding the abstract pollset (apr_pollfd_t
array) in
its entirety after calling poll() is harmful to Apache, which is
only
going to process the first match.
I am unable to parse this at all. If you are talking about the
current
implementation, then one of the advantages is that you don't need to
do
that anymore.
Here is the code I referred to as rebuilding the abstract pollset
(apr_pollfd_t array):
for (i = 0; i num; i++) {
aprset[i].rtnevents = get_revent(pollset[i].revents);
}
*yeah, calling from Apache to APR is more expensive than an internal
APR call, but are we digging that deep to find the benefit?
This is a straight bit-mask check, and in most cases, can be optimized
out to a no-op. We have to write the optimization, but 99% of the time,
the function call can be removed. As for the first/all rtnevents
decision, Apache should be using all of the results. We don't
currently, but we could and should.
RE: EINVAL vs APR_EINVAL in apr_xlate_open()
From: Karl Fogel [mailto:[EMAIL PROTECTED] Justin Erenkrantz [EMAIL PROTECTED] writes: In APR calls, if EINVAL is produced, it should use APR_EINVAL. And, for checking for error, the caller should use APR_STATUS_IS_EINVAL(s). -- justin Well, this is *us* generating the EINVAL by fiat, not getting it from some lower-down system call. Maybe we should still go with APR_EINVAL, on the principle of least surprise, though. I'm not sure; trusting people with more APR experience to know a good answer here. Just for completeness. We should be returning APR_EINVAL, because that is the safe code. However, on all platforms that implement EINVAL, EINVAL and APR_EINVAL are equivalent. And, I can't think of any platforms that don't define EINVAL. Ryan
RE: [design] work around new apr_poll leakage?
Okay, so this is essentially exactly what we had before I re-wrote
apr_poll() a few weeks ago. There are some differences, such as keeping
the allocated structure, but at the end of the day any implementation
based on this design will have the same problem that the previous design
had, namely performance.
So, the problem that this design is trying to solve is a memory leak. I
would rather just focus on fixing that problem. So, there are a couple
of things here, 1) The apr_pool_t that is in the current code is only
there because I needed it for the apr_palloc(), if that call is removed
so should the apr_pool_t. 2) This is a multi-pronged approach. The
assumption is that modern platforms will get the most benefit, but all
platforms will have the leak removed. With no further ado, the approach
I would suggest.
The idea is that we will still end up creating the pollfd_t on every
call to apr_poll(), this does have a performance penalty, but on modern
platforms, that performance penalty is constant. The only real
difference is how you allocate the array. There are three options:
1) C99 specifies Variable Length Arrays (VLAs) specifically to solve
this problem. If your compiler supports it (gcc does on Linux at least,
as does MSVC I believe), then we create an array on the stack with a
size of the num argument. (Obviously, we would need some autoconf
macros to determine if the platform supports VLAs.)
2) For platforms that don't support VLAs, we create an array of size
FOO (value to be determined later). Because this is allocated on the
stack, we just need to fill it out and call poll().
3) If the caller needs more than FOO elements, we use malloc() and
free() to allocate the pollfd_t array. This is an isolated call to
malloc/free, only required on platforms that don't support VLAs, and can
be optimized out if the user sets FOO correctly at compile time. (For
Apache, the default will most likely be 10, but we can experiment for
the best value).
Obviously FOO will need to be named better, but that is only important
if we take this route.
Ryan
Ok, we have real problems with the palloc in the new design of
apr_poll().
But it's fundamentally better than falling back on poll() because our
design
just sucks, performance-wise. Ryan speaks of alloca, but that won't
work on all platforms, most especially stack-impaired platforms such
as
Netware without the ability to grow the stack.
So here's a design suggestion. Take the apr_pollfd_t element;
struct apr_pollfd_t {
apr_pool_t *p;
apr_datatype_e desc_type;
apr_int16_t reqevents;
apr_int16_t rtnevents;
apr_descriptor desc;
};
If we pull the apr_pool_t *p [which is WAY overkill if you have
several
dozen to hundreds of descriptors you might be polling against] and
create a brand new apr_pollfd_set_t [also transparent!]...
typedef struct {
apr_pool_t *p;
apr_pollfd_internal_t *int;
apr_pollfd_t *first;
int count;
} apr_pollfd_set_t;
Modify apr_poll_setup to return a new apr_pollfd_set_t, with *first
already pointing at a newly allocated apr_pollfd_t array, and set up
apr_pollfd_internal_t to NULL.
The *int pointer tracks APR's internal count and arrays that it needs.
If the count to an invocation of apr_poll() exceeds our internally
allocated
count, THEN we go and allocate a larger internal structure. But not
for
every pass into apr_poll().
ADVANTAGES
. we stop the leak. Once we grow to some given *int pollset size,
we won't be growing anymore.
. we lighten the apr_pollfd_t elements by one pool pointer.
. using *first, we can change our offset into a huge array of
elements.
. using count, we can resize the array we are interested in.
. neat feature, we actually can have several apr_pollfd_set_t
indexes floating around, neatly pointing into different parts
of a huge apr_pollfd_t array.
DISADVANTAGES
. if not using apr_poll_setup(), the user is absolutely responsible
for initializing *v to NULL.
. two structures to think about. Not complex structures, but two,
none the less.
Comments or ideas? This was idle brainstorming.
Bill
.
RE: [design] work around new apr_poll leakage?
Congratulations. You have just designed the interface that was removed two weeks ago. :-( BTW, that interface was removed because it was too bulky and complex and too slow. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA -Original Message- From: Brian Pane [mailto:[EMAIL PROTECTED] Sent: Thursday, July 18, 2002 11:28 PM To: dev@apr.apache.org Subject: Re: [design] work around new apr_poll leakage? William A. Rowe, Jr. wrote: At 11:21 PM 7/18/2002, Brian Pane wrote: I'm almost in complete agreement with this proposal. We need a wrapper object to hold the temp pollset so that we can re-use it for all subsequent poll calls on the same fd set. Notice that apr_pollfd_internal_t *int; below _IS_ entirely opaque... [and using a reserved keyword, the member name REALLY sucked... how about simply *private.] Right, but the pointer to the vector of apr_pollfd_t objects is transparent. And that's the one that will cause problems if we try to use something like /dev/poll, because the set of descriptors in that vector can get out of synch with the descriptors that have been registered with /dev/poll. (Yes, we could avoid that problem by including within the apr_pollfd_internal_t a list of the descriptors that we used on the last poll call for this apr_pollfd_set_t, which would make it possible to detect added/removed descriptors since the last call. But that would require an O(n)-time scan through the entire vector on each request, which defeats the scalability advantages of using a faster poll interface.) Speaking of linear-time code, I just checked the current apr_poll(), and it's doing an O(n)-time loop to copy from the apr_pollfd_t structs to the pollfd structs. That's a major scalability problem for larger values of n. Fortunately, your new design offers a way to fix the O(n) problem: build the pollfd vector once, and re-use it on subsequent calls if the apr_pollfd_t vector hasn't changed. (The transparency of the apr_pollfd_t vector and apr_pollfd_set_t makes this a little tricky, though.) --Brian
RE: Hehe, sure is a lot of code now
I run an old linux box in my basement that firewalls the ADSL runs the house network has an apache for some very low volume family domains. I got a ton of virtual hosts set up and the apache that came with the SuSe of late 2000 was getting weird and messing up the vhosts and probably is a security risk, so I grabbed 2.0.39 today and got it going. What's amusing is, this is an old P100 (no, that's not a typo). Type ./configure --prefix whatever and go for a coffee. Then type make and go for lunch and a shower. I swear between apr and apr-util and httpd it must have spent a solid minute just on the iconv guesswork in configure. Mind you, the box *runs* 2.0.39 just fine. Snicker. I wonder if I now have the record for the oldest/slowest computer ever to build apache2. -Tim Nope, my home server is a P100. Been running 2.0 since the early betas. Ryan
RE: [design] work around new apr_poll leakage?
Ryan Bloom wrote: Congratulations. You have just designed the interface that was removed two weeks ago. :-( Exactly. The old API had the right design: an abstract poll object with accessor functions, so that the poll wrapper can run in O(1) time. The new implementation results in an O(n) time wrapper (due to the need to copy n pollfd objects on every poll call), not to mention the memory leak. BTW, that interface was removed because it was too bulky and complex and too slow. Please go ahead and revert the new implementation. If we need to tune the old implementation, let's do that. No, the old design was completely bogus. As proof, Trawick vetoed even using the damned thing inside of APR. I don't care if you think that we don't have to use it, if a developer of APR believes that the API is too ugly to use in their code, then the API is borked. Damn, I hated the API and I designed it. The current API is correct, because it gives the user back control of their memory. You have a problem with looping through the pollset each time you call apr_poll? I have a problem with requiring 6 functions to manipulate the pollset. You can't do it with less than 6, we tried, and we always ended up with 6. Oh wait, 7 if you want both sockets and files. You are optimizing for apps with large pollsets. Fine, go ahead. Create a wrapper structure with an opaque structure as Will suggested. You want to re-use your pollset, figure out a solution. But for apps (like Apache, and most other non-async apps) with small pollsets, the current implemented with a few tweaks is the correct implementation. Ryan
RE: [design] work around new apr_poll leakage?
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Ryan Bloom [EMAIL PROTECTED] writes: No, the old design was completely bogus. As proof, Trawick vetoed even using the damned thing inside of APR. I listed two conditions for the veto: 1) something more complex than calling poll() in that situation 2) something slower than calling poll() in that situation It seems to me that the current code meets both conditions, though not as obnoxiously as code using the older apr_poll() interface would have. Can you clarify this please? Does that mean that you are vetoing the current code too? BTW, I don't understand how you can say that calling apr_poll() is more complex than calling poll(), since they have the same API. If your saying that they _only_ solution you will accept is calling poll() itself, then that is completely bogus. Ryan
RE: [design] work around new apr_poll leakage?
Fine. Congratulations. Yet again we are stopped from making any progress because people refuse to work together and instead we have to use fscking vetos to play together. The old API sucks. It is unusable, causes mistakes, and doesn't allow for a fully featured API. But let's put it back. I'm going away for a few more days. I was a lot happier when I wasn't reading this e-mail list. Ryan -- Ryan Bloom [EMAIL PROTECTED] [EMAIL PROTECTED] -Original Message- From: Brian Pane [mailto:[EMAIL PROTECTED] Sent: Monday, July 22, 2002 8:07 AM To: [EMAIL PROTECTED] Cc: dev@apr.apache.org Subject: Re: [design] work around new apr_poll leakage? Ryan Bloom wrote: The current API has a memory leak that makes it unsuitable for general-purpose use (including the httpd). That's why I vetoed it. Thatis just BS. This conversation started today with a solution to the memory leak. The conversation started with the showstopper in the httpd STATUS. My veto from three days ago still stands: http://marc.theaimsgroup.com/?l=apr-devm=102705248212752 I'm happy to see the increased discussion of potential fixes, but that discussion alone doesn't resolve the veto. The only two ways to resolve the veto are to: remove the new poll code, or replace it with a variant that doesn't have the memory leak and O(n) scalability problem. --Brian
Re: Is --enable-utf8 working everywhere?
On Wed, 17 Jul 2002, Greg Stein wrote: On Wed, Jul 17, 2002 at 05:39:52PM -0400, Jim Jagielski wrote: Ulrich Drepper wrote: On Wed, 2002-07-17 at 11:00, Sascha Schumann wrote: int *func_call(); #define errno *func_call() =20 I don't see the problem with return errno; though. The problem is not return errno; it is return errno ? errno : EINVAL; Are you suggesting something will change the value of 'errno' during this code, such that the errno being tested isn't the value being returned?? I don't think Ulrich is concerned with *correctness*. His comments are about resulting code size and performance. I think Ulrich hasn't been clear in that regard, but you (Jim) are also (seemingly) tending to be a bit obstinate in wanting to do it your way :-) Yeah, I definately didn't understand Ulrich's concerns until he specifically stated what he was trying to solve. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
RE: Is --enable-utf8 working everywhere?
From: Ulrich Drepper [mailto:[EMAIL PROTECTED] On Tue, 2002-07-16 at 17:49, Blair Zajac wrote: We don't need to make a copy of errno. Oh yes, we do. When it comes to interaction with the C library you can believe me. errno is no simple variable on almost all systems. So, use the patch as I wrote it, please. You most definitely do not need to make a copy of errno. As proof, we don't make copies of it anywhere in APR, and it works on every platform we have tried. While errno is not a simple variable, compilers are smart enough to treat it as if it is a simple variable, so just 'return errno' works everywhere. Ryan
RE: Is --enable-utf8 working everywhere?
From: Ryan Bloom [mailto:[EMAIL PROTECTED] From: Ulrich Drepper [mailto:[EMAIL PROTECTED] On Tue, 2002-07-16 at 17:49, Blair Zajac wrote: We don't need to make a copy of errno. Oh yes, we do. When it comes to interaction with the C library you can believe me. errno is no simple variable on almost all systems. So, use the patch as I wrote it, please. You most definitely do not need to make a copy of errno. As proof, we don't make copies of it anywhere in APR, and it works on every platform we have tried. While errno is not a simple variable, compilers are smart enough to treat it as if it is a simple variable, so just 'return errno' works everywhere. BTW, I thought of this after I hit send. Errno MUST be treated as a simple variable, or all of the older apps that used it as such would break with new compilers. Ryan
RE: Earth to APR????
++1! Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA -Original Message- From: David Reid [mailto:[EMAIL PROTECTED] Sent: Monday, July 15, 2002 4:36 PM To: APR Dev List Subject: Earth to APR Can we all come down off the ceiling for a bit and get our feet back on the ground? I can't help but wonder why we are making this whole thing so complicated when people have been using computers and libraries using time quite happily for a number of years without getting as involved as this has gotten. But then maybe I've borrowed some of Cliff's crack? BTW, this is getting embarrassing to watch - please can we bring this to a head somehow? david
RE: a new proposal Re: Earth to APR????
I'll say it again. Leave apr_time_t alone. If a program wants 1 second resolution, then they should be using time_t. apr_time_t specifically provides microsecond resolution. Different solutions for different problems. :-) Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA -Original Message- From: Brian Pane [mailto:[EMAIL PROTECTED] Sent: Monday, July 15, 2002 5:35 PM To: APR Dev List Subject: a new proposal Re: Earth to APR Actually, now that I think about it, I have a solution in mind: we (APR) should get out of the business of providing microsecond time manipulation. Here's my rationale... What added value can APR provide to applications that need to work with time? - Getting the current time in a portable manner? Yes. - Converting timestamps to date/time form in a portable manner? Yes. - Doing arithmetic on times? Not really. Every solution we've found for representing a microsecond-resolution time has a performance tradeoff somewhere: either in extracting seconds (current apr_time_t), or in extracting microseconds (busec representation), or in addition and subtraction (struct representation). Which of these operations can we afford to make slower in order to speed up the other ops? I don't know. It depends on the application. Based on these observations, I think it's important that APR be able to provide a portable way to look up the current time with microsecond resolution, but APR need not do all the rest of its time operations in microseconds. Here's a design that I think would be a reasonable compromise: - apr_time_t becomes a 64-bit uint representing seconds since the epoch. (Maybe it's typedef'ed to time_t. Maybe not. Let's worry about that *after* we figure out the desired semantics.) - apr_time_now() returns a time_t, but it also returns the microseconds as a separate value for apps that need them: apr_status_t apr_time_now(apr_time_t *sec, apr_uint32_t *usec); Note: We could also keep apr_time_now() unchanged and add a different function that would provide seconds and microseconds. There are pros and cons for both. - All the time-maniuplation functions that currently use apr_time_t continue to use apr_time_t. (As far as I can tell, none of them really need microseconds.) - Interval times are expressed as an integer count of microseconds. (We *might* want to let apr_poll() take an integer count of milliseconds to avoid a division by 1,000, since poll is such an important special case.) With this model, we can add one more field to the httpd's request_rec to hold the microseconds part of the request_time. The httpd core won't care about it, but custom modules that need more granular time (e.g., for special-purpose logging or session ID generation) will have access to it. Apps that need to work with microsecond resolution can choose their own time representation, based on whatever is best suited to the needs of the app: structure, binary microseconds, even convert to floating-point. --Brian
RE: more notes on the apr_time_t issue
On Sun, 14 Jul 2002, Ryan Bloom wrote: How to resolve Aaron's veto? The #ifndef NEW_TIME_FMT thing is interesting. :) How does that help me though? Wouldn't that mean that APR would be linked with the new time implementation and the older app would link in just fine even though the representation had changed? It alerts you to what we have done. It is up to you to fix it thought. You are right that it doesn't help with binary compat, but we haven't committed to binary compat until 1.0 is released BTW. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
RE: IO abstraction layer for files
We had an IO abstraction layer a long time ago. The original one was bulky, hard to debug, and didn't solve the problem it was designed to solve, namely filtering. When the filters were introduced, the IOLs were removed, because they were no longer needed. I guess my only question, is what problem are you trying to solve with an IOL? Until we have an answer to that, there is really no way to know if they are a good idea or not. Every time that they have come up in the past, the reason was never enough to justify adding the complexity and performance problems. Remember that in general, an IOL would mean that every time you wanted to read to or write from a file or network, you would do a pointer de-ref and call an extra function. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA -Original Message- From: Ian Holsman [mailto:[EMAIL PROTECTED] Sent: Sunday, July 14, 2002 5:54 PM To: dev@apr.apache.org Subject: IO abstraction layer for files I know this topic has come up several times in the past, but I can't remember the outcome/reasoning on why we didn't want to re-introduce the IO Layer for file operations.. TIA Ian
RE: more notes on the apr_time_t issue
From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED] At 03:21 PM 7/14/2002, Ryan Bloom wrote: So my names are wrong, but the concept is the same. More macros than you can keep straight to fill in the time structure. The advantage to the current model, is that there is one macro: APR_SEC_FROM_USEC. No, that's not entirely fair, and no, we can have an APR_TIME_PER_SEC value. I'm not suggesting we hide this scalar behind a curtain. The other macros are for convenience, not to be a pain in the ass. Others disagree, but once we hit binary compatibility, these can never change. The macros, however, are considerably faster on all platforms, they don't rely on the compiler doing all the optimization. That's why the macros are infinitely preferable to the constant. They are [will also be] vetted so that the correct behavior is on the library, not the developer's math. The macros are no faster than the constant. The implementation is faster. Completely different. In fact, you added one more macro, the nsec variant, so now you have _10_ macros to get information out of a simple time type. That doesn't seem incredibly extreme to you? Not in terms of performance. We have a four real macros; apr_time_sec_get(time) [which is the same as...] apr_time_to_sec(time) apr_time_from_sec(sec) apr_time_xsec_get(time) [gets only the fraction of a second] apr_time_to_xsec(time) [converts complete value by scale] apr_time_from_xsec(xsec) [converts complete value by scale] apr_time_xmake(sec, xsec); So xsec can be an nsec, usec or msec, depending on the app. So WHAT? They all follow an incredibly simple pattern. BTW, nsec is completely bogus. There isn't an OS that I know of that reports nsec intervals. Windows comes the closest with 100 nsec chunks, but that isn't nsec. Having a macro that purports to report nsecs when NONE of our OSes can accurately report them is bogus. Four macros, by four different scales, equals 16 macros. And, at the end of the day, the values stored are useful once you leave APR. BTW, the apr_time_xmake(sec, xsec) macro is completely wrong IMHO. In order to use it, I need to provide multiple values, but that doesn't make any sense, because we have four time scales. Either provide a single value, or four. Two just doesn't make sense. I have no problem fixing the ambiguity between _xsec_get and _to_xsec, since the difference is subtle. Perhaps apr_time_frac_to_xsec(time) instead of _xsec_get? (saying, only the fractional portion of a second?) Perhaps an example of the difference, because I don't see it. 22 functions/macros to deal with time. 8 of those are because we are storing time in a format that nobody else expects. We have those today. We have them because we deal in uniform apr_time_t RIGHT NOW TODAY. This whole proposal changes nothing in that department. Sure it does. We have 1 macro today, now we need 10! Quit it with the ten bs :) Ok, four macros of three-four flavors, instead of one macro const value. We've been using those macros for several weeks, or had you missed that? There is no other impact in complexity here, IMHO. Quite honestly, I have tried to ignore the macro-ization that has been done recently, although whenever I see ten commits in a row to implement them, I do curse loudly. I would never use those macros with today's implementation, because they just get in the way of me knowing what is going on. great, that means that EVERY single call that deals with time in EVERY single APR app needs to change. And, it means that calling those functions will just become a painful exercise for new programmers. Let's see, I need to pass an apr_time value, which is a binary usec implementation, now, how do I get binary usec's again? I guess I'll go read the docs to find out. Programming used to make sense, because the types made sense and for simple things, like timeouts, programmers could use simple values. Now, our timeouts are incredibly complex, which makes using them painful. :-( No, this is _NO_ more difficult than going back to the docs to figure out what APR_FOO_FLAG values you must use. Look at any sources, and you will see examples (including apr's own code, once Brian and I are finished cleaning up.) You're right, it is no more difficult, it is equally as difficult. My complaint is that the concept of time is FAR simpler than file or network operations. I don't mind complex API's for complex problems, but time is a simple complex. It is an offset from a set point in time, or an offset from an arbitrary point in time. Why is such a simple concept causing such a complex solution? 2. we already have this issue on win32, and I don't like where we are right now. Win32 is using msec, but we pass and return usec values. That's just sure to introduce bugs. I'm going to cache *two* values in the socket_t structure, s
RE: Why not POSIX time_t?
For clarity, there is only one reason that we aren't just using POSIX's time_t. While Windows has time_t, it doesn't use time_t's internally. Instead, it uses a completely different epoch with 100 nano-second resolution. The only other reason for apr_time_t is to get usec resolution. From: Aaron Bannert [mailto:[EMAIL PROTECTED] On Sun, Jul 14, 2002 at 07:27:04PM -0700, Brian Pane wrote: The reason we don't use a struct (timeval or any variant thereof) is that doing addition and subtraction on the struct is much slower, more complicated, and (if people try to do their own match on the struct directly) more error-prone than doing the same ops on a scalar. How exactly is the subtraction slower?I'm not at all sure what you mean by people matching on the struct directly... In order to do subtraction or struct matching, you have to deal with carry-over, because you have two numbers to add or compare. There is a big part of me that believes that Apache should just use time_t, while APR continues to implement apr_time_t for other applications. Time_t is used if you want second resolution, and apr_time_t is used for more granularity. BTW, this whole conversation started because we wanted to speed up Apache. Has anybody considered taking a completely different tack to solve this problem? I know there is a patent on this, but I am willing to ignore it, and I am pretty sure that we can get the patent owner to let us use it. (I won't say who owns the patent, but if the owner wants to stand up, it will be obvious why I say this). We could create a separate thread to keep track of the current time every second. That way, the computation is completely removed from request processing it is just a memory access. On platforms without threads, we just go back to getting the time during request processing. That will hurt performance for those platforms, but I am not too concerned about that. Ryan
RE: more notes on the apr_time_t issue
On Sun, 14 Jul 2002, William A. Rowe, Jr. wrote: At 09:54 PM 7/14/2002, Ryan Bloom wrote: The macros are no faster than the constant. The implementation is faster. Completely different. This statement really confused me. Please explain. You made the statement that the macros were considerably faster than the current scalar. That isn't true. The macros that are currently implemented are equivalant to the constant scalar. The new macros that implement things in binary time are faster than the current scalar, but that has nothing to do with whether it is macro-ized or not, it is all about the implementation. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: [PATCH] possible optimization in apr_poll()?
This will work, and should be portable. Just a question, how big a
performance improvement is this? Have we hit the point where we are
optimizing code just to optimize code?
I would not support this change, simply because I don't see it making a
huge difference. We are better off fixing the apr_time_t implementation
and then looking for things like this.
Ryan
Is this at all portable? I'm making the approximation that n10 is
about the same as a /1000, and since apr_poll() isn't guaranteed to be
that accurate, this should be a good chance for an optimization, right?
-aaron
Index: srclib/apr/poll/unix/poll.c
===
RCS file: /home/cvs/apr/poll/unix/poll.c,v
retrieving revision 1.6
diff -u -r1.6 poll.c
--- srclib/apr/poll/unix/poll.c 13 Jul 2002 06:31:52 - 1.6
+++ srclib/apr/poll/unix/poll.c 15 Jul 2002 03:46:25 -
@@ -128,7 +128,7 @@
}
if (timeout 0) {
-timeout /= 1000; /* convert microseconds to milliseconds */
+timeout = 10; /* approximate /= 1000 to convert to milliseconds
*/
}
i = poll(pollset, num, timeout);
--
___
Ryan Bloom [EMAIL PROTECTED]
550 Jean St
Oakland CA 94610
---
RE: more notes on the apr_time_t issue
Evidently Justin has found the second use case. And if you are trying to preserve time when udate'ing() on a file, you better know the original time if you want to roundtrip. But that is a *BSD thing only, and they provide the POSIX values already. Four macros, by four different scales, equals 16 macros. And, at the end of the day, the values stored are useful once you leave APR. s/the values/none of the values/ ... I presume. yes. BTW, the apr_time_xmake(sec, xsec) macro is completely wrong IMHO. In order to use it, I need to provide multiple values, but that doesn't make any sense, because we have four time scales. Either provide a single value, or four. Two just doesn't make sense. You are being somewhat goofy or clueless. I have two numbers. Perhaps they are seconds and microseconds, perhaps seconds and milliseconds. Both are very common. In either case, you want an APR time. Plug them both in (from a tm struct, or whatever) and you get a result. yes, but what happens when I have three numbers? Or when my two numbers are milliseconds and microseconds? Why have three macros when one can do it? I have no problem fixing the ambiguity between _xsec_get and _to_xsec, since the difference is subtle. Perhaps apr_time_frac_to_xsec(time) instead of _xsec_get? (saying, only the fractional portion of a second?) Perhaps an example of the difference, because I don't see it. Skipping the algebraic differences? Simply you have two things; Give me a timeout in milliseconds == apr_time_as_msec(apr_time_value) Fill me a tm structure; tm.sec == apr_time_sec_get(apr_time_value) tm.usec == apr_time_usec_get(apr_time_value) There is a big difference here, the first apr_time_as_msec gave you the WHOLE time, integers and fraction of a second. The second case, _get, got you only the intregal seconds and the intregal usec fraction of a second. If you want _frac_ in the macro, that's fine. Got it thanks for the explanation. Sorry you find them obtuse. Perhaps Aaron's pointer to the Veto early, veto often, prevent people from wasting their efforts shouldn't be changed at all. I'd suggest you snooze, you lose, but of course this is only two weeks old, not two years old. I am going to ignore this flame bait. Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
Re: Faster time_now() [Was: Why not POSIX time_t?]
On Mon, 15 Jul 2002, William A. Rowe, Jr. wrote: At 10:03 PM 7/14/2002, Ryan Bloom wrote: BTW, this whole conversation started because we wanted to speed up Apache. Has anybody considered taking a completely different tack to solve this problem? I know there is a patent on this, but I am willing to ignore it, and I am pretty sure that we can get the patent owner to let us use it. (I won't say who owns the patent, but if the owner wants to stand up, it will be obvious why I say this). We could create a separate thread to keep track of the current time every second. That way, the computation is completely removed from request processing it is just a memory access. On platforms without threads, we just go back to getting the time during request processing. That will hurt performance for those platforms, but I am not too concerned about that. Such code, of course, certainly doesn't belong in apr. That's a higher level construct that would fit well in http, or any other app that needs such performance tweaks. The app would have to decide how to live if it doesn't have threads available. And there is the matter of IP :-) Of course this doesn't belong in APR, but the only reason the whole apr_time_t discussion came up was to fix a performance problem in Apache. If that performance problem hadn't come up, would we have even looked at changing the current implementation? Ryan ___ Ryan Bloom [EMAIL PROTECTED] 550 Jean St Oakland CA 94610 ---
RE: cvs commit: apr/network_io/win32 sendrecv.c
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] wrowe 2002/07/14 23:34:33 Modified:network_io/win32 sendrecv.c Log: The last transpostion of -timeout into apr_time fixed a bug identified by TANAKA Koichi [EMAIL PROTECTED], where we used our old ms based timeout within the timeval structure usec member. This patch fixes the only exception to needing the timeout_ms. NOTICE that we first test that *timeout* is positive, since only then will timeout_ms contain anything worth using. This is the exact kind of code that is likely to introduce bugs, and why we shouldn't be duplicating the timeout value inside of APR. At some point, somebody is going to commit code to APR on Windows, that doesn't use both values. Ryan
RE: Why not POSIX time_t?
If you mean using only second-resolution times, that's an option, but not for the httpd. One of the big problems with 1.3 was that the request time was only stored with 1-second resolution. We used to have to add in custom, redundant time lookups to get better resolution whenever we needed to instrument request duration or log request times with higher precision. But http only requires 1 sec resolution. If you want better than that, then you will have a performance hit. No you won't. It's *faster* to get the time with microsecond resolution than to get it with second resolution. (gettimeofday() is faster than time().) Then get the time with microsecond resolution in the thread, but only save second resolution. This way, the 64-bit division is not happening in the middle of request processing. But what you just said is that time code is not a performance problem. No, I said the retrieval of the time (the thing for which you're proposing a separate thread) is not a performance bottleneck. So, why are we optimzing this again? Because 64-bit divisions are a performance problem, as seen in FirstBill's recent benchmarks. Looking up the time with microsecond resolution is not the problem. The problem is that we do gratuitous 64-bit math once we've retrieved the time. And the binary microsecond change is a valid remedy for that problem. Yes it is, but it isn't the ONLY valid remedy. Since that change is causing a lot of grief, I am asking us to consider other remedies. Ryan
RE: cvs commit: apr renames_pending
From: Thom May [mailto:[EMAIL PROTECTED] * [EMAIL PROTECTED] ([EMAIL PROTECTED]) wrote : wrowe 2002/07/15 00:42:35 Modified:.renames_pending Log: Two down. Don't know if I agree with the entire host of these becoming apr_socket_ foo, but, well... I won't argue. Please do. That was why I commited them, in the hopes of some discussion... Shoot, I've been in every other discussion this week. I'll just say this. The original names were to keep APR and POSIX names in sync. That was to leverage the huge number of POSIX programmers out there. We only used different names from POSIX when we were implementing something that POSIX doesn't have, or when our implementation didn't look anything like POSIX. Just a thought. Ryan
RE: Why not POSIX time_t?
++1 here. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA -Original Message- From: David Reid [mailto:[EMAIL PROTECTED] Sent: Monday, July 15, 2002 8:00 AM To: APR Dev List Subject: Re: Why not POSIX time_t? +1 from me. Let's finish this nonsense! david New proposal... leave apr_time_t exactly as it is. The performance problem is with how we are converting an apr_time_t into a value with 1 second resolution (ie, doing 64 bit divisions). I propose we introduce some new macros (or functions) to efficiently remove resolution from apr_time_t and do an approximate (not precise) conversion to seconds. Aaron (I believe) has already demonstrated this idea. This is the least invasive, easiest to understand and workable solution IMHO. Bill If you mean using only second-resolution times, that's an option, but not for the httpd. One of the big problems with 1.3 was that the request time was only stored with 1-second resolution. We used to have to add in custom, redundant time lookups to get better resolution whenever we needed to instrument request duration or log request times with higher precision. But http only requires 1 sec resolution. If you want better than that, then you will have a performance hit. No you won't. It's *faster* to get the time with microsecond resolution than to get it with second resolution. (gettimeofday() is faster than time().) Then get the time with microsecond resolution in the thread, but only save second resolution. This way, the 64-bit division is not happening in the middle of request processing. But what you just said is that time code is not a performance problem. No, I said the retrieval of the time (the thing for which you're proposing a separate thread) is not a performance bottleneck. So, why are we optimzing this again? Because 64-bit divisions are a performance problem, as seen in FirstBill's recent benchmarks. Looking up the time with microsecond resolution is not the problem. The problem is that we do gratuitous 64-bit math once we've retrieved the time. And the binary microsecond change is a valid remedy for that problem. Yes it is, but it isn't the ONLY valid remedy. Since that change is causing a lot of grief, I am asking us to consider other remedies. Ryan
RE: network_io/win32/poll.c
That version isn't used anymore, and it should be removed. I believe wrowe left it in the directory for reference purposes, and was going to remove it in time. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA -Original Message- From: Brad Nicholes [mailto:[EMAIL PROTECTED] Sent: Monday, July 15, 2002 10:10 AM To: dev@apr.apache.org Subject: network_io/win32/poll.c Should the Win32 version of poll.c in the network_io directory be removed from CVS? If Win32 still uses there own version, shouldn't it be moved to poll/win32? Brad Brad Nicholes Senior Software Engineer Novell, Inc., the leading provider of Net business solutions http://www.novell.com
RE: network_io/win32/poll.c
Feel free to 'cvs rm' it. If it is only there for reference, people can get to it from the Attic. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA -Original Message- From: Brad Nicholes [mailto:[EMAIL PROTECTED] Sent: Monday, July 15, 2002 10:22 AM To: dev@apr.apache.org; [EMAIL PROTECTED] Subject: RE: network_io/win32/poll.c If it isn't used any more and is need only for reference, can we at least move it somewhere else? Since NetWare uses the Win32 Network_io source and the unix poll.c, it confuses our make files to find the same named file in both directories. I keep having to manually delete the file before I build so that I don't get the wrong copy. Brad Brad Nicholes Senior Software Engineer Novell, Inc., the leading provider of Net business solutions http://www.novell.com Ryan Bloom [EMAIL PROTECTED] Monday, July 15, 2002 11:16:01 AM That version isn't used anymore, and it should be removed. I believe wrowe left it in the directory for reference purposes, and was going to remove it in time. Ryan -- Ryan Bloom [EMAIL PROTECTED] 645 Howard St. [EMAIL PROTECTED] San Francisco, CA -Original Message- From: Brad Nicholes [mailto:[EMAIL PROTECTED] Sent: Monday, July 15, 2002 10:10 AM To: dev@apr.apache.org Subject: network_io/win32/poll.c Should the Win32 version of poll.c in the network_io directory be removed from CVS? If Win32 still uses there own version, shouldn't it be moved to poll/win32? Brad Brad Nicholes Senior Software Engineer Novell, Inc., the leading provider of Net business solutions http://www.novell.com
