Re: Working directory of script is / !
On Wed, 30 Jul 2003, Stas Bekman wrote: Perrin Harkins wrote: On Tue, 2003-07-29 at 07:23, Stas Bekman wrote: That's correct. This is because $r-chdir_file in compat doesn't do anything. The reason is that under threaded mpm, chdir() affects all threads. Of course we could check whether the mpm is prefork and do things the old way, but that means that the same code won't work the same under threaded and non-threaded mpms. Hence the limbo. Still waiting for Arthur to finish porting safecwd package, which should resolve this problem. When he does finish it, won't we make the threaded MPM work just like this? It seems like it would be reasonable to get prefork working properly, even if the threaded MPM isn't ready yet. It's a tricky thing. If we do have a complete implementation then it's cool. If not then we have a problem with people testing their code on prefork mpm and then users getting the code malfunctioning on the threaded mpms. I think we could have a temporary subclass of the registry (e.g.: ModPerl::RegistryPrefork) which will be removed once the issue is resolved. At least it'll remind the developers that their code won't work on the threaded mpm setups. However if they make their code working without relying on chdir then they can use Modperl::Registry and the code will work everywhere. What's wrong with having the chdir code check for the threaded mpm, and, if it detects it, generate a warning that describes the situation? Admittedly, I have a difficult time understanding someone who tests under one mpm, and then releases under another mpm without testing. I realize there are people who do this sort of thing; I'm merely stating that I have difficulty understanding them. Ed
RE: Compiled-in but not recognized
On Sun, 11 Aug 2002, Colin wrote: -Original Message- From: Ged Haywood [mailto:[EMAIL PROTECTED]] Sent: Sunday, August 11, 2002 6:02 PM Subject: Re: Compiled-in but not recognized Hi there, On Sun, 11 Aug 2002, Colin wrote: I know this is a recurring problem but bear with me ... :) httpd -l Compiled-in modules: http_core.c mod_so.c mod_perl.c pwd? I think that Ged was suggesting you might have multiple httpd binaries on your system, and was suggesting that you verify you're running the binary you think you're running. It's really annoying when you're trying to debug a program, and the program you're running is not the one you're adding the debugging statements to. However, I suspect most of us have done it on occasion. Ed How the #@*! is it getting past all those debug statements without hitting any?!?! - Me
Re: PerlChildInitHandler doesn't work inside VirtualHost?
On Thu, 8 Aug 2002, Rick Myers wrote: On Aug 09, 2002 at 12:16:45 +1000, Cees Hek wrote: Quoting Jason W May [EMAIL PROTECTED]: Running mod_perl 1.26 on Apache 1.3.24. I've found that if I place my PerlChildInitHandler inside a VirtualHost block, it is never called. It doesn't really make sense to put a PerlChildInitHandler inside a VirtualHost directive. Why? The Eagle book says this is a perfectly valid concept. Well, for one thing, it would only call the handler if a request to that virtual host was the first request for that child. Assuming it works; I'd think this would be a good canidate for a case that's never been tested before, due to the fact that it would not call the handler if the request that initiated the child was not to that virtual host... It would fail to work in all cases if Apache does not recognize what triggered the child until after child init. Looking over pages 59 through 61, 72 and 73, this appears to me to be the case. Yes, it does explicitly say that it's ok in virtual host blocks, but it doesn't say it works. Ed
Re: mod perl load average too high
That looks like there's something that occasionally goes off and starts spinning, given the low memory usage and the fact that some processes using little cpu are also not swapped out. I suspect that one of your pages has a potential infinite loop that's being triggered. Try and catch at what point the load suddenly starts rising, and check what pages were accessed around that time. They're where you should start looking. Note that you should probably focus on the access and error log lines that correspond with processes that are using excessive amounts of cpu. Ed On Tue, 6 Aug 2002, Anthony E. wrote: I'm using apache 1.3.26 and mod_perl 1.27 My apache processes seem to be taking up more and more system resources as time goes on. Can someone help me determine why my server load is going up? When i first start apache, my load average is about .02, but after a couple of hours, it goes up to 4 or 5, and after a couple of days, has been as high as 155. I have the following directives configured in httpd.conf: MaxKeepAliveRequests 100 MinSpareServers 5 MaxSpareServers 20 StartServers 10 MaxClients 200 MaxRequestsPerChild 5000 Here is a snip of 'top' command: 6:28pm up 46 days, 23:03, 2 users, load average: 2.24, 2.20, 1.98 80 processes: 74 sleeping, 6 running, 0 zombie, 0 stopped CPU0 states: 99.3% user, 0.2% system, 0.0% nice, 0.0% idle CPU1 states: 100.0% user, 0.0% system, 0.0% nice, 0.0% idle Mem: 1029896K av, 711884K used, 318012K free, 0K shrd, 76464K buff Swap: 2048244K av, 152444K used, 1895800K free 335796K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 25893 nobody16 0 10188 9.9M 3104 R95.5 0.9 21:55 httpd 25899 nobody16 0 9448 9448 3104 R95.3 0.9 63:27 httpd 25883 nobody 9 0 10468 10M 3096 S 2.5 1.0 0:16 httpd 25895 nobody 9 0 10116 9.9M 3104 S 2.1 0.9 0:15 httpd 25894 nobody 9 0 10240 10M 3104 S 1.9 0.9 0:16 httpd 25898 nobody 9 0 10180 9.9M 3100 S 1.7 0.9 0:13 httpd Also, I notice in my error_log i get this entry quite frequently: 26210 Apache::DBI new connect to 'news:1.2.3.4.5userpassAutoCommit=1PrintError=1' What can i do to keep the server load low? = Anthony Ettinger [EMAIL PROTECTED] http://apwebdesign.com home: 415.504.8048 mobile: 415.385.0146 __ Do You Yahoo!? Yahoo! Health - Feel better, live better http://health.yahoo.com
Re: [announce] mod_perl-1.99_01
On Mon, 8 Apr 2002, Stas Bekman wrote: Ged Haywood wrote: Compilations should be SILENT unless something goes wrong. The build process takes time, if you don't give an indication of what's going on users will try to do funky things. Since the build process is comprised of many small sub-processes you cannot really use something like completion bar. As someone said, redirect the output to a temporary location. But, add to that one of those little | bars, which turns one position every time another build step completes (each file compiled, each dependancy file built, etc.). However, in the case of an error, I would want the whole thing available. Possibly something along the lines of, the last build step and all output from then on printed to stdout (or stderr), ended with, For the full build log, see /tmp/mod_perl.build.3942 or some such. Also remember that mod_perl build prints very little, it's the compilation messages that generated most of the output. p.s. I don't recall seeing silent build processes at all. The only ones I've seen went too far the other way. I especially loved the one which used a shell script, which started out with a dozen large shell functions, then an 'exec /dev/null 2/dev/null', then a half-dozen more large shell functions, and ending with 'main $'. When the shell script finished, its caller checked its exit code, and reported 'done.' or 'failed.' as appropriate. Admittedly, I wouldn't have minded too much, except that I'd gotten the latter answer. Ed
Re: Apache::DBI or What ?
On Sun, 24 Mar 2002, Andrew Ho wrote: What would be ideal is if the database would allow you to change the user on the current connection. I know PostgreSQL will allow this using the command line interface psql tool (just do \connect database user), but I'm not sure if you can do this using DBI. Does anyone know if any datbases support this sort of thing? This occurred to me in the case of Oracle (one of my co-workers was facing a very similar problem in the preliminary stages of one of his designs), and I actually had asked our DBAs about this (since the Oracle SQL*Plus also allows you to change users). As I suspected (from the similar connect terminology), our DBAs confirmed that Oracle just does a disconnect and reconnect under the hood. I would bet the psql client does the same thing. First, I'll suggest that there are hopefully other areas you can look at optimizing that will get you a bigger bang for your time - in my test environment (old hardware), it takes 7.4 ms per disconnect/reconnect/rebind and 4.8 ms per rebind. Admittedly, I'm dealing with LDAP instead of SQL, and I've no idea how they compare. If the TCP connection were retained, this could still be a significant win. *Any* reduction in the connection overhead is an improvement. If there are a million connects per day, and this saves a milli-second per connect (believable to me, as at least three packets don't need to be sent - syn, syn ack, and fin. My TCP's a bit fuzzy, but I think there's a couple more, and there's also the mod_perl disconnect/reconnect overhead), that's over 15 minutes of response time and about 560,000,000 bits of network bandwidth (assuming the DB is not on the same machine) saved. Admittedly, at 100Mb/s, that's only 6 seconds. It may, in some cases, still be necessary to move access control from the DB into ones application, so one can maintain a single connection which never rebinds, but I think it's better to utilize the security in the DB instead of coding ones own - more eyes have looked over it. We're talking about a fairly small unit of time; it may very well be better to throw money if you are near your performance limit. Ed
Re: Performace...
On Sun, 24 Mar 2002, Kee Hinckley wrote: At 2:27 PM -0500 3/23/02, Geoffrey Young wrote: you might be interested in Joshua Chamas' ongoing benchmark project: [EMAIL PROTECTED]">http://mathforum.org/epigone/modperl/sercrerdprou/[EMAIL PROTECTED] http://www.chamas.com/bench/ he has the results from a benchmark of Apache::Registry and plain handlers, as well as comparisons between HTML::Mason, Embperl, and other templating engines. Although there are lots of qualifiers on those benchmarks, I consider them rather dangerous anyway. They are Hello World benchmarks, in which startup time completely dominates the time. The things that That explains why Embperl did so poorly compared to PHP, yet when we replaced our PHP pages with Embperl, our benchmarks using real user queries, sending the same queries through the old and new pages, the new pages showed a 50% performance boost. Note: that gain was enough to saturate our test network. Our purpose for the benchmark was to determine if it was an improvement or not, not to determine the exact improvement, so we don't really know what the real gain was. The same machines do several other tasks, and our monitoring at the time of change was not very sophisticated, so we only really know it was a big win. Something on the order of 37 load issues the week before the change, most of which were fairly obviously web overload, and two the week after (those two being very obviously associated with other services the boxes are running.) Ed
Re: 'Pinning' the root apache process in memory with mlockall
Danger: Rant ahead. Proceed with caution. On Sat, 23 Mar 2002, Stas Bekman wrote: See the discussion on the [EMAIL PROTECTED] list, http://marc.theaimsgroup.com/?t=10165973081r=1w=2 where it was said that it's a very bad idea to use mlock and variants. Moreover the memory doesn't get unshared when the parent pages are paged out, it's the reporting tools that report the wrong information and of course mislead the the size limiting modules which start killing the processes. As a conclusion to this thread I've added the following section to the performance chapter of the guide: =head3 Potential Drawbacks of Memory Sharing Restriction It's very important that the system won't be heavily engaged in swapping process. Some systems do swap in and out every so often even if they have plenty of real memory available and it's OK. The following applies to conditions when there is hardly any free memory available. So if the system uses almost all of its real memory (including the cache), there is a danger of parent's process memory pages being swapped out (written to a swap device). If this happens the memory usage reporting tools will report all those swapped out pages as non-shared, even though in reality these pages are still shared on most OSs. When these pages are getting swapped in, the sharing will be My Solaris 2.6 box, while in this situation, was swapping hard, as measured by my ears, by iostat, and by top (both iowait and the memory stats). Note that mlockall does not restrict memory sharing, it restricts swapping a certain portion of memory. This will prevent this memory from ever being needlessly unshared. In the discussion you referred to, all of the people saying this was a bad idea were using terms like, I think. None of them had the situation themselves, so have a difficult time coming to terms with it. None of them had related former experience using this. Something like this really needs to be tested by someone who has the issue, and has the ability to do benchmarks with real data streams. If they find it seems to work well, then they should test it on production systems. Anyone else talking about it is simply that much hot air, myself included. (I *could* test it, but I don't have enough of a problem to put a priority on it. If we were waiting for me to get time, we'd be waiting a long time.) Yes, I agree, it's better to never swap. But if we take the attitude that we won't use tools to help us when times are tight, then get rid of swap entirely. Locking memory is all about being selective about what you will and won't swap. Yes, I agree, it'd be better to mlock those bits of memory that you really care about, but that's hard to do when that memory is allocated by software you didn't write. (In this case, I'd really like to mlock all the memory that perl allocated but did not free in BEGIN sections (including, of course, use blocks). I would also like to compact that first, but that could be even more difficult.) As far as the logic regarding 'let the OS decide' - the admin of the system has the ability to have a much better understanding of how the system resources are used. If I have one section of memory which is used 95% of the time by 75% of my active processes, I really don't want that memory to swap out just because another program that'll only run for a minute wants a bit more memory, if it can take that memory from anywhere else. When doing individual page-ins, memory managers tend to worry only about those processes that they are trying to make runable now; they're not going to go and load that page back on to every other page map that shares it just because they also use it. So even though that memory is loaded back into memory, all those processes will still have to swap it back. For them to do otherwise would be irresponsible, unless the system administrator clearly doesn't know how to system administrate, or has chosen not to. The OS is supposed to handle the typical case; having one segment of memory used by dozens of processes actively is not the typical case. This does not happen on end-user machines; this only happens on servers. Theoretically speaking, servers are run by people who can analyze and tune; mlock and mlockall are tools available to them to do such tuning. reported back to normal after a certain amount of time. If a big chunk of the memory shared with child processes is swapped out, it's most likely that CApache::SizeLimit or CApache::GTopLimit will notice that the shared memory floor threshold was crossed and as a result kill those processes. If many of the parent process' pages are swapped out, and the newly created child process is already starting with shared memory below the limit, it'll be killed immediately after serving a single request (assuming that we the C$CHECK_EVERY_N_REQUESTS is set to one). This is a very bad situation which will eventually lead to a state where the system won't
Re: Berkeley DB 4.0.14 not releasing lockers under mod_perl
Does shutting down apache free up your locks?
(As an aside, I'm not sure I'll ever get over undef being proper closing
of a database connection; it seems so synonomous to free([23]). I
expect something like $db-db_close() or something.)
Ed
On Thu, 21 Mar 2002, Dan Wilga wrote:
At 2:03 PM -0500 3/21/02, Aaron Ross wrote:
I'm testing with the Perl script below, with the filename ending
.mperl (which, in my configuration, causes it to run as a mod_perl
registry script).
I would re-write it as a handler and see if Apache::Registry is partly
to blame.
I tried doing it as a handler, using the configuration below (and the
appropriate changes in the source) and the problem persists. So it
doesn't seem to be Registry's fault.
Location /dan
SetHandler perl-script
PerlHandler DanTest
/Location
source code
#!/usr/bin/perl
package DanTest;
use strict;
use BerkeleyDB qw( DB_CREATE DB_INIT_MPOOL DB_INIT_CDB );
my $dir='/home/httpd/some/path';
sub handler {
system( rm $dir/__db* $dir/TESTdb );
foreach( 1..5 ) {
my $env = open_env($dir);
my %hash;
my $db = open_db( TESTdb, \%hash, $env );
untie %hash;
undef $db;
undef $env;
}
print HTTP/1.1 200\nContent-type: text/plain\n\n;
print `db_stat -c -h $dir`;
print \n;
}
sub open_env {
my $env = new BerkeleyDB::Env(
-Flags=DB_INIT_MPOOL|DB_INIT_CDB|DB_CREATE,
-Home= $_[0],
);
die Could not create env: $! .$BerkeleyDB::Error. \n if !$env;
return $env;
}
sub open_db {
my( $file, $Rhash, $env ) = @_;
my $db_key = tie( %{$Rhash}, 'BerkeleyDB::Btree',
-Flags=DB_CREATE,
-Filename=$file,
-Env=$env );
die Can't open $file: $! .$BerkeleyDB::Error.\n if !$db_key;
return $db_key;
}
1;
Dan Wilga [EMAIL PROTECTED]
Web Technology Specialist http://www.mtholyoke.edu
Mount Holyoke CollegeTel: 413-538-3027
South Hadley, MA 01075Seduced by the chocolate side of the Force
RE: loss of shared memory in parent httpd
I believe I have the answer... The problem is that the parent httpd swaps, and any new children it creates load the portion of memory that was swaped from swap, which does not make it copy-on-write. The really annoying thing - when memory gets tight, the parent is the most likely httpd process to swap, because its memory is 99% idle. This issue aflicts Linux, Solaris, and a bunch of other OSes. The solution is mlockall(2), available under Linux, Solaris, and other POSIX.1b compliant OSes. I've not experimented with calling it from perl, and I've not looked at Apache enough to consider patching it there, but this system call, if your process is run as root, will prevent any and all swapping of your process's memory. If your process is not run as root, it returns an error. The reason turning off swap works is because it forces the memory from the parent process that was swapped out to be swapped back in. It will not fix those processes that have been sired after the shared memory loss, as of Linux 2.2.15 and Solaris 2.6. (I have not checked since then for behavior in this regard, nor have I checked on other OSes.) Ed On Thu, 14 Mar 2002, Bill Marrs wrote: It's copy-on-write. The swap is a write-to-disk. There's no such thing as sharing memory between one process on disk(/swap) and another in memory. agreed. What's interesting is that if I turn swap off and back on again, the sharing is restored! So, now I'm tempted to run a crontab every 30 minutes that turns the swap off and on again, just to keep the httpds shared. No Apache restart required! Seems like a crazy thing to do, though. You'll also want to look into tuning your paging algorithm. Yeah... I'll look into it. If I had a way to tell the kernel to never swap out any httpd process, that would be a great solution. The kernel is making a bad choice here. By swapping, it triggers more memory usage because sharing removed on the httpd process group (thus multiplied)... I've got MaxClients down to 8 now and it's still happening. I think my best course of action may be a crontab swap flusher. -bill
Re: [OT] RE: modperl growth
On Tue, 5 Feb 2002, Dave Rolsky wrote: On Mon, 4 Feb 2002, Andrew Ho wrote: One last thing that is hard is where is your DocumentRoot? This is a huge problem for web applications being installable out of the box. Perl can't necessarily figure that out by itself, either. You take a guess and then ask the user to confirm. And you can't guess you just ask. That's a good strategy (assuming a missing if in there somewhere). It can be augmented with the tactic of check for a running apache, see where it gets its config file from, and parse the config file to get the initial guess. (Note that I wouldn't want this to be a final guess; I'm using mod_perl in a virtual host config; the main apache config doesn't use it, and has a completely unrelated docroot (/usr/local/apache/htdocs as opposed to /home/appname/public_html)) There's nothing wrong with an interactive installer. What kills mod_perl apps is they simply have a README or INSTALL that says Copy all the template files to a directory called 'app-root' under your document root. My what? Which files are templates? I don't know this unix stuff; copy doesn't work right. I think we've all probably heard these words before... I guess my point is that installation is hard. Rather than trying to make it work for everybody out of the box, you should make it work for the typical case out of the box, and then provide hooks for installing it in custom places. I think the best installer is an interactive installer that tries really hard to provide good defaults. I agree; while I frequently leave unimportant considerations alone (note my main docroot above), I tend to have very poor luck with the works with the typical case out of the box, and then provides hooks which change with every bloo^W^W^W^W^Wfor installing it in custom places. I won't go into speculations why. Ed
Re: performance coding project? (was: Re: When to cache)
On Sat, 26 Jan 2002, Perrin Harkins wrote: It all depends on what kind of application do you have. If you code is CPU-bound these seemingly insignificant optimizations can have a very significant influence on the overall service performance. Do such beasts really exist? I mean, I guess they must, but I've never seen a mod_perl application that was CPU-bound. They always seem to be constrained by database speed and memory. I've seen one. However, it was much like a normal performance problem - the issue was with one loop which ran one line which was quite pathological. Replacing loop with an s///eg construct eliminated the problem; there was no need for seemlingly insignificant optimizations. (Actually, the problem was *created* by premature optimization - the coder had utilized code that was more efficient than s/// in one special case, to handle a vastly different instance.) However, there could conceivably be code which was more of a performance issue, especially when the mod_perl utilizes a very successful cache on a high traffic site. On the other hand how often do you get a chance to profile your code and see how to improve its speed in the real world. Managers never plan for debugging period, not talking about optimizations periods. Unless there's already a problem, and you have a good manager. We've had a couple of instances where we were given time (on the schedule, before the release) to improve speed after a release. It's quite rare, though, and I've never seen it for a mod_perl project. Ed
Re: UI Regression Testing
On Sat, 26 Jan 2002, Gunther Birznieks wrote: I agree that testing is great, but I think it is quite hard in practice. Also, I don't think programmers are good to be the main people to write their own tests. It is OK for programmers to write their own tests but frequently it is the user or a non-technical person who is best at doing the unexpected things that are really were the bug lies. My experience is that the best testers come from technical support, although this is not to suggest that all technical support individuals are good at this; even among this group, it's rare. Users or other non-technical people may find a few more bugs, but frequently, the non-technical people don't have the ability to correctly convey how to reproduce the problems, or even what the problem was. I clicked on the thingy, and it didn't work. This being said, users and tech support can't create unit tests; they're not in a position to. Finally, unit tests do not guarantee an understanding of the specs because the business people generally do not read test code. So all the time spent writing the test AND then writing the program AND ONLY THEN showing it to the users, then you discover it wasn't what the user actually wanted. So 2x the coding time has been invalidated when if the user was shown a prototype BEFORE the testing coding commenced, then the user could have confirmed or denied the basic logic. For your understanding of the spec, you use functional tests. If your functional test suite uses test rules which the users can understand, you can get the users to double-check them. For example, at work, we use a suite which uses a rendered web page as its test output, and the input can be sent to a web page to populate a form; this can be read by most people who can use the application. Unit software is a means of satisfying a spec, but it doesn't satisfy the spec itself - if it did, you'd be talking about the entire package, and therefore refering to functional testing. (At least, this is the way I distinguish between them.) Admittedly, we are a bit lacking in our rules, last I checked. Ed
Re: Single login/sign-on for different web apps?
On Wed, 16 Jan 2002, Paul Lindner wrote: On Wed, Jan 16, 2002 at 06:56:37PM -0500, Vsevolod Ilyushchenko wrote: 3) Perl-based applications can just use the module and the common key to decrypt the contents of the cookie to find the authenticated username. If the cookie is not present redirect to the central authentication page, passing in the URL to return to after authentication. Hmmm... Can I do it securely without using Kerberos? I think so. Looks like if I use https instead of http, people won't be able to steal my (encoded) session information as it is transmitted. And I can also add the IP address to the cookie information. But the cookies file might be readable by other people! If they can steal that file and change the IP address of another machine to yours, they can pretend they are you! I wonder if there is a way out of this... Yes, you use the timestamp. Just reauthenticate the user when they try to do 'sensitive' activities. No, use session cookies - they're not stored on disk. If you need the system to retain knowledge through a browser shutdown, you can use a timestamped cookie to retain the user ID, but don't have it allow them to do anything other than not have to type their user ID in again (password screen has user ID filled out for them.) One can also mark the cookies such that they'll only be transmitted over https. $cookie = CGI::Cookie-new(-name = 'login', -value = tgape::setcookiepassword($uid, $pass), -domain = '.tgape.org', -path = '/', -secure = 1, ); If you feel the need to timestamp your session cookies, make the cookie include an encrypted timestamp. For example you might allow someone to view their bank balance if they typed their password within the last 2 hours. Transferring money might require a valid password within the last 10 minutes.. Ah, but many systems will refresh a cookie on activity. So they view their balance, get a new cookie, and then transfer money. Of course, the best authentication system for banking I've seen is from UBS. They send you a scratchlist of around 100 numbers. Every time you login you use one of the numbers and cross it off. Very slick. All I need to do is find where you left the list. Written passwords are not anywhere near as secure as memorized passwords, unless the person carrying them around is really conscious about security concerns. Ed
RE: mod_perl beginners list
On Tue, 15 Jan 2002, Robert Landrum wrote: At 10:22 PM + 1/15/02, Matt Sergeant wrote: On Tue, 15 Jan 2002, Robert Landrum wrote: I've seen nothing on this list that suggests that new users shouldn't ask questions. If they don't ask questions because they're afraid of the response they might get, then maybe they should stay home and leave the programming to those people who have mettle to ask. I know where the sentiment comes from, but I really hope people don't read that and stay away in fear. Really folks, we're friendly here, so long as you play by the rules: quote cleanly, don't post HTML, and ask politely. Absolutly. My response was addressing Joe's statment that users are too intimidated to post. I disagree. True programmers know no fear. True programmers know no fear of computers. However, any programmer who knows fear of RTFM would likely not post to any perl mailing list that didn't have beginner or newbie in the name, due to having experience with such lists before ever hearing about mod_perl. Count me as someone who would be interested in being on the list, though not necessarily very active. Ed
Re: Single login/sign-on for different web apps?
No. There are very important reasons why Apache by default puts an ACL restricting .ht* from being viewable. (Basically, the password encryption used in said file is moderately easily cracked via brute force.) One could use a file distributed using rsync(1) or some such (preferably with RSYNC_RSH=ssh). However, that's still a bit on the unsecure side, unless you really do trust everyone who is running one of these web servers. Ed On Wed, 16 Jan 2002, Medi Montaseri wrote: I wonder if one could change the HTTP Server's behavior to process a distributed version of AuthUserFile and AuthGroupFile. That instead of AuthUserFile /some/secure/directory/.htpasswd One would say AuthUserFile http://xyz.com/some/directory/htpasswd; Then write a GUI (web) inteface to this password and group file and you have distributed authentication system. Ed Grimm wrote: On Wed, 16 Jan 2002, Medi Montaseri wrote: I think Netegrity single sing-on system modifies the HTTP server (possible with mod_perl) to overload or override its native authoentication and instead contact a Host, Database or LDAP to get the yes or no along with expiration data it then sends its finding to the CGI by sending additonal HTTP Header info. A CGI program can then go from there... Something like this. Basically, it has modules, plugins, or access instructions, as appropriate, for various web servers, to configure them to use it. I know it gives an LDAP interface, and I'm assuming it gives an LDAPS interface; I'm not sure what others it may present. I might not have this 100%, but perhaps we can learn from those commercial products. Someone suggested using LDAP and RDBMS, my question is why both, why not just RDBMS. Why not just LDAP? As someone working on rolling out a single sign-on solution with LDAPS, I really want to know... (We're planning on getting Netegrity for its distributed administration stuff; at that time, we'll start using its web server configuration stuff for any new web servers. Until that time, we're rolling out LDAPS, and we're not currently planning on converting systems we roll out in the interm to Netegrity.) Incidentally, we're being a bunch of lazy bums, compared to the rest of y'all. We're considering single sign-on to mean they only need to keep track of one userid and password (unless they need to access classified or otherwise restricted stuff.) If they go to different sites and have to log on again, we don't currently care. (Basically, we have too many sites created by too many groups. We'll share the same login between servers run by the same group, but beyond that, security concerns outweigh user convinience.) Ed Aaron Johnson wrote: We are working on/with a similar system right now. We have an application that is written in Perl, but the people visiting will most likely be signing on at a different point then our applications sign in page. Our system was built to use its own internal database for authentication and their app/login uses a different method. The design requirements were that each side would have to do as little possible to modify there application to work in our single sign on solution. We have the luxury of not being overly concerned with the security aspect so please keep that in mind. We setup a nightly sync program that verifies the data in the current database vs. their login user information database. Here is a less then detailed summary of how the system operates. 1) The user logs into the application through their application and they are sent a cookie that contains the user name. 2) All links to our application are sent to a single page on their end with the full url of the page they want as part of the query string. 3) They verify that the user is valid using whatever method they want. 4) The user is then redirected to a special page in our application that expects the query string to contain two items, the user name and the final URL to go to. 5) Our application verifies the HTTP_REFFERER and the query string contains valid values. 6) Our application checks the database for a user matching the name sent in. Then if the user already has a session if they do then they are redirected to the correct page, otherwise it does a lookup in our system to create a session for the user based on the incoming user name and then redirects to the final URL. Now a user can go between the two applications without concern since they have a cookie for each domain. If the user comes to our site the reverse of the above occurs. This allowed us to plug into existing applications without a lot of rework. It is also fairly language/platform independent. As stated above I know there are some large security issues with this approach
Re: Single login/sign-on for different web apps?
On Wed, 16 Jan 2002, Medi Montaseri wrote: I think Netegrity single sing-on system modifies the HTTP server (possible with mod_perl) to overload or override its native authoentication and instead contact a Host, Database or LDAP to get the yes or no along with expiration data it then sends its finding to the CGI by sending additonal HTTP Header info. A CGI program can then go from there... Something like this. Basically, it has modules, plugins, or access instructions, as appropriate, for various web servers, to configure them to use it. I know it gives an LDAP interface, and I'm assuming it gives an LDAPS interface; I'm not sure what others it may present. I might not have this 100%, but perhaps we can learn from those commercial products. Someone suggested using LDAP and RDBMS, my question is why both, why not just RDBMS. Why not just LDAP? As someone working on rolling out a single sign-on solution with LDAPS, I really want to know... (We're planning on getting Netegrity for its distributed administration stuff; at that time, we'll start using its web server configuration stuff for any new web servers. Until that time, we're rolling out LDAPS, and we're not currently planning on converting systems we roll out in the interm to Netegrity.) Incidentally, we're being a bunch of lazy bums, compared to the rest of y'all. We're considering single sign-on to mean they only need to keep track of one userid and password (unless they need to access classified or otherwise restricted stuff.) If they go to different sites and have to log on again, we don't currently care. (Basically, we have too many sites created by too many groups. We'll share the same login between servers run by the same group, but beyond that, security concerns outweigh user convinience.) Ed Aaron Johnson wrote: We are working on/with a similar system right now. We have an application that is written in Perl, but the people visiting will most likely be signing on at a different point then our applications sign in page. Our system was built to use its own internal database for authentication and their app/login uses a different method. The design requirements were that each side would have to do as little possible to modify there application to work in our single sign on solution. We have the luxury of not being overly concerned with the security aspect so please keep that in mind. We setup a nightly sync program that verifies the data in the current database vs. their login user information database. Here is a less then detailed summary of how the system operates. 1) The user logs into the application through their application and they are sent a cookie that contains the user name. 2) All links to our application are sent to a single page on their end with the full url of the page they want as part of the query string. 3) They verify that the user is valid using whatever method they want. 4) The user is then redirected to a special page in our application that expects the query string to contain two items, the user name and the final URL to go to. 5) Our application verifies the HTTP_REFFERER and the query string contains valid values. 6) Our application checks the database for a user matching the name sent in. Then if the user already has a session if they do then they are redirected to the correct page, otherwise it does a lookup in our system to create a session for the user based on the incoming user name and then redirects to the final URL. Now a user can go between the two applications without concern since they have a cookie for each domain. If the user comes to our site the reverse of the above occurs. This allowed us to plug into existing applications without a lot of rework. It is also fairly language/platform independent. As stated above I know there are some large security issues with this approach. Aaron Johnson Vsevolod Ilyushchenko wrote: Hi, Have you ever ran into the problem of putting up many separate web apps on several machines in your company/university/whatever that are written from scratch or downloaded frow the Web and each of which has its own user database? What would you think is a good way to make the system seem more cohesive for the users? It occurs to me that 1) for the single login they all should use the same user database (obviously :), and 2) for the single sign-on there must be a way of storing the session information. That is, once I login in the morning to the first app, I get a cookie, a ticket or something similar, and then, as I go from app to app, I will not have to re-enter my credentials because they are supplied by a cookie. Apache::Session sounds like the right tool for the job. (Did I hear Kerberos? :) Has anyone had experince with this kind of app integration? The downside I see is that once I settle on a particular scheme to do it, I will have to build this scheme into every app that
Re: Single login/sign-on for different web apps?
On Thu, 17 Jan 2002, Gunther Birznieks wrote: Of course, the best authentication system for banking I've seen is from UBS. They send you a scratchlist of around 100 numbers. Every time you login you use one of the numbers and cross it off. Very slick. Does that really work in practice? That sounds really annoying. Is this for business banking or for retail? How do they get the next 100 numbers to the user? Do they mail it out when they've used 90? It sounds like it would be less annoying to use certificates and some plug-in token there is going to be that much extra work to deal with a password sheet. Alternately, for a high-tech approach, RSA makes a nice product called a SecurID token (Well, one of mine says Security Dynamics on the back, but the new ones definitely say RSA). Actually, they make two, one nice, one not nice. The nice one has a keypad where you enter in a pin, press a button, and it generates a temporary id based on its serial number, your pin, and the current time interval; the time interval changes every minute or two. The not nice one has no keypad; it works like the other would if you didn't enter a pin. I know of several companies that use these; they tend to work fairly well. (I had one break on me, but I gave it a lot of abuse first; it lasted almost half of its battery span in spite of not being taken care of.) Ed
