Re: [AOLSERVER] AOLserver 4.5.1 (CVS) crashing bug on x64 with fix
Many thanks for the patch. it looks good. it applied with
some minor related cleanups.
-gustaf neumann
On 17.08.11 19:06, Jin Choi wrote:
Hello. I've tracked down and fixed a crashing bug in AOLserver on x64. I
believe it is correct.
AOLserver was crashing with
Fatal: munmap(0x, 0) failed: Invalid argument
I tracked down the only call to munmap in NsUnMap in nsd/fastpath.c. It takes
the return value of NsMap and tests it against NULL. NsMap is just returning
the return value from mmap(). The man page says:
RETURN VALUE
On success, mmap() returns a pointer to the mapped area. On error, the
value MAP_FAILED (that is, (void *) -1) is returned, and errno is set
appropriately.
Index: nsd/fastpath.c
===
RCS file: /cvsroot/aolserver/aolserver/nsd/fastpath.c,v
retrieving revision 1.26
diff -u -r1.26 fastpath.c
--- nsd/fastpath.c 19 Apr 2006 17:48:47 - 1.26
+++ nsd/fastpath.c 16 Aug 2011 17:40:51 -
@@ -521,7 +521,7 @@
}
if (servPtr-fastpath.mmap) {
map = NsMap(fd, 0, stPtr-st_size, 0,arg);
- if (map != NULL) {
+ if (map != MAP_FAILED) {
close(fd);
fd = -1;
result = Ns_ConnReturnData(conn, status, map,
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear all, i did some more digging/googling in this issue and i share the opinion that - at least for the time being - Tcl_Finalize() could be omitted on windows versions without too much harm. Some background: The Tcl manpage says: *Tcl_Finalize* is similar to*Tcl_Exit* except that it does not exit from the current process. It is useful for cleaning up when a process is finished using*Tcl* but wishes to continue executing, and when*Tcl* is used in a dynamically loaded extension that is about to be unloaded. On some systems*Tcl* is automatically notified when it is being unloaded, and it calls*Tcl_Finalize* internally; on these systems it not necessary for the caller to explicitly call*Tcl_Finalize*. However, to ensure portability, your code should always invoke*Tcl_Finalize* when *Tcl* is being unloaded, to ensure that the code will work on all plat- forms.*Tcl_Finalize* can be safely called more than once. For aolserver, it is questionable for me why we need Tcl_Finalize() (the primarily purpose for Tcl_Finalize according to its documentation is that the process wants to continue without Tcl), furthermore there seems to be some magic involved, that some systems call Tcl_Finalize() ... automatically (hinting most likely the windows situation with the assembly code). Since finalize tries to unload Tcl, there seems to be some race conditions in this area on windows, at least when there are still are multiple threads around. E.g. [1] says: Because DLL notifications are serialized, entry-point functions should not attempt to communicate with other threads or processes. Deadlocks may occur as a result. Neither aolserver 4.0.10 nor naviserver call Tcl_Finalize(), so i guess we can live with a 4.5.1+ version under windows without it. -gustaf neumann [1] http://msdn.microsoft.com/en-us/library/ms682583.aspx On 06.08.11 16:28, Maurizio Martignano wrote: It is me again... Well I noticed that the change I suggested about Tcl_Finalize did not make it into CVS HEAD. If it doesn't go there, I am afraid I will have to anyhow introduce it myself in my distribution. I need to have a working system. With that call still in, the service can't (CANNOT) be stopped gracefully. This is a matter of testing: Take the system, make it run with a real OpenACS based application (how about ]po[, or xowiki) and see how it works and see how it interacts with the system... Does it start? Does it run? Does it stop properly? For the time being in Windows 64 that function needs to be out. Thanks a lot, Maurizio Thank you, Maurizio *From:*AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] *On Behalf Of *Gustaf Neumann *Sent:* 06 August 2011 10:28 *To:* AOLSERVER@LISTSERV.AOL.COM *Subject:* Re: [AOLSERVER] Aolserver Progress - Some few examples Maurizio, Tcl_Finalize() is supposed to work, and if it does now work something is still broken in the windows version. Omitting Tcl_Finalize() is removeing the symptom, not the cause. It is not unlikely that something else will have the same problem due to this cause. When Tcl_Finalize() is not run, the registered exit handlers are not executed. How serious this is depends on the exit handlers. You are right, that the memory leak does not matter due to the shutdown. The difference is like between a graceful and an ungraceful shutdown. -gustaf On 05.08.11 16:29, Maurizio Martignano wrote: Dear Gustav, I understand your concerns about Tcl_Finalize... but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test Have Aolserver / nsd running on a big application... observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples.... All's well that ends well
On 07.08.11 17:37, Maurizio Martignano wrote:
Dear Gustav,
Thank you.
All's well that ends well...
I'm not sure all the changes I suggested are still in the
codebase... especially the ones in RED
The reason, i have not committed theses two suggested
changes to the code base is that these are incorrect and
would break compilation on the non-windows platforms. Both
suggested changes are in #ifdef branches for compilations
without _WIN32 set (therefore irrelevant for Win32 and
Win64). Please check more carefully first.
-gustaf neumann
It is not up to Aolserver/nsd include system to define
SOCKET as int on Windows.
Thank you again,
Maurizio
*include/ns.h -- 222 **ß***
*#define*O_BINARY0
// Commented out by M. _Martignano_ on the 05/08/2011
// #define SOCKET _int_
*#define*INVALID_SOCKET (-1)
I removed the define, in Windows (32/64) SOCKET should be
defined somewhere else, and not redefined here.
*include/ns.h -- 674*
// Type changed from _int_ to SOCKET by M. _Martignano_ on
the 05/08/2011
NS_EXTERN SOCKET *Ns_ConnSock*(Ns_Conn*conn);
Self explanatory.
*nsd/conn.c - 615*
// Type changed from _int_ to SOCKET by M. _Martignano_ on
the 05/08/2011
SOCKET
*Ns_ConnSock*(Ns_Conn*conn)
{
Self explanatory
*nsd/nsd.h - 75*
// Conditional compilation clause added by M. _Martignano_
on the 05/08/2011
*#ifndef*_WIN32
*struct*pollfd {
*int*fd;
*short*events;
*short*revents;
};
*#endif*
struct pollfd is already defined
*nsd/nsd.h - 312*
// _int_ trigger[2] changed into SOCKET trigger[2] by M.
_Martignano_ on the 05/08/2011
SOCKET trigger[2]; /* _Wakeup_ trigger socket. */
Well, we all know about this... don't we?
*nsd/nsmain.c -- 633*
// Conditional compilation clause added by M. _Martignano_
on the 05/08/2011
*#ifndef*_WIN32
Tcl_Finalize();
*#endif*
Semantic. Tcl_Finalize never ends on Windows so I removed it.
*nsd/queue.c -- 42*
// Function added by M. _Martignano_ on the 05/08/2011
*#ifdef*_WIN32
*static**double*round(*double*x) { *return*floor(x + 0.5); }
*#endif*
Well believe it or not round does not exist. So I ehm
re-implemented it based on floor.
*nspd/listen.c -- 80
*// Type changed from _int_ to SOCKET by M. _Martignano_
on the 05/08/2011
SOCKETsock, new;
Self explanatory
*nspd/pd.h -- 52 **ß***
*#include*unistd.h
// Conditional compilation added by M. _Martignano_ on the
05/08/2011
*#ifndef*_WIN32
Well this follows the same reasoning used for ns.h. Anyhow
this is the weakest of all my changes and all the module
should be carefully looked at. I am personally not using
it in my Windows distributions.
*nsperm/nsperm.c -- 583*
// Conditional compilation added by M. _Martignano_ on the
05/08/2011
*#ifndef*_WIN32
*if*(inet_aton(net, ip) == 0 || inet_aton(slash+1, mask)
== 0) {
*#else*
*if*(inet_pton(AF_INET,net, ip) == 0 ||
inet_pton(AF_INET,slash+1, mask) == 0) {
*#endif*
Well with Micosoft Visual C 10 inet_aton seems not to be
there any longer. Luckily we can still rely on inet_pton.
*From:*AOLserver Discussion
[mailto:AOLSERVER@LISTSERV.AOL.COM] *On Behalf Of *Gustaf
Neumann
*Sent:* 07 August 2011 15:23
*To:* AOLSERVER@LISTSERV.AOL.COM
*Subject:* Re: [AOLSERVER] Aolserver Progress - Some few
examples
Dear all,
i did some more digging/googling in this issue and i share
the opinion that - at least for the time being -
Tcl_Finalize() could be omitted on windows versions
without too much harm. Some background:
The Tcl manpage says:
*Tcl_Finalize* is similar to*Tcl_Exit* except that it does not exit
from
the current process. It is useful for cleaning up when a process is
finished using*Tcl* but wishes to continue executing, and when*Tcl*
is
used in a dynamically loaded extension that is about to be unloaded.
On some systems*Tcl* is automatically notified when it is being
unloaded, and it calls*Tcl_Finalize* internally; on these systems it
not
necessary for the caller to explicitly call*Tcl_Finalize*. However, to
ensure portability, your code should always invoke*Tcl_Finalize*
when
*Tcl* is being unloaded, to ensure that the code will work on all
plat-
forms.*Tcl_Finalize* can be safely called more than once.
For aolserver, it is questionable for me why we need
Tcl_Finalize() (the primarily purpose for Tcl_Finalize
according to its documentation is that the process wants
to continue without Tcl), furthermore there seems to
be some magic involved, that some systems call
Tcl_Finalize() ... automatically (hinting most likely the
windows situation with the assembly code). Since finalize
tries to unload Tcl, there seems to be some race
conditions in this area on windows, at least when there
are still are multiple threads around. E.g. [1] says:
Because DLL notifications are serialized, entry-point
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Maurizio, Tcl_Finalize() is supposed to work, and if it does now work something is still broken in the windows version. Omitting Tcl_Finalize() is removeing the symptom, not the cause. It is not unlikely that something else will have the same problem due to this cause. When Tcl_Finalize() is not run, the registered exit handlers are not executed. How serious this is depends on the exit handlers. You are right, that the memory leak does not matter due to the shutdown. The difference is like between a graceful and an ungraceful shutdown. -gustaf On 05.08.11 16:29, Maurizio Martignano wrote: Dear Gustav, I understand your concerns about Tcl_Finalize... but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test Have Aolserver / nsd running on a big application... observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Maurizio and all... i have updated cvs on sourceforge with most of your patches. A few points are questionable (see below). For me, it is still unclear, why 4.5.1 worked for you, but not the head version not. As far i can see, all socket usages were int the same way in 4.5.1, the variable triggers in nsd.h was defined like this at least since 2004. Maybe it was luck that the win version worked in 4.5.1 (different memory layout, different compilers / compiler options, ...) A few comments to the patches (i have omitted these): nsd/nsmain.c +#ifndef _WIN32 Tcl_Finalize(); +#endif === If one does no Tcl_Finalize() one introduces a memory leak. nsproxy/nsproxy.c -Tcl_FindExecutable(argv[0]); === The call to Tcl_FindExecutable() is required (at least in Tcl 8.5), otherwise tcl will crash (at least under unix like operating systems) +#define uint32_t unsigned long +#define uint16_t unsigned short + +typedef void * caddr_t; === These defines should be most probably within an #ifdef static void -FatalExit(char *func) +NSP_FatalExit(char *func) === What's wrong with the static name FatalExit()? -static Proc *firstClosePtr = NULL; +static Proc *firstClosePtr; === What's wrong with the initialization of the static variable? I have not updated the nsproxy changes, since these need more work. -gustaf neumann On 05.08.11 08:14, Maurizio Martignano wrote: Dear all, here you are with a first patch file and the zip containing the code base I'm using. The nspd module requires more work, but I'm not using it. The files in the zip archive do compile and seems to be working Windows 32 and Windows 64. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Maurizio, i guess, everything is fine with you with the head version on sourceforge. For me, it is still unclear, why 4.5.1 worked for you, but not the head version not. As far i can see, all socket usages were int the same way in 4.5.1, the variable triggers in nsd.h was defined like this at least since 2004. Maybe it was luck that the win version worked in 4.5.1 (different memory layout, different compilers / compiler options, ...) [MM] No Just Alzheimer... Me. When I first did the compilation I spotted the problem and I fixed it by changing int trigger[2] into SOCKET trigger[2]. Then I forgot completely about it Then the issue showed up again. when I downloaded the code again ok, this explains it. === If one does no Tcl_Finalize() one introduces a memory leak. [MM] The process/service is about to end anyhow. I believe that mo matter what the poor process/service does all its memory will be released by the OS. So this is not an issue. Tcl_finalize shuts down Tcl; it calls the registered exit handlers and then it shuts down various Tcl subsystems. It is certainly not recommended to to omit, but i have not checked the exact consequences for aolserver. If it hangs, it is an indication that there is still something wrong in tcl and/or aolserver. Keep an eye open in this respect. nsproxy/nsproxy.c -Tcl_FindExecutable(argv[0]); === The call to Tcl_FindExecutable() is required (at least in Tcl 8.5), otherwise tcl will crash (at least under unix like operating systems) [MM] Not my change, please look at my newest email and sorry if I cause you some inconvenience. It was included in your first patch (maybe erratic). If this is not needed, fine. All the best, and many thanks for the feedback -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Problem with file uploads larger than maxinput
Brian many thanks, i commited your change to the CVS on sourceforge. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Problem with file uploads larger than maxinput
Dear all, A few minutes ago, i have committed a change to the sourceforge aolserver repository that should should fix the problem finally. The primary problem was not that the driver replied directly to the client, but the fact that client (e.g. firefox) has not yet sent the full request while the server had already replied and closed the connection. Since the handling of the header happens in aolserver before the full request from the client is received, the server could close the connection before the client has sent the full request. So, although the client (e.g. firefox) has already received a proper reply from aolserver, it complained about the fact, that it could not sent the full request. This explains, why the error handling worked reliably with small upload files but not with large ones in current browsers; it explains as well, that simple clients (like nc) worked always reliable. Additionally, i have moved the reply handling to the connection threads, and made some more cosmetical changes. Brian, please check again. -gustaf neumann On 08.07.11 13:49, Fenton, Brian wrote: Dear Gustaf many thanks for the reply. Yes, I can confirm your results - if I set a limit of 10k and upload a file of 15k, it works perfectly for me. Would be great if you could look into this. If not, maybe you could give me a few pointers where to change the request handling thread. many thanks Brian From: AOLserver Discussion [AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann [neum...@wu-wien.ac.at] Sent: 08 July 2011 12:20 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Problem with file uploads larger than maxinput Dear Brian, Thanks with the config files, they are fine. With some more testing, i found some more insights: The version in CVS head returns always the right result to a client (tested with nc -v localhost 8006 POST.txt), no matter of the size of the uploaded content, but works only reliably in browsers (firefox, chrome) for rather small requests (set your maxupload to 5k, upload a 10 k file), and does reliable NOT work with upload files 20MB (which is the interesting case). So, the problem looks like a timing/buffering problem to me. RFC 2616 says e.g. If the server chooses to close the connection immediately after sending the response, it SHOULD send a Connection header including the connection-token close. The reply of aolserver contains the connection: close, so this seems standard compliant. The driver has to close the connection immediately after sending the reply, so, the moral of the story seems to be that we have to get the response handling out of the driver and require a solution via the request handling thread - what's the best solution anyhow. So far, i don't understand, why the size of the upload request changes the behavior. maybe, i can look into this next week. -gustaf neumann On 06.07.11 13:08, Fenton, Brian wrote: Dear Gustaf thanks for the reply. I have tried it with the latest OpenACS config.tcl and also another older one I use (both attached). I just assumed I was doing something wrong, didn't want to sound as though I was criticising your work. kind regards Brian From: AOLserver Discussion [AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann [neum...@wu-wien.ac.at] Sent: 05 July 2011 23:47 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Problem with file uploads larger than maxinput Strange, it works for me. can you send me your config file? Concerning the right place: As it was discussed here, it would be certainly better to move the reply-sending to a request handling thread (or a spooling thread like in naviserver), simply to be sure that the driver is never blocking. This should not be complicated, but i simply did not have to time, and i see no reason, why replying from the driver should not work. -gustaf -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Problem with file uploads larger than maxinput
Dear Brian, Thanks with the config files, they are fine. With some more testing, i found some more insights: The version in CVS head returns always the right result to a client (tested with nc -v localhost 8006 POST.txt), no matter of the size of the uploaded content, but works only reliably in browsers (firefox, chrome) for rather small requests (set your maxupload to 5k, upload a 10 k file), and does reliable NOT work with upload files 20MB (which is the interesting case). So, the problem looks like a timing/buffering problem to me. RFC 2616 says e.g. If the server chooses to close the connection immediately after sending the response, it SHOULD send a Connection header including the connection-token close. The reply of aolserver contains the connection: close, so this seems standard compliant. The driver has to close the connection immediately after sending the reply, so, the moral of the story seems to be that we have to get the response handling out of the driver and require a solution via the request handling thread - what's the best solution anyhow. So far, i don't understand, why the size of the upload request changes the behavior. maybe, i can look into this next week. -gustaf neumann On 06.07.11 13:08, Fenton, Brian wrote: Dear Gustaf thanks for the reply. I have tried it with the latest OpenACS config.tcl and also another older one I use (both attached). I just assumed I was doing something wrong, didn't want to sound as though I was criticising your work. kind regards Brian From: AOLserver Discussion [AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann [neum...@wu-wien.ac.at] Sent: 05 July 2011 23:47 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Problem with file uploads larger than maxinput Strange, it works for me. can you send me your config file? Concerning the right place: As it was discussed here, it would be certainly better to move the reply-sending to a request handling thread (or a spooling thread like in naviserver), simply to be sure that the driver is never blocking. This should not be complicated, but i simply did not have to time, and i see no reason, why replying from the driver should not work. -gustaf -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] URL handling produce segfault
Alexey, both URLs work nicely for me (using 4.5.1 from sourceforge under Mac OS X); the URLs are rewritten finally into /?q=тавр*+2011 Can it be that you have some unusual encoding settings? -gustaf neumann On 04.07.11 11:54, Alexey Pechnikov wrote: 1. It's work fine: http://localhost:8100/json/object/search/?q=%D1%82%D0%B0%D0%B2%D1%80*+2010 2. It's fail: http://localhost:8100/json/object/search/?q=%D1%82%D0%B0%D0%B2%D1%80*+2011 AOL Log: [-default:0-] Fatal: received fatal signal 11 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] URL handling produce segfault
a segfault is a segfault is a segfault no matter whether is happens in aolserver, in tcl or in an extension. you might have a chance to recover from segfaults in nsproxy, but certainly, the best approach is to get rid of these... -gn On 05.07.11 21:25, Alexey Pechnikov wrote: Is it possible to prevent (or detailed logging) the AOL segfault in case of tcl extension segfault? I use runit supervisor and AOL is restarting automatically but debugging in this situation is not easy. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Problem with file uploads larger than maxinput
Strange, it works for me. can you send me your config file? Concerning the right place: As it was discussed here, it would be certainly better to move the reply-sending to a request handling thread (or a spooling thread like in naviserver), simply to be sure that the driver is never blocking. This should not be complicated, but i simply did not have to time, and i see no reason, why replying from the driver should not work. -gustaf -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Problem with file uploads larger than maxinput
Dear all,
As a small contribution, i added a patch to sourceforge cvs.
The patch returns the 413 error message via a new function
Ns_ConnReturnEntityToLarge(), which is written in the same
style as Ns_ConnReturnNotFound(). The patch is somewhat
minimal and handles this issue entirely in the driver (also,
forcing the log entry). I have paid no special attention to
the custom response page.
-gn
On 23.06.11 19:14, Dossy Shiobara wrote:
That was what I was thinking -- driver marks the request
as exceeding the limit, and setting the response status
code to 413. The benefits that I see (if implemented the
way I'm imagining) --
1) access logging of requests w/ 413 status code
2) custom response page via ns/server/${server}/redirects.
If you and Brian could work up a patch, that'd be wonderful!
Does anyone see any problems with this approach? Any
reason not to do it? I don't suppose it can possibly
break any kind of backward compatibility, as no code right
now can even be written to handle such a scenario, anyway.
Of course, once we decide on a behavior, and folks code
against the implementation ... changing/fixing it becomes
more expensive for everyone, so if there's any kind of
worry about how this is going to work, lets iron out the
details now ;)
On 6/23/11 12:29 PM, Enrique Catalan wrote:
IMHO, I agree with Dossy, to use the driver thread to
check the hard
limits and instead of dropping the connection, just mark
the HTTP
request and let the request handler to return the 413. I
also think
the template could be configured in the 'ns_section
ns/server/${server}/redirects' , isn't it ?
If you all agree with this, Brian and I can help to get a
patch.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Feature request: ns_querygetarray function
Am 10.05.11 17:59, schrieb Alexey Pechnikov:
Problems:
1. When names and number of the array elements are not known we can't
use ns_querygetall
why not using ns_getform? like e.g.:
===form-action.tcl
foreach {att value} [ns_set array [ns_getform]] {
lappend a($att) $value
}
ns_return 200 text/plain keys: [array names a]\nkeys+values: [array get a]
==
-gustaf neumann
--
Univ.Prof. Dr. Gustaf Neumann
Institute of Information Systems and New Media
WU Vienna
Augasse 2-6, A-1090 Vienna, AUSTRIA
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] maxrun setting in aol4.5.1
Hi Majid, maxrun is one way to limit the maximum number of concurrent connections. It allows to limit certain kinds of requests (actually the queueing of incoming requests). See the naviserver man page for an example http://naviserver.sourceforge.net/n/naviserver/files/ns_limits.html If you have max_threads set to 40, you won't have at any time more than 40 connection threads running (no matter, whether you set the default maxrun to 40 or 500 or leave the default of 100). If you have no special intentions with maxruns, i would recommend to leave it to the default. In your snippet, you set just some Tcl variables. In several cases it is not obvious how these are used to set the actual config variables of aolserver. For example, the setting of max_keep_alive wonders me: The keep-alive of the server is controlled via the config variable keepwait, which is a value specified in secs. It is somewhat strange that you seem to bring this in relation with max_threads, ... but maybe, your naming is just misleading. -gustaf neumann On 15.04.11 23:48, Majid Khan wrote: Hi All, I am done with the basic understanding of the configuration of aol4.5.1 and I have set the following variable for my configuration. Since we have full openACS files with alot of customization and too much additional TCL files so I have set the connecitons per threads 1000 and after that it will destroy which to me initially looks good enough though I didn't go live but I am planning to go with that. I was just confused on one of the parameter that is maxruns which I read is that it is the concurrent connection to the server and I found one of the example where they set maxruns=maxthreads so my question is should I do the same thing doesn't it mean that it will just accept 40 concurrent connections in my case? however we have a very busy site and we don't want that it should be just limited to 40 so I am planing to set it to 500 please correct me if my understanding is incorrect also suggest me if there is anything which needs to be tuned more in my below settings. set max_threads 40 set max_conns 1000 ;# or ConnsPerThread, i.e. after how many requests processed by a given thread should it be destroyed, 0 means never. set min_threads 5 set thread_timeout 120 ;# destroy an idle thread after this many idle seconds set max_busy_threads 10 set max_wait 100 ;# max number of threads waiting to be serviced set max_wait_time 120 ;# seconds to timeout while waiting to be serviced set max_keep_alive [expr $max_threads * 3] set stack_size 2097152 ;# 2 MB set max_upload_limit [expr 20 * 1024 * 1024] ;# 20 MB limit set max_socks [expr $max_threads + $max_wait + $max_busy_threads] ;# MaxSock setting to max_threads sum plus MaxWait plus MaxBusyThreads set nsv_buckets 48 Best Regards, Majid. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] maxrun setting in aol4.5.1
I still don't get, what your are doing with MaxKeepAlive,
but i don't have to. Be aware, that aolserver does not use
it as a config variable (just do a fgrep -Ri MaxKeepAlive
. in the src directory). If your application uses
MaxKeepAlive (from tcl, or from one of your c-modules), it
is fine. -gustaf
On 16.04.11 12:05, Majid Khan wrote:
Hi Gaustaf,
Thanks for clarifying the maxrun param. About Tcl
variables this is how I am using them:
ns_section ns/encodings
ns_param .adp utf-8
ns_param .byt utf-8
set myserver_root /myserver
set max_threads 40
set max_conns 1000 ;# or ConnsPerThread, i.e. after how
many requests processed by a given thread should it be
destroyed, 0 means never.
set min_threads 5
set thread_timeout 120 ;# destroy a thread after this many
idle seconds
set max_busy_threads 10
set max_wait 100 ;# max number of threads waiting to be
serviced
set max_wait_time 120 ;# seconds to timeout while waiting
to be serviced
set max_keep_alive [expr $max_threads * 3]
set stack_size 2097152 ;# 2MB Limit
set max_upload_limit [expr 20 * 1024 * 1024] ;# 20 MB limit
set max_socks [expr $max_threads + $max_wait +
$max_busy_threads] ;# MaxSock setting to max_threads sum
plus MaxWait plus MaxBusyThreads
set nsv_buckets 48
ns_section ns/parameters
ns_param User myuser
ns_param ServerLog /myserver/log/error.log
ns_param LogRoll on
*ns_param MaxKeepAlive $max_keep_alive*
ns_param LogMaxBackup 10
ns_param ListenBacklog 32
ns_param Home /myserver
ns_param StackSize $stack_size
ns_param auxconfigdir /myserver/parameters
ns_param crashcmd ns_crash
ns_param OutputCharset utf-8
ns_param dnscache off
ns_param smtphost $smtphost
ns_param inputcharset utf-8
ns_param URLCharset utf-8
ns_section ns/threads
# use more than 1 processor (Solaris)
ns_param SystemScope on
ns_param mutexmeter true
ns_section ns/server/myserver
ns_param MinThreads $min_threads
ns_param MaxThreads $max_threads
ns_param MaxConns $max_conns
ns_param ThreadTimeout $thread_timeout
ns_param MaxBusyThreads $max_busy_threads
ns_param MaxWait $max_wait
ns_param MaxWaitTime $max_wait_time
ns_param DirectoryFile { index.html }
ns_param UrlStats off
ns_param MaxUrlStats 1000
ns_param PageRoot $myserver_root/www
ns_param globalstats false
ns_param enabletclpages true
ns_param checkmodifiedsince off
ns_section ns/server/myserver/fastpath
ns_param cache false
ns_param cachemaxentry 16384
ns_param cachemaxsize 512
ns_section ns/server/myserver/redirects
ns_param 404 /404.adp
#ns_param 500 /500.htm
ns_section ns/server/myserver/adp/parsers
ns_param adp {.adp ;# The simple parser looks for \%
... \%}
#fancy=.adp ;# The fancy parser does a lot more.
ns_section ns/server/bayt/module/nssock
ns_param Address $ip
ns_param Hostname $domain
ns_param Port $port
ns_param maxinput $max_upload_limit
ns_param MaxSock $max_socks
About KeepWait what you mentioned is different then
maxkeepalive. The idea is if any client which uses a
keep-alive connection (For HTTP/1.1 clients, persistent
connections or keep-alive are the default unless otherwise
specified.) it will be counted as a single request
regardless of how many request has been sent using the
same connection which improves the performance because it
would not use any new TCP connection instead. So in my
case I have set maxkeepalive as 120 and there is no such
relation between max_threads variable with that it is just
my logic to have it multiply by 3 otherwise you can ignore
and set it as whatever you want. I didn't set keepwait
which I should thanks for mentioning that I will set it up
to 30 sec which means after 30 secs hang up clients while
waiting for an HTTP request in a connection in a
keep-alive situation. Setting to a high value may cause
performance problems in heavily loaded servers. The higher
the timeout, the more server connections will be kept
occupied waiting on connections with idle clients. The
purpose of writing Gustaf in such a detailed way is just
because I wanted to make sure what I understand is correct
and others can benefit of it in our community to
understand the logic of the config params.
One last thing which I found I should do is about the DNS
tuning, so I have taken the following code from
http://www.aolserver.com/docs/admin/config-reference.tcl.txt
and will put it in my config file.
# DNS tuning
ns_param dnscachetrue ;# In-memory cache of
DNS lookups
ns_param dnscachetimeout 60;# How long to keep
hostnames in cache
dnscachetimeout is I think in mins.
Regards,
Majid.
On Sat, Apr 16, 2011 at 11:02 AM, Gustaf Neumann
neum...@wu-wien.ac.at mailto:neum...@wu-wien.ac.at wrote:
Hi Majid,
maxrun is one way to limit the maximum number of
concurrent connections. It allows to limit certain
kinds of requests (actually the queueing of incoming
requests). See the naviserver man page for an example
http://naviserver.sourceforge.net/n/naviserver/files
Re: [AOLSERVER] What does 'exiting: exceeded max connections per thread' mean?
Aolserver uses maxconns (set via ns_pools), naviserver uses connsperthread. If this configuration parameter is set to 0, a connection thread will never exit due to an exceeded number of connections handled by this thread. The spread value does not change this. -gn On 20.02.11 08:55, aT wrote: Does ConnsPerThread = 0 , have any affect on this ? Should it be 0 , meaning as many connection per thread as possible ? On 02/16/2011 08:26 AM, Gustaf Neumann wrote: The thread exiting code is based on maxconns and timeout. If e.g. maxconns is is very low (e.g. 20), the default spread of 20 does not make a lot of change. Furthermore it is more important, when many connection threads are configured (e.g. 80), and the there is much library code (e.g. fully configured OpenACS). We use maxconns 1000 + the default spread. -gn On 16.02.11 00:26, Peter Sadlon wrote: If you have a lot of .tcl files then it will take longer for the new threads to be created (so if you are using OpenACS then it can take awhile depending on the RAM and CPU power of your server) Increase the connections per thread to 1000 and see how that works for you. There is (in theory) a way to set each thread to be +/- 20% the number of max connections. This would prevent all the threads from exiting and being recreated at around the same time, however, even when I tried using that (and I forget how to do it off the top of my head) it never seemed to work, all my connections would exit at the same time so I never looked into it further since I don't really have a lot of .tcl files to load and new thread creation was rather fast. Perhaps someone else can speak to varying the number of connections per thread. Date: Tue, 15 Feb 2011 15:17:08 -0800 From: thejackschm...@gmail.com Subject: Re: [AOLSERVER] What does 'exiting: exceeded max connections per thread' mean? To: AOLSERVER@LISTSERV.AOL.COM From my own experience, it seems that I get a ton of that like all the threads (or a majority of them) have exceeded max connections and it seems that it does take a while before they start serving again. I'd like to know if this is normal or should the turn around time be a lot quicker... Thanks! -- -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] What does 'exiting: exceeded max connections per thread' mean?
The thread exiting code is based on maxconns and timeout. If e.g. maxconns is is very low (e.g. 20), the default spread of 20 does not make a lot of change. Furthermore it is more important, when many connection threads are configured (e.g. 80), and the there is much library code (e.g. fully configured OpenACS). We use maxconns 1000 + the default spread. -gn On 16.02.11 00:26, Peter Sadlon wrote: If you have a lot of .tcl files then it will take longer for the new threads to be created (so if you are using OpenACS then it can take awhile depending on the RAM and CPU power of your server) Increase the connections per thread to 1000 and see how that works for you. There is (in theory) a way to set each thread to be +/- 20% the number of max connections. This would prevent all the threads from exiting and being recreated at around the same time, however, even when I tried using that (and I forget how to do it off the top of my head) it never seemed to work, all my connections would exit at the same time so I never looked into it further since I don't really have a lot of .tcl files to load and new thread creation was rather fast. Perhaps someone else can speak to varying the number of connections per thread. Date: Tue, 15 Feb 2011 15:17:08 -0800 From: thejackschm...@gmail.com Subject: Re: [AOLSERVER] What does 'exiting: exceeded max connections per thread' mean? To: AOLSERVER@LISTSERV.AOL.COM From my own experience, it seems that I get a ton of that like all the threads (or a majority of them) have exceeded max connections and it seems that it does take a while before they start serving again. I'd like to know if this is normal or should the turn around time be a lot quicker... Thanks! -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] nsproxy
Do you have /usr/lib/aolserver4/bin/nsproxy installed? try: file /usr/lib/aolserver4/bin/nsproxy OpenACS has a wrapper for exec to redirect every exec-call to nsproxy. Works pretty well. http://fisheye.openacs.org/browse/OpenACS/openacs-4/packages/acs-tcl/tcl/proxy-procs.tcl?r=HEAD -gustaf neumann On 29.01.11 19:14, Bernhard van Woerden wrote: I'm trying to run ns_proxy from the control port on AOLserver 4.5.1 Tcl 8.5 but get nsd:nscp 61 set handle [ns_proxy get myproxy] exec failed: no such file or directorycould not get 1 handle from pool myproxy: could not create child process and in the log [-nscp:1-] Notice: nscp: nsd 61: set handle [ns_proxy get myproxy] [-nscp:1-] Error: exec /usr/lib/aolserver4/bin/nsproxy: execve() failed: No such file or directory I was hoping to use nsproxy to run some TclCurl stuff that I think is segfaulting and taking down the nsd process but also some exec's to other binaries. Any ideas as to how can I get this working ? Many Thanks Bernhard -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Does not work ns_return + zlib
On 01.12.10 21:31, Tom Jackson wrote: Personally I also wouldn't assume that AOLserver works perfectly with Tcl8.6. Me neither, but Tcl 8.6 is still in an early state. Unless you absolutely need 8.5, I would stick with the latest 8.4 version. For one thing 8.4 is faster than either. Well, it depends. My recommendation is to use Tcl 8.5. From our experience from running large OpenACS apps (600.000 loc Tcl on our production environment, delivering up to 220 GB/day, up to 2.500+ concurrent users), the performance difference between 8.4 and 8.5 is very little as long one is using just 8.4 functionality. Tcl 8.5 has several improvements to make programs more readably (e.g. the in operator in expressions), more safe (e.g. the expand operator) and leads to more stable multi-threaded programs (e.g. c-stack protection, checking stack boundaries instead of plain crashing as in 8.4), provides bignums, and many many more really useful fixes and improvements. The OpenACS community has decided some time ago to require 8.5 for future releases. Please don't listen to the Tcl gurus, they are behind the curve...actually they are actively slowing down the curve. Come on, stop that bashing. Jeff was referring to http://code.activestate.com/lists/tcl-core/9805/ On real problems, the Tcl-core people are very helpful and highly responsive... -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10?
It seems as if you have invalid character data in your database. http://networking.itags.org/internet-explorer/28465/ http://php.net/manual/de/function.urlencode.php it is most probably the easiest approach to fix these in the database. With postgres, you could make a dump, change the quotes in the dump and reload. -gustaf neumann On 02.12.10 08:39, Janine Ohmer wrote: Sort of... the column in question is of type text, and the only function I can find that will convert from text to bytea is decode, not encode. So here's what I did: select decode(answer_3, 'escape') from public_places where public_place_id=1012; The interesting part of the result is Park\302\222s, which is supposed to be Park's. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver 4.5.1 crashing with max connections per thread messages
Dear Björn,
While i don't see the direct connection between your changed
settings and error with EAGAIN, there are apparently
misconfiguration in the snippet of your config file which
are related to the changed settings:
1) If you have maxthreads defined as 10, then your first
db-pool should have at least 10 db-connections. Note, that
you might have in your app as well scheduled procs that will
use as well db-connections; 15 might be a good value.
2) In the pools parameter, you have to enlist you pools
(pools * is not allowed for new aolservers, use in your
example pool1,pool2,pool3 instead of the *).
For an example of the config file, see:
http://cvs.openacs.org/browse/OpenACS/openacs-4/etc/config.tcl?r=HEAD
-gustaf neumann
On 01.12.10 16:25, Björn Þór Jónsson wrote:
Hi,
The server has been rock stable since I changed
ns_param maxconnections 5
to
ns_param maxconnections 100
and from
ns_param maxthreads 5
to
ns_param maxthreads 10
ulimit -u says: unlimited
and /etc/security/limits.conf has all lines commented out.
Thanks for the help :)
/Björn
2010/11/22 Gustaf Neumann neum...@wu.ac.at
mailto:neum...@wu.ac.at
Dear Björn,
The error indicates that /pthread_create /returned EAGAIN
(paraphrased as Resource temporarily unavailable).
This error indicates that
the system lacked the necessary resources to create
another thread, or the system-imposed limit on the
total
number of threads in a process {PTHREAD_THREADS_MAX}
would be exceeded. (from
http://linux.die.net/man/3/pthread_create).
so, for the user, under which the server runs, check
ulimit -u,
limits.conf, etc. Can it be that you switched to a new
machine with lower
limits than before?
What is you setting of maxthreads?
-gustaf neumann
Am 18.11.10 17:13, schrieb Björn Þór Jónsson:
Hi,
After recently upgrading from AOLserver 4.5.0 to
4.5.1 and from nspostgres-4.0 to nspostgres-4.1 the
server is repeatedly crashing (when it gets hammered
by the google bots). The error.log has many entries
like these before the server dies:
[17/Nov/2010:02:18:42][700.3218660208][-default:6195-] Notice:
exiting: exceeded max connections per thread
[17/Nov/2010:02:18:43][700.3217636208][-default:6193-] Notice:
exiting: exceeded max connections per thread
[17/Nov/2010:02:18:44][700.3219172208][-default:6196-] Notice:
exiting: exceeded max connections per thread
[17/Nov/2010:02:18:45][700.3218148208][-default:6194-] Error:
Tcl exception:
adp flush failed: connection closed
abort exception raised
while processing connection #31907:
GET ...
Host: localhost:8006
...
nsthreads: pthread_create failed in NsCreateThread:
Resource temporarily unavailable [this is the last
line in the log before the crash]
This is the database section of the AOLserver config
file:
ns_section ns/db/drivers
ns_param postgres nspostgres.so
ns_section ns/db/pools
ns_param pool1 Pool 1
ns_param pool2 Pool 2
ns_param pool3 Pool 3
ns_section ns/db/pool/pool1
ns_param maxidle10
ns_param maxopen10
ns_param connections5
ns_param extendedtableinfo true
ns_param driver postgres
ns_param datasource localhost::${db_name}
ns_param user $user_account
ns_section ns/db/pool/pool2
ns_param maxidle10
ns_param maxopen10
ns_param connections5
ns_param extendedtableinfo true
ns_param driver postgres
ns_param datasource localhost::${db_name}
ns_param user $user_account
ns_section ns/db/pool/pool3
ns_param maxidle10
ns_param maxopen10
ns_param connections5
ns_param extendedtableinfo true
ns_param driver postgres
ns_param datasource localhost::${db_name}
ns_param user $user_account
ns_section ns/server/${server}/db
ns_param pools *
ns_param defaultpoolpool1
The server is running on Ubuntu 10.04.1 LTS
2.6.32-25-generic-pae #45-Ubuntu SMP Sat Oct 16
21:01:33 UTC 2010 i686 GNU/Linux
Is there anything I should configure differently or
has any other ideas what might be causing this?
/Björn
--
Björn Þór Jónsson
http://bthj.is
-- AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an
email to lists...@listserv.aol.com
mailto:lists...@listserv.aol.com
Re: [AOLSERVER] AOLserver 4.5.1 crashing with max connections per thread messages
Dear Björn,
The error indicates that /pthread_create /returned EAGAIN
(paraphrased as Resource temporarily unavailable).
This error indicates that
the system lacked the necessary resources to create
another thread, or the system-imposed limit on the total
number of threads in a process {PTHREAD_THREADS_MAX}
would be exceeded. (from http://linux.die.net/man/3/pthread_create).
so, for the user, under which the server runs, check ulimit -u,
limits.conf, etc. Can it be that you switched to a new machine with lower
limits than before?
What is you setting of maxthreads?
-gustaf neumann
Am 18.11.10 17:13, schrieb Björn Þór Jónsson:
Hi,
After recently upgrading from AOLserver 4.5.0 to 4.5.1 and from
nspostgres-4.0 to nspostgres-4.1 the server is repeatedly crashing
(when it gets hammered by the google bots). The error.log has many
entries like these before the server dies:
[17/Nov/2010:02:18:42][700.3218660208][-default:6195-] Notice:
exiting: exceeded max connections per thread
[17/Nov/2010:02:18:43][700.3217636208][-default:6193-] Notice:
exiting: exceeded max connections per thread
[17/Nov/2010:02:18:44][700.3219172208][-default:6196-] Notice:
exiting: exceeded max connections per thread
[17/Nov/2010:02:18:45][700.3218148208][-default:6194-] Error: Tcl
exception:
adp flush failed: connection closed
abort exception raised
while processing connection #31907:
GET ...
Host: localhost:8006
...
nsthreads: pthread_create failed in NsCreateThread: Resource
temporarily unavailable [this is the last line in the log before the
crash]
This is the database section of the AOLserver config file:
ns_section ns/db/drivers
ns_param postgres nspostgres.so
ns_section ns/db/pools
ns_param pool1 Pool 1
ns_param pool2 Pool 2
ns_param pool3 Pool 3
ns_section ns/db/pool/pool1
ns_param maxidle10
ns_param maxopen10
ns_param connections5
ns_param extendedtableinfo true
ns_param driver postgres
ns_param datasource localhost::${db_name}
ns_param user $user_account
ns_section ns/db/pool/pool2
ns_param maxidle10
ns_param maxopen10
ns_param connections5
ns_param extendedtableinfo true
ns_param driver postgres
ns_param datasource localhost::${db_name}
ns_param user $user_account
ns_section ns/db/pool/pool3
ns_param maxidle10
ns_param maxopen10
ns_param connections5
ns_param extendedtableinfo true
ns_param driver postgres
ns_param datasource localhost::${db_name}
ns_param user $user_account
ns_section ns/server/${server}/db
ns_param pools *
ns_param defaultpoolpool1
The server is running on Ubuntu 10.04.1 LTS
2.6.32-25-generic-pae #45-Ubuntu SMP Sat Oct 16 21:01:33 UTC 2010 i686
GNU/Linux
Is there anything I should configure differently or has any other
ideas what might be causing this?
/Björn
--
Björn Þór Jónsson
http://bthj.is
-- AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in
the email message. You can leave the Subject: field of your email blank.
--
Univ.Prof. Dr. Gustaf Neumann
Institute of Information Systems and New Media
WU Vienna
Augasse 2-6, A-1090 Vienna, AUSTRIA
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Site hanging
Dear Janine The symptoms sound similar to bug #1615787 https://sourceforge.net/tracker/?func=detailaid=1615787group_id=3152atid=103152 this bug is fixed in the 4.0 branch (not sure, if you are using a version before this fix) and in aolserver 4.5.1 best regards -gustaf neumann 2007-10-19 Gustaf Neumann neum...@wu-wien.ac.at * nsd/driver.c: * nsd/queue.c: Better fix for BUG #1615787 in aolserver 4.5. Although the original fix apparently helped for www.openacs.org, it appears to simply change the timing, but did not fix the problem itself. The problem was that under limited resource configurations (e.g. maxthreads 5, maxconnections 3) and heavy traffic, incoming requests were queued but not processed in new connection threads. The situation was especially bad, when the number of queued requests was larger than maxconnections, since after processing n requests the server was idling with a high number of queued requests. 2006-12-15 Dossy Shiobara do...@panoptic.com * nsd/driver.c: [BUG #1615787] Fix deadlock bug when driver thread stalls under specific conditions. Thanks to Jeff Rogers and Gustaf Neumann. On 06.11.10 17:23, Janine Ohmer wrote: Another day, another problem. ;) This time it's a somewhat more recent (but still old) OpenACS site, that has always run under AOLserver 4.0.10. Overnight it got hit pretty hard by the Yahoo crawler, and it has stopped responding. Netstat shows 117 connections, 78 in CLOSE_WAIT state and the rest in SYN_RECV. This site would hang up very occasionally on my old system; I would just restart it, since it didn't happen often enough to worry about. But it has only been running for about 16 hours, and that is way too soon to be doing this. The new system is more powerful than the old one, and runs a much newer 64 bit Linux, vs a very old 32 bit one. It's entirely possible that my configuration (connections) is not ideal, but it hasn't changed so I would expect it to run at least as well on the new system as it did on the old. For now I will just restart it but I'm sure this will happen again. Any suggestions on how to troubleshoot? I have never had much luck over the years figuring this out; I usually just end up installing keepalive to restart, but that is really not ideal. thanks, janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Has anyone built nsd3.3+ad13 on 64-bit Linux?
If you want to compile the vintage version, i would will have to compile either with -m32 or to backport the 64bit fixes of the last 10 years, which are most likely necessary for aolserver33 as well. Search e.g. through http://www.ohloh.net/p/aolserver/commits?query=64-bitcommit=Update http://www.ohloh.net/p/aolserver/commits?query=64-bitcommit=Update and similar search patterns best regards -gustaf neumann On 05.11.10 23:03, Janine Ohmer wrote: Followup to this... The original build error was: (cd /usr/local/src/aol33+ad13/aolserver/tcl7.6/unix; make CFLAGS='-I../include -D_REENTRANT=1 -DNDEBUG=1 -g -fPIC -Wall -Wno-unused -mcpu=x86_64 -DHAVE_CMMSG=1 -DUSE_FIONREAD=1 -DHAVE_COND_EINTR=1' libtcl7.6.a) make[2]: Entering directory `/usr/local/src/aol33+ad13/aolserver/tcl7.6/unix' gcc -c -I../include -D_REENTRANT=1 -DNDEBUG=1 -g -fPIC -Wall -Wno-unused -mcpu=x86_64 -DHAVE_CMMSG=1 -DUSE_FIONREAD=1 -DHAVE_COND_EINTR=1 -I./../generic -I. -DHAVE_UNISTD_H=1 -DHAVE_SYS_TIME_H=1 -DTIME_WITH_SYS_TIME=1 -DHAVE_TM_ZONE=1 -DHAVE_TM_GMTOFF=1 -DHAVE_TIMEZONE_VAR=1 -DSTDC_HEADERS=1 -DNEED_MATHERR=1 -DHAVE_SYS_IOCTL_H=1 -DTCL_SHLIB_EXT=\.so\ ./../generic/panic.c `-mcpu=' is deprecated. Use `-mtune=' or '-march=' instead. ./../generic/panic.c:1: error: bad value (x86_64) for -mtune= switch I fixed this with: #CFLAGS+=$(GCCOPT) -mcpu=$(shell uname -m) CFLAGS+=$(GCCOPT) -m64 -mtune=generic in aolserver/include/Makefile.global. Now everything builds, but it starts to run and then seg faults. *sigh* I also tried CFLAGS+=$(GCCOPT) -O2 -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic (everything from the new Makefile.global that the old one doesn't have) But it still seg faults. janine On Nov 5, 2010, at 1:58 PM, Janine Ohmer wrote: I just posted the following to at openacs.org, and wanted to see if anyone who lurks here but not there might have any ideas: Due to unforeseen circumstances (server death :) I'm moving some very old sites to a new system running 64 bit Redhat (old system was 32 bit). A couple of these sites were running under nsd3.3+ad13. To my great surprise most of nsd worked without having to be rebuilt, but the postgres driver did not work and I was unsuccessful at building a new one. Not wanting to invest a lot of time on an ancient webserver, I went ahead with the move and set the 3.2.5 sites up under AOLserver 4.0.10, also quite old but the client didn't want me to upgrade anything more than was absolutely necessary. Unfortunately, they don't work. There are no errors in the error log on startup but the page load dies with: invalid command name template::adp_parse There don't seem to be many people reporting this error. I've seen this thread: http://openacs.org/forums/message-view?message_id=26808 But have double-checked the config file (several times!) and Home is set correctly. However, this makes me think that the problem is probably due to changes to AOLserver's module/tcl scripts. So does anyone know how to either a) run 3.2.5 under AOLserver 4 or b) get nsd3.3+ad13 built and working on 64 bit Linux? Thanks in advance! --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLServer Terminal Escape Sequence in Logs Command Injection Vulnerability
as i said, a quick fix to close the logging exploit. The information loss (changing ESC to the bell character 7) is very little; under normal operation, you should never have a bell character in the log file, and now, if you see one, it should ring a bell the incoming url might be url-encoded or not. if one url-encodes the logged url, then there is as well an information loss. One could certainly just url-encode the escape character, but there is as well an information loss on these. One could grep safely for the bell character, but this is probably not very intuitive. Note, that it is not only sufficient to escape the url, the attack could as well come from other http reply header fields. One should implement a DStringAppendSanitized() function in nslog.c to allow multi-character substitutions. -gustaf On 09.09.10 14:41, Dossy Shiobara wrote: As a short-term solution, this is probably adequate, but there's information loss -- it'd be nice to indicate the original byte sequence somehow in the log entry by escaping characters so that log analysis tools could detect such attacks, etc. Perhaps the right answer is to log the URI with proper URL-encoding, so that it would be logged as %1B instead of the literal byte. On 9/9/10 8:18 AM, Gustaf Neumann wrote: i have just now committed a quick fix for the problem into the aolserver/nslog/nslog.c into the sourceforge module. please check, if this is in all cases sufficient. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLServer Terminal Escape Sequence in Logs Command Injection Vulnerability
The problem is a very old one, known at least since the 1980ies. There are discussions, whether this qualifies as a security problem of some software writing to a logfile or as a problem of the terminal emulators (which it actually is). IMHO, this discussion is worthless. Some vendors of webservers even seem to refuse to fix this. Actually, every software writing unfiltered to some terminals is dangerous, even a grep. The real security problem are some terminal emulations, but there are many of these. The evil thing is that writing to the log file at the first place does nothing bad, but looking at the logfile with some standard tools can rise the problem - it is a sleeper attack. If the log file is e.g. accessible via NFS, the machines of developer developers watching the log file are under attack. As mentioned above this is not only a problem of aolserver, but potentially of more or less all logging programs or programs writing unfiltered to a terminal, such as e.g. database management systems, etc. If you're super-paranoid, don't tail -f access log files or other files that an attacker can write to through accessing a service on your system. Get into the habit of cat -v $FILENAME to make sure non-printable characters are escaped, intead of being fed unfiltered to your terminal program. or tail -f ... | cat -v -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Handling threads the right way
Am 30.06.10 00:23, schrieb Sep Ng: Basically we had aolservers running and while serving pages, it's also doing some heavy load processing from a ton of scheduled custom written procedures. Aolserver crashes and segmentation faults are fairly frequent and the logs at the time pointed to these running threads as a probable cause. Possibly a configuration issue, but I remember we tried fiddling with those numbers and it never really helped. From my experience, the two most common causes for crashes in aolserver are a) c-based extensions (tcl extensions and/or aolserver modules) which are not thread safe (e.g. using non thread-safe libraries) or which have bugs (e.g. double frees of memory), or b) running out of memory. The aolserver threads are typically heavy-weight, the zippy memory allocator over-allocates memory and does not free it. All threads use the same memory block, which is limited on most 32bit machines to 2GB. When you hit the limit, the server crashes. A solution for (b) is to compile everything (tcl, aolserver, modules) with 64 bit. If you are using just Tcl + Aolserver commands, the server should never crash with recent versions of aolserver and tcl (e.g. tcl 8.4.19, 8.5.8). What C extensions or aolserver modules are you using? The following heuristic might help in the search of the problem. If your crashes are random and independent of the load (e.g. happen with already a little load) then bugs in the C code of the extensions are likely. If crashes happens during thread end, bugs in the memory management of the c-extensions are likely (tcl deletes an interpreter with all associated memory). If the crashes happen under heavy load, thread-safety of some component is likely to be the problem (e.g. we had a long time problems with our aolserver, which ran stable up to about 600 concurrent users, then it started to crash. One of the problem was the kerberos library, which was not reentrant, although it claimed to be so). So basically, what I'm currently beating my head over is to build a much cleaner and better way of handling all the load but in so doing, I'm not entirely sure whether or not to spawn a lot of threads for the jobs, or basically keep it to a minimum. Judging from Andrew's post, it would likely be better to reuse threads but I'm not entirely sure how that happens. I mean, everytime you'd invoke ns_thread begin/begindetached, you are creating a new thread already, no? How do you reuse them? In general, you have two options: you can use aolserver-threads and threads created by the tcl thread library (libthread), which can be loaded as a module into the aolserver (when compiled with the appropriate flags). I prefer libthread, since all such threads run in an event loop. This makes it very easy to send to these threads commands which are implemented via event-loops. One can as well to implement this way easy communication between the threads. It is essentially the same programming model that can be used by using tcl+libthread outside of the aolserver. This is probably a stupid question, but is there a distinction between the threads and connection threads in aolserver? I know the connection threads are probably the connections to the database (I think). no, the connection threads handle incoming HTTP requests, they have a connection to the web-client. These are the threads controlled in the config file via e.g, maxthreads. Typically, a connection thread is created on demand (incoming request) and lives, until either it times out (it received no requests for a certain time) or it has served a maximum number of requests (all configurable in the config file). When it reaches the end of its live-cycle, it is terminated (and maybe recreated afterwards by new demand). The threads of aolserver are typically heavyweight, since every thread contains a full tcl interpreter. The weight depends on the size of the blueprint, which is determined essentially by all tcl-procs available at runtime. If you are using e.g. OpenACS with many OpenACS packages defined, these might be several thousand procs. When a thread starts, it reads the blue-print with all procs, which might take a while. If you are using just Aolserver with a few tcl-modules, the cost of starting a new thread (essentially loading the blueprint) might not be so bad, and you might start for every jobs a new thread. But this certainly depends on your needs and the kind of performance you might want to reach. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Has anyone had experience on 64-bit AOLserver for possible memory leaks?
Am 14.06.10 09:24, schrieb Sep Ng: Thanks for the swift reply Gustaf. I have Debian Etch on 64-bit as the OS platform and I'm thinking that OS choice should not particularly affect performance, right? i doubt that there will be a perceivable difference between Linux versions, but this depends, how fine you measure :). openacs.org runs aolserver 4.5.1 with Ubuntu 8.04.3 LTS since more than a year without any problems. $ file /usr/local/aolserver/bin/nsd /usr/local/aolserver/bin/nsd: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), for GNU/Linux 2.6.8, dynamically linked (uses shared libs), not stripped -gustaf -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] how to parse Content-Type text/xml form post
Grab the post data and use an xml parser to get the content There are multiple xml parsers for tcl available that you can use. I would recommend tdom, which supports among other things xpath (see http://www.tdom.org/) -gustaf neumann Am 07.04.10 06:40, schrieb Brad Chick: This should be fairly simple, but I having issues parsing a form post that is of Content-Type text/xml. The methods I see to handle the form data all try to parse the body into key/value pairs. This is not what I need. Rather, I just need the xml payload. any help would be appreciated. Thanks -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] how to parse Content-Type text/xml form post
Is there something wrong with ns_conn content ? Am 07.04.10 09:03, schrieb Bas Scheffers: I think the question is *how* to grab the raw post data? On Wednesday, April 7, 2010 4:12pm, Gustaf Neumannneum...@wu-wien.ac.at said: Grab the post data and use an xml parser to get the content There are multiple xml parsers for tcl available that you can use. I would recommend tdom, which supports among other things xpath (see http://www.tdom.org/) -gustaf neumann Am 07.04.10 06:40, schrieb Brad Chick: This should be fairly simple, but I having issues parsing a form post that is of Content-Type text/xml. The methods I see to handle the form data all try to parse the body into key/value pairs. This is not what I need. Rather, I just need the xml payload. any help would be appreciated. Thanks -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] differences between ns_httppost and sockets? (fix/insight for thread: SSL data truncation?)
Am 04.02.10 11:46, schrieb Torben Brosten:
John Caruso et al,
Could xotcl-core[2] be the work around/solution to this SSL
ns_httpsget bug[1]?
The mentioned bug [1] looks like a buffering problem to me.
Since OpenSSL has its own buffering layer, the interaction
can be tricky and might as well differ depending on the OpenSSL
version. It has been a while, since i fiddled with this in another
context (pound + streaming connections).
I am not aware of the problem in the tls implementation,
but we are not using tls intensively. In general, the http-client
functions of xotcl-core provide an alternative
http/https implementation based directly on the Tcl I/O
functionality. So, this is indeed a different integration
with OpenSSL, which can be tested/debugged also outside
of aolserver.
While the aolserver built-in functions work
mostly without add-ons (nsopenssl), xotcl-core functions require
(1) the tcl thread library (in this context for event
driven I/O and mutex functionality), (2) tls
and (3) xotcl.
-gustaf neumann
cheers,
Torben
1. On July 15 2009 John Caruso wrote:
We've run into a bug with AOLserver 4.5.1 / nsopenssl 3.0beta26. The
bug is fully documented here:
https://sourceforge.net/tracker/?func=detailaid=2822117group_id=3152atid=103152
But the short version is that when using the nsopenssl client-side
routines (e.g. ns_httpsget), the result may be truncated if the
client starts reading before all of the data has been received. This
bug ONLY occurs with an AOLserver client (any version) running
against an AOLserver 4 / nsopenssl 3.0beta26 server. We've
reproduced the bug on RHEL4, RHEL5, and Mac OS X.
The bug is easily demonstrated by copying the file I've attached to
this message (sslbug.tcl) to the top-level context of a web server
running AOLserver 4.x/nsopenssl 3.0beta26 and then navigating to
https://server/sslbug.tcl. If you comment out the ns_httpsget and
use ns_httpget instead, you'll see that the bug disappears.
We've done a lot of instrumenting of nsopenssl/AOLserver, but haven't
been able to track down the root cause. It seems likely that it's
related to data buffering, which seems like it would be occurring
within AOLserver or Tcl...but the issue is definitely specific to
SSL, which implies that it's something in nsopenssl 3.0beta26.
Does anyone have any idea what might be causing this problem?
- John
-- AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in
the email message. You can leave the Subject: field of your email
blank.
2. On Feb 4, 2010, at 1:21 AM, Gustaf Neumann wrote:
Dear Brian and all,
xotcl-core of OpenACS contains a full set of HTTP client
support, including POST requests and https (via the
tcl tls package). Below is the description included in the
source
best regards
-gustaf neumann
===
# Defined classes
# 1) HttpCore (common base class)
# 2) HttpRequest (for blocking requests + timeout support)
# 3) AsyncHttpRequest (for non-blocking requests + timeout support)
# 4) HttpRequestTrace (mixin class)
# 5) Tls (mixin class, applicable to various protocols)
#
##
#
# 1 HttpRequest
#
# HttpRequest is a class to implement the client side
# for the HTTP methods GET and POST.
#
# Example of a GET request:
#
# set r [::xo::HttpRequest new -url http://www.openacs.org/]
#
# The resulting object $r contains all information
# about the requests, such as e.g. status_code or
# data (the response body from the server). For details
# look into the output of [$r serialize]. The result
# object $r is automatically deleted at cleanup of
# a connection thread.
#
# Example of a POST request with a form with var1 and var2
# (providing post_data causes the POST request).
#
# set r [::xo::HttpRequest new \
# -url http://yourhost.yourdomain/yourpath \
# -post_data [export_vars {var1 var2}] \
# -content_type application/x-www-form-urlencoded]
#
# More recently, we added timeout support for blocking http
# requests. By passing a timeout parameter, you gain control
# on the total roundtrip time (in milliseconds, ms):
#
# set r [::xo::HttpRequest new \
# -url http://www.openacs.org/ \
# -timeout 1500]
#
# Please, make sure that you use a recent distribution of tclthread
# ( 2.6.5 ) to have the blocking-timeout feature working
# safely. This newly introduced feature makes use of advanced thread
# synchronisation offered by tclthread that needed to be fixed in
# tclthread = 2.6.5. At the time of this writing, there was no
# post-2.6.5 release of tclthread, hence, you are required to obtain a
# CVS snapshot, dating at least 2008-05-23. E.g.:
#
# cvs -z3 -d:pserver:anonym...@tcl.cvs.sourceforge.net:/cvsroot
Re: [AOLSERVER] differences between ns_httppost and sockets?
Dear Brian and all,
xotcl-core of OpenACS contains a full set of HTTP client
support, including POST requests and https (via the
tcl tls package). Below is the description included in the
source
best regards
-gustaf neumann
===
# Defined classes
# 1) HttpCore (common base class)
# 2) HttpRequest (for blocking requests + timeout support)
# 3) AsyncHttpRequest (for non-blocking requests + timeout support)
# 4) HttpRequestTrace (mixin class)
# 5) Tls (mixin class, applicable to various protocols)
#
##
#
# 1 HttpRequest
#
# HttpRequest is a class to implement the client side
# for the HTTP methods GET and POST.
#
# Example of a GET request:
#
# set r [::xo::HttpRequest new -url http://www.openacs.org/]
#
# The resulting object $r contains all information
# about the requests, such as e.g. status_code or
# data (the response body from the server). For details
# look into the output of [$r serialize]. The result
# object $r is automatically deleted at cleanup of
# a connection thread.
#
# Example of a POST request with a form with var1 and var2
# (providing post_data causes the POST request).
#
# set r [::xo::HttpRequest new \
# -url http://yourhost.yourdomain/yourpath \
# -post_data [export_vars {var1 var2}] \
# -content_type application/x-www-form-urlencoded]
#
# More recently, we added timeout support for blocking http
# requests. By passing a timeout parameter, you gain control
# on the total roundtrip time (in milliseconds, ms):
#
# set r [::xo::HttpRequest new \
# -url http://www.openacs.org/ \
# -timeout 1500]
#
# Please, make sure that you use a recent distribution of tclthread
# ( 2.6.5 ) to have the blocking-timeout feature working
# safely. This newly introduced feature makes use of advanced thread
# synchronisation offered by tclthread that needed to be fixed in
# tclthread = 2.6.5. At the time of this writing, there was no
# post-2.6.5 release of tclthread, hence, you are required to obtain a
# CVS snapshot, dating at least 2008-05-23. E.g.:
#
# cvs -z3 -d:pserver:anonym...@tcl.cvs.sourceforge.net:/cvsroot/tcl co \
# -D 20080523 -d thread2.6.5~20080523 thread
#
# Provided that the Tcl module tls (see e.g. http://tls.sourceforge.net/)
# is available and can be loaded via package require tls into
# the aolserver, you can use both TLS/SSL secured or unsecured requests
# in the synchronous/ asynchronous mode by using an
# https url.
#
# set r [::xo::HttpRequest new -url https://learn.wu-wien.ac.at/]
#
##
#
# 2 AsyncHttpRequest
#
# AsyncHttpRequest is a subclass for HttpCore implementing
# asynchronous HTTP requests without vwait (vwait causes
# stalls on aolserver). AsyncHttpRequest requires to provide a listener
# or callback object that will be notified upon success or failure of
# the request.
#
# Asynchronous requests are much more complex to handle, since
# an application (a connection thread) can submit multiple
# asynchronous requests in parallel, which are likely to
# finish after the current request is done. The advantages
# are that the spooling of data can be delegated to a spooling
# thead and the connection thread is available for handling more
# incoming connections. The disadvantage is the higher
# complexity, one needs means to collect the received data.
#
#
# The following example uses the background delivery thread for
# spooling and defines in this thread a listener. This generic
# listener can be subclasses in applications.
#
# When using asynchronous requests, make sure to specify a listener
# for the callbacks and delete finally the request object in the
# bgdelivery thread.
#
# ::bgdelivery do ::xo::AsyncHttpRequest new \
# -url https://oacs-dotlrn-conf2007.wu-wien.ac.at/conf2007/; \
# -mixin ::xo::AsyncHttpRequest::SimpleListener
# -proc finalize {obj status value} { my destroy }
#
##
#
# 3 HttpRequestTrace
#
# HttpRequestTrace can be used to trace one or all requests.
# If activated, the class writes protocol data into
# /tmp/req-somenumber.
#
# Use
#
# ::xo::HttpCore instmixin add ::xo::HttpRequestTrace
#
# to activate trace for all requests,
# or mixin the class into a single request to trace it.
#
Am 03.02.10 14:50, schrieb Fenton, Brian:
Good news - I found a public webservice at http://rpc.geocoder.us/service/soap/
and ns_http works great with that with my AOLserver version.
So, does anybody have any advice re using an SSL version? Are sockets the way
to go to do this?
thanks
Brian
From: AOLserver Discussion [aolser...@listserv.aol.com] On Behalf Of Fenton,
Brian [brian.fen...@quest.ie]
Sent: 03 February 2010 11:58
Re: [AOLSERVER] differences between ns_httppost and sockets?
Brian,
we started using OpenACS around 5.0, so i cannot comment
on version 4.6 (actually, i am wondering, what reasons you have for
not upgrading to a more recent version). In general, xotcl-core
has very little dependencies on OpenACS, the http client
support has none. There is a dependency in the integration
of xotcl-core with the OpenACS object system
(it requires OpenACS 5.2), so loading the package
might require some fiddleing around.
all the best
-gustaf neumann
Am 04.02.10 12:26, schrieb Fenton, Brian:
Dear Gustaf,
I would dearly love to avail of the xotcl functionality but I suspect my system
won't meet the dependencies.
The system is running an OpenACS 4.6 system (8 years old) and I know the
OpenACS xotcl package arrived later than that.
What are the dependencies for xotcl-core?
I wasn't aware of the tcl tls package - I'll take a look at that.
many thanks!
Brian
From: AOLserver Discussion [aolser...@listserv.aol.com] On Behalf Of Gustaf
Neumann [neum...@wu.ac.at]
Sent: 04 February 2010 09:21
To: AOLSERVER@LISTSERV.AOL.COM
Subject: Re: [AOLSERVER] differences between ns_httppost and sockets?
Dear Brian and all,
xotcl-core of OpenACS contains a full set of HTTP client
support, including POST requests and https (via the
tcl tls package). Below is the description included in the
source
best regards
-gustaf neumann
===
# Defined classes
# 1) HttpCore (common base class)
# 2) HttpRequest (for blocking requests + timeout support)
# 3) AsyncHttpRequest (for non-blocking requests + timeout support)
# 4) HttpRequestTrace (mixin class)
# 5) Tls (mixin class, applicable to various protocols)
#
##
#
# 1 HttpRequest
#
# HttpRequest is a class to implement the client side
# for the HTTP methods GET and POST.
#
# Example of a GET request:
#
# set r [::xo::HttpRequest new -url http://www.openacs.org/]
#
# The resulting object $r contains all information
# about the requests, such as e.g. status_code or
# data (the response body from the server). For details
# look into the output of [$r serialize]. The result
# object $r is automatically deleted at cleanup of
# a connection thread.
#
# Example of a POST request with a form with var1 and var2
# (providing post_data causes the POST request).
#
# set r [::xo::HttpRequest new \
# -url http://yourhost.yourdomain/yourpath \
# -post_data [export_vars {var1 var2}] \
# -content_type application/x-www-form-urlencoded]
#
# More recently, we added timeout support for blocking http
# requests. By passing a timeout parameter, you gain control
# on the total roundtrip time (in milliseconds, ms):
#
# set r [::xo::HttpRequest new \
# -url http://www.openacs.org/ \
# -timeout 1500]
#
# Please, make sure that you use a recent distribution of tclthread
# ( 2.6.5 ) to have the blocking-timeout feature working
# safely. This newly introduced feature makes use of advanced thread
# synchronisation offered by tclthread that needed to be fixed in
# tclthread= 2.6.5. At the time of this writing, there was no
# post-2.6.5 release of tclthread, hence, you are required to obtain a
# CVS snapshot, dating at least 2008-05-23. E.g.:
#
# cvs -z3 -d:pserver:anonym...@tcl.cvs.sourceforge.net:/cvsroot/tcl co \
# -D 20080523 -d thread2.6.5~20080523 thread
#
# Provided that the Tcl module tls (see e.g. http://tls.sourceforge.net/)
# is available and can be loaded via package require tls into
# the aolserver, you can use both TLS/SSL secured or unsecured requests
# in the synchronous/ asynchronous mode by using an
# https url.
#
# set r [::xo::HttpRequest new -url https://learn.wu-wien.ac.at/]
#
##
#
# 2 AsyncHttpRequest
#
# AsyncHttpRequest is a subclass for HttpCore implementing
# asynchronous HTTP requests without vwait (vwait causes
# stalls on aolserver). AsyncHttpRequest requires to provide a listener
# or callback object that will be notified upon success or failure of
# the request.
#
# Asynchronous requests are much more complex to handle, since
# an application (a connection thread) can submit multiple
# asynchronous requests in parallel, which are likely to
# finish after the current request is done. The advantages
# are that the spooling of data can be delegated to a spooling
# thead and the connection thread is available for handling more
# incoming connections. The disadvantage is the higher
# complexity, one needs means to collect the received data.
#
#
# The following example uses the background delivery thread for
# spooling and defines
Re: [AOLSERVER] upload progress
Tom Jackson schrieb: John, I'm just going to venture a guess. I hope that Jim D. or someone else more familiar with the internals will set me straight. The problem with upload progress monitoring is that uploads are finished before a conn thread is allocated. Uploads are done in the driver thread, or a worker thread before the conn thread is given control. Tom, the typical meachnism for upload progress bars is that separate (ajax) queries are used to query the state of an running upload, which is identified by some unique ID (e.g. X-Progress-ID in http://wiki.nginx.org/NginxHttpUploadProgressModule, or some other heuristics. e.g. based on URL and peer addr). So, one needs just a thread-safe introspection meachnism, which checks in the spooler the state (such as ns_upload_stats in naviserver) and returns it back. The spooler hast just to update these statistics when it recieves data. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] large file uploads
bgdelivery - as implemented - works only for downloads, not for uploads. uploads are technically different, since the c-level driver handles these already. Acually, i tend to believe, that naviserver does the right thing (i have not tried, since we are running in 64bit mode since a while). what naviserver version have you tried? Have you set the driver parameter maxupload to 0 to enable file spooling? Acutally, if the upload sizemaxupload, spooling to file happens in naviserver. When the spooler finishes, the file are available under [ns_conn contentfile], which returns the name of the spooled file. Without looking into the details, i would not be wondering, if the spooling to disc works fine, but the multipart processing happens in memory (what you mentioned as form-handler) You might be able to use an exernal mime-decoder such as http://www.freesoft.org/CIE/FAQ/mimedeco.c (possibly via nsproxy). -gustaf neumann Steve Manning schrieb: Don't think you can use it for uploads but Gustaf's Background Delivery thread might help you to serve these files more efficiently. Its discussed here http://www.openacs.org/xowiki/weblog-portlet?ptag=bgdelivery - Steve On Wed, 2009-11-18 at 18:20 -0800, John Buckman wrote: On Nov 18, 2009, at 4:34 PM, Hossein Sharifi wrote: Are you using AOLserver 4.5x on a 64-bit platform? nsd will crash on 32-bit platforms when memory usage (or log files) exceed 2GB. I had to upgrade for a similar issue involving memory usage. Although I think that it would be ideal to port the Naviserver code at some point. No, I'm on a 32bit linux. I guess the solution is to upgrade to a 64bit OS and have way more memory than the max file size. I built naviserver today, with their spooler thread, and tested large file uploads. Naviserver is very efficient at memory when handling the file upload, but it does hold the entire uploaded file in memory as it hands the file off to my form handler. So, it has the same peak memory usage as aolserver, though it needs the peak memory for less time. Still, not an optimal solution. -john -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com mailto:lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Steve Manning Systems Engineer Du Maurier Ltd Tel: +44 (0)116 284 9661 Email: st...@dumaurier.co.uk mailto:st...@dumaurier.co.uk Any views expressed in this email and any attachments are the senders own and do not represent the views of Du Maurier Limited. This email and any attachments should only be read by those persons to whom it is addressed. Accordingly, we disclaim all responsibility and accept no liability (including negligence) for the consequences of any person other than the intended recipients acting , or refraining from acting, on such information. If you have received this email in error, please accept our apologies and we simply request that you delete the email and any attachments. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this email is strictly prohibited. Du Maurier Limited, Tel +44 (0)116 2849661. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Problems Installing on Leopard (Mac 10.5.8)
Dear Cesáreo, for compiling under Mac OS X Leopard or newer, use the following flags to compile Tcl: ./configure --prefix=/opt/aolserver --enable-threads --disable-corefoundation --enable-symbols best regards -gustaf neumann Cesáreo García Rodicio schrieb: Hi! I'm trying to build from source Aolserver 4.5.1 in /usr/local/aolserver under Mac OS X Leopard. First of all, I had problems with make (it didn't find tclsh), but it was solved as in http://www.mail-archive.com/aolserver@listserv.aol.com/msg10030.html Finally I did get to install it in /usr/local/aolserver (bare aolserver, so far, no extra modules). My problem is when I test nsd: roraima:aolserver root# ./bin/nsd -t base.tcl -u nobody -g web roraima:aolserver root# The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec(). Break on __THE_PROCESS I added --enable-64bit to tcl configure script and I rebuilt it. Same problem ( forked ) I was seeking it in this list (some things) but I've only found the --enable-64bit thing and it didn't work for me Some clues? Thanks Cesáreo PD: Macports had some problems with make (suppose tcl executable) roraima:aolserver-4.5.1 root# uname -a Darwin roraima.local 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:55:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_I386 i386 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Problems Installing on Leopard (Mac 10.5.8)
You have to compile Tcl and all tcl based modules with --disable-corefoundation and you have to make sure that you are linking against this version of tcl. best regards -gustaf neumann Cesáreo García Rodicio schrieb: Dear Gustaf, Thanks for your answer but, sadly, it didn't work. Same problem: ... The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec(). ... The building process ends, apparently, OK. This is what I'm doing: TCL building - 1. ./configure --prefix=$AOLSERVER_INST --enable-threads --disable-corefoundation --enable-symbols 2. make test 3. make make install AOS building - 1. ./configure --with-tcl=$TCL_LIB --prefix=$AOLSERVER_INST --enable-symbols 2. $TCL_EXEC/tclsh8.4 nsconfig.tcl -install $AOLSERVER_INST -debug 4. make make install ¿More ideas? Thanks Cesáreo El 19/octubre/09 1:42, Gustaf Neumann escribió: Dear Cesáreo, for compiling under Mac OS X Leopard or newer, use the following flags to compile Tcl: ./configure --prefix=/opt/aolserver --enable-threads --disable-corefoundation --enable-symbols best regards -gustaf neumann Cesáreo García Rodicio schrieb: Hi! I'm trying to build from source Aolserver 4.5.1 in /usr/local/aolserver under Mac OS X Leopard. First of all, I had problems with make (it didn't find tclsh), but it was solved as in http://www.mail-archive.com/aolserver@listserv.aol.com/msg10030.html Finally I did get to install it in /usr/local/aolserver (bare aolserver, so far, no extra modules). My problem is when I test nsd: roraima:aolserver root# ./bin/nsd -t base.tcl -u nobody -g web roraima:aolserver root# The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec(). Break on __THE_PROCESS I added --enable-64bit to tcl configure script and I rebuilt it. Same problem ( forked ) I was seeking it in this list (some things) but I've only found the --enable-64bit thing and it didn't work for me Some clues? Thanks Cesáreo PD: Macports had some problems with make (suppose tcl executable) roraima:aolserver-4.5.1 root# uname -a Darwin roraima.local 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:55:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_I386 i386 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Univ.Prof. Dr. Gustaf Neumann Institute of Information Systems and New Media WU Vienna Augasse 2-6, A-1090 Vienna, AUSTRIA -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Fatal signal 11 error?
Paul, you might be interested in following the discussion on the thread Fatal: received fatal signal 11 - new error after years! http://www.openacs.org/forums/message-view?message_id=2902965 on the OpenACS forums.allthough, the discussion there has nothing to do with virtualization. best regards -gustaf neumann Jade Rubick schrieb: Hi Paul: We found that to debug these, it's best to recompile with debug symbols on -- that way you can get an idea of what is causing the problem. Jade Jade Rubick | Director of Development | TRUiST 2201 Wisconsin Ave NW, Suite 250 | Washington, DC 20007 | www.truist.com http://www.truist.com | +1 202 903 2564 The information contained in this email/document is confidential and may be legally privileged. Access to this email/document by anyone other than the intended recipient(s) is unauthorized. If you are not an intended recipient, any disclosure, copying, distribution, or any action taken or omitted to be taken in reliance to it, is prohibited. On Mon, Sep 7, 2009 at 8:18 AM, Paul Griffiths anothe...@yahoo.com mailto:anothe...@yahoo.com wrote: We are running a custom version of acs 4.3 and the aolserver keeps restarting at random times throughout the day. We noticed this as we have upgraded to new virtualized hardware. The server comes right back so it's not evident from a user perspective. I was looking at the error logs and see that the ‘fatal signal 11’ shows up just before the restart. (grep 'fatal signal 11' -C 5 error.log) Any insight, experience, or suggestions are welcome. Best, Paul -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com mailto:lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver Crash
William Scott Jordan schrieb: Hey all! I've had a few times recently where AOLserver has crashed under high loads. Each time, I see a line in the logs that looks something like unable to alloc 4895393 bytes. What was the size of AOLserver before the crash (most likely around 2GB)? What is the footprint per connection thread (how many packages are you loading, are you using OpenACS, DotLrn)? How many connection threads did you define? Any guesses on what's causing this? In most cases, reduction on the numer of max connection threads helps. One can certainly create even with a single thread this problem (e.g. inifinite loop appending to a tcl variable). Once, you are close to 2GB, an exec of an external program can cause crossing this boundary. In general, there the following ways to proceed: - find the memory wasting loop like above (rather unlikely problem) - reduce memory consumption - compile tcl, aolserver and all used modules with 64bit -gustaf neumann -William -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Univ.Prof. Dr. Gustaf Neumann Institute of Information Systems and New Media WU Vienna Augasse 2-6, A-1090 Vienna, AUSTRIA -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_adp_parse issue
Francesco Petrarch schrieb: Thanks for the suggestions, however they have not provided any positive results. ns_param StackSize 2048000 - only succeeding in causing aolserver to crash with could not allocate memory' errors after a day of running i would not give up fast on that. What ist your stack-size setting, when the system crashes? Set it to a value between your current value and 2mb. From your other posting i conclude, that it normally crashes more frequent, since you mentioned ... estimate about 100 times from 100,000 page requests a day right now.. The could not allocate memory shows that you are most likely running a 32bit version which gets a size close to 2GB. You could reduce resource consumption (e.g. less connection threads) or compile with 64 bit. best regards -gustaf neumann -- Univ.Prof. Dr. Gustaf Neumann Institute of Information Systems and New Media WU Vienna Augasse 2-6, A-1090 Vienna, AUSTRIA -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Is ns_info threads safe to use?
Sep Ng schrieb: Is there a way in AOLserver to do live monitoring on threads? We're sort of hoping to get info on what thread ids are running as of the moment on aolserver. The xotcl-request-monitor watches running requests, essentially via defining filters/traces for requsts and using a monitor thread for keeping track of the starts and ends of requests. If there are starts recorded without ends, it knows these requests are still running in some threads. This approach does not depened on ns_info, we use it on all our production sites. Originally i had one version for pure aolserver and one for OpenACS; since a while i just work on the OpenACS version (which is available via the public cvs repository of OpenACS). best regards -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Question on two AOLserver tickets
Tom Jackson schrieb: On Thu, 2009-05-14 at 18:08 -0400, Jim Davidson wrote: Good idea. Maybe it would make sense to disable it by default with some config flag to enable? I was thinking the same, but I wasn't sure how many people actually use this command. i would even recommend to push this to a compile option. The (sub)command could be always available but return in installed versions an empty result. The docs could say that ... this (sub)command is intended for debugging/developing aolserver and is normally deactivated. In order to avtivate it, compile aolsever with flag XXX. Be aware that this (sub)command is not thread save and can kill the aolserver when used with multiple threads -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Diagnosing an AOLserver performance problem
Janine Sisk wrote: Unfortunately I have no idea; they process their own stats and so far have not shared (I have asked but they keep forgetting). ... With just those changes I'm still seeing the nsd process consume 50%+ of the CPU from time to time but it looks like it's happening for a shorter period of time most of the time, so that is at least an improvement. From my experience, you can obtain much more improvement by looking for bottlenecks in the application than by tuning a few parameter in the aolserver configuration. So far, i did not see any case of slow behavior like you are describing attributable to aolserver. Your problem is most likely more an application performance problem than a aolserver performance problem. If you have an application running since several years, it is not unlikely that some tables have grown quite large, such that missing indices and sequentual scans might have become a problem over the years. Furthermore, many OpenACS user don't care much about the performance of scheduled jobs, but these can lead exactly to the behavior you are observing. I would recommend to monitor the behavior more closely, by using e.g. the xotcl request monitor and the aolserver statistics http://aolserver.cvs.sourceforge.net/viewvc/aolserver/aolserver/examples/config/stats.tcl?view=markup Since you seem to run quite an old version of OpenACS: Another option, i would consider is to upgrade OpenACS. We fixed many performance problems over the last years in the base framework. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_httptime format error in AOL 4.5.1 [solved]
Dear Alexey,
Glad the problem is resolved. i commited a slightly
modified version of the patch (moved the static definition
to the right place in the file, made sure that year is always
4 digits) to CVS head on sourceforge. The patch makes
perfectly sense for non en* locale settings.
It would have saved all of us time if you would have
told us earlier that you are using a patched version.
I have checked the debian patches, there are no
other .c or .h files in the change set, so hopefully
this source of confusion is closed.
Many thanks for your help.
best regards
-gustaf neumann
PS: Tom, can you commit your improvements to queue.c
as well to cvs head?
Alexey Pechnikov schrieb:
Hello!
I'm sorry but the problem is produced by patch for non-english locale.
This patch use wrong format string:
snprintf(buf, 40, %s, %d %s %d %02d:%02d:%02d GMT,
I did fix the bug and I'm think this patch may be added to upstream.
Patch is here:
http://mobigroup.ru/files/aol4.5.1/httptime.c.diff
Thanks for help!
=
--- httptime.c.orig 2003-01-18 22:24:20.0 +0300
+++ httptime.c 2009-05-12 20:14:24.0 +0400
@@ -27,6 +27,9 @@
* version of this file under either the License or the GPL.
*/
+static char *weekdays_names[7] =
+{ Sun, Mon, Tue, Wed, Thu, Fri, Sat };
+
/*
* time.c --
@@ -92,11 +95,14 @@
}
/*
- * This will most likely break if the locale is not an english one.
+ * Using snprintf instead of strftime to always use english names
* The format is RFC 1123: Sun, 06 Nov 1997 09:12:45 GMT
*/
-strftime(buf, 40, %a, %d %b %Y %H:%M:%S GMT, tmPtr);
+snprintf(buf, 40, %s, %02d %s %d %02d:%02d:%02d GMT,
+weekdays_names[tmPtr-tm_wday], tmPtr-tm_mday,
+month_names[tmPtr-tm_mon], tmPtr-tm_year + 1900,
+tmPtr-tm_hour, tmPtr-tm_min, tmPtr-tm_sec);
Ns_DStringAppend(pds, buf);
return pds-string;
Best regards, Alexey Pechnikov.
http://pechnikov.tel/
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Fwd: [AOLSERVER] ns_httptime format error in AOL 4.5.1
Mr Spock would say: fascinating.
We know at least, that the environment is indeed
not the problem. Can you check, whether both programs
use the same strftime function?
such as:
$ nm time-format| fgrep strftime
U strftime@@GLIBC_2.2.5
$ nm /usr/local/src/aolserver-4.5.1/nsd/libnsd.so | fgrep strf
U strftime@@GLIBC_2.2.5
-gustaf neumann
Alexey Pechnikov schrieb:
This return the correct result:
catch {exec /tmp/time-format} msg
ns_log Notice $msg
[12/May/2009:14:56:47][15868.3082901168][-main-] Notice: Thu, 01 Jan 1970 00:00:00 GMT
Best regards, Alexey Pechnikov.
http://pechnikov.tel/
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
Univ.Prof. Dr. Gustaf Neumann
Institute of Information Systems and New Media
WU Vienna
Augasse 2-6, A-1090 Vienna, AUSTRIA
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Diagnosing an AOLserver performance problem
Janine Sisk schrieb: When I first started working on this problem, the settings were ns_param maxconnections 5 ns_param maxthreads 5 ns_param minthreads 5 maxconnections of 5 is for most applications to small. The term maxconnections is misleading since it does not mean maximum number of incoming connections, but is a counter per thread, when it will shut down. With the value above, every thread will exit and be recreated after 5 requests. If the application is well written, no garbage should be left after the request, therefore the value can be set to e.g. 1000. I changed minthreads to 0, which dramatically sped up server restart time, and increased connections/threads first to 50/10 and then 100/20. I didn't see much of an increase in the last change so I left it there. when start with minthreads 5 takes long, you seem to have quite a slow machine. Do you have some usage figures, how many concurrent users you have to serve? By concurrent, i mean running requests at the same time. You can as well increase the threadtimeout to reduce the number of thread restarts ns_param threadtimeout 120 regarding the pools: I would recommend to comment out the maxidle and maxopen values (were needed for old aolserver versions) and - more important - to reduce the number of connections. When you have maxthreads at a low value (e.g. 20), you should not need more than e.g. 20 connections for the first two pools. ns_section ns/db/pool/pool1 ns_param maxidle10 ns_param maxopen10 ns_param connections300 ns_param verbose$debug maybe the 600+ connections you have makes things slower than needed. best regards -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Fwd: [AOLSERVER] ns_httptime format error in AOL 4.5.1
Dear Alexey, From the output, i start to believe that the setting of LANG/LOCALE is not the problem, but i am getting the suspicion that there is a bug in the c-library of your operating system. What operating system do you use? Rationale: aolserver uses strftime(..., ..., ... %d ..., ...) to format the day of month. Since strftime() is defined in the C-library, the bug should be there. -gustaf neumann Alexey Pechnikov schrieb: Hello! On Sunday 10 May 2009 23:42:08 Tom Jackson wrote: I wonder if you have to set this before your start nstclsh? $ LANG=en_US.UTF-8 nstclsh % ns_httptime 0 Thu, 1 Jan 1970 00:00:00 GMT You might also see what my replacement proc produces, or just try in nstclsh: $ nstclsh % set seconds 0 0 % clock format $seconds -format %a, %d %b %Y %H:%M:%S %Z -gmt 3 Thu, 01 Jan 1970 00:00:00 GMT But I just noticed something strange: % ns_parsehttptime [ns_httptime 0] invalid time: Thu, 01 Jan 1970 00:00:00 GMT % ns_parsehttptime Thu, 01 Jan 1970 00:00:00 GMT invalid time: Thu, 01 Jan 1970 00:00:00 GMT % ns_parsehttptime Thu, 1 Jan 1970 00:00:00 GMT invalid time: Thu, 1 JAn 1970 00:00:00 GMT On the last one, notice that the month gets converted to JAn. Hm, I'm get the same result for my ru_RU.UTF-8 locale. Best regards, Alexey Pechnikov. http://pechnikov.tel/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Diagnosing an AOLserver performance problem
Janine, the values make sense, since the 4.5.1 does not omit the cleanups but instead of doing these at the same time, it distributes these (by default by a randomization factor of 20%). The longer the server runs the spikes should go down, since the cleanup times will differ more. Can you post the config file from that site? -gustaf neumann Janine Sisk schrieb: Ok... I just upgraded all of the sites on this system. Things look a bit better, but I still occasionally see an nsd pop up to 50%+ for a few seconds. It doesn't seem to last as long, but that may just be because it's Sunday (and Mother's Day in the US), so traffic is always going to be a bit lighter. Is there anything else I can try? janine On May 10, 2009, at 4:54 AM, Gustaf Neumann wrote: Janine, my recommendation is to give aolserver 4.5.1 a try. We had a similar observation especially when many connection threads are configured. Depending on your query-patterns, it might easily happen with earlier versions that more or less all connection threads will be refreshed (restarted) at the same time. The more threads are defined the worse it will become. Aolserver 4.5.1 addresses this problem by providing a randomization spread. best regards -gustaf neumann Janine Sisk schrieb: I host a couple of sites based on an old version of OpenACS with a proprietary (and fairly primitive) CMS. I didn't write any of it, and up to now have not had to look at things very closely. But a combination of increased traffic and more stuff running on the system has made performance problems that were there all along rear some really ugly heads, and I need to figure out what's going on. From watching the output of top I can see that both Postgres and AOLserver are doing their share of CPU hogging. I've just turned on log_min_duration_statement in Postgres, to see where it's spending most of its time. I'm less sure of what to do for AOLserver. I've increased maxconnections and maxthreads and that seemed to help a little, but not enough. It has been a long time since I last needed to tune AOLserver and I'm a bit at a loss. Any suggestions on where to start? I'd love to know what it's doing when it suddenly jumps up to using 86% of the cpu, then drops back to almost nothing again. BTW, this is AOLserver 4.0.10, with this patch applied: http://sourceforge.net/tracker/index.php?func=detailgroup_id=3152atid=103152aid=1615787 (it didn't seem to help) I can try upgrading if necessary but nothing I've read has made me think that 4.5 will give me any better performance, so for the moment I'm erring on the side of changing as little as possible. thanks, janine --- Janine Sisk President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Sisk President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Error after AOLserver upgrade
Dear Janine,
please check:
http://www.openacs.org/forums/message-view?message_id=488620
The problem was fixed in OpenACS more than 2 years ago.
http://fisheye.openacs.org/browse/OpenACS/openacs-4/packages/acs-tcl/tcl/form-processing-procs.tcl?r1=1.55r2=1.56
Hope this helps,
-gustaf neumann
Janine Sisk schrieb:
Apologies in advance for posting with no research... was just checking
email before running out the door and found an error report from the
client I upgraded over the weekend. When I get back I will employ The
Google and look at the code and all that, but I wanted to toss this
out to see if anyone knew the answer. This is in the proprietary CMS
code that I didn't write, so it's possible they are doing something
that no-one has run into before.
bad variable name af_to_sql(form_name__publish_date): upvar won't
create a scalar variable that looks like an array element
while executing
global $name
(procedure ad_form line 308)
invoked from within
ad_form -extend -name form_name -form {
big snip
---
Janine Sisk
President/CEO of furfly, LLC
503-693-6407
-- AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in
the email message. You can leave the Subject: field of your email blank.
--
Univ.Prof. Dr. Gustaf Neumann
Institute of Information Systems and New Media
WU Vienna
Augasse 2-6, A-1090 Vienna, AUSTRIA
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Fwd: [AOLSERVER] ns_httptime format error in AOL 4.5.1
Alexey Pechnikov schrieb:
My operation system is debian lenny.
Indeed, very strange.
To pin down the problem, compile the progam below and check
the output. It should show the same bug on your system.
I don't have a lenny system around to test, but at least in Ubuntu
8.0.4, everything works as expected.
I don't see, how the locale settings can influence the %d formatting
of stftime. But anyhow, it might be worth to check the output of
locale -k ru_RU.UTF-8 LC_TIME
on your system
hope, we are getting closer
-gustaf neumann
== time-format.c
#include time.h
#include stdio.h
int main() {
time_t clock = 0;
struct tm time, *tmPtr=time;
char buf[40];
tmPtr = gmtime_r(clock,tmPtr);
strftime(buf, 40, %a, %d %b %Y %H:%M:%S GMT, tmPtr);
fprintf(stderr,%s\n,buf);
}
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] AOLServer+pound problem
Dear Alexey, my recommendation is to try nginx. We have up to 3000 concurrently open tcp-connetions, the systems is very reponsive. On the same system we ran into problems at about 800 connections with pound; then we reduced the stack size, that helped until 1500 connections, then we switched. nginx is more work to configure. at least the english documentation is sometimes tough to read, but here you have an advantage. nginx is very stable and fully featured. -gustaf neumann Alexey Pechnikov schrieb: Hello! I'm now have problem with AOLServer (version 4.5.1) and pound (2.4.3.1). For 500+ active users my pound SSL proxy is not effective (for Core Quad server Load Average = 2.0 and system idle is about 0.5 but response time is more than 1 second! AOL can serve pages fast but pound is bottle neck). I need cookie-based virtual hosts. Is nginx good for me? P.S. Russian is my own language and can read nginx documentation. As example my pound configuration: Service HeadDeny X-SSL-.* HeadRequire Host:.*offline.mts.mobigroup.ru.* HeadRequire Cookie: .*session=branch%3Dstableunit%3D1 BackEnd Address 127.0.0.1 Port8001 End End Service HeadDeny X-SSL-.* HeadRequire Host:.*offline.mts.mobigroup.ru.* HeadRequire Cookie: .*session=branch%3Dstableunit%3D3 BackEnd Address 127.0.0.1 Port8003 End End ... Service HeadDeny X-SSL-.* HeadRequire Host:.*offline.mts.mobigroup.ru.* BackEnd Address 127.0.0.1 Port8000 End End Best regards, Alexey Pechnikov. http://pechnikov.tel/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Problems with aolserver and memory crash
If the limit is ok (ayan's mail), you might run into the maximum segment size on 32bit. If this is the case, recompile everything with 64bit enabled. -gustaf neumann Agustin Lopez schrieb: Hello! I have in production six aolserver serving one OpenACS cluster. The aolserver works well. We have only one big problem. When I try to do some big memory operations which need 400 MB, probably the server crash with the log message unable to realloc XX bytes or unable to alloc XX bytes Why this crash? The server have memory for that operation. Any way to check the free memory? Any solution? My versions are: aolserver 4.5.1 tcllib-1.11.1 thread2.6.5 xotcl-1.6.2 If you need some more info, please tell me. Best regards, Agustin -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- Univ.Prof. Dr. Gustaf Neumann Institut of Information Systems and New Media WU Vienna Augasse 2-6, A-1090 Vienna, AUSTRIA -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] missing access log entries
Dear Tom,
your rewrite of the two functions below look fine to me. The structure
is much clearer now, results of the authorization handling are handled
now consistently. I am useing these functions since a few days on one
of my servers and found nothing unusual. Many thanks to Tom!
Andrew, did you test this version as well for your test-cases?
If thise works for you as well (i would assume so), i would think
that version should go into CVS.
best regards
-gustaf neumann
Tom Jackson schrieb:
I think I finally figured it out, and it now appears to me that
including the access logging trace where it is makes perfect sense. What
didn't make sense, and was a bug, was the meaning of the value of
status at the point trace filters, server traces and void traces ran.
In effect, status must indicate the success/failure of sending a
response to the client. The current code didn't do that. Here is my
rewrite of the two affected functions, with comments added:
/*
*--
*
* ConnAuthorize --
*
* Try to authorize a connection.
*
* Results:
* NS_OK on successful authorization,
* NS_FILTER_RETURN on authorization failure, or
* NS_ERROR on error.
*
* Side effects:
* Connection request is authorized. On failure an alternative
* response may be sent to the client.
*
*--
*/
static int
ConnAuthorize(Conn *connPtr)
{
Ns_Conn *conn = (Ns_Conn *) connPtr;
NsServer *servPtr = connPtr-servPtr;
intstatus;
status = Ns_AuthorizeRequest(servPtr-server,
connPtr-request-method, connPtr-request-url,
connPtr-authUser, connPtr-authPasswd, connPtr-peer);
switch (status) {
case NS_OK:
break;
case NS_FORBIDDEN:
/*
* NS_OK indicates that a response was sent to the client
* a this point, we must fast-forward to traces,
* so we convert the NS_OK to NS_FILTER_RETURN.
*/
if ((status = Ns_ConnReturnForbidden(conn)) == NS_OK) {
status = NS_FILTER_RETURN;
}
break;
case NS_UNAUTHORIZED:
/*
* NS_OK indicates that a response was sent to the client
* a this point, we must fast-forward to traces,
* so we convert the NS_OK to NS_FILTER_RETURN.
*/
if ((status = Ns_ConnReturnUnauthorized(conn)) == NS_OK) {
status = NS_FILTER_RETURN;
}
break;
case NS_ERROR:
default:
status = NS_ERROR;
break;
}
return status;
}
/*
*--
*
* ConnRun --
*
* Run a valid connection.
*
* Results:
* None.
*
* Side effects:
* Connection request is read and parsed and the cooresponding
* service routine is called.
*
*--
*/
static void
ConnRun(Conn *connPtr)
{
Ns_Conn *conn = (Ns_Conn *) connPtr;
NsServer *servPtr = connPtr-servPtr;
inti, status;
Tcl_Encodingencoding = NULL;
/*
* Initialize the connection encodings and headers.
*/
connPtr-outputheaders = Ns_SetCreate(NULL);
encoding = NsGetInputEncoding(connPtr);
if (encoding == NULL) {
encoding = NsGetOutputEncoding(connPtr);
if (encoding == NULL) {
encoding = connPtr-servPtr-urlEncoding;
}
}
Ns_ConnSetUrlEncoding(conn, encoding);
if (servPtr-opts.hdrcase != Preserve) {
for (i = 0; i Ns_SetSize(connPtr-headers); ++i) {
if (servPtr-opts.hdrcase == ToLower) {
Ns_StrToLower(Ns_SetKey(connPtr-headers, i));
} else {
Ns_StrToUpper(Ns_SetKey(connPtr-headers, i));
}
}
}
/*
* Run the request.
*/
while (1) {
/*
* Proxy requests replace request logic
*/
if (connPtr-request-protocol != NULL
connPtr-request-host != NULL) {
status = NsConnRunProxyRequest((Ns_Conn *) connPtr);
break;
}
/*
* Each stage of request processing can return one of three
* possible results:
* NS_OK means continue to next stage
* NS_FILTER_RETURN means skip to traces
* NS_ERROR means skip to 500 response attempt
*/
status = NsRunFilters(conn, NS_FILTER_PRE_AUTH);
if (status != NS_OK) {
break;
}
status = ConnAuthorize(connPtr);
if (status != NS_OK) {
break;
}
status = NsRunFilters(conn, NS_FILTER_POST_AUTH);
if (status != NS_OK) {
break;
}
status = Ns_ConnRunRequest(conn);
break;
}
if (status == NS_ERROR) {
status
Re: [AOLSERVER] missing access log entries
Tom Jackson schrieb: Hey, I just noticed that my patch to queue.c misses one case. If an authorization proc returns NS_ERROR, a 500 response is returned, but no trace filters run. This seems like a bug, ... i was as well looking at the result of Ns_AuthorizeRequest, and not at the result of a filter proc. More filters and the traces are run, when the status is NS_OK or NS_FILTER_RETURN. If i look at the code, i wonder, what happens with filters and traces for status codes like NS_FORBIDDEN, NS_UNAUTHORIZED, or NS_SHUTDOWN? Since the status code is handled in the same proc, we have to deal with it consistently To be consistent, the NS_ERROR/default block should be: case NS_ERROR: default: Ns_ConnReturnInternalError(conn); status = NS_OK; break; That is still just one case. If i read the code correctly even with the patch above, the trace filters and the traces are not run when AuthorizeRequest returns NS_FORBIDDEN or NS_UNAUTHORIZED. Therefore, these entries won't show up in the access log-file, which looks as a bug to me. The return codes NS_OK and NS_FILTER_RETURN (which can be set by a filter, by the request or the authorization) mean in the 4.5 code-line: NS_OK: run filters with NS_FILTER_TRACE, run filters with NS_FILTER_VOID_TRACE, run traces NS_FILTER_RETURN run filters with NS_FILTER_TRACE, which is something i don't understand. By looking into the code, i see that a VOID_TRACE filter is registered by the NsTclRegisterTraceObjCmd (when a trace is registered). Why we have actually TRACE and VOID_TRACE? Wouldn't be a single TRACE enough? Shouln't be servertraces marked as deprecated in favor of TRACE filters? (or vice versa)? nslog is the only place, in the common code, where a servertrace is registered. This could be replaced by a call to register a TRACE filter. Then, we should simplify the distinction between NS_OK and NS_FILTER_TRACE to a single case, where the traces are always run. That would be a nice simplificaton and would make it easier to achieve a consitent behavior. -gustaf neumann PS: i wrote this just by reading the code. There might be certainly some imporant cases/reasones missing. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] missing access log entries
Tom Jackson schrieb: Andrew, Okay, great! The changes should probably be divided into two patches: good idea. While the changes in queue.c are somewhat non-obvious the changes related to adprequest.c and adpeval.c are more clear to me. They address the original problem. i was not able to follow the full discussion and most likely one needs more thoughts and largers test cases to consider. Do i understand correctly: - when either the POST_AUTH filter or the request returns != NS_OK the server returns a 500 error - if there was an earlier ns_return XXX, it is ignored. NsRunFilters retuns either NS_OK or NS_FILTER_BREAK, NS_FILTER_TRACE or NS_FILTER_RETURN. Don't we have to deal with the other non-ok cases? More filters and the traces are run, when the status is NS_OK or NS_FILTER_RETURN. If i look at the code, i wonder, what happens with filters and traces for status codes like NS_FORBIDDEN, NS_UNAUTHORIZED, or NS_SHUTDOWN? Have you checked what happens, when a Ns_ConnRunRequest ended in Tcl with a return, break or continue? -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] missing access log entries
Andrew Steets schrieb: The patch I sent earlier seems to fulfill these needs, but I am worried about corner cases where LogTrace (from the nslog module) could blow up. Nothing about the state of the Conn * seems to be guaranteed when the ConnCleanup callbacks are called. Dear Andrew, i think most (all?) of the repondents seems to agree that writing in the about case to the access log file. For me there are still two quesions open: a) is it possoble to call ns_adp_abort at some time, where the server might crash (in normal operations, everthing looks fine to me, problems might occur in when called from some traces; other calls are likely to have similar problems) b) the patch replaces the call to the regular server trace by a connection cleanup call. this means, at least in 4.5.*, ns_adp_abort seems to cancel all traces (also these registered with ns_register_trace). Is this desired? From Tom's website: http://rmadilo.com/files/nsapi/ns_adp_abort.html the doc of ns_adp_abort says ... Every ns_returnxxx call in an ADP should be followed with a call to ns_adp_abort With this recommendation, cancelling traces seem wrong to me; or at least, this should be documented. We don't use traces, all of OpenACS does not use it, so this is no current issue for us. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] missing access log entries
Andrew Steets schrieb: I'm inclined to agree with you that the current behavior is a bug, but it raises the question: should there be such a function that says this connection wasn't handled, don't even log it - or, should ALL connections always be logged, even if it's aborted? As Scott suggested, we should probably log everything, at least for some reasonable value of everything. Even if you switch the access log trace to the cleanup callback, you still don't get access entries for clients who connect but don't issue a well formed HTTP request. I don't have a huge problem with that, and I think it would be difficult to log those types of events. i am as well in favor of loging the adp_aborts, since this seems a used idiom. There is no advantage in omitting such entries in the log file, but it might be hard to figure out what happens without that. For example, in our production system, we use nginx, which logs these requests on its own. A problem show up, when we try to debug a situation, where some requests present in the nginx log are missing in the aolserver log. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] TCL 8.5 and ns_proxy segfault [closed] + 2 new non-critical bugs
Alexey Pechnikov schrieb: Patch really did solve the problem. I did test AOL4.5.1+Tcl8.5.6 build on one my low-loaded productions server by 4 hour now and this work. Thank you very much! Good to get confirmation. I have commited the changes for both discussed problems (buffer overflow and 8.5 initialization problem) to CVS HEAD at sourceforge. But I did find a few additional problems: Currently, i have two tough deadlines, so i can't look into these in more depth. It might be a good idea to report the two bugs on the tracker to keep them on the radar http://sourceforge.net/tracker2/?func=browsegroup_id=3152atid=103152 -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] TCL 8.5 and ns_proxy segfault
Alexey Pechnikov schrieb: Hello! I did try to build AOL 4.5.1 with TCL 8.5. AOL is work, but nsproxy module now doesn't work. $ /usr/lib/aolserver4/bin/nsproxy Segmentation fault I did copy strace for no-work nspoxy (TCL 8.5) and work (TCL 8.4). i can confirm, that aolserver 4.5.1 compiled with tcl 8.5 crashes, when nsproxy is being called. The strace is not very informative. gdb is better. The stack trace below indicates that the problem might point to a problem with the thread associated data. GetData() in nsd/tclinit.c calls Tcl_AsyncCreate(), which crashes when it tries to get the thread data. #1 0x0a094807 in ThreadStorageGetHashTable () #2 0x0a09490e in TclpThreadDataKeyGet () #3 0x0a093961 in Tcl_GetThreadData () #4 0x0a00be49 in Tcl_AsyncCreate () #5 0x000adb75 in GetData () at tclinit.c:1850 #6 0x000ad8e9 in GetCacheEntry (servPtr=0x2a010) at tclinit.c:1654 #7 0x000ad738 in PopInterp (server=0x0) at tclinit.c:1547 #8 0x000abeee in Ns_TclAllocateInterp (server=0x0) at tclinit.c:368 #9 0x000abe03 in Ns_TclCreateInterp () at tclinit.c:250 #10 0x7f0a in Ns_ProxyMain (argc=1, argv=0xb71c, init=0x3f69 MyInit) at nsproxylib.c:329 #11 0x3f63 in main (argc=1, argv=0xb71c) at nsproxy.c:45 not good. -gn -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] TCL 8.5 and ns_proxy segfault
Well, i found at least one problem in ns_proxy so far, but it seems
unrelated
to Tcl 8.5 and Alexey's problem.
nsproxy defines the structure Proxy as follows:
typedef struct Proxy {
char id[16];/* Proxy unique string id. */
Proc *procPtr;/* Running child process, if any. */
} Proxy;
later, the id is initialized by
sprintf(proxyPtr-id, %s-proxy-%d, poolPtr-name, poolPtr-nextid++);
The sprintf will cause a problem, if the pool name has more than 16-9 =
7 bytes.
Since pool name is passed in via tcl, and its length is not checked,
wierd things
are likely to happen (the procPtr will be overwritten).
For example, OpenACS does a
set handle [ns_proxy get exec_proxy]
where len(exec_proxy)7.
The quick fix is to change char id[16] to id[32], but actually, one should
check the length...
Unfortunately, this does not help in the original problem.
-gustaf neumann
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] TCL 8.5 and ns_proxy segfault
Dear Alexey,
below is a fix for the nsproxy problem that you reported.
The problem comes from a different initialization semantics
in Tcl 8.5, that hits the nsproxy binary. The added call
should work with Tcl 8.4 as well; please report back
such i can commit the fix to CVS.
best regards
-gustaf neumann
===
RCS file: /cvsroot/aolserver/aolserver/nsproxy/nsproxy.c,v
retrieving revision 1.1
diff -u -r1.1 nsproxy.c
--- nsproxy.c19 Apr 2006 18:55:09 -1.1
+++ nsproxy.c7 Mar 2009 23:45:04 -
@@ -42,6 +42,7 @@
int
main(int argc, char **argv)
{
+Tcl_FindExecutable(argv[0]);
return Ns_ProxyMain(argc, argv, MyInit);
}
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] AOL + SQLite+nsproxy question
Alexey, don't use binaries of drivers compiled for 4.0.* with 4.5.* and vice versa; this can easily lead to crashes. Make as well sure, that all involved binaries are compiled against the same version of tcl. nspostgres does work with 4.0.10 and 4.5.0 and 4.5.1 with all recent versions of postgres. We use it in 20+ installations without any problems. -gustaf neumann Alexey Pechnikov schrieb: Hello! On Sunday 01 March 2009 21:05:30 Don Baccus wrote: http://www.google.com/search?hl=enq=nspostgres+received+fatal +signal+11btnG=Search nspostgres doesn't work as you can see. I know for a fact that Iuri, the last person who shows up for that search, got it working and in fact has used it in a development project in which I was mentoring him. I did find advise to set pgbin variable in config but this didn't help. ns_section ns/db/driver/postgres ns_parampgbin /usr/lib/postgresql/8.1/bin/psql AOL 4.0.10 + nspostgres work fine. But AOL 4.5 + nspostgres (I did try to use versions nspostgres compiled for AOL 4.0 and 4.5) doesn't work. There is AOL log: [01/Mar/2009:20:49:09][746.3057605520][-conn:1-] Notice: dbinit: sql(127.0.0.1:5000:offline1): 'select count(*) as count fromnotice.messages where not is_send and user_id=2954 and type='user'' [01/Mar/2009:20:49:09][746.3057744784][-conn:0-] Notice: Querying 'select count(*) as count from notice.messages where not is_send and user_id=2955 and type='user';' [01/Mar/2009:20:49:09][746.3057744784][-conn:0-] Notice: dbinit: sql(127.0.0.1:5000:offline1): 'select count(*) as count fromnotice.messages where not is_send and user_id=2955 and type='user'' [01/Mar/2009:20:49:11][746.3057605520][-conn:1-] Fatal: received fatal signal 11 Best regards. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Exec error produce error Fatal: received fatal signal 11.
dear alexey,
Strange, both commands behave as expected on my machine with aolserver
4.5.1.
Can it be that your machine is running out of memory? Can it be that you
have
mixed tcl versions during compilation?
btw, you should use nsproxy instead of exec to call soffice.
best regards
-gustaf neumann
Alexey Pechnikov schrieb:
Hello!
AOL version 4.5 may be killed my simple code:
catch {exec soffice -invisible -headless -norestore
macro:///Standard.Converter.ConvertToPDF($tmpdocname) -display :10}
I rewrite command as
exec soffice -invisible -headless -norestore
macro:///Standard.Converter.ConvertToPDF($tmpdocname) -display :10
/dev/null
It's work but is not stable.
And this code work wrong:
if {[file exists $tmpresname]==1} {
ns_returnfile 200 [ns_guesstype $tmpresname] $tmpresname
} else {
puts File $tmpresname not exists
}
We can see message in AOL log:
File /tmp/fileiBmXN0/doc.pdf not exists
But
$ ls -lh /tmp/fileiBmXN0/doc.pdf
-rw-r--r-- 1 www-data www-data 81K 2009-03-02 00:38 /tmp/fileiBmXN0/doc.pdf
Is AOL 4.5 development and unstable release? I did think AOL 4.5 is production
but AOL 4.0.10 work fine and AOL 4.5 generate a lot of different errors.
Best regards.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] AOLserver 4.5.1 released on February 2, 2009
Fenton, Brian schrieb: Congratulations indeed. On a related note, AOLserver.com doesn't appear to be responding. Can someone bring it up please? i can't help here, but i noticed that www.aolserver.com (or http://aolserver.sourceforge.net/) works. For documentation, please use http://panoptic.com/wiki/aolserver/AOLserver_Wiki for the time being. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Virtual hosting
While it is true that for plain proxing, pound is simpler and easier to configure, nginx scales better (e.g. when you have e.g. 1000+ concurrent connecitions) and offers a large scale of modules etc. We switched from pound to nginx two years ago, and the decision was right for us. While for pound, every connection is a separate thread, nginx uses asynchronous connecitions (+ configurable multiple threads). best regards -gustaf neumann Alexey Pechnikov schrieb: Hello! В сообщении от Saturday 07 February 2009 06:00:39 Alex написал(а): Apache for proxying nginx nginx for proxing?!! pound Best regards, Alexey. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Error code: ssl_error_rx_record_too_long
Check out http://openacs.org/xowiki/tag/bgdelivery aolserver 4.5 head and naviserver has this built in. best regards -gustaf neumann Scott Goodwin schrieb: Thank you for posting the resolution to this problem. The bgdelivery capability in OpenACS looks interesting, and I noticed this in the OpenACS bgdelivery code: ns_log notice AOLserver is not patched for bgdelivery, NOT loading bgdelivery Are there patches to the AOLserver executable required to make bgdelivery possible? /s. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver Crash!
Rami, it looks to me as if the problem is due to a c-extension you are using and happens after a thread exit. When a thread exists, it frees among other things the associated tcl interpreter. At this time, all c extensions have to unload cleanly as well. Note that ns_eval creates and destroys as well a thread. The problem might be due to some leftover, due to uninitialized memory, etc. This is inherently cumbersome to debug in C code, since sometimes, you might be lucky and the server survives the real bug, some other times not. Furthermore, the bug and the crash are often in two different parts of the code. Maybe, you have simply luck on the other machines. i would recommend the following: a) try to make the crash happen reproducible, in as simple as possible setup. I would recommend to stress thread destroys (e.g. setting maxconnections to e.g. 2, test with calls doing an ns_eval) b) reduce all c-extensions (do you have to use nsmysql and nsoracle?) In the best of all possible worlds, you might not need all c-modules in the crash case, so dropping might help to detect the culprit. From some distant, my first suspicion falls to nsmysql, i am not sure how frequent this is used. hope this help a little -gustaf neumann Rami Jadaa schrieb: Hi Scott, Thanks for your reply. I don't think that I can send the log as it will be so big , as AOlserver initiates and load a lot of ACS code... And for the checksum, we did the following: Using pound, we shifted the load going to this webserver to another server on another machine where it uses a different local copy of the same application, and then after the reload, the server were we shifted the load to crashed, and the old one didn't!! So i can take out he doubt on file corruption, right? Environment : Aolserver 4.0.10 , fetched from CVS almost 6 months back . nsoracle Oracle Driver version 2.8a1 nsmysql CVS Oracle 10gR2 Libraries AMD x86_64 RHEL 4 Curently tcl 8.4.16 also tried tcl 8.4.11 Please help as this is driving me crazy :( Thanks in advance -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Data corruption with fastpath caching
Titi Alailima schrieb: This sounds like the problem. Not a bug with fastpath, Come on folks, the discussion wether or the behavior shown by John Caruso is a bug or not is completely fruitless. Most aolserver users are not novices, and if it takes some of us several weeks to find, what the problem is, we should act and not insist, that it is no bug. In my opinion, switching the caching index from inodes to file names (like in windows) is a very reasonable solution getting rid of most problems (although less cache efficient). It would be additionally a nice feature to provide a configuration option for getting back the current behavior (for people with tons of links). This option would guarantee backward compatibiity. This way, one could savely let fastpath switched on by default. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Fun, free applications for building personal sites.
Jeff Rogers schrieb: One starting point I'd be willing to help on here is an openid client (since I've already started work on it). Hi Jeff, A few weeks ago i have made as well a small openid client (using OpenACS, but not integrated with OpenACS authorities, cookies, etc). git clone git://alice.wu-wien.ac.at/openid Currently, this is just an openid 1.* implementation. Our goal is to provide support for openID 2.0 in the near future and integrate it with the OpenACS authorization system. Martin Matuska (working in my institute) is now looking into these matters. To use it, you need OpenACS 5.4 (maybe 5.3 is sufficient as well) and a recent version of xotcl-core and tdom. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Fix for ns_tmpnam under Windows
fixed. -gustaf neumann Titi Alailima schrieb: There is a missing variable declaration in this patch for i, the for-loop index. Anyone want to make this fix and commit it? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] Fix for ns_tmpnam under Windows
Dear AOLserver community There was a problem with ns_tmpnam, when the aolserver was compiled under windows (esp. Vista). The Tcl commandns_tmpnam is implemented in C using tmpnam(), which exists under WIN32, but behaves differently as on unix counterparts and is practically unusable. The problem is that tmnam() under Windows generates a filename for the root directory of the actual drive (at least under Vista, no permissions under normal conditions) and ignores the TMP environment variable. See some background info form Microsoft: http://msdn2.microsoft.com/en-us/library/hs3e7355(VS.80).aspx I have commited a patch to CVS head to address this problem. http://aolserver.cvs.sourceforge.net/aolserver/aolserver/nsd/tclfile.c?r1=1.25r2=1.26 The patch is local and documents the intended behavior and background in detail. Please crosscheck. best regards -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] ns_share (was Active participation)
Brett Schwarz schrieb: Just curious...did you have to do anything special to get it working inside aolserver? There are two ways of using tcl libthread in aolsever: (a) configure it for plain tcl (standard), load into aolserver with package require; you get tsv:: etc. (b) configure with --with-aolserver, load it as an aolserver module from the startup script; in this case libthread overloads nsv etc. with (a) you have the advantage with standard tcl compatibility, with (b) one can run existing aolserver applications (OpenACS, ...) on it, still getting the performance gain, and - for me the most important benefit - you get threads that can easily communitcate via the event-loop used in the tcl thread library. -gustaf neumann - Original Message From: Jay Rohr [EMAIL PROTECTED] To: AOLSERVER@LISTSERV.AOL.COM Sent: Wednesday, April 16, 2008 2:17:38 PM Subject: Re: [AOLSERVER] ns_share (was Active participation) just FYI I am using the tsv (from the thread package) functions within aolserver and, at least for my uses, it is faster than the nsv equivalents. (and it is a namespace :-)) It also has a number of advantages, I can run the same code outside of aolserver, the tsv::lock allows me to run a block of code under the shared mutex, and it has a more complete set of list functions. Jay Jim Davidson wrote: Hi, Brent Welch wrote some ns_share code which worked with Tcl variable traces to emulate the original code. It works except for a few edge cases but is generally considered less efficient and flexible than the new nsv_* commands. So, when I say we gave up on ns_share I mean we stopped applying our patches to the Tcl core for ns_share, allowing us to use the Tcl source un-altered. -Jim On Apr 14, 2008, at 1:37 PM, Brett Schwarz wrote: True -- the original threading in AOLserver 2.0 way back in 1995 (yes, 1995) was based on a thread-safe hacked Tcl 7.3. I can't recall where I got it -- I think someone at UCB did the work. We later hacked Tcl 7.4 and 8.1 on our own before enough support was in the Tcl core and we were ready to give up on ns_share which was a significant hack that couldn't reasonably be brought forward. I was just looking at file.tcl, and it still uses ns_share. Is that ok? Or should that be changed? Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] can someone make this change
John Buckman schrieb: the utils/*.tcl files in CVS all need: #!/usr/local/bin/tclsh prepended at the top. Currently, they don't have this, and thus are run as shell scripts. hmm, shouldn't this be a /usr/bin/env tclsh instead? -gustaf -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] can someone make this change
Cite from http://wiki.tcl.tk/812: There seems to be a growing consensus that the best way to make a Tcl script executable as if it were a Bash or Sh script is to put the following on the very top line of your Tcl (or Tk) script: #!/usr/bin/env tclsh well, the hack with the Tcl comment with the continuation line works as well, but i am not sure that i would call it the way it should be done. -gustaf neumann John Buckman schrieb: This looks to be the way it should be done (from tests/new/all.tcl) $ more tests/new/all.tcl #!/bin/sh # the next line restarts using tclsh \ exec tclsh $0 $@ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] writing an aolserver 4 module frustration
Mark, i have compiled and installed your posted version of nsperl2 on Mac OS X 10.5.2 against aolserver 4.5 from head, and it does not crash. Are you sure, you linked nsperl2 against the same tcl version, with the same alloc implementation? What other modules are you loading? The invalid block message shows typically up, when there is a double free issued. -gustaf Mark Aufflick schrieb: Slight prototype fix to my .c file, but no change to the issue/ On Sun, Mar 23, 2008 at 7:45 PM, Mark Aufflick [EMAIL PROTECTED] wrote: Hi all, So I'm trying to write a new loadable module. From a combination of example modules I thought I was set, but I have this frustrating roadblock I can't figure out. As soon as I have a Tcl_CreateCommand in my code, the server startup crashes with alloc: invalid block. I have attached my .c and Makefile if anyone has time to cast their eye over it? I think I'm missing some difference with Aolserver 4 since Vlad's nsfortune shows the same issue and I assume that used to work once! Unless I have a problem with my aolserver still, but nslog works fine and that obviously does Tcl_CreateCommand... Here is my log output: ...snip... [23/Mar/2008:19:31:28][79157.2691797108][-main-] Notice: nslog: opened '/opt/aolserver/servers/server1/modules/nslog/access.log' [23/Mar/2008:19:31:28][79157.2691797108][-main-] Notice: modload: loading '/opt/aolserver/bin/nscp.so' [23/Mar/2008:19:31:28][79157.2691797108][-main-] Notice: nscp: listening on 127.0.0.1:8003 [23/Mar/2008:19:31:28][79157.2691797108][-main-] Notice: nscp: added user: [23/Mar/2008:19:31:28][79157.2691797108][-main-] Notice: modload: loading '/opt/aolserver/bin/nsperl2.so' [23/Mar/2008:19:31:28][79157.2691797108][-main-] Notice: nsperl2: loaded [23/Mar/2008:19:31:28][79157.2691797108][-main-] Notice: before [23/Mar/2008:19:31:28][79157.2691797108][-main-] Notice: after [23/Mar/2008:19:31:28][79157.2691797108][-main-] Notice: conf: [ns/server/server1]enabletclpages = 0 [23/Mar/2008:19:31:28][79157.2691797108][-main-] Notice: default thread pool: minthreads 5 maxthreads 20 idle 0 current 0 maxconns 20 queued 0 timeout 1 alloc: invalid block: 0x207740: 0 0 0 Abort trap The before, after shows the Tcl_CreateCommand returns. The stack on exit looks something like: #0 0x93ef6cb8 in __kill () #1 0x93f9b7c8 in abort () #2 0x0a05b428 in Tcl_PanicVA () #3 0x0a05b460 in Tcl_Panic () #4 0x0a0697a8 in TclpFree () #5 0x0a0105b0 in Tcl_DeleteCommandFromToken () #6 0x0a055c38 in TclTeardownNamespace () #7 0x0a010d94 in DeleteInterpProc () #8 0x000cfe4c in Ns_TclDestroyInterp (interp=0x38410) at tclinit.c:521 #9 0x000d1be8 in PushInterp (itPtr=0x48810) at tclinit.c:1625 #10 0x000cfc9c in Ns_TclDeAllocateInterp (interp=0x38410) at tclinit.c:402 #11 0x000d17c0 in NsTclInitServer (server=0x4c0d0 server1) at tclinit.c:1405 #12 0x000c1d20 in NsInitServer (server=0x4c0d0 server1, initProc=0x3f38 ServerInit) at server.c:219 #13 0x000b7ba0 in Ns_Main (argc=3, argv=0xba4c, initProc=0x3f38 ServerInit) at nsmain.c:524 #14 0x3f1c in main (argc=3, argv=0xba4c) at main.c:64 Unfortunately as I mentioned in my compile issue email I have not been able to compile against a debug-symbol Tcl so I can't trace past Ns_TclDestroyInterp. Thanks for your time - I'm going nuts here! Mark. -- Mark Aufflick contact info at http://mark.aufflick.com/about/contact -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Compiling 4.5 on Mac OS X 10.5 leopard issues
Dear Mark, this problem was discussed earlier here in the mailing list. I have commited a fix to the head version in the sourceforge cvs for the problem, which changes the semantics for setrlimit for compiles on Leopard and above. It passes now in case of unlimited number of open files OPEN_MAX files to setrlimit. -gustaf neumann Mark Aufflick schrieb: Anyhoo - once built, nsd segfaults, predictably in the dynamic loading stage: ... I get this even if I just try nsd --help I need some --help :) It seems that people have this running on Intel ok, and I have compiled Aolserver 4.x on earlier Mac OS X versions ok, but the huge string of defines to gcc has me a bit stumped about where to look. Thanks. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] 32- vs. 64-bit mem bloat
Dear Alex, Didn't you mention earlier, that the bloat happens only on one of your machines? Was this a wrong indication? If you have a well behaved system, you can increase maxconnections to e.g. 1 (the parameter denotes the number of connections to be served by a connection thread before it restarts) and set minthreads to eg. 5. This does not solve the memory-leak problem but should slowdown the memory hunger, if connection thread restart causes the growth. I assume, you are using the standard zippy memory allocator from tcl. I would not recommend to restart based on the size, since the server will do this at a time you might not want it (e.g. during a demo). A nightly reboot is the better option. The size certainly depends on the amount of tcl code times number of threads and the state management (use of connection-spanning tcl vars, or use of nsv). Concerning tcllib and MD5: tcllib contains actually 3 versions of md5: a) a tcl-based code b) a version based on critcl (see tcllib*/md5/md5c.tcl) c) a version based on trf During startup, it tries to use the best version. (b) and (c) are accelerated versions using c-code, this might explain the speed differences you are experiencing. I would recommend to test first with the tcl-only version and check the memory growth. If it has no effect, it seems safe to use the accelerated versions. Concerning xotcl: - there is no special handling needed concening memory freeing. as long you take care, that only those objects survive a connection that you want. Typical applications load the frequently uses classes during startup to speed processing of requests. - xotcl-core of OpenACS has support to ease deallocation of temporary objects at the end of a connection (e.g. Object o -destroy_on_cleanup) - [Class allinstances] returns the classes. To check the objects, use [Object allinstances] Concerning Sig 11: This seems to come from libnssession.so. I am not sure, how frequently this is used (we don't use it). It seems as if the source for the crash is there. Is it possible for you to test without nssession (crashes, memory consumption)? -gustaf Hello Tom, Gustaf I am still trying to pin down the memory leak problem. Could you answer few more questions, please? 1. It looks like that nsd process grows ~1.5 - 2.0 MB every time a thread dies due to timeout or number of connections (and new one starts). Changing configuration just makes the bloating process slower of faster. I restart the nsd process when it reaches 0.5GB in size, but I hope that's a reasonable number. I tried valgrind it, but couldn't make any conclusions. Any better ideas how to debug it? 2. TclLib shall not harm IMHO, it's Tcl- only, no binary code. Please confirm that. Besides, I need it only for MD5, and standalone md5 Tcl implementations for some reason seems to be much slower. If TclLib is risky, what would be the best package/module for MD5 ? mhash may be? 3. XoTcl - I verified that I do not leave any objects which I created, they all destroyed. (at least that's what [Class allinstances] reports) However, Gustaf noted that it still may need special handling. What kind of handling? Could you point me to the code in openacs that does that? For me it would be a big hit if I need to rewrite without XoTcl, so I would like to make it working. 4. Memory leaks aside, nsd quite frequently dies on Sig 11. Sometimes just a few seconds after being started. I remember having similar issues some year+ ago, but those were fixed quickly. Anyone experiences the same? I didnt' debug it in depth yet, that's what gdb said when I quickly tried it: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread -1296041040 (LWP 32284)] 0xb7338ba5 in Ns_CookieGet () from /aolserver_bin/aolserver4.5/lib/libnssession.so Thanks, ~ Alex. On Tue, Mar 4, 2008 at 3:29 AM, Gustaf Neumann [EMAIL PROTECTED] wrote: Dear Alex, The good news is that it bloats only on one server, so the problem must on that configuration. There were a couple of memory leaks reported for postgres 7.4, not sure if these hit you (google around): http://www.postgresql.org/docs/7.4/static/release-6-5.html do you have a chance to install on the affected machine as well postgres 8.2 (maybe in addition to 7.4)? it would be interesting to see, if you still see a bloat. i found a small glitch in your crude script: The last line of the xotcl install should read: execute make install-aol instead of execute make install -gustaf neumann Alex schrieb: Tom, Gustaf, All Yes, that happens even if the same page is requested. Could you tell me more about pre-queue filters? The interesting thing is that I only see this memory leak issue on one of my servers. All servers run x86 Debian, though might be slightly differently dist-upgraded, Postgres version differs 7.4 vs 8.2 The website code itself, nsd config files are exactly the same on all servers. Over
Re: [AOLSERVER] Install problem
Dossy Shiobara schrieb:
On 2008.03.04, Michael Andrews [EMAIL PROTECTED] wrote:
OK - I'll take a look tonight. I think that a 0 as the threadtimeout
value use to mean no timeout. I guess I'll need to look at the C to
see what the changes were.
Is this file even needed anymore?
IMHO, the semantics around timeouts should ALWAYS be:
= -1 == infinite
0 == immediate
= 1 == n units
this value is used in queue.c, where incoming connections
are enqueued. Not sure, if 0 is desirable (busy server, when
nothing arrives) or -1 (block would make shutdown more
complex).
If i remember correct, my check was done in consistency with
socktimeout, which is checked as well for n1
if (!Ns_ConfigGetInt(path, socktimeout, n) || n 1) {
n = 30;/* 30 seconds. */
}
For implementing -1 for infinite - if desired - more changes
would be needed, since the value is used in Ns_IncrTime() which
is agnostic for the timeout value semantics mentioned above.
Not checking the input values (as it was before) is not
a good idea either, since these values will break the server.
best regards
-gustaf neumann
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] authuser
Jeff Rogers schrieb: Does anyone (other than me) think there would be value in allowing the conn username to be updated, i.e., by a new 'ns_conn setauthuser' subommand? i would appreciate it. -gn -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] 32- vs. 64-bit mem bloat
Dear Alex, The good news is that it bloats only on one server, so the problem must on that configuration. There were a couple of memory leaks reported for postgres 7.4, not sure if these hit you (google around): http://www.postgresql.org/docs/7.4/static/release-6-5.html do you have a chance to install on the affected machine as well postgres 8.2 (maybe in addition to 7.4)? it would be interesting to see, if you still see a bloat. i found a small glitch in your crude script: The last line of the xotcl install should read: execute make install-aol instead of execute make install -gustaf neumann Alex schrieb: Tom, Gustaf, All Yes, that happens even if the same page is requested. Could you tell me more about pre-queue filters? The interesting thing is that I only see this memory leak issue on one of my servers. All servers run x86 Debian, though might be slightly differently dist-upgraded, Postgres version differs 7.4 vs 8.2 The website code itself, nsd config files are exactly the same on all servers. Over the weekend I tried to rebuild tcl, tcllib, xotcl, aolserver. The details how I did it can be found here http://alex-andr.org/notes/tiki-index.php?page=AOLServer+setup+with+XOTcl I used latest versions however, tcl 8.4.18, tcllib 1.10, xotcl 1.6.0 I even made a crude script that fetches the sources and builds a stand-alone aolserver installation, in order to make sure the build results are reproducible. http://alex-andr.org/notes/tiki-index.php?page=nsd+setup+script Still, even when all [hopefully] binaries are being copied to all servers, only one of them experience the same issue with huge memory leak on each page request. Any insightful thoughts? Thanks, ~ Alex. On Fri, Feb 29, 2008 at 6:23 PM, Tom Jackson [EMAIL PROTECTED] wrote: Actually 2MB / request sounds like a massive memory leak. Maybe something isn't getting cleaned up? Question: does this happen if you request the same page over and over? I discovered one way to get a massive leak in 4.5: register a prequeue filter. Note that both XoTCL and tcllib are compiled third party modules, which is usually where you get memory leaks, not in the modules which have been used with AOLserver for years. My guess would be tcllib, or just failing to release something big. tom jackson On Friday 29 February 2008 11:47, Alex wrote: What would be a normal size of 32 bit nsd process? How can it be reduced? One of my servers only has 1G or ram, and I am forced to restart nsd every so often, when it uses up almost all the memory. Running 4.5 + XoTCL + tcllib, it seams to grow up to 2MB on each page request . -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Install problem
Xavier Bourguignon schrieb: Hello Gustaf, yes I compiled tcl8.4.18 myself with --enable-threads. I will redo everything and see what happens, will keep you guys posted on this. what do you get when doing a nm /home/xbourgui/programs/tcl8.4//lib/libtcl8.4.so | fgrep Tcl_GetMemory ? You should see something like 0a064e91 T Tcl_GetMemoryInfo The problem occurs in your listing with nstrheadtest, which is just a testing routine, not needed for using aolserver. If only this linkage fails, it would be no problem. But, if Tcl_GetMemoryInfo() is not defined in libtcl8.4, you will have as well problems with linking nsd, which would be serious. -gustaf neumann Thank you On 04/03/2008, Gustaf Neumann [EMAIL PROTECTED] wrote: Dear Xavier, There is a known problem with Tcl_GetMemoryInfo() in Tcl 8.5 (see e.g. http://aspn.activestate.com/ASPN/Mail/Message/tcl-core/3610677 or http://sourceforge.net/tracker/index.php?func=detailaid=1868171group_id=10894atid=110894). however, since you are using tcl 8.5, this seems wierd. Did you compile Tcl yourself? Did you compile with --enable-threads? best regards -gustaf neumann Xavier Bourguignon schrieb: I ran make clean, then make all and I got this on make all: gcc -O2 -Wall -Wno-implicit-int -fno-strict-aliasing -fPIC -O2 -fno-strict-aliasing -pipe -nostartfiles -I/home/xbourgui/Desktop/aolserver-4.5.0/include -I/home/xbourgui/programs/tcl8.4//include -DNO_CONST -DHAVE_LIMITS_H=1 -DHAVE_UNISTD_H=1 -DHAVE_SYS_PARAM_H=1 -DUSE_THREAD_ALLOC=1 -D_REENTRANT=1 -D_THREAD_SAFE=1 -DHAVE_PTHREAD_ATTR_SETSTACKSIZE=1 -DHAVE_PTHREAD_ATFORK=1 -DTCL_THREADS=1 -DPEEK_XCLOSEIM=1 -D_LARGEFILE64_SOURCE=1 -DTCL_WIDE_INT_TYPE=long\ long -DHAVE_STRUCT_STAT64=1 -DHAVE_OPEN64=1 -DHAVE_LSEEK64=1 -DHAVE_TYPE_OFF64_T=1 -DHAVE_GETCWD=1 -DHAVE_OPENDIR=1 -DHAVE_STRSTR=1 -DHAVE_STRTOL=1 -DHAVE_STRTOLL=1 -DHAVE_STRTOULL=1 -DHAVE_TMPNAM=1 -DHAVE_WAITPID=1 -DHAVE_GETPWUID_R_5=1 -DHAVE_GETPWUID_R=1 -DHAVE_GETPWNAM_R_5=1 -DHAVE_GETPWNAM_R=1 -DHAVE_GETGRGID_R_5=1 -DHAVE_GETGRGID_R=1 -DHAVE_GETGRNAM_R_5=1 -DHAVE_GETGRNAM_R=1 -DHAVE_GETHOSTBYNAME_R_6=1 -DHAVE_GETHOSTBYNAME_R=1 -DHAVE_GETHOSTBYADDR_R_8=1 -DHAVE_GETHOSTBYADDR_R=1 -DUSE_TERMIOS=1 -DHAVE_SYS_TIME_H=1 -DTIME_WITH_SYS_TIME=1 -DHAVE_TM_ZONE=1 -DHAVE_GMTIME_R=1 -DHAVE_LOCALTIME_R=1 -DHAVE_TM_GMTOFF=1 -DHAVE_TIMEZONE_VAR=1 -DHAVE_ST_BLKSIZE=1 -DSTDC_HEADERS=1 -DHAVE_SIGNED_CHAR=1 -DHAVE_LANGINFO=1 -DHAVE_SYS_IOCTL_H=1 -DPACKAGE_NAME=\\ -DPACKAGE_TARNAME=\\ -DPACKAGE_VERSION=\\ -DPACKAGE_STRING=\\ -DPACKAGE_BUGREPORT=\\ -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_TIMEGM=1 -DHAVE_DRAND48=1 -DHAVE_RANDOM=1 -DHAVE_POLL=1 -DHAVE_GETADDRINFO=1 -DHAVE_GETNAMEINFO=1-c -o nsthreadtest.o nsthreadtest.c nsthreadtest.c: In function 'DumperThread': nsthreadtest.c:336: warning: implicit declaration of function 'Tcl_GetMemoryInfo' nsthreadtest.c: At top level: nsthreadtest.c:51: warning: 'RCSID' defined but not used /home/xbourgui/programs/tcl8.4/bin/tclsh8.4 /home/xbourgui/Desktop/aolserver-4.5.0/util/nsremove.tcl nsthreadtest gcc -o nsthreadtest nsthreadtest.o \ -L. -lnsthread -L/home/xbourgui/programs/tcl8.4//lib -ltcl8.4 -ldl -lpthread -lieee -lm -lgcc_s -Wl,--export-dynamic -Wl,-rpath,/home/xbourgui/programs/tcl8.4/lib -Wl,-rpath,/lib nsthreadtest.o: In function `DumperThread': nsthreadtest.c:(.text+0x2a8): undefined reference to `Tcl_GetMemoryInfo' collect2: ld returned 1 exit status make[1]: *** [nsthreadtest] Error 1 make[1]: Leaving directory `/home/xbourgui/Desktop/aolserver-4.5.0/nsthread' make: *** [build] Error 1 So I don't know what is going on, but it seems that something is wrong with one of the files. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Install problem
Tom Jackson schrieb: Gustaf, is the intent just to flag potentially dangerous values? yes -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Install problem
Dear Xavier, There is a known problem with Tcl_GetMemoryInfo() in Tcl 8.5 (see e.g. http://aspn.activestate.com/ASPN/Mail/Message/tcl-core/3610677 or http://sourceforge.net/tracker/index.php?func=detailaid=1868171group_id=10894atid=110894). however, since you are using tcl 8.5, this seems wierd. Did you compile Tcl yourself? Did you compile with --enable-threads? best regards -gustaf neumann Xavier Bourguignon schrieb: I ran make clean, then make all and I got this on make all: gcc -O2 -Wall -Wno-implicit-int -fno-strict-aliasing -fPIC -O2 -fno-strict-aliasing -pipe -nostartfiles -I/home/xbourgui/Desktop/aolserver-4.5.0/include -I/home/xbourgui/programs/tcl8.4//include -DNO_CONST -DHAVE_LIMITS_H=1 -DHAVE_UNISTD_H=1 -DHAVE_SYS_PARAM_H=1 -DUSE_THREAD_ALLOC=1 -D_REENTRANT=1 -D_THREAD_SAFE=1 -DHAVE_PTHREAD_ATTR_SETSTACKSIZE=1 -DHAVE_PTHREAD_ATFORK=1 -DTCL_THREADS=1 -DPEEK_XCLOSEIM=1 -D_LARGEFILE64_SOURCE=1 -DTCL_WIDE_INT_TYPE=long\ long -DHAVE_STRUCT_STAT64=1 -DHAVE_OPEN64=1 -DHAVE_LSEEK64=1 -DHAVE_TYPE_OFF64_T=1 -DHAVE_GETCWD=1 -DHAVE_OPENDIR=1 -DHAVE_STRSTR=1 -DHAVE_STRTOL=1 -DHAVE_STRTOLL=1 -DHAVE_STRTOULL=1 -DHAVE_TMPNAM=1 -DHAVE_WAITPID=1 -DHAVE_GETPWUID_R_5=1 -DHAVE_GETPWUID_R=1 -DHAVE_GETPWNAM_R_5=1 -DHAVE_GETPWNAM_R=1 -DHAVE_GETGRGID_R_5=1 -DHAVE_GETGRGID_R=1 -DHAVE_GETGRNAM_R_5=1 -DHAVE_GETGRNAM_R=1 -DHAVE_GETHOSTBYNAME_R_6=1 -DHAVE_GETHOSTBYNAME_R=1 -DHAVE_GETHOSTBYADDR_R_8=1 -DHAVE_GETHOSTBYADDR_R=1 -DUSE_TERMIOS=1 -DHAVE_SYS_TIME_H=1 -DTIME_WITH_SYS_TIME=1 -DHAVE_TM_ZONE=1 -DHAVE_GMTIME_R=1 -DHAVE_LOCALTIME_R=1 -DHAVE_TM_GMTOFF=1 -DHAVE_TIMEZONE_VAR=1 -DHAVE_ST_BLKSIZE=1 -DSTDC_HEADERS=1 -DHAVE_SIGNED_CHAR=1 -DHAVE_LANGINFO=1 -DHAVE_SYS_IOCTL_H=1 -DPACKAGE_NAME=\\ -DPACKAGE_TARNAME=\\ -DPACKAGE_VERSION=\\ -DPACKAGE_STRING=\\ -DPACKAGE_BUGREPORT=\\ -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_TIMEGM=1 -DHAVE_DRAND48=1 -DHAVE_RANDOM=1 -DHAVE_POLL=1 -DHAVE_GETADDRINFO=1 -DHAVE_GETNAMEINFO=1-c -o nsthreadtest.o nsthreadtest.c nsthreadtest.c: In function 'DumperThread': nsthreadtest.c:336: warning: implicit declaration of function 'Tcl_GetMemoryInfo' nsthreadtest.c: At top level: nsthreadtest.c:51: warning: 'RCSID' defined but not used /home/xbourgui/programs/tcl8.4/bin/tclsh8.4 /home/xbourgui/Desktop/aolserver-4.5.0/util/nsremove.tcl nsthreadtest gcc -o nsthreadtest nsthreadtest.o \ -L. -lnsthread -L/home/xbourgui/programs/tcl8.4//lib -ltcl8.4 -ldl -lpthread -lieee -lm -lgcc_s -Wl,--export-dynamic -Wl,-rpath,/home/xbourgui/programs/tcl8.4/lib -Wl,-rpath,/lib nsthreadtest.o: In function `DumperThread': nsthreadtest.c:(.text+0x2a8): undefined reference to `Tcl_GetMemoryInfo' collect2: ld returned 1 exit status make[1]: *** [nsthreadtest] Error 1 make[1]: Leaving directory `/home/xbourgui/Desktop/aolserver-4.5.0/nsthread' make: *** [build] Error 1 So I don't know what is going on, but it seems that something is wrong with one of the files. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Not sure if aolserver compile correclty
Nima,
how large is the nsd, when it dies? If the server is really compiled
with 64bit,
and it eats up all memory, then either you have not much memory+swap, or
it must be really huge.
Maybe you can add something like
ps -eo pid,user,vsize,args | awk 'aolserver-current\/bin\/nsd/ {print $3}'
into a crontab entry, running every 10 minutes, to monitor the size,
appending the size to some log file.
What do you get, if you do a
file your-aolserver/bin/nsd
-gustaf neumann
Nima Mazloumi schrieb:
I've been using aolserver 64bit now for 2 weeks. After 5 days it crashes
because all memory was consumed. therefore I restart the server every night.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Ns_RegisterAtReady, NsRunAtReadyProcs
Tom Jackson schrieb: I'm not sure if you are disputing the bug or not, but I'm somewhat tired of this. I've been testing various patches for the last few weeks in addition to my own code. It seems that everyone knows I've been working on this for a few weeks or more, and everytime I identify a problem, a new patch with a new problem is placed into CVS. Well, i think, i have identified and pinpointed a couple of bugs, and you did as well. Your are probably as well aware, that i am working on the code, so i commited over time a couple of improvements based on different load scenarios etc. Yes, i tried rather to fix the code than to come up with a different approach (and was as well surprised the original code has much problems). Each time I try to point out that this isn't a simple issue, but patches show up with a one dimensional solution or no testing to back them up. I'll commit my code over the weekend replacing the current code. My fix will be a bug fix. The bugs are these: 1. unable to respond to load changes (up and down) 2. unable to respond to configuration changes (up and down) 3. poor, incorrect or missing synchronization code. 4. queued doesn't reflect current queued requests. 5. threads exit unnecessarily on timeout, then new threads to replace 6. current patches cause signal 11 under easily repeated conditions. If anyone wants to stick to the current code, let me know, and I'll just keep my changes here. well, this list of issues is not complete (e.g. queuing in the driver) and rather vague (1-3). For (4) you seem to propose a change in semantics for the reported value, for (5) you proposal seems to be ignore maxconns, (6) is not easy to repeat, i was not able to do so. What makes you believe that (4) is a bug? The total number of processed queries is certainly interested for e.g. implementing a performance monitor. If you siltently change the semantics, you will break such applications. Therefore, it is better to add another variable (e.g. currently_queued) than altering something you find not useful. Btw, with the current code, i get on my home pc (intel imac, mac os x) up to 5000 to 6000 requests per second on small files (maxhreads 5) in cases without driver stalls. With maxthreads 10 i get about twice the value (e.g.g 10169.49 [#/sec]). Certainly, these values are hard to compare since they depend on OS and hardware. Server Software:AOLserver/4.5.0 Server Hostname:localhost Server Port:8001 Document Path: /hires_image-learn.gif Document Length:6339 bytes Concurrency Level: 10 Time taken for tests: 2.111 seconds Complete requests: 12000 Failed requests:0 Broken pipe errors: 0 Total transferred: 78638204 bytes HTML transferred: 76093356 bytes Requests per second:5684.51 [#/sec] (mean) Time per request: 1.76 [ms] (mean) Time per request: 0.18 [ms] (mean, across all concurrent requests) Transfer rate: 37251.64 [Kbytes/sec] received Connnection Times (ms) min mean[+/-sd] median max Connect:0 00.0 0 2 Processing: 0 10.3 1 6 Waiting:0 10.3 1 6 Total: 0 10.4 1 6 My code is well tested with small, large and very large files, low, and high maxconns, low-high threads, and also using [ns_conn channel] to send the sock to a worker thread. I run the queue under continuous loads and vary the threads/maxconns to see the response. note that different load it is not primarily an issue of small and large files but more an issue of fast and slow requests (fast requests are typically small files, but slow requests might return as well small responses, but require a many CPU cycles) and as well an issue of slow and fast thread start up time (aolserver vs. openacs). I have observed quite different issues in such situations. I have no objections in replacing the current code in CVS, as long - it fixes the issues we have discussed, - it does not introduce new issues, - it respects configuration values, - it is performance-wise no drawback. best regards -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Ns_RegisterAtReady, NsRunAtReadyProcs
Tom Jackson schrieb:
I've pointed out several times that I'm looking at the operation of the queue
from the point of view of performance (both economy and responsiveness), not
the specific numbers (min, max, timeout, maxconns) being maintained.
...
So for instance, if Andrew thinks it is a bug for minthreads to be violated,
it is best to not use a thread timeout. That fixes the issue instantly.
to obey minthreads helps to improve performance, since when all
connection threads are
gone, there will be a delay for thread creation upon a new request. I
agree with
Andrew that it is a bug if - as it was before - minthreads are only
available at server start,
and disappear as soon the threads timeout.
If minthreads is taken as a sacred number, then your solution doesn't cut it.
The thread exits and then recreates another one. This is merely cosmetic, and
it only appears correct if you don't notice that the number actually went
below minthreads.
If there was a correct solution, it would prevent the violation of this lower
limit, not fix it up after it happens.
i do not agree here. maxconns should be obeyed as well. If a thread has
served the
specified number of items, it should exit. If the number of threads is
reduced
below minthreads, a fresh thread should be created. Keeping the thread
instead
alife is not correct. If one does not like the behavior, one could
specify maxconns=99.
As an example, just look at the original code. How did threads escape exit at
startup if there is a timeout?
388 if (poolPtr-threads.current = poolPtr-threads.min) {
389 timePtr = NULL;
390 } ...
So there was already some thought at preventing thread exit based upon a
timeout, if minthreads would be violated. So the question is: why isn't this
code working?
simply because it does not consider that maxconn cycles may lead as well
to thread exits.
There are a number of bugs in this code, and fixing it up at the end, after a
thread has exited doesn't remove the actual bugs.
An individual thread sits in a while loop waiting for a queued request this
loop, and the following check is currently:
status = NS_OK;
while (!poolPtr-shutdown
status == NS_OK
poolPtr-queue.wait.firstPtr == NULL) {
/*
nothing is queued, we wait for a queue entry
*/
status = Ns_CondTimedWait(poolPtr-cond, poolPtr-lock, timePtr);
}
if (poolPtr-queue.wait.firstPtr == NULL) {
msg = timeout waiting for connection;
break;
}
Status starts out as NS_OK, the entire while loop is skipped if there is a
waiting request, poolPtr-queue.wait.firstPtr != NULL.
this is fine, since the waiting request can be immediately processed
without the
condwait.
Once the wait is done, the conditions are again checked, the loop is skipped
if status becomes NS_TIMEOUT, or if there is a waiting request (or shutdown).
The problem is that we made the decision to exit on timeout before we knew if
exiting would violate the minthread condition.
So we could do a wait, then again check if we will violate this condition, and
if so, update status to NS_OK, and avoid unnecessary exit. If we move the
timeout code inside the loop, we also avoid repeating/executing this anywhere
else.
this won't get rid of the exits due to maxconns. The code after your
snippet assumes, that
it will be reached only when there is a request ready for processing.
This is what was
happening under Apache Bench. Send a 1000 requests, as long as concurrency is
a little bigger than current threads, things get stuck. It doesn't
necessarily appear to be a thread exit issue. Other problems also show up.
The driver thread also does sock cleanup, which gets stuck. These stuck socks
are dropped connections, they just appear to be in a queue. In fact, they are
not waiting to be serviced, the sock is just waiting for cleanup.
I have noticed this as well, but have not tried to fix this. Sometimes
the driver
stops queueing requests, although some connection threads are idling around.
Maybe this is the problem that jeff rogers had already fixed for 4.0?
Another issue was the fact that the driver thread was not doing a
condBroadcast on thread create, and if thread create is skipped, it just does
a condSignal. Apparently it is not a good idea to just do condSignal.
there was another problem in the book-keeping of idle etc. which i have
fixed
today (committed to cvs). The problem was that under heavy concurrency,
NsQueueConn() did not know if there are conneciton threads in a cond
wait or not. A condSignal is the right thing. It does not make sense to
broadcast and wake up multiple waiting threads for a single request.
i was not able to recreate the crash that you have reported in the other
mail.
However, with ab with small files and pure aolserver (without openacs),
i could verify that some of the counters were wrong. The decision to
Re: [AOLSERVER] Ns_RegisterAtReady, NsRunAtReadyProcs
Tom Jackson schrieb: I've been working on a slightly different angle here, trying to avoid thread exit, and to allow threads to range usefully between min and max. The current code in CVS has a number of problems which lead to the server not creating more threads when there are queued requests and the number of threads is below max (sometimes below min!). concerning the code in cvs head: the server creates now worker threads when a - thread exits and more requests are queued, and no other thread is idle, or when a - thread exits and the number of connection threads falls below minthreads, both cases unless shutdown. The handled thread exists are typically due to exceeding maxconns or the idle timeout. This is what my tests show. What are the problems? Or are you taking about the version in cvs before these changes? Of course, before these changes, minthreads was respected only at startup, queued requests were not processed. My first solution was based on a strategy, where a thread did not exit, when there was more to do (more queued requests), allthough it should exit due to maxconns. However, a user specifying maxconns might have good reasons for doing so, therefore the value must be respected. Avoiding exit in this situation is not viable option IMHO. It can be discussed, whether the idle timeout should be obeyed e.g. in situations where number of threads is below minthreads. In the code in cvs the thread exits and is recreated as in other cases, when it falls below minthreads. It would be more efficient to avoid the timeout in such situations, but one might end up with quite old worker threads, which are more sensitive to growth. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Ns_RegisterAtReady, NsRunAtReadyProcs
Tom Jackson schrieb: But there was something a little more interesting to track down: what happens when a thread exits? ... Apparently, all the interps associated with the thread exit. certainly, every worker thread has its own tcl interpreter associated. When the interpreter exits, the tcl interpreter with all its state exists as well. This impacts the current discussion because I'm having trouble finding an easy way to avoid thread exit because of a timeout. But this information seems to simplify the problem. Why exit a thread just because of a timeout? i don't understand. what timeouts exactly are you taking about? In the code are several timeouts for various purposes... -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Ns_RegisterAtReady, NsRunAtReadyProcs
Andrew Piskorski schrieb: My only point here was that I'm going to stop looking at the timeout parameter, and timed out threads as an issue. If threads timeout, the number of threads in a threadpool will drop below minthreads, usually to zero. But isn't that a bug? If not, just what is minthreads supposed to really mean? you are right, if someone specified minthreads, they should be maintained. i have added an additional condition to the thread recreation to maintain the specified minimal number of threads. maybe Tom has something more sophisticated in mind. this is again a minimal invasive fix, similar to the fix for assuring the processing of the queued requests, when all connection threads exit. for more details, see: http://aolserver.cvs.sourceforge.net/aolserver/aolserver/nsd/queue.c?view=log best regards -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Ns_RegisterAtReady, NsRunAtReadyProcs
Tom Jackson schrieb: The if statement on line 507 checks the Pool structure outside of a mutex lock. Good observation. The code was not correct, although the potential damage was small. Fixed by now. -gustaf -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Ns_RegisterAtReady, NsRunAtReadyProcs
Tom Jackson schrieb:
On Friday 19 October 2007 09:12, Gustaf Neumann wrote:
Actually, if minthreads is set to a value 0 (it was not set), then idle
threads should care about queued requests. Maybe the
easier fix is to set minthreads per default to 1. I will
try this, when i am back home (in a couple of hours).
AOLserver does not respect minthreads except at startup. This is part of the
same issue: Nothing except a request can create a thread. It seems like on
thread exit, a little accounting process could go on to bring threads.current
up to threads.min, this might require more than one thread creation, maybe
not.
fully agreed.
i continued a little on the isue and commited a patch
to CVS, which respects
maxconns always. Instead of performing
in boundary situations in the worst case more than the
configured maxconns requests, the code creates now
a new connection thread automatically after the exit of a thread
coming to the end of it work cycle, when jobs are pending
and no other thread is able to process these.
i made as well a small variant of this (not in CVS), which
defines a new command NsCheckPools() in pools.c
which iterates over pools and starts a thread in the
same situation as above.
NsCheckPools could be extended to check for the existance of
minthreads etc. However, for this patch, i wanted to be
as little invasive as necessary, and added the thread-revival
code to the place, where a thread exits.
For the new thread generation, i had to parameterize
NsCreateConnThread() to avoid a resource deadlock
in Ns_ThreadJoin().
In your patch, you change the while loop test:
+ while (poolPtr-threads.maxconns = 0
+ || ncons-- 0
+ || poolPtr-queue.wait.num 1) {
Shouldn't the loop continue with poolPtr-queue.wait.num 0 ?
0 or 1, both is fine here. the condition trigger for the new cases only
when 1 is used.
Your patch looks like a great fix...I just still don't understand why the
server would completely lock up. As long as you have requests coming in,
seems like a new thread would get created. I wonder if what is happening is
that Apache Bench simple stops sending new requests because none of the
requests are finishing. If it can block, I wonder if simply visiting with a
browser would kick things off again?
one can use the browser to kick things off again, but if there are
already say 50 requests
in the queue, and the browser hangs in the first request. The bad thing
is, that in the
meantime, some other bulky requests are queued as well. So the queue
might be
acutally permanently growing.
for my testing, i use:
ab -c 20 -n 35 http://localhost:8003/file-storage/view/dotlrn-2.3.1.tgz
with maxthreads 5 and maxconns 3 I get reliable hangs, where
the last thread exists with about 5 to 15 queued requests. Without the
recent patches, the queue is processed until it is empty.
btw, the reason, when the patch helps, is no miracle. It is completely
clear why the server hangs. This bug is not a classical deadlock (it
is possible that the queued reuests are processed), but shares some
properties of a life lock (some other requests prevent the processing
of some other requests, at least for a while). It is not a race condition
either. I don't believe that TriggerDriver() is the right place to
handle the problem, since the driver is happily accepting requests
in the bug situation. It can certainly be, that the fixed bug is different
from the bug the Jeff Rogers fixed originally.
-gustaf neumann
PS: i am pretty sure that this is the same bug as on openacs.org.
Around the time of the bug, somebody in spain was apparently
benchmarking his internet connection, downloading dotlrn*.tar*
in multiple parallel sessions. Once i saw that the user
was trying to download 1200 copies of dotlrn.tar in about 10 minutes.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Ns_RegisterAtReady, NsRunAtReadyProcs
Dossy Shiobara schrieb: On 2007.10.13, Tom Jackson [EMAIL PROTECTED] wrote: This all looks great, except for the fact that the NsRunAtReadyProcs proc is never called. But, you're absolutely right: we need to figure this out, one way or another. Thanks for finding this. The question, why the patch helped is indeed a miracle. Maybe it changed the timing or causes a different serialization due to a mutex. However, it did NOT solve the underlying problem. Activated by Tom's question, i spent some more time on the problem and managed to reproduce the problem reliably. The problem actually happend when the number of available connection threads is configured very low, as well as the the number of requests per connection thread (e.g. 5 and 3). When the server accepts a high number of requests, these are queued even without guaranteeing resources to process these. The situation is especially bad with bursty requests. When a connection thread exits e.g. after 3 requests, and there are still many requests queued, there is no mechanism to automatically trigger new connection threads in this situation. When a client connects in this situation to the server, some old (maybe obsolete) requests are taken from the queue, but after this, the server might sit idle even with a high number of requests queued. I have commited a patch to cvs head, that processes these pending requests, even when maxconns is exceeded. Maybe it is better to restart exiting connection threads automatically or to limit the number of queued requests at first hand; or maybe there is a callback solution. However. I have removed the old - useless - patch from the head version in CVS. best regards -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Ns_RegisterAtReady, NsRunAtReadyProcs
Tom Jackson schrieb: I can try running my tests again with 3-5 threads, what other parameters were there? Does this get stuck running OpenACS on any machine, or just on openacs.org? As written before, i set maxthreads to 5 and maxconnections to 3 to trigger the problem. I did not mess with openacs.org, but reproduced the problem under mac os x. Also, Gustaf, the maxconnections should be higher...with maxconnections=3 you are exiting a connection thread every three requests, is this how you want it to work? yes, i wanted to trigger the bug. When the problem occured, openacs.org had different settings (maxthreads 5 maxconnections 10). Very interesting, I'll look into this more today. One thing I didn't try was a configuration where maxconnections was lower than the number of simultaneous requests from Apache Bench. That is not the point. Run ab with more parallel requests than maxthreads, such that requests queue up. As long as maxconnections is high enough, the connection threads will process the queued requests. If the number of queued requests is higher than the number of maxconns, matters degrade. Having one or two queued requests on a thread exit (due to maxconn exceeded) is no real problem, since on the next request, a new thread is started, which will care about the queued request as well. That means, that under good conditions, it is possible that there are e.g. 10 requests queued, but a slow and steady heartbeat of request will cause the queue to empty itself, such that aolserver recovers. The worst case scenario is actually maxconnections 1 (stop every connection thread after every request). In this situation (without my patch) if a queue piles up, it is never shrinking. Actually, if minthreads is set to a value 0 (it was not set), then idle threads should care about queued requests. Maybe the easier fix is to set minthreads per default to 1. I will try this, when i am back home (in a couple of hours). How exactly does the patch work? It appears to simply extend the tour-of-duty for a thread, yes but does the exiting thread ever get to trigger the queue to create new threads? no, i have not modified anything in this regard. When a thread processes all queued requests before exiting, there is no urgent need to create a new connection request. The next incoming request will create a new thread on demand. Once threads die, it is nearly impossible to get the number of threads back up, i have not observed this in my test, the problem appears with reaching maxthreads. This does not exclude, that there might be another problem. -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] More on lost of memory of aolserver process
Hi Brian, Fenton, Brian schrieb: Thanks very much for the replies. The developers reported back that the crashes were caused by a report getting repeatedly called that was returning massive amounts of data to the browser. Sounds a bit odd to me, but I don't have any more data at the moment. When I get more, I'll report back here. if this large amount of data is not a bug, you will run always into such memory problems, since all aolserver threads run in the same (limited) address space of a single process. I would suggest to considerate the following options: - spool data to a file and return the file - use HTTP streaming via ns_write - move to 64bit -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] More on lost of memory of aolserver process
Hi Brian, Fenton, Brian schrieb: I hope nobody minds me bumping this old thread. Gustaf, I used your script on a production server that appears to be leaking memory. Can you please help me interpret it? This was the total at startup TOTAL: pid 10686 vsz 592489 namespaces 543 vars 96 var_bytes 10118 array_elements 313 cmds 266 procs 4511 then just before the crash TOTAL: pid 10686 vsz 1112509 namespaces 543 vars 86 var_bytes 8771 array_elements 256 cmds 266 procs 4469 unable to alloc 495960171 bytes well the script was written to detect, if someone adds variables into namespaces, or appends permanently to variables, which are not reclaimed during cleanup of a connection. This is what i called application level leaks. in your case, this does not seem to be the case (no additional name spaces or vars, all figures are rather less than at start-time). The growth in vsz is not good (but not unusual) but the size of the alloc is something to worry. Growth of vsz might be due to fragmentation of memory (zippy is optimized for minimal locks, not minimal memory footprint) or due to the number of threads you are using (maybe there were less threads at startup time than on crash time?) Would you consider the less than double vsz figure a significant change? Is it of any significance that the other values decreased (vars var_bytes etc)? So would this suggest a C-level or application-level problem? The main question is: why does it try to allocate 500MB? what was the last request? do you use some image libraries? The strange thing about this particular case is that the developers tell me that nothing changed on this server in quite some time. And yet it mysteriously started crashing last week. if the allocation of 500MB sounds for your apps unusual to you, i would suggest to try to figure out, what happens here in the last request in deail. Hope this helps -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks
to detach a stream from a connection thread and - transfer it to a spooling thread and - reimplement there something like an ns_fcopy working on streams without making the visible to the user. But why? The committed changes aka this mess is working since a few years reliably in production. i just looked it up: yesterday background delivery was used more than 200.000 times on a single server here. Are you trying to argue that the change is reducing the reliability of the aolserver/openacs? Talk to to nima, who was close to giving up on openacs due to performance problems, especially on days when many file downloads happened. I should also mention that the code in the aolserver module works together with the request-monitor modules of openacs, where you see the ongoing foreground and background activities. And how does this mess change, if there is a module just implementing the two subcommands? you just end up with more maintenance, more installation effort (updating aolserver startup scripts, updating build processes, updating FAQs). Maybe it helps the ego of some developer if one can say use my module, something, i don't care about. Using the code as a module (minus the ability to transfer the channel), here is an example of use (the string bgwrite could easily be changed to anything else, I chose it before I realized that the code doesn't do any bgwriting.): How about trying to understand what the code does before making a module? -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks
Tom Jackson schrieb: Okay, after looking further into this patch, I see that it doesn't actually add any functionality to AOLserver. It looks like you would have to install a newer version of OpenACS to use this. as i wrote in my earlier mail, the patch is simple and small and adds just two subcommands to ns_conn. Thanks to dossy, the patch is commited to cvs. head. The applications to ns_conn channel are on the tcl layer and are quite simple to use. Look into the xotcl-core package (xotcl-core/tcl/bgdelivery-procs.tcl) in the openacs cvs repository. With the patch, xotcl-core and libthread, one can replace e.g. ns_returnfile 200 $mime_type $filename by ad_returnfile_background 200 $mime_type $filename in cr_write_content in acs-content-repository/tcl/revision-procs.tcl to use the background delivery for content sent from the content repository (e.g. file store). best regards -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver vs lighthttpd, benchmarks
Dossy Shiobara schrieb: On 2007.09.27, Jeff Rogers [EMAIL PROTECTED] wrote: [...] It seems at first glance that it would make more sense to hand the task of writing to the connection back to the driver thread once the connection thread is done with it, [...] I don't know if the change was attempted, but it was certainly considered and acknowledged as ultimately the right thing to do but as I mentioned in a previous message, AOLserver's write speed hasn't been a significant bottleneck for AOL, so the work was not pursued. If anyone has real and actual needs to do large file serving out of AOLserver directly and would like to either (a) make the change and contribute it, or (b) fund its development, I would think it would be welcomed. This is very similar what the background delivery in openacs does. Most of this can be done on the tcl layer with little effort. I have developed a small patch for the aolserver that extends ns_conn by two additional subcommands channel and contentsentlength, where the first one returns the tcl channel name for a connection and the second one is for passing the content-length to the log file. The background delivery code passes the channel to a event aware thread (using tcl libthread), which spools the file to via tcl fcopy. The single delivery thread can easily handle a few hundred simultaneous deliveries without sweat (probably much more). The motivation of the patch was mostly to avoid blocking of connection threads, especially, when large files are delivered to users with slow connections. This is especially important for openacs, where the threads are rather fat (a few thousand tcl procs are in the blueprint), such that the number of configured threads is rather small. There are as well other useful applications for the patch, see e.g. http://www.openacs.org/xowiki/weblog-portlet?ptag=bgdelivery I have asked dossy for commit permissions to add my patches to the aolserver on Sept 22, 2006 and sent him the patches as well, but my impression was that there was very little or no interest. The patch is as well incoporated in naviserver since about this time. The most developed install script of openacs adds the patch by default http://www.cognovis.de/developer/en/aolserver_install -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] aolserver focus
Daniël Mantione schrieb:
Again, a practical situation: How many PHP packages support such headers?
(Even OpenACS doesn't support them, so you would have to fix OpenACS too.)
i am not sure, where this discussion is supposed to lead to. A couple of
years ago, i argued
here on the list to have support on the aolserver side for exactly this
task. The
argument was, it is simple enough to do it on the application layer, no
need for direct
support in aolserver. Up to a certain point, this was true, i ended up
with te following
code for openacs, which maintains its own variant of ns_conn.
ad_conn -set peer_addr [ns_conn peeraddr]
if { [ns_config -bool ns/parameters ReverseProxyMode 0] } {
set addr [lindex [ns_set iget [ns_conn headers] x-forwarded-for] end]
if {[string length $addr] 0} {
ad_conn -set peeraddr $addr
}
}
two points:
a) i still believe, that a aolserver newbies (if this species exists) will
have troubles to handle this case: they rather look for a configure
option than for a doit-yourself solution
b) the power of aolserver + tcl is that once it is done it works
for all apps, no need to load a certain PHP/Perl/Ruby/... modules
etc. Having Tcl as a basic glue language ensures that e.g. the
loaded modules work together, which is in a multi-language environment
not always the case
Can you (socially) convince those users to rewrite the PHP apps for you to
use your OpenACS?
most certainly not. It is as well not easy the other way around.
To achieve good code, this is most likely a complete re-design
matching the framework.
-gustaf
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to [EMAIL PROTECTED]
with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
