Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear all, i did some more digging/googling in this issue and i share the opinion that - at least for the time being - Tcl_Finalize() could be omitted on windows versions without too much harm. Some background: The Tcl manpage says: *Tcl_Finalize* is similar to*Tcl_Exit* except that it does not exit from the current process. It is useful for cleaning up when a process is finished using*Tcl* but wishes to continue executing, and when*Tcl* is used in a dynamically loaded extension that is about to be unloaded. On some systems*Tcl* is automatically notified when it is being unloaded, and it calls*Tcl_Finalize* internally; on these systems it not necessary for the caller to explicitly call*Tcl_Finalize*. However, to ensure portability, your code should always invoke*Tcl_Finalize* when *Tcl* is being unloaded, to ensure that the code will work on all plat- forms.*Tcl_Finalize* can be safely called more than once. For aolserver, it is questionable for me why we need Tcl_Finalize() (the primarily purpose for Tcl_Finalize according to its documentation is that the process wants to continue without Tcl), furthermore there seems to be some magic involved, that some systems call Tcl_Finalize() ... automatically (hinting most likely the windows situation with the assembly code). Since finalize tries to unload Tcl, there seems to be some race conditions in this area on windows, at least when there are still are multiple threads around. E.g. [1] says: Because DLL notifications are serialized, entry-point functions should not attempt to communicate with other threads or processes. Deadlocks may occur as a result. Neither aolserver 4.0.10 nor naviserver call Tcl_Finalize(), so i guess we can live with a 4.5.1+ version under windows without it. -gustaf neumann [1] http://msdn.microsoft.com/en-us/library/ms682583.aspx On 06.08.11 16:28, Maurizio Martignano wrote: It is me again... Well I noticed that the change I suggested about Tcl_Finalize did not make it into CVS HEAD. If it doesn't go there, I am afraid I will have to anyhow introduce it myself in my distribution. I need to have a working system. With that call still in, the service can't (CANNOT) be stopped gracefully. This is a matter of testing: Take the system, make it run with a real OpenACS based application (how about ]po[, or xowiki) and see how it works and see how it interacts with the system... Does it start? Does it run? Does it stop properly? For the time being in Windows 64 that function needs to be out. Thanks a lot, Maurizio Thank you, Maurizio *From:*AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] *On Behalf Of *Gustaf Neumann *Sent:* 06 August 2011 10:28 *To:* AOLSERVER@LISTSERV.AOL.COM *Subject:* Re: [AOLSERVER] Aolserver Progress - Some few examples Maurizio, Tcl_Finalize() is supposed to work, and if it does now work something is still broken in the windows version. Omitting Tcl_Finalize() is removeing the symptom, not the cause. It is not unlikely that something else will have the same problem due to this cause. When Tcl_Finalize() is not run, the registered exit handlers are not executed. How serious this is depends on the exit handlers. You are right, that the memory leak does not matter due to the shutdown. The difference is like between a graceful and an ungraceful shutdown. -gustaf On 05.08.11 16:29, Maurizio Martignano wrote: Dear Gustav, I understand your concerns about Tcl_Finalize... but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test Have Aolserver / nsd running on a big application... observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples.... All's well that ends well
Dear Gustav, Thank you. Alls well that ends well Im not sure all the changes I suggested are still in the codebase especially the ones in RED It is not up to Aolserver/nsd include system to define SOCKET as int on Windows. Thank you again, Maurizio include/ns.h 222 ß #define O_BINARY0 // Commented out by M. Martignano on the 05/08/2011 // #define SOCKET int #define INVALID_SOCKET (-1) I removed the define, in Windows (32/64) SOCKET should be defined somewhere else, and not redefined here. include/ns.h 674 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 NS_EXTERN SOCKET Ns_ConnSock(Ns_Conn *conn); Self explanatory. nsd/conn.c - 615 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 SOCKET Ns_ConnSock(Ns_Conn *conn) { Self explanatory nsd/nsd.h - 75 // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 struct pollfd { int fd; short events; short revents; }; #endif struct pollfd is already defined nsd/nsd.h - 312 // int trigger[2] changed into SOCKET trigger[2] by M. Martignano on the 05/08/2011 SOCKET trigger[2]; /* Wakeup trigger socket. */ Well, we all know about this dont we? nsd/nsmain.c 633 // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Tcl_Finalize(); #endif Semantic. Tcl_Finalize never ends on Windows so I removed it. nsd/queue.c 42 // Function added by M. Martignano on the 05/08/2011 #ifdef _WIN32 static double round(double x) { return floor(x + 0.5); } #endif Well believe it or not round does not exist. So I ehm re-implemented it based on floor. nspd/listen.c 80 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 SOCKETsock, new; Self explanatory nspd/pd.h 52 ß #include unistd.h // Conditional compilation added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Well this follows the same reasoning used for ns.h. Anyhow this is the weakest of all my changes and all the module should be carefully looked at. I am personally not using it in my Windows distributions. nsperm/nsperm.c 583 // Conditional compilation added by M. Martignano on the 05/08/2011 #ifndef _WIN32 if (inet_aton(net, ip) == 0 || inet_aton(slash+1, mask) == 0) { #else if (inet_pton(AF_INET,net, ip) == 0 || inet_pton(AF_INET,slash+1, mask) == 0) { #endif Well with Micosoft Visual C 10 inet_aton seems not to be there any longer. Luckily we can still rely on inet_pton. From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 07 August 2011 15:23 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Dear all, i did some more digging/googling in this issue and i share the opinion that - at least for the time being - Tcl_Finalize() could be omitted on windows versions without too much harm. Some background: The Tcl manpage says: Tcl_Finalize is similar to Tcl_Exit except that it does not exit from the current process. It is useful for cleaning up when a process is finished using Tcl but wishes to continue executing, and when Tcl is used in a dynamically loaded extension that is about to be unloaded. On some systems Tcl is automatically notified when it is being unloaded, and it calls Tcl_Finalize internally; on these systems it not necessary for the caller to explicitly call Tcl_Finalize. However, to ensure portability, your code should always invoke Tcl_Finalize when Tcl is being unloaded, to ensure that the code will work on all plat- forms. Tcl_Finalize can be safely called more than once. For aolserver, it is questionable for me why we need Tcl_Finalize() (the primarily purpose for Tcl_Finalize according to its documentation is that the process wants to continue without Tcl), furthermore there seems to be some magic involved, that some systems call Tcl_Finalize() ... automatically (hinting most likely the windows situation with the assembly code). Since finalize tries to unload Tcl, there seems to be some race conditions in this area on windows, at least when there are still are multiple threads around. E.g. [1] says: Because DLL notifications are serialized, entry-point functions should not attempt to communicate with other threads or processes. Deadlocks may occur as a result. Neither aolserver 4.0.10 nor naviserver call Tcl_Finalize(), so i guess we can live with a 4.5.1+ version under windows without it. -gustaf neumann [1] http://msdn.microsoft.com/en-us/library/ms682583.aspx On 06.08.11 16:28, Maurizio Martignano wrote: It is me again Well I noticed that the change I suggested about Tcl_Finalize did not make it into CVS
Re: [AOLSERVER] Aolserver Progress - Some few examples.... All's well that ends well
On 07.08.11 17:37, Maurizio Martignano wrote: Dear Gustav, Thank you. All's well that ends well... I'm not sure all the changes I suggested are still in the codebase... especially the ones in RED The reason, i have not committed theses two suggested changes to the code base is that these are incorrect and would break compilation on the non-windows platforms. Both suggested changes are in #ifdef branches for compilations without _WIN32 set (therefore irrelevant for Win32 and Win64). Please check more carefully first. -gustaf neumann It is not up to Aolserver/nsd include system to define SOCKET as int on Windows. Thank you again, Maurizio *include/ns.h -- 222 **ß*** *#define*O_BINARY0 // Commented out by M. _Martignano_ on the 05/08/2011 // #define SOCKET _int_ *#define*INVALID_SOCKET (-1) I removed the define, in Windows (32/64) SOCKET should be defined somewhere else, and not redefined here. *include/ns.h -- 674* // Type changed from _int_ to SOCKET by M. _Martignano_ on the 05/08/2011 NS_EXTERN SOCKET *Ns_ConnSock*(Ns_Conn*conn); Self explanatory. *nsd/conn.c - 615* // Type changed from _int_ to SOCKET by M. _Martignano_ on the 05/08/2011 SOCKET *Ns_ConnSock*(Ns_Conn*conn) { Self explanatory *nsd/nsd.h - 75* // Conditional compilation clause added by M. _Martignano_ on the 05/08/2011 *#ifndef*_WIN32 *struct*pollfd { *int*fd; *short*events; *short*revents; }; *#endif* struct pollfd is already defined *nsd/nsd.h - 312* // _int_ trigger[2] changed into SOCKET trigger[2] by M. _Martignano_ on the 05/08/2011 SOCKET trigger[2]; /* _Wakeup_ trigger socket. */ Well, we all know about this... don't we? *nsd/nsmain.c -- 633* // Conditional compilation clause added by M. _Martignano_ on the 05/08/2011 *#ifndef*_WIN32 Tcl_Finalize(); *#endif* Semantic. Tcl_Finalize never ends on Windows so I removed it. *nsd/queue.c -- 42* // Function added by M. _Martignano_ on the 05/08/2011 *#ifdef*_WIN32 *static**double*round(*double*x) { *return*floor(x + 0.5); } *#endif* Well believe it or not round does not exist. So I ehm re-implemented it based on floor. *nspd/listen.c -- 80 *// Type changed from _int_ to SOCKET by M. _Martignano_ on the 05/08/2011 SOCKETsock, new; Self explanatory *nspd/pd.h -- 52 **ß*** *#include*unistd.h // Conditional compilation added by M. _Martignano_ on the 05/08/2011 *#ifndef*_WIN32 Well this follows the same reasoning used for ns.h. Anyhow this is the weakest of all my changes and all the module should be carefully looked at. I am personally not using it in my Windows distributions. *nsperm/nsperm.c -- 583* // Conditional compilation added by M. _Martignano_ on the 05/08/2011 *#ifndef*_WIN32 *if*(inet_aton(net, ip) == 0 || inet_aton(slash+1, mask) == 0) { *#else* *if*(inet_pton(AF_INET,net, ip) == 0 || inet_pton(AF_INET,slash+1, mask) == 0) { *#endif* Well with Micosoft Visual C 10 inet_aton seems not to be there any longer. Luckily we can still rely on inet_pton. *From:*AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] *On Behalf Of *Gustaf Neumann *Sent:* 07 August 2011 15:23 *To:* AOLSERVER@LISTSERV.AOL.COM *Subject:* Re: [AOLSERVER] Aolserver Progress - Some few examples Dear all, i did some more digging/googling in this issue and i share the opinion that - at least for the time being - Tcl_Finalize() could be omitted on windows versions without too much harm. Some background: The Tcl manpage says: *Tcl_Finalize* is similar to*Tcl_Exit* except that it does not exit from the current process. It is useful for cleaning up when a process is finished using*Tcl* but wishes to continue executing, and when*Tcl* is used in a dynamically loaded extension that is about to be unloaded. On some systems*Tcl* is automatically notified when it is being unloaded, and it calls*Tcl_Finalize* internally; on these systems it not necessary for the caller to explicitly call*Tcl_Finalize*. However, to ensure portability, your code should always invoke*Tcl_Finalize* when *Tcl* is being unloaded, to ensure that the code will work on all plat- forms.*Tcl_Finalize* can be safely called more than once. For aolserver, it is questionable for me why we need Tcl_Finalize() (the primarily purpose for Tcl_Finalize according to its documentation is that the process wants to continue without Tcl), furthermore there seems to be some magic involved, that some systems call Tcl_Finalize() ... automatically (hinting most likely the windows situation with the assembly code). Since finalize tries to unload Tcl, there seems to be some race conditions in this area on windows, at least when there are still are multiple threads around. E.g. [1] says: Because DLL notifications are serialized, entry-point
Re: [AOLSERVER] Aolserver Progress - Some few examples.... All's well that ends well
Dear Gustav, You ARE ABSOLUTELY RIGHT! And I AM DEFINETELY A VICTIM OF ALZHEIMER. APOLOGIES Ciao, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 07 August 2011 19:02 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples All's well that ends well On 07.08.11 17:37, Maurizio Martignano wrote: Dear Gustav, Thank you. Alls well that ends well Im not sure all the changes I suggested are still in the codebase especially the ones in RED The reason, i have not committed theses two suggested changes to the code base is that these are incorrect and would break compilation on the non-windows platforms. Both suggested changes are in #ifdef branches for compilations without _WIN32 set (therefore irrelevant for Win32 and Win64). Please check more carefully first. -gustaf neumann It is not up to Aolserver/nsd include system to define SOCKET as int on Windows. Thank you again, Maurizio include/ns.h 222 ß #define O_BINARY0 // Commented out by M. Martignano on the 05/08/2011 // #define SOCKET int #define INVALID_SOCKET (-1) I removed the define, in Windows (32/64) SOCKET should be defined somewhere else, and not redefined here. include/ns.h 674 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 NS_EXTERN SOCKET Ns_ConnSock(Ns_Conn *conn); Self explanatory. nsd/conn.c - 615 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 SOCKET Ns_ConnSock(Ns_Conn *conn) { Self explanatory nsd/nsd.h - 75 // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 struct pollfd { int fd; short events; short revents; }; #endif struct pollfd is already defined nsd/nsd.h - 312 // int trigger[2] changed into SOCKET trigger[2] by M. Martignano on the 05/08/2011 SOCKET trigger[2]; /* Wakeup trigger socket. */ Well, we all know about this dont we? nsd/nsmain.c 633 // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Tcl_Finalize(); #endif Semantic. Tcl_Finalize never ends on Windows so I removed it. nsd/queue.c 42 // Function added by M. Martignano on the 05/08/2011 #ifdef _WIN32 static double round(double x) { return floor(x + 0.5); } #endif Well believe it or not round does not exist. So I ehm re-implemented it based on floor. nspd/listen.c 80 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 SOCKETsock, new; Self explanatory nspd/pd.h 52 ß #include unistd.h // Conditional compilation added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Well this follows the same reasoning used for ns.h. Anyhow this is the weakest of all my changes and all the module should be carefully looked at. I am personally not using it in my Windows distributions. nsperm/nsperm.c 583 // Conditional compilation added by M. Martignano on the 05/08/2011 #ifndef _WIN32 if (inet_aton(net, ip) == 0 || inet_aton(slash+1, mask) == 0) { #else if (inet_pton(AF_INET,net, ip) == 0 || inet_pton(AF_INET,slash+1, mask) == 0) { #endif Well with Micosoft Visual C 10 inet_aton seems not to be there any longer. Luckily we can still rely on inet_pton. From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 07 August 2011 15:23 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Dear all, i did some more digging/googling in this issue and i share the opinion that - at least for the time being - Tcl_Finalize() could be omitted on windows versions without too much harm. Some background: The Tcl manpage says: Tcl_Finalize is similar to Tcl_Exit except that it does not exit from the current process. It is useful for cleaning up when a process is finished using Tcl but wishes to continue executing, and when Tcl is used in a dynamically loaded extension that is about to be unloaded. On some systems Tcl is automatically notified when it is being unloaded, and it calls Tcl_Finalize internally; on these systems it not necessary for the caller to explicitly call Tcl_Finalize. However, to ensure portability, your code should always invoke Tcl_Finalize when Tcl is being unloaded, to ensure that the code will work on all plat- forms. Tcl_Finalize can be safely called more than once. For aolserver, it is questionable for me why we need Tcl_Finalize() (the primarily purpose for Tcl_Finalize according to its documentation is that the process wants to continue without Tcl), furthermore there seems to be some magic involved, that some systems call
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Maurizio, Tcl_Finalize() is supposed to work, and if it does now work something is still broken in the windows version. Omitting Tcl_Finalize() is removeing the symptom, not the cause. It is not unlikely that something else will have the same problem due to this cause. When Tcl_Finalize() is not run, the registered exit handlers are not executed. How serious this is depends on the exit handlers. You are right, that the memory leak does not matter due to the shutdown. The difference is like between a graceful and an ungraceful shutdown. -gustaf On 05.08.11 16:29, Maurizio Martignano wrote: Dear Gustav, I understand your concerns about Tcl_Finalize... but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test Have Aolserver / nsd running on a big application... observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Gustav, I understand perfectly than omitting the function is removing the symptom and not the cause, but process/service wise having Tcl_Finalize in that particular place (where the process/service is about to end) or not having it doesn't make any difference. The Operating System will take care of all resources de-allocation. Nevertheless, exactly for the reason you mentioned, that there may be some problems and that these problems may not be present only in that particular piece of code, I'll have a look into the function behavior. I suspect that it might be a matter of the order with which things are de-allocated. but I might be wrong. I'll come back on this. Thank you for your point, I will follow your recommendation. Ciao, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 06 August 2011 10:28 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Maurizio, Tcl_Finalize() is supposed to work, and if it does now work something is still broken in the windows version. Omitting Tcl_Finalize() is removeing the symptom, not the cause. It is not unlikely that something else will have the same problem due to this cause. When Tcl_Finalize() is not run, the registered exit handlers are not executed. How serious this is depends on the exit handlers. You are right, that the memory leak does not matter due to the shutdown. The difference is like between a graceful and an ungraceful shutdown. -gustaf On 05.08.11 16:29, Maurizio Martignano wrote: Dear Gustav, I understand your concerns about Tcl_Finalize. but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test.. Have Aolserver / nsd running on a big application. observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
YES. I do agree that executing or not executing the exit handlers may make a difference.. Just to help me in my troubleshooting can you tell me if and where these handlers are registered. I am digging into Tcl_Finalize.. J Thank you, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 06 August 2011 10:28 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Maurizio, Tcl_Finalize() is supposed to work, and if it does now work something is still broken in the windows version. Omitting Tcl_Finalize() is removeing the symptom, not the cause. It is not unlikely that something else will have the same problem due to this cause. When Tcl_Finalize() is not run, the registered exit handlers are not executed. How serious this is depends on the exit handlers. You are right, that the memory leak does not matter due to the shutdown. The difference is like between a graceful and an ungraceful shutdown. -gustaf On 05.08.11 16:29, Maurizio Martignano wrote: Dear Gustav, I understand your concerns about Tcl_Finalize. but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test.. Have Aolserver / nsd running on a big application. observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
It is me again. Well I noticed that the change I suggested about Tcl_Finalize did not make it into CVS HEAD. If it doesn't go there, I am afraid I will have to anyhow introduce it myself in my distribution. I need to have a working system. With that call still in, the service can't (CANNOT) be stopped gracefully. This is a matter of testing: Take the system, make it run with a real OpenACS based application (how about ]po[, or xowiki..) and see how it works and see how it interacts with the system. Does it start? Does it run? Does it stop properly? For the time being in Windows 64 that function needs to be out. Thanks a lot, Maurizio Thank you, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 06 August 2011 10:28 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Maurizio, Tcl_Finalize() is supposed to work, and if it does now work something is still broken in the windows version. Omitting Tcl_Finalize() is removeing the symptom, not the cause. It is not unlikely that something else will have the same problem due to this cause. When Tcl_Finalize() is not run, the registered exit handlers are not executed. How serious this is depends on the exit handlers. You are right, that the memory leak does not matter due to the shutdown. The difference is like between a graceful and an ungraceful shutdown. -gustaf On 05.08.11 16:29, Maurizio Martignano wrote: Dear Gustav, I understand your concerns about Tcl_Finalize. but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test.. Have Aolserver / nsd running on a big application. observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Dossy, Your proposal of your wrapper sounds good to me. Why do not we insert that in the codebase? Till we understand better the issue? Next week I am going to redo some testing also in Win32 and I will let you know.. Thank you very much, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 06 August 2011 17:39 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Could you build AOLserver with debugging symbols and run nsd.exe under a debugger, with Tcl_Finalize *NOT* commented out/removed (i.e., as it currently is in CVS HEAD) and confirm where this 0x4015 exception is actually happening? Not calling Tcl_Finalize at process exit means any callbacks registered with Tcl_CreateExitHandler will not fire. It is absolutely wrong to not call Tcl_Finalize for this reason. If this is causing a problem on Win64 at the moment, as a temporary measure, you could wrap the call to Tcl_Finalize with the appropriate #ifndef: #if !defined(_WIN64) Tcl_Finalize(); #endif // !_WIN64 But, on all other platforms where invoking Tcl_Finalize does work, it should absolutely be done. On 8/6/11 9:35 AM, Maurizio Martignano wrote: Dear Gustav (and all the others) I did some digging. And here are the results.. 1. Tcl_Finalize gets properly called in Windows 64 by tclsh (ok tlcsh85t.exe) at exit time 2. From within Aolserver it doesn't even get actually called, but at the act of calling an exception is generated inside the C/C++ runtime: Faulting application name: nsd.exe, version: 0.0.0.0, time stamp: 0x4e3d1e32 Faulting module name: MSVCR100.dll, version: 10.0.30319.415, time stamp: 0x4d26d15a Exception code: 0x4015 Fault offset: 0x00075fe9 Faulting process id: 0x1114 Faulting application start time: 0x01cc542b82cdba6b Faulting application path: C:\aolserver\bin\nsd.exe Faulting module path: C:\Windows\system32\MSVCR100.dll Report Id: e90de38e-c01e-11e0-9d90-cef6f702c08b 3. Looking at the code in TCL I believe (and here I repeat I BELIEVE) the problem is in the TCL DLL initialization code: tclWin32Dll.c case DLL_PROCESS_DETACH: /* * Protect the call to Tcl_Finalize. The OS could be unloading us from * an exception handler and the state of the stack might be unstable. */ if defined(HAVE_NO_SEH) !defined(_WIN64) __asm__ __volatile__ ( /* * Construct an EXCEPTION_REGISTRATION to protect the call to * Tcl_Finalize */ leal %[registration], %%edx \n\t movl %%fs:0, %%eax\n\t movl %%eax,0x0(%%edx) \n\t /* link */ leal 1f, %%eax\n\t movl %%eax,0x4(%%edx) \n\t /* handler */ movl %%ebp,0x8(%%edx) \n\t /* ebp */ movl %%esp,0xc(%%edx) \n\t /* esp */ movl %[error], 0x10(%%edx) \n\t /* status */ /* * Link the EXCEPTION_REGISTRATION on the chain */ movl %%edx,%%fs:0 \n\t /* * Call Tcl_Finalize */ call _Tcl_Finalize\n\t /* * Come here on a normal exit. Recover the EXCEPTION_REGISTRATION * and store a TCL_OK status */ movl %%fs:0, %%edx\n\t movl %[ok],%%eax\n\t movl %%eax,0x10(%%edx) \n\t jmp2f \n /* * Come here on an exception. Get the EXCEPTION_REGISTRATION that * we previously put on the chain. */ 1:\t movl %%fs:0, %%edx\n\t movl 0x8(%%edx), %%edx\n /* * Come here however we exited. Restore context from the * EXCEPTION_REGISTRATION in case the stack is unbalanced. */ 2:\t movl 0xc(%%edx), %%esp\n\t movl 0x8(%%edx), %%ebp\n\t movl 0x0(%%edx), %%eax\n\t movl %%eax,%%fs:0 \n\t : /* No outputs */ : [registration] m (registration), [ok] i (TCL_OK), [error] i (TCL_ERROR) : %eax, %ebx, %ecx, %edx, %esi, %edi, memory ); #else #ifndef HAVE_NO_SEH __try { #endif Tcl_Finalize(); #ifndef HAVE_NO_SEH } __except (EXCEPTION_EXECUTE_HANDLER) { /* empty handler body. */ } #endif #endif break 4. My personal point of view a. This portion of code is too target specific, it uses assembly, some of these things may become obsolete or wrong when for instance moving
Re: [AOLSERVER] Aolserver Progress - Some few examples.... - Errata Corrige
Hello all I did some tests on Windows 32. Tcl_Finalize prevents the proper stopping of the service also on Windows 32. So the proper mod was and still is: // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Tcl_Finalize(); #endif Tcl_Finalize doesn't work also on the 32 bit code. Ciao, Maurizio Dear Dossy, Your proposal of your wrapper sounds good to me. Why do not we insert that in the codebase? Till we understand better the issue? Next week I am going to redo some testing also in Win32 and I will let you know.. Thank you very much, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 06 August 2011 17:39 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Could you build AOLserver with debugging symbols and run nsd.exe under a debugger, with Tcl_Finalize *NOT* commented out/removed (i.e., as it currently is in CVS HEAD) and confirm where this 0x4015 exception is actually happening? Not calling Tcl_Finalize at process exit means any callbacks registered with Tcl_CreateExitHandler will not fire. It is absolutely wrong to not call Tcl_Finalize for this reason. If this is causing a problem on Win64 at the moment, as a temporary measure, you could wrap the call to Tcl_Finalize with the appropriate #ifndef: #if !defined(_WIN64) Tcl_Finalize(); #endif // !_WIN64 But, on all other platforms where invoking Tcl_Finalize does work, it should absolutely be done. On 8/6/11 9:35 AM, Maurizio Martignano wrote: Dear Gustav (and all the others) I did some digging. And here are the results.. 1. Tcl_Finalize gets properly called in Windows 64 by tclsh (ok tlcsh85t.exe) at exit time 2. From within Aolserver it doesn't even get actually called, but at the act of calling an exception is generated inside the C/C++ runtime: Faulting application name: nsd.exe, version: 0.0.0.0, time stamp: 0x4e3d1e32 Faulting module name: MSVCR100.dll, version: 10.0.30319.415, time stamp: 0x4d26d15a Exception code: 0x4015 Fault offset: 0x00075fe9 Faulting process id: 0x1114 Faulting application start time: 0x01cc542b82cdba6b Faulting application path: C:\aolserver\bin\nsd.exe Faulting module path: C:\Windows\system32\MSVCR100.dll Report Id: e90de38e-c01e-11e0-9d90-cef6f702c08b 3. Looking at the code in TCL I believe (and here I repeat I BELIEVE) the problem is in the TCL DLL initialization code: tclWin32Dll.c case DLL_PROCESS_DETACH: /* * Protect the call to Tcl_Finalize. The OS could be unloading us from * an exception handler and the state of the stack might be unstable. */ if defined(HAVE_NO_SEH) !defined(_WIN64) __asm__ __volatile__ ( /* * Construct an EXCEPTION_REGISTRATION to protect the call to * Tcl_Finalize */ leal %[registration], %%edx \n\t movl %%fs:0, %%eax\n\t movl %%eax,0x0(%%edx) \n\t /* link */ leal 1f, %%eax\n\t movl %%eax,0x4(%%edx) \n\t /* handler */ movl %%ebp,0x8(%%edx) \n\t /* ebp */ movl %%esp,0xc(%%edx) \n\t /* esp */ movl %[error], 0x10(%%edx) \n\t /* status */ /* * Link the EXCEPTION_REGISTRATION on the chain */ movl %%edx,%%fs:0 \n\t /* * Call Tcl_Finalize */ call _Tcl_Finalize\n\t /* * Come here on a normal exit. Recover the EXCEPTION_REGISTRATION * and store a TCL_OK status */ movl %%fs:0, %%edx\n\t movl %[ok],%%eax\n\t movl %%eax,0x10(%%edx) \n\t jmp2f \n /* * Come here on an exception. Get the EXCEPTION_REGISTRATION that * we previously put on the chain. */ 1:\t movl %%fs:0, %%edx\n\t movl 0x8(%%edx), %%edx\n /* * Come here however we exited. Restore context from the * EXCEPTION_REGISTRATION in case the stack is unbalanced. */ 2:\t movl 0xc(%%edx), %%esp\n\t movl 0x8(%%edx), %%ebp\n\t movl 0x0(%%edx), %%eax\n\t movl %%eax,%%fs:0 \n\t : /* No outputs */ : [registration] m (registration), [ok] i (TCL_OK), [error] i (TCL_ERROR) : %eax, %ebx, %ecx, %edx, %esi, %edi, memory ); #else
Re: [AOLSERVER] Aolserver Progress - Some few examples.... - Errata Corrige
I vaguely remember never figuring this out either and deciding to ifdef it out. In practice it doesn't do much -- I've never come across a on-exit handler that really needed fire. Curious if anyone has. Jim Sent from a phone On Aug 6, 2011, at 3:29 PM, Maurizio Martignano maurizio.martign...@spazioit.com wrote: Hello all I did some tests on Windows 32. Tcl_Finalize prevents the proper stopping of the service also on Windows 32. So the proper mod was and still is: // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Tcl_Finalize(); #endif Tcl_Finalize doesn’t work also on the 32 bit code. Ciao, Maurizio Dear Dossy, Your proposal of your wrapper sounds good to me. Why do not we insert that in the codebase? Till we understand better the issue? Next week I am going to redo some testing also in Win32 and I will let you know…. Thank you very much, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 06 August 2011 17:39 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Could you build AOLserver with debugging symbols and run nsd.exe under a debugger, with Tcl_Finalize *NOT* commented out/removed (i.e., as it currently is in CVS HEAD) and confirm where this 0x4015 exception is actually happening? Not calling Tcl_Finalize at process exit means any callbacks registered with Tcl_CreateExitHandler will not fire. It is absolutely wrong to not call Tcl_Finalize for this reason. If this is causing a problem on Win64 at the moment, as a temporary measure, you could wrap the call to Tcl_Finalize with the appropriate #ifndef: #if !defined(_WIN64) Tcl_Finalize(); #endif // !_WIN64 But, on all other platforms where invoking Tcl_Finalize does work, it should absolutely be done. On 8/6/11 9:35 AM, Maurizio Martignano wrote: Dear Gustav (and all the others) I did some digging… And here are the results…. 1. Tcl_Finalize gets properly called in Windows 64 by tclsh (ok tlcsh85t.exe) at exit time 2. From within Aolserver it doesn’t even get actually called, but at the act of calling an exception is generated inside the C/C++ runtime: Faulting application name: nsd.exe, version: 0.0.0.0, time stamp: 0x4e3d1e32 Faulting module name: MSVCR100.dll, version: 10.0.30319.415, time stamp: 0x4d26d15a Exception code: 0x4015 Fault offset: 0x00075fe9 Faulting process id: 0x1114 Faulting application start time: 0x01cc542b82cdba6b Faulting application path: C:\aolserver\bin\nsd.exe Faulting module path: C:\Windows\system32\MSVCR100.dll Report Id: e90de38e-c01e-11e0-9d90-cef6f702c08b 3. Looking at the code in TCL I believe (and here I repeat I BELIEVE) the problem is in the TCL DLL initialization code: tclWin32Dll.c case DLL_PROCESS_DETACH: /* * Protect the call to Tcl_Finalize. The OS could be unloading us from * an exception handler and the state of the stack might be unstable. */ if defined(HAVE_NO_SEH) !defined(_WIN64) __asm__ __volatile__ ( /* * Construct an EXCEPTION_REGISTRATION to protect the call to * Tcl_Finalize */ leal %[registration], %%edx \n\t movl %%fs:0, %%eax\n\t movl %%eax,0x0(%%edx) \n\t /* link */ leal 1f, %%eax\n\t movl %%eax,0x4(%%edx) \n\t /* handler */ movl %%ebp,0x8(%%edx) \n\t /* ebp */ movl %%esp,0xc(%%edx) \n\t /* esp */ movl %[error], 0x10(%%edx) \n\t /* status */ /* * Link the EXCEPTION_REGISTRATION on the chain */ movl %%edx,%%fs:0 \n\t /* * Call Tcl_Finalize */ call _Tcl_Finalize\n\t /* * Come here on a normal exit. Recover the EXCEPTION_REGISTRATION * and store a TCL_OK status */ movl %%fs:0, %%edx\n\t movl %[ok],%%eax\n\t movl %%eax,0x10(%%edx) \n\t jmp2f \n /* * Come here on an exception. Get the EXCEPTION_REGISTRATION that * we previously put on the chain. */ 1:\t movl %%fs:0, %%edx\n\t movl 0x8(%%edx), %%edx\n /* * Come here however we exited. Restore context
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Maurizio and all... i have updated cvs on sourceforge with most of your patches. A few points are questionable (see below). For me, it is still unclear, why 4.5.1 worked for you, but not the head version not. As far i can see, all socket usages were int the same way in 4.5.1, the variable triggers in nsd.h was defined like this at least since 2004. Maybe it was luck that the win version worked in 4.5.1 (different memory layout, different compilers / compiler options, ...) A few comments to the patches (i have omitted these): nsd/nsmain.c +#ifndef _WIN32 Tcl_Finalize(); +#endif === If one does no Tcl_Finalize() one introduces a memory leak. nsproxy/nsproxy.c -Tcl_FindExecutable(argv[0]); === The call to Tcl_FindExecutable() is required (at least in Tcl 8.5), otherwise tcl will crash (at least under unix like operating systems) +#define uint32_t unsigned long +#define uint16_t unsigned short + +typedef void * caddr_t; === These defines should be most probably within an #ifdef static void -FatalExit(char *func) +NSP_FatalExit(char *func) === What's wrong with the static name FatalExit()? -static Proc *firstClosePtr = NULL; +static Proc *firstClosePtr; === What's wrong with the initialization of the static variable? I have not updated the nsproxy changes, since these need more work. -gustaf neumann On 05.08.11 08:14, Maurizio Martignano wrote: Dear all, here you are with a first patch file and the zip containing the code base I'm using. The nspd module requires more work, but I'm not using it. The files in the zip archive do compile and seems to be working Windows 32 and Windows 64. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Gustav, Thank you so much for you feedback. I just distributed a new mail with an explanation for the patches.. Sorry if it arrives too late.. I will answers your questions here below. Once again, sorry for the bad timing. Ciao, Maurizio From: Gustaf Neumann [mailto:neum...@wu-wien.ac.at] Sent: 05 August 2011 14:23 To: AOLserver Discussion Cc: Maurizio Martignano Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Dear Maurizio and all... i have updated cvs on sourceforge with most of your patches. A few points are questionable (see below). For me, it is still unclear, why 4.5.1 worked for you, but not the head version not. As far i can see, all socket usages were int the same way in 4.5.1, the variable triggers in nsd.h was defined like this at least since 2004. Maybe it was luck that the win version worked in 4.5.1 (different memory layout, different compilers / compiler options, ...) [MM] No.. Just Alzheimer. Me. When I first did the compilation I spotted the problem and I fixed it by changing int trigger[2] into SOCKET trigger[2]. Then I forgot completely about it.. Then the issue showed up again... when I downloaded the code again.. A few comments to the patches (i have omitted these): nsd/nsmain.c +#ifndef _WIN32 Tcl_Finalize(); +#endif [MM] As simple as that: TCL_Finalize never ends on Windows, so it prevents the process/service to stop properly. === If one does no Tcl_Finalize() one introduces a memory leak. [MM] The process/service is about to end anyhow. I believe that mo matter what the poor process/service does all its memory will be released by the OS. So this is not an issue. nsproxy/nsproxy.c -Tcl_FindExecutable(argv[0]); === The call to Tcl_FindExecutable() is required (at least in Tcl 8.5), otherwise tcl will crash (at least under unix like operating systems) [MM] Not my change, please look at my newest email and sorry if I cause you some inconvenience. same situations for the following changes.. Sorry... +#define uint32_t unsigned long +#define uint16_t unsigned short + +typedef void * caddr_t; === These defines should be most probably within an #ifdef static void -FatalExit(char *func) +NSP_FatalExit(char *func) === What's wrong with the static name FatalExit()? -static Proc *firstClosePtr = NULL; +static Proc *firstClosePtr; === What's wrong with the initialization of the static variable? [MM] absolutely nothing.. May be my bad usage of patch, again apologies. I have not updated the nsproxy changes, since these need more work. [MM] Perfect -gustaf neumann On 05.08.11 08:14, Maurizio Martignano wrote: Dear all, here you are with a first patch file and the zip containing the code base I'm using. The nspd module requires more work, but I'm not using it. The files in the zip archive do compile and seems to be working Windows 32 and Windows 64. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Maurizio, i guess, everything is fine with you with the head version on sourceforge. For me, it is still unclear, why 4.5.1 worked for you, but not the head version not. As far i can see, all socket usages were int the same way in 4.5.1, the variable triggers in nsd.h was defined like this at least since 2004. Maybe it was luck that the win version worked in 4.5.1 (different memory layout, different compilers / compiler options, ...) [MM] No Just Alzheimer... Me. When I first did the compilation I spotted the problem and I fixed it by changing int trigger[2] into SOCKET trigger[2]. Then I forgot completely about it Then the issue showed up again. when I downloaded the code again ok, this explains it. === If one does no Tcl_Finalize() one introduces a memory leak. [MM] The process/service is about to end anyhow. I believe that mo matter what the poor process/service does all its memory will be released by the OS. So this is not an issue. Tcl_finalize shuts down Tcl; it calls the registered exit handlers and then it shuts down various Tcl subsystems. It is certainly not recommended to to omit, but i have not checked the exact consequences for aolserver. If it hangs, it is an indication that there is still something wrong in tcl and/or aolserver. Keep an eye open in this respect. nsproxy/nsproxy.c -Tcl_FindExecutable(argv[0]); === The call to Tcl_FindExecutable() is required (at least in Tcl 8.5), otherwise tcl will crash (at least under unix like operating systems) [MM] Not my change, please look at my newest email and sorry if I cause you some inconvenience. It was included in your first patch (maybe erratic). If this is not needed, fine. All the best, and many thanks for the feedback -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
On Fri, Aug 05, 2011 at 02:22:44PM +0200, Gustaf Neumann wrote: For me, it is still unclear, why 4.5.1 worked for you, but not the head version not. As far i can see, all socket usages were int the same way in 4.5.1, the variable triggers in nsd.h was defined like this at least since 2004. Maurizio explained that earlier: He had misremembered, and actually DID have to make the int to SOCKET changes in the earlier 4.5.1 version as well for it to work on Win64. And someone else (Dossy I think) explained why that makes sense: The definition of SOCKET is different on Win64. So all these versions of AOLserver have been working ok on 32 bit Windows, but were presumably broken on 64 bit Windows all along. -- Andrew Piskorski a...@piskorski.com http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Gustav, I understand your concerns about Tcl_Finalize. but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test.. Have Aolserver / nsd running on a big application. observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio From: Gustaf Neumann [mailto:neum...@wu-wien.ac.at] Sent: 05 August 2011 15:52 To: Maurizio Martignano Cc: 'AOLserver Discussion' Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Dear Maurizio, i guess, everything is fine with you with the head version on sourceforge. For me, it is still unclear, why 4.5.1 worked for you, but not the head version not. As far i can see, all socket usages were int the same way in 4.5.1, the variable triggers in nsd.h was defined like this at least since 2004. Maybe it was luck that the win version worked in 4.5.1 (different memory layout, different compilers / compiler options, ...) [MM] No.. Just Alzheimer. Me. When I first did the compilation I spotted the problem and I fixed it by changing int trigger[2] into SOCKET trigger[2]. Then I forgot completely about it.. Then the issue showed up again... when I downloaded the code again.. ok, this explains it. === If one does no Tcl_Finalize() one introduces a memory leak. [MM] The process/service is about to end anyhow. I believe that mo matter what the poor process/service does all its memory will be released by the OS. So this is not an issue. Tcl_finalize shuts down Tcl; it calls the registered exit handlers and then it shuts down various Tcl subsystems. It is certainly not recommended to to omit, but i have not checked the exact consequences for aolserver. If it hangs, it is an indication that there is still something wrong in tcl and/or aolserver. Keep an eye open in this respect. nsproxy/nsproxy.c -Tcl_FindExecutable(argv[0]); === The call to Tcl_FindExecutable() is required (at least in Tcl 8.5), otherwise tcl will crash (at least under unix like operating systems) [MM] Not my change, please look at my newest email and sorry if I cause you some inconvenience. It was included in your first patch (maybe erratic). If this is not needed, fine. All the best, and many thanks for the feedback -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear all, Id like to provide you with very few examples to explain what I was talking about: These problems manifested themselves in the Win64 version driver.c void NsWaitDriversShutdown(Ns_Time *toPtr) { Driver *drvPtr = firstDrvPtr; int status = NS_OK; while (drvPtr != NULL) { Ns_MutexLock(drvPtr-lock); while (!(drvPtr-flags DRIVER_STOPPED) status == NS_OK) { status = Ns_CondTimedWait(drvPtr-cond, drvPtr-lock, toPtr); } Ns_MutexUnlock(drvPtr-lock); if (status != NS_OK) { Ns_Log(Warning, driver: shutdown timeout: %s, drvPtr-module); } else { Ns_Log(Notice, driver: stopped: %s, drvPtr-module); Ns_ThreadJoin(drvPtr-thread, NULL);ß WAITS FOREVER drvPtr-thread = NULL; } drvPtr = drvPtr-nextPtr; } } And then: static void TriggerDriver(Driver *drvPtr) { if (send(drvPtr-trigger[1], , 1, 0) != 1) { ß ALWAYS ALWAYS RETURNS AN ERROR Ns_Fatal(driver: trigger send() failed: %s, ns_sockstrerror(ns_sockerrno)); } } Ns.h typedef struct Driver { /* * Visible in Ns_Driver. */ void *arg; /* Driver callback data. */ char *server; /* Virtual server name. */ char *module;/* Driver module. */ char*name;/* Driver name, e.g., nssock. */ char*location; /* Location, e.g, http://foo:9090; */ char*address; /* Address in location. */ intsendwait; /* send() I/O timeout. */ intrecvwait;/* recv() I/O timeout. */ int bufsize; /* Conn bufsize (0 for SSL) */ int sndbuf; /* setsockopt() SNDBUF option. */ int rcvbuf; /* setsockopt() RCVBUF option. */ /* * Private to Driver. */ struct Driver *nextPtr; /* Next in list of drivers. */ struct NsServer *servPtr; /* Driver virtual server. */ char *fullname; /* Full name, i.e., server/module. */ int flags; /* Driver state flags. */ Ns_Thread thread;/* Thread id to join on shutdown. */ Ns_Mutex lock; /* Lock to protect lists below. */ Ns_Cond cond;/* Cond to signal reader threads, * driver query, startup, and shutdown. */ inttrigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) Ns_DriverProc *proc; /* Driver callback. */ int opts; /* Driver options. */ etc . I have the feeling that we are not facing Windows specific stuff, but somehow some general errors, mistakes that happen to manifest themselves only on Windows (64). Hope it clarifies the situation, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
On Aug 4, 2011, at 12:24 AM, Maurizio Martignano wrote: inttrigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) A Unix pipe is just a pair of file descriptors, and a file descriptor in Unix is just an integer. Windows pipes are implemented differently, of course, and the semantics aren't exactly the same, either. Can you tell from CVS who put this code in? I'm sure it was done from ignorance of Windows vs. Unix differences regarding pipes, not intentionally. I have the feeling that we are not facing Windows specific stuff, but somehow some general errors, mistakes that happen to manifest themselves only on Windows (64). Not in this case, no, that's perfectly reasonable Unix code. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
It's probably safer to define this as SOCKET, but windows.h says SOCKET is: typedef u_int SOCKET; And: typedef unsigned intu_int; Since Windows is LLP64 and most Unix-like systems are LP64, I don't understand how AOLserver's defining trigger[2] as (int) is the problem -- Windows might complain about some signed/unsigned thing at compile time, but in both cases, (int) is 32 bits. On 8/4/11 3:24 AM, Maurizio Martignano wrote: inttrigger[2]; /* Wakeup trigger pipe. */ ßWhy is this an int when it was a SOCKET (any justification) -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Don, I went back to my archives This is the situation: 1. the code in CVS had always int trigger[2]; 2. I took the version 4.5.1 from the tar ball dated 2009-02-02 and I did the change SOCKET int trigger[2]; to make it work 3. then I recently took the Aolserver code from CVS Head and forgot to make the above change and so I started to observe problems in the networking code 4. re-inserted the change and everything seems to work... but I will test it and very deeply. 5. I have to disagree with your statement A Unix pipe is just a pair of file descriptors, and a file descriptor in Unix is just an integer. If I look at the code base (and please do the same with me)... check nscp/nscp.c nsd/binder.c, fd.c, listen.c, sock.c, tclfile.c, tclhttp.c, urlopen.c nsext/nsext.c nspd/nspd.c nsproxy/nsproxylib.c nssock/nssock.c nssl/nssl.c Well not always sockets are declared/defines as SOCKET as they should be (and this I am afraid also in Unix) This is a problem of proper data type selection and usage, it is mostly a consistency problem. I would humbly suggest that one of the code maintainers takes a walk on the code base and makes sure that everywhere we need a socket we have a real SOCKET socket and not an int socket. The code will be more consistent and more portable to Windows. Ciao, Maurizio PS: the attached text file shows where the keyword SOCKET is currently used... -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 14:53 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 12:24 AM, Maurizio Martignano wrote: inttrigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) A Unix pipe is just a pair of file descriptors, and a file descriptor in Unix is just an integer. Windows pipes are implemented differently, of course, and the semantics aren't exactly the same, either. Can you tell from CVS who put this code in? I'm sure it was done from ignorance of Windows vs. Unix differences regarding pipes, not intentionally. I have the feeling that we are not facing Windows specific stuff, but somehow some general errors, mistakes that happen to manifest themselves only on Windows (64). Not in this case, no, that's perfectly reasonable Unix code. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. * Socket descriptor or INVALID_SOCKET on error. SOCKET SOCKET sock = INVALID_SOCKET; ./binder.c SOCKET sock; static Sock *SockAccept(SOCKET lsock, Driver *drvPtr); static int Poll(PollData *pdataPtr, SOCKET sock, int events, Ns_Time *timeoutPtr); Ns_QueueWait(Ns_Conn *conn, SOCKET sock, Ns_QueueWaitProc *proc, SOCKET lsock; if (lsock != INVALID_SOCKET) { if (lsock != INVALID_SOCKET) { Poll(PollData *pdataPtr, SOCKET sock, int events, Ns_Time *timeoutPtr) SockAccept(SOCKET lsock, Driver *drvPtr) if (sockPtr-sock == INVALID_SOCKET) { setsockopt(sockPtr-sock, SOL_SOCKET, SO_SNDBUF, setsockopt(sockPtr-sock, SOL_SOCKET, SO_RCVBUF, sockPtr-sock = INVALID_SOCKET; ./driver.c ns_sockpair(SOCKET *socks) SOCKET sock; if (sock == INVALID_SOCKET || if (socks[1] == INVALID_SOCKET || if (socks[0] == INVALID_SOCKET) { ./fd.c SOCKET new, sock; if (sock == INVALID_SOCKET) { if (sock == INVALID_SOCKET) { ListenProc(SOCKET sock, void *arg, int why) SOCKET new; if (new != INVALID_SOCKET) { ./listen.c * 0 or SOCKET_ERROR. ns_socknbclose(SOCKET sock) return SOCKET_ERROR; SOCKET ns_sockdup(SOCKET sock) return INVALID_SOCKET; return (SOCKET) dup; ./nswin32.c static SOCKET SockConnect(char *host, int port, char *lhost, int lport, int async); static SOCKET SockSetup(SOCKET sock); Ns_SockRecv(SOCKET sock, void *buf, int toread, int timeout) Ns_SockSend(SOCKET sock, void *buf, int towrite, int timeout) Ns_SockWait(SOCKET sock, int what, int seconds) Ns_SockWaitEx(SOCKET sock, int what, int ms) SOCKET SOCKET SOCKET sock; if (sock == INVALID_SOCKET) { if (sock != INVALID_SOCKET listen(sock, backlog) != 0) { sock = INVALID_SOCKET; SOCKET Ns_SockAccept(SOCKET lsock, struct sockaddr *saPtr, int *lenPtr) SOCKET sock; if (sock != INVALID_SOCKET) { SOCKET SOCKET SOCKET sock; if (sock != INVALID_SOCKET
Re: [AOLSERVER] Aolserver Progress - Some few examples....
It is not a matter of understanding It is a matter of testing On Windows 64 int trigger[2] doesnt work whereas SOCKET trigger[2] does work. On top of that in several other places SOCKET has been used, so if for no other reason, I suggest one of the code maintainers takes a proper walk on the code base and make sure that all sockets are SOCKET sockets and not int sockets Ciao, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 04 August 2011 15:51 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples It's probably safer to define this as SOCKET, but windows.h says SOCKET is: typedef u_int SOCKET; And: typedef unsigned intu_int; Since Windows is LLP64 and most Unix-like systems are LP64, I don't understand how AOLserver's defining trigger[2] as (int) is the problem -- Windows might complain about some signed/unsigned thing at compile time, but in both cases, (int) is 32 bits. On 8/4/11 3:24 AM, Maurizio Martignano wrote: inttrigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
On Win64, can you tell me what sizeof(SOCKET) and sizeof(int) are? Try this simple program: #include windows.h #include winsock2.h int main(int argc, char[] *argv) { printf(sizeof(SOCKET) = %d, sizeof(int) = %d\n, sizeof(SOCKET), sizeof(int)); return 0; } I just learned that on Win64, Microsoft *changed* the definition of SOCKET from u_int to UINT_PTR: /* * The new type to be used in all * instances which refer to sockets. */ #ifdef _WIN64 typedef UINT_PTRSOCKET; #else typedef u_int SOCKET; #endif Which means 64-bit SOCKET on Win64 vs. 32-bit SOCKET on Win32 or most Unix-like systems. ... The short answer is, yes, anywhere we have a variable that contains a handle to a socket, we should use the SOCKET type, not a naked int declaration, for portability. On 8/4/11 10:28 AM, Maurizio Martignano wrote: It is not a matter of understanding It is a matter of testing On Windows 64 int trigger[2] doesn't work whereas SOCKET trigger[2] does work. On top of that in several other places SOCKET has been used, so if for no other reason, I suggest one of the code maintainers takes a proper walk on the code base and make sure that all sockets are SOCKET sockets and not int sockets... -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dossy, It is irrelevant... Absolutely irrelevant.. With int trigger[2] static void TriggerDriver(Driver *drvPtr) { if (send(drvPtr-trigger[1], , 1, 0) != 1) { The send doesn't work and always returns error With SOCKET trigger[2]; It DOES Work... Back to your question: The program should be: #include windows.h #include winsock.h int main(int argc, char* argv[]) { printf(sizeof(SOCKET) = %d, sizeof(int) = %d\n, sizeof(SOCKET), sizeof(int)); return 0; } Its output when compiled as Windows 64 is: c:\Itmpcl test.c Microsoft (R) C/C++ Optimizing Compiler Version 16.00.30319.01 for x64 Copyright (C) Microsoft Corporation. All rights reserved. test.c Microsoft (R) Incremental Linker Version 10.00.30319.01 Copyright (C) Microsoft Corporation. All rights reserved. /out:test.exe test.obj c:\Itmptest sizeof(SOCKET) = 8, sizeof(int) = 4 and when compiled with Windows 32 C:\Itmpcl test.c Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86 Copyright (C) Microsoft Corporation. All rights reserved. test.c Microsoft (R) Incremental Linker Version 10.00.30319.01 Copyright (C) Microsoft Corporation. All rights reserved. /out:test.exe test.obj C:\Itmptest sizeof(SOCKET) = 4, sizeof(int) = 4 C:\Itmp I hope you understand my point now (thank you for your educational example which just proves it.) NOW BACK TO US 1. The Aolserver code base inconsistently uses int and SOCKET when declaring /defining variables to be used as Sockets. 2. This inconsistency it is not a Windows/Unix problem - it is just a plain inconsistency 3. We are just lucky that Windows 64 exposed the problem. 4. I do recommend that one of the code maintainers takes a walk on code base and makes all the sockets declarations/definitions consistent. Thank you, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 04 August 2011 16:56 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Win64, can you tell me what sizeof(SOCKET) and sizeof(int) are? Try this simple program: #include windows.h #include winsock2.h int main(int argc, char[] *argv) { printf(sizeof(SOCKET) = %d, sizeof(int) = %d\n, sizeof(SOCKET), sizeof(int)); return 0; } I just learned that on Win64, Microsoft *changed* the definition of SOCKET from u_int to UINT_PTR: /* * The new type to be used in all * instances which refer to sockets. */ #ifdef _WIN64 typedef UINT_PTRSOCKET; #else typedef u_int SOCKET; #endif Which means 64-bit SOCKET on Win64 vs. 32-bit SOCKET on Win32 or most Unix-like systems. ... The short answer is, yes, anywhere we have a variable that contains a handle to a socket, we should use the SOCKET type, not a naked int declaration, for portability. On 8/4/11 10:28 AM, Maurizio Martignano wrote: It is not a matter of understanding It is a matter of testing On Windows 64 int trigger[2] doesn't work whereas SOCKET trigger[2] does work. On top of that in several other places SOCKET has been used, so if for no other reason, I suggest one of the code maintainers takes a proper walk on the code base and make sure that all sockets are SOCKET sockets and not int sockets. -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
On Aug 4, 2011, at 7:20 AM, Maurizio Martignano wrote: 5. I have to disagree with your statement A Unix pipe is just a pair of file descriptors, and a file descriptor in Unix is just an integer. Feel free to disagree with the official Linux documentation then: http://www.kernel.org/doc/man-pages/online/pages/man2/pipe.2.html Note int pipefd[2] file descriptors in Unix are integers. Dossy says this is the same in Windows (I wouldn't know), perhaps the problem lies in the semantics, IIRC windows doesn't support anonymous pipes, for instance. (But I could be wrong about that … but I'm not wrong about Unix fds being ints) Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Don, If you follow the last discussions even Dossy agrees that a SOCKET is not an int on Windows 64 All of this depends on the week type system of C, were types with different names, supposed to be used for different needs are considered equivalent is their size is the same. If we had used Ada none of this would have had happened: types with different names are different no matter what their size is. Anyhow in the base code 95% of the times or even more sockets are declared as SOCKET sockets. Here and there they are declared as int. This is an inconsistency and it should be removed. I do beg the community to do this little change because it is in the benefit and interest of everybody. Hope it helps, Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 18:07 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 7:20 AM, Maurizio Martignano wrote: 5. I have to disagree with your statement A Unix pipe is just a pair of file descriptors, and a file descriptor in Unix is just an integer. Feel free to disagree with the official Linux documentation then: http://www.kernel.org/doc/man-pages/online/pages/man2/pipe.2.html Note int pipefd[2] file descriptors in Unix are integers. Dossy says this is the same in Windows (I wouldn't know), perhaps the problem lies in the semantics, IIRC windows doesn't support anonymous pipes, for instance. (But I could be wrong about that . but I'm not wrong about Unix fds being ints) Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
On Aug 4, 2011, at 9:55 AM, Maurizio Martignano wrote: All of this depends on the week type system of C, were types with different names, supposed to be used for different needs are considered equivalent is their size is the same. If we had used Ada none of this would have had happened: types with different names are different no matter what their size is. If we were using Ada a file descriptor in Unix would still be described as an integer. I'm not saying that the Unix code you've uncovered is portable between Unix and Windows. I'm just pointing out that pipes are defined as an array of two integers in Unix, so that the code isn't wrong for Unix, as you originally claimed. I did so hoping it would increase your understanding, i.e. your claim that it appears to be a bug even in Unix is incorrect. If you want to make progress here, just accept that the code is perfectly good Unix code and then figure out how to make the code work for both Unix and Windows, instead of trying to argue incorrectly that the code's not correct for Unix. It's not portable, but it's correct for Unix. Thank you. Anyhow in the base code 95% of the times or even more sockets are declared as SOCKET sockets. Here and there they are declared as int. This is an inconsistency and it should be removed. I do beg the community to do this little change because it is in the benefit and interest of everybody. I'm sure that the community will accept a patch that declares the pipe in a way that makes both Unix and Windows happy if you'll provide one. Meanwhile, quit complaining because I pointed out that, in Unix, int pipefd[2] is the correct declaration for a pipe. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dossy Shiobara wrote: It's probably safer to define this as SOCKET, but windows.h says SOCKET is: The source comment is misleading, because trigger is set up as a socket pair, not as a pipe. Not sure why it's this way, but there it is. And ns_sockpair is already prototyped as ns_sockpair(SOCKET *socks) so if this is a problem it really should be triggering a compiler warning about it. In any case, since unix defines SOCKET as int, it should make zero difference on the unix side to change the structure definition. -J typedef u_int SOCKET; And: typedef unsigned int u_int; Since Windows is LLP64 and most Unix-like systems are LP64, I don't understand how AOLserver's defining trigger[2] as (int) is the problem -- Windows might complain about some signed/unsigned thing at compile time, but in both cases, (int) is 32 bits. On 8/4/11 3:24 AM, Maurizio Martignano wrote: int trigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/| can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Don In Aolserver source code 95% of more of the times sockets are declared as SOCKET; the other times as int. This is an inconsistency and is a fact. If you wanted to develop only for Unix why did you use SOCKET in some occasions and int in some others? The source code is inconsistent and it just happens to work on Unix because there SOCKET and int have the same size. And this is also a fact. But I believe we should stop here, I admit all the faults you want, but please let's use SOCKET everywhere Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 19:25 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 9:55 AM, Maurizio Martignano wrote: All of this depends on the week type system of C, were types with different names, supposed to be used for different needs are considered equivalent is their size is the same. If we had used Ada none of this would have had happened: types with different names are different no matter what their size is. If we were using Ada a file descriptor in Unix would still be described as an integer. I'm not saying that the Unix code you've uncovered is portable between Unix and Windows. I'm just pointing out that pipes are defined as an array of two integers in Unix, so that the code isn't wrong for Unix, as you originally claimed. I did so hoping it would increase your understanding, i.e. your claim that it appears to be a bug even in Unix is incorrect. If you want to make progress here, just accept that the code is perfectly good Unix code and then figure out how to make the code work for both Unix and Windows, instead of trying to argue incorrectly that the code's not correct for Unix. It's not portable, but it's correct for Unix. Thank you. Anyhow in the base code 95% of the times or even more sockets are declared as SOCKET sockets. Here and there they are declared as int. This is an inconsistency and it should be removed. I do beg the community to do this little change because it is in the benefit and interest of everybody. I'm sure that the community will accept a patch that declares the pipe in a way that makes both Unix and Windows happy if you'll provide one. Meanwhile, quit complaining because I pointed out that, in Unix, int pipefd[2] is the correct declaration for a pipe. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Hi It's a socket so it can be monitored by select and poll. It should be SOCKET, I think it was in the past. On windows lib-c file handles returned by _open aren't the same as sockets. You can see this in the libc source Microsoft provides. They can't be monitored with select. The ns_sockpair is a pipe-like thing on windows by using socket calls to actually listen and connect on a localhost network socket. Net: seems I does need to change to SOCKET and that should have no effect on unix. Warning: I'm going on memory, not looking at the code. Jim Sent from a phone On Aug 4, 2011, at 11:34 AM, Jeff Rogers dv...@diphi.com wrote: Dossy Shiobara wrote: It's probably safer to define this as SOCKET, but windows.h says SOCKET is: The source comment is misleading, because trigger is set up as a socket pair, not as a pipe. Not sure why it's this way, but there it is. And ns_sockpair is already prototyped as ns_sockpair(SOCKET *socks) so if this is a problem it really should be triggering a compiler warning about it. In any case, since unix defines SOCKET as int, it should make zero difference on the unix side to change the structure definition. -J typedef u_int SOCKET; And: typedef unsigned int u_int; Since Windows is LLP64 and most Unix-like systems are LP64, I don't understand how AOLserver's defining trigger[2] as (int) is the problem -- Windows might complain about some signed/unsigned thing at compile time, but in both cases, (int) is 32 bits. On 8/4/11 3:24 AM, Maurizio Martignano wrote: int trigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/| can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Maurizio, I think we're all in agreement at this point. Could you put together a patch? -J Maurizio Martignano wrote: Don In Aolserver source code 95% of more of the times sockets are declared as SOCKET; the other times as int. This is an inconsistency and is a fact. If you wanted to develop only for Unix why did you use SOCKET in some occasions and int in some others? The source code is inconsistent and it just happens to work on Unix because there SOCKET and int have the same size. And this is also a fact. But I believe we should stop here, I admit all the faults you want, but please let's use SOCKET everywhere Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 19:25 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 9:55 AM, Maurizio Martignano wrote: All of this depends on the week type system of C, were types with different names, supposed to be used for different needs are considered equivalent is their size is the same. If we had used Ada none of this would have had happened: types with different names are different no matter what their size is. If we were using Ada a file descriptor in Unix would still be described as an integer. I'm not saying that the Unix code you've uncovered is portable between Unix and Windows. I'm just pointing out that pipes are defined as an array of two integers in Unix, so that the code isn't wrong for Unix, as you originally claimed. I did so hoping it would increase your understanding, i.e. your claim that it appears to be a bug even in Unix is incorrect. If you want to make progress here, just accept that the code is perfectly good Unix code and then figure out how to make the code work for both Unix and Windows, instead of trying to argue incorrectly that the code's not correct for Unix. It's not portable, but it's correct for Unix. Thank you. Anyhow in the base code 95% of the times or even more sockets are declared as SOCKET sockets. Here and there they are declared as int. This is an inconsistency and it should be removed. I do beg the community to do this little change because it is in the benefit and interest of everybody. I'm sure that the community will accept a patch that declares the pipe in a way that makes both Unix and Windows happy if you'll provide one. Meanwhile, quit complaining because I pointed out that, in Unix, int pipefd[2] is the correct declaration for a pipe. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
We are reasoning too much... 1. Compiling the code on Windows 64 made clear there's some inconsistency in the code... 2. This inconsistence is on how sockets are declared: 95% and more of the times as SOCKET and the rest of the times as int 3. On UNIX and WIN32 no problem cause SOCKET and int have the same size 4. On WIN64 there's a problem cause SOCKET and int do not have the same size and the helpful example Dossy made and I compiled on Win32 and Win34 elegantly shows that. 5. We have a good opportunity to clean the code, making it more consistent and more portable... 6. I insist on the need to make this little walk on the code base to make it consistent... We should use everywhere SOCKET. What are we waiting for? :-) :-) :-) -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Jim Davidson Sent: 04 August 2011 20:22 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Hi It's a socket so it can be monitored by select and poll. It should be SOCKET, I think it was in the past. On windows lib-c file handles returned by _open aren't the same as sockets. You can see this in the libc source Microsoft provides. They can't be monitored with select. The ns_sockpair is a pipe-like thing on windows by using socket calls to actually listen and connect on a localhost network socket. Net: seems I does need to change to SOCKET and that should have no effect on unix. Warning: I'm going on memory, not looking at the code. Jim Sent from a phone On Aug 4, 2011, at 11:34 AM, Jeff Rogers dv...@diphi.com wrote: Dossy Shiobara wrote: It's probably safer to define this as SOCKET, but windows.h says SOCKET is: The source comment is misleading, because trigger is set up as a socket pair, not as a pipe. Not sure why it's this way, but there it is. And ns_sockpair is already prototyped as ns_sockpair(SOCKET *socks) so if this is a problem it really should be triggering a compiler warning about it. In any case, since unix defines SOCKET as int, it should make zero difference on the unix side to change the structure definition. -J typedef u_int SOCKET; And: typedef unsigned int u_int; Since Windows is LLP64 and most Unix-like systems are LP64, I don't understand how AOLserver's defining trigger[2] as (int) is the problem -- Windows might complain about some signed/unsigned thing at compile time, but in both cases, (int) is 32 bits. On 8/4/11 3:24 AM, Maurizio Martignano wrote: int trigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/| can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Rusty, I started very politely, gently... Stressing I was seeing that the code base was kind of separating, moving away from Windows support... (I did see how SOCKETwere used). Then I provided the examples Then I stressed the int trigger[2]; Then I made it clear and I am sorry that this was seen as a porting issue and not as an inconsistency... so I stressed again I am sorry if I have been... well as you describe in your mail but at least the point has been made Thank you. Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Rusty Brooks Sent: 04 August 2011 20:55 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Fine, SOCKET should be used everywhere. But also, you're kind of being a dick. Don't do that. Rusty On Aug 4, 2011, at 1:26 PM, Maurizio Martignano wrote: Don In Aolserver source code 95% of more of the times sockets are declared as SOCKET; the other times as int. This is an inconsistency and is a fact. If you wanted to develop only for Unix why did you use SOCKET in some occasions and int in some others? The source code is inconsistent and it just happens to work on Unix because there SOCKET and int have the same size. And this is also a fact. But I believe we should stop here, I admit all the faults you want, but please let's use SOCKET everywhere Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 19:25 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 9:55 AM, Maurizio Martignano wrote: All of this depends on the week type system of C, were types with different names, supposed to be used for different needs are considered equivalent is their size is the same. If we had used Ada none of this would have had happened: types with different names are different no matter what their size is. If we were using Ada a file descriptor in Unix would still be described as an integer. I'm not saying that the Unix code you've uncovered is portable between Unix and Windows. I'm just pointing out that pipes are defined as an array of two integers in Unix, so that the code isn't wrong for Unix, as you originally claimed. I did so hoping it would increase your understanding, i.e. your claim that it appears to be a bug even in Unix is incorrect. If you want to make progress here, just accept that the code is perfectly good Unix code and then figure out how to make the code work for both Unix and Windows, instead of trying to argue incorrectly that the code's not correct for Unix. It's not portable, but it's correct for Unix. Thank you. Anyhow in the base code 95% of the times or even more sockets are declared as SOCKET sockets. Here and there they are declared as int. This is an inconsistency and it should be removed. I do beg the community to do this little change because it is in the benefit and interest of everybody. I'm sure that the community will accept a patch that declares the pipe in a way that makes both Unix and Windows happy if you'll provide one. Meanwhile, quit complaining because I pointed out that, in Unix, int pipefd[2] is the correct declaration for a pipe. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
We are in violent agreement... It was never my intention to raise the discussion to this level. I just observed the code. I may have used tones a bit too strong or too stressing... I never used bad words... I am going to provide a patch that will remove the inconsistencies I tried to explain... It will be very a very limited set of changes but it will make Aolserver more portable. Funny we are ready to fight and willing to kill each other on int trigger[2] versus SOCKET trigger[2] argument BTW: being called a dick is not very pleasant, and the only reason I do not react on this is that I'm interested in making Aolserver portable more than anything else. Thank you for your understanding and kind support. -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 21:52 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 11:26 AM, Maurizio Martignano wrote: If you wanted to develop only for Unix why did you use SOCKET in some occasions and int in some others? Damnit, that was never the point. You said that you thought that the code was incorrect for Unix, as well, a bug lurking that just hadn't caught anyone yet. That's flat out wrong. That declaration is standard Unix, and as I tried to point out previously, the mistake was probably made by someone who wasn't aware that it's not portable to all versions of Windows. End of story. That's all I said. All this other stuff you're ranting about is totally unrelated to my point. As Rusty said, you're being a dick. I don't bother trying to help dicks. So I'll no longer try to help you. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Hi, I'm looking at the code now -- definitely needs to be SOCKET in nsd.h. The reason can be seen in ns_sockpair in fd/sock.c where the code for a socket pair is done. It's just a wrapper around Unix socketpair() but has a bunch of extra code to do the loopback-connect thing on Windows. The socks are passed by reference -- this code thinks it's 16 bytes (i.e., 2 8-byte long ints) but it's getting 2 4-bytes from the trigger in the driver code. It's could be overwriting some more of the driver structure -- not clear. Note that ns_pipe != ns_sockpair. An ns_pipe is used to create process pipelines with the lib-c code on Windows. On Windows, it can't be used a s trigger pipe. The fix should be just updating nsd.h with SOCKET. I'd do it but I don't have any of my dev environment setup -- something I was going to finally dig into in a few weeks. Anyone else with commit could handle the code update. Compiling and testing on 64-bit Windows would be more work and require the Visual Studio stuff -- the Express C++ stuff appears to only support 32-bit. BTW: The basic strategy for Windows was to make it look as much like Unix as possible, as lightly as possible. This is why you get some weirdness like the opendir/readdir/closedir emulation code stuffed in the Windows thread library (nsthread/winthread.c). There's also quite a bit of ifdef WIN stuff in the process exec code (nsd/exec.c) and some file management code (nsd/fd.c). The use of trigger pipes is a Unix way of monitoring events -- I think on Windows you could WaitForMultipleHandles with all sorts of things, including network and process sync stuff, but since the goal was to be Unix-like, instead of modifying the event-driven code to eliminate socketpair's for wakeup, this ns_sockpair emulation was used instead. Also, in later versions Windows was being ignored -- you see this in the oversight of SOCKET for the driver trigger we've been talking about and the NsMap code not yet implemented for Windows. I agree a scan of all SOCKETs not being declared are SOCKET's makes good sense. -Jim On Aug 4, 2011, at 8:28 AM, Maurizio Martignano wrote: It is not a matter of understanding It is a matter of testing On Windows 64 int trigger[2] doesn’t work whereas SOCKET trigger[2] does work. On top of that in several other places SOCKET has been used, so if for no other reason, I suggest one of the code maintainers takes a proper walk on the code base and make sure that all sockets are SOCKET sockets and not int sockets… Ciao, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 04 August 2011 15:51 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples It's probably safer to define this as SOCKET, but windows.h says SOCKET is: typedef u_int SOCKET; And: typedef unsigned intu_int; Since Windows is LLP64 and most Unix-like systems are LP64, I don't understand how AOLserver's defining trigger[2] as (int) is the problem -- Windows might complain about some signed/unsigned thing at compile time, but in both cases, (int) is 32 bits. On 8/4/11 3:24 AM, Maurizio Martignano wrote: inttrigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
I am doing the scan and I am preparing the patch. I will have also to do the testing as some of my customers do require the Windows 64 version. And I may offer to do this testing also in the future, non on a continuous basis, but every now and then. Ciao and thanks, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Jim Davidson Sent: 05 August 2011 00:43 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Hi, I'm looking at the code now -- definitely needs to be SOCKET in nsd.h. The reason can be seen in ns_sockpair in fd/sock.c where the code for a socket pair is done. It's just a wrapper around Unix socketpair() but has a bunch of extra code to do the loopback-connect thing on Windows. The socks are passed by reference -- this code thinks it's 16 bytes (i.e., 2 8-byte long ints) but it's getting 2 4-bytes from the trigger in the driver code. It's could be overwriting some more of the driver structure -- not clear. Note that ns_pipe != ns_sockpair. An ns_pipe is used to create process pipelines with the lib-c code on Windows. On Windows, it can't be used a s trigger pipe. The fix should be just updating nsd.h with SOCKET. I'd do it but I don't have any of my dev environment setup -- something I was going to finally dig into in a few weeks. Anyone else with commit could handle the code update. Compiling and testing on 64-bit Windows would be more work and require the Visual Studio stuff -- the Express C++ stuff appears to only support 32-bit. BTW: The basic strategy for Windows was to make it look as much like Unix as possible, as lightly as possible. This is why you get some weirdness like the opendir/readdir/closedir emulation code stuffed in the Windows thread library (nsthread/winthread.c). There's also quite a bit of ifdef WIN stuff in the process exec code (nsd/exec.c) and some file management code (nsd/fd.c). The use of trigger pipes is a Unix way of monitoring events -- I think on Windows you could WaitForMultipleHandles with all sorts of things, including network and process sync stuff, but since the goal was to be Unix-like, instead of modifying the event-driven code to eliminate socketpair's for wakeup, this ns_sockpair emulation was used instead. Also, in later versions Windows was being ignored -- you see this in the oversight of SOCKET for the driver trigger we've been talking about and the NsMap code not yet implemented for Windows. I agree a scan of all SOCKETs not being declared are SOCKET's makes good sense. -Jim On Aug 4, 2011, at 8:28 AM, Maurizio Martignano wrote: It is not a matter of understanding It is a matter of testing On Windows 64 int trigger[2] doesnt work whereas SOCKET trigger[2] does work. On top of that in several other places SOCKET has been used, so if for no other reason, I suggest one of the code maintainers takes a proper walk on the code base and make sure that all sockets are SOCKET sockets and not int sockets Ciao, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 04 August 2011 15:51 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples It's probably safer to define this as SOCKET, but windows.h says SOCKET is: typedef u_int SOCKET; And: typedef unsigned intu_int; Since Windows is LLP64 and most Unix-like systems are LP64, I don't understand how AOLserver's defining trigger[2] as (int) is the problem -- Windows might complain about some signed/unsigned thing at compile time, but in both cases, (int) is 32 bits. On 8/4/11 3:24 AM, Maurizio Martignano wrote: inttrigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com