Re: [AOLSERVER] Memory problem with Aolserver/TCL/OpenACS
Thanks a lot Janine. Cheers, Maurizio From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Janine Sisk Sent: 09 March 2007 20:06 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Memory problem with Aolserver/TCL/OpenACS We had a site that was doing this, and the fix in this post worked for us: http://dossy.org/archives/000108.html It was rather perplexing; only one of our many hosted sites had the problem, and it just started exhibiting it one day with no obvious change to account for it. Based on Dossy's description that shouldn't be possible. janine On Mar 9, 2007, at 1:38 AM, Maurizio Martignano wrote: Dear all, I've got a question (possibly a newbie's one) on the memory allocated by Aolserver (4.0.10) when running OpenACS (5.1.5). When monitoring (with top or atop) the memory used by Aolserver running an OpenACS based Web application it is possible to notice that the value of virtual memory allocated by the nsd process always grows and never ever gets smaller. I suspected this continuous growth of the allocated virtual memory was due either to tDOM or the Aolserver memory allocator. I made sure in my application that every time tDOM is used its allocated memory gets explicitly released, but still no luck. So I guess the problem could reside in the Aolserver memory allocator. Any clue, any suggestion? Am I doing something wrong? I'd like not to be forced to schedule a nightly restart of the server just to keep the memory allocation problem under control. Thanks in advance for your help, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver 4.5.0 on windows not exiting service
Hello all. I've included Titi's suggestion in the current version of the Win32- OpenACS port. So now NSD is a well behaved windows service. Thanks for your help, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Compiling 4.5 on Win32 using Visual Studio Express Edition 2008
Hello there, Please have a look at the following Win32-OpenACS port: http://www.friendlybits.com/en/inf_tec_en/win32openacs_en/ This port has been compiled using Visual Studio 2008 Professional Edition and I believe you should be able to compile it also with the Express Edition. The distribution contains both the binary files and the source codes. Building/compilation instructions can be found on the web page. Happy compiling Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Shedis Sent: 26 April 2008 06:09 To: AOLSERVER@LISTSERV.AOL.COM Subject: [AOLSERVER] Compiling 4.5 on Win32 using Visual Studio Express Edition 2008 Hello, TCL: 8.5.2 AOLserver: aolserver-HEAD-20071203 This is one of my many attempts to but in the past 6 months i have never managed to compile. Once successful, i wanted to write up the process for other like me out there. But it never happend. Except for one case where with lots of back and forth emails from Dossy help me to compile on suse 10.1. But it was a lot of test and trial and i lost track of how i actually started. But i have to admit that once nsd was started it just works. Guys why not try to document the compilation process and try to align it with 'it just compiles' notion. is it too far fetched. For example a clear instructions for: a. compiling on suse 10.1 b. compiling on suse 10.2 c. compiling on win32 2003 sp1, with vs express edition 2008 etc I'm trying to compile 4.5 using visual studio express edition 2008 but keep getting the following error. Does any one know how to fix this? Steps i'm using to compile: 1. C:\Program Files\Microsoft Visual Studio 9.0\Common7\Tools \vsvars32.bat 2. nmake -f makefile.vc INSTALLDIR=c:\aolserver OPTS=threads,thrdalloc 3. nmake -f makefile.vc INSTALLDIR=c:\aolserver OPTS=threads,thrdalloc install 4. c:\aolserver\bin\tclsh85t.exe nsconfig.tcl Configuring include/ns.mak with variables: AOLSERVER=c:\aolserver DEBUG=0 TCLSH=c:\aolserver\bin\tclsh85t.exe TCLINC=c:\aolserver\include TCLLIB=c:\aolserver\lib\tcl85t.lib 5. nmake install make install: nsd Microsoft (R) Program Maintenance Utility Version 9.00.21022.08 Copyright (C) Microsoft Corporation. All rights reserved. c:\aolserver\bin\tclsh85t.exe C:\source\aolserver- HEAD-20071203\util\nsi nstall.tcl -d c:\aolserver\bin init.tcl installed: C:/aolserver/bin/init.tcl cl /EHsc /W3 /nologo /c /MD /O2 /D _WINDOWS /D TCL_THREADS=1 /D WI N32 /D FD_SETSIZE=128 /D NO_CONST=1 /D _MBCS /D NDEBUG /I C: \source\a olserver-HEAD-20071203\include /I c:\aolserver\include /c / Foadpcmds.o adpcmd s.c adpcmds.c c:\source\aolserver-head-20071203\nsd\nsd.h(71) : warning C4005: 'POLLIN' : macr o redefinition C:\Program Files\Microsoft SDKs\Windows\v6.0A\include \winsock2.h(1495) : see previous definition of 'POLLIN' c:\source\aolserver-head-20071203\nsd\nsd.h(72) : warning C4005: 'POLLOUT' : mac ro redefinition C:\Program Files\Microsoft SDKs\Windows\v6.0A\include \winsock2.h(1499) : see previous definition of 'POLLOUT' c:\source\aolserver-head-20071203\nsd\nsd.h(73) : warning C4005: 'POLLPRI' : mac ro redefinition C:\Program Files\Microsoft SDKs\Windows\v6.0A\include \winsock2.h(1496) : see previous definition of 'POLLPRI' c:\source\aolserver-head-20071203\nsd\nsd.h(74) : warning C4005: 'POLLHUP' : mac ro redefinition C:\Program Files\Microsoft SDKs\Windows\v6.0A\include \winsock2.h(1503) : see previous definition of 'POLLHUP' c:\source\aolserver-head-20071203\nsd\nsd.h(75) : error C2011: 'pollfd' : 'struc t' type redefinition C:\Program Files\Microsoft SDKs\Windows\v6.0A\include \winsock2.h(1506) : see declaration of 'pollfd' NMAKE : fatal error U1077: 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN \cl.EXE' : return code '0x2' Stop. NMAKE : fatal error U1077: 'c:\aolserver\bin\tclsh85t.exe' : return code '0x1' Stop. Thanks in advance. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Compiling 4.5 on Win32 using Visual Studio Express Edition 2008
I just forgot to mention... Compiling with Visual Studio 2008 is just as easy (or as difficult) as compiling with gcc on a Linux/Unix box. Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Shedis Sent: 26 April 2008 06:09 To: AOLSERVER@LISTSERV.AOL.COM Subject: [AOLSERVER] Compiling 4.5 on Win32 using Visual Studio Express Edition 2008 Hello, TCL: 8.5.2 AOLserver: aolserver-HEAD-20071203 This is one of my many attempts to but in the past 6 months i have never managed to compile. Once successful, i wanted to write up the process for other like me out there. But it never happend. Except for one case where with lots of back and forth emails from Dossy help me to compile on suse 10.1. But it was a lot of test and trial and i lost track of how i actually started. But i have to admit that once nsd was started it just works. Guys why not try to document the compilation process and try to align it with 'it just compiles' notion. is it too far fetched. For example a clear instructions for: a. compiling on suse 10.1 b. compiling on suse 10.2 c. compiling on win32 2003 sp1, with vs express edition 2008 etc I'm trying to compile 4.5 using visual studio express edition 2008 but keep getting the following error. Does any one know how to fix this? Steps i'm using to compile: 1. C:\Program Files\Microsoft Visual Studio 9.0\Common7\Tools \vsvars32.bat 2. nmake -f makefile.vc INSTALLDIR=c:\aolserver OPTS=threads,thrdalloc 3. nmake -f makefile.vc INSTALLDIR=c:\aolserver OPTS=threads,thrdalloc install 4. c:\aolserver\bin\tclsh85t.exe nsconfig.tcl Configuring include/ns.mak with variables: AOLSERVER=c:\aolserver DEBUG=0 TCLSH=c:\aolserver\bin\tclsh85t.exe TCLINC=c:\aolserver\include TCLLIB=c:\aolserver\lib\tcl85t.lib 5. nmake install make install: nsd Microsoft (R) Program Maintenance Utility Version 9.00.21022.08 Copyright (C) Microsoft Corporation. All rights reserved. c:\aolserver\bin\tclsh85t.exe C:\source\aolserver- HEAD-20071203\util\nsi nstall.tcl -d c:\aolserver\bin init.tcl installed: C:/aolserver/bin/init.tcl cl /EHsc /W3 /nologo /c /MD /O2 /D _WINDOWS /D TCL_THREADS=1 /D WI N32 /D FD_SETSIZE=128 /D NO_CONST=1 /D _MBCS /D NDEBUG /I C: \source\a olserver-HEAD-20071203\include /I c:\aolserver\include /c / Foadpcmds.o adpcmd s.c adpcmds.c c:\source\aolserver-head-20071203\nsd\nsd.h(71) : warning C4005: 'POLLIN' : macr o redefinition C:\Program Files\Microsoft SDKs\Windows\v6.0A\include \winsock2.h(1495) : see previous definition of 'POLLIN' c:\source\aolserver-head-20071203\nsd\nsd.h(72) : warning C4005: 'POLLOUT' : mac ro redefinition C:\Program Files\Microsoft SDKs\Windows\v6.0A\include \winsock2.h(1499) : see previous definition of 'POLLOUT' c:\source\aolserver-head-20071203\nsd\nsd.h(73) : warning C4005: 'POLLPRI' : mac ro redefinition C:\Program Files\Microsoft SDKs\Windows\v6.0A\include \winsock2.h(1496) : see previous definition of 'POLLPRI' c:\source\aolserver-head-20071203\nsd\nsd.h(74) : warning C4005: 'POLLHUP' : mac ro redefinition C:\Program Files\Microsoft SDKs\Windows\v6.0A\include \winsock2.h(1503) : see previous definition of 'POLLHUP' c:\source\aolserver-head-20071203\nsd\nsd.h(75) : error C2011: 'pollfd' : 'struc t' type redefinition C:\Program Files\Microsoft SDKs\Windows\v6.0A\include \winsock2.h(1506) : see declaration of 'pollfd' NMAKE : fatal error U1077: 'C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN \cl.EXE' : return code '0x2' Stop. NMAKE : fatal error U1077: 'c:\aolserver\bin\tclsh85t.exe' : return code '0x1' Stop. Thanks in advance. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Compiling 4.5 on Win32 using Visual Studio Express Edition 2008
Dear all, When this community produces a new stable release of Aolserver, let it be 4.5.1 or 4.5.x, I'll be more than happy to produce a new version of the Win32-OpenACS distribution containing it. Cheers and thanks, Maurizio -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Dossy Shiobara Sent: 28 April 2008 16:47 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Compiling 4.5 on Win32 using Visual Studio Express Edition 2008 On 2008.04.28, Titi Alailima [EMAIL PROTECTED] wrote: Speaking of Maurizio's work, since he is basing his work on 4.5.0, is there any chance we could start a stable branch of 4.5.0-based code? Seems like 4.5.0 is getting some real traction, but the current HEAD sounds a little like a moving target, so I'd like to see something that basically just has bug fixes to 4.5.0. I don't care if we call it 4.5.0.x or 4.5.x (and look at the HEAD more as 4.6 alpha), but I don't like having to comb through the CVS and find the bug fixes I need and patch them in manually, and I've heard enough on this list to realize it would be foolish to try to take something to production using the HEAD. So, here's what I'm thinking ... * Push out one final 4.0.11 release as the last in the 4.0.x series. * Merge down fixes from CVS HEAD into the 4.5.x branch, then push out a 4.5.1 release. There is already a aolserver_v45_bp branch for 4.5.x. In theory, CVS HEAD can be viewed as 4.6.x, as you point out. Should the version strings be changed to reflect this? -- Dossy Shiobara | [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Getting a handle on memory usage
Dear all, Sometime ago I made some tests on the memory leaks of aolserver+tcs+openacs. I'd like to share my two cents worth of what I believe I found, hoping it might help. Aolserver uses TCL, and TCL's got three different memory managers: 1. Standard: using the malloc, free, etc... primitives of the OS (not so fast and not using a lot of memory) 2. Zippy: multithreaded primitives specially developed for TCL (very fast but using plenty of memory - this is the default mechanism) 3. VTMalloc: another multithreaded implementation, developed by third parties, and somehow in between Standard and Zippy for what concerns efficiency and memory occupation. When using Zippy I noticed that the memory occupied by Aolserver (nsd) always grows, without being released. On the contrary when using the Standard memory allocator every now and then memory gets released when not used. The total memory occupied by Aolserver keeps growing but at a slower pace that in the previous case. How to enable TCL Standard memory allocator? Just call the following configure command: configure -enable-threads Then edit the Makefile, look for AC_FLAGS and remove from it the -DUSE_THREAD_ALLOC=1 define (i.e. delete the string -DUSE_THREAD_ALLOC=1). Compile everything, and see how it goes With this change in place, I've seen some saving on the memory occupied by nsd (around 30%). Another point to look at very carefully is the overall system configuration (config.tcl): Parameters like maxconnections maxdropped maxthreads minthreads threadtimeout and maxidle maxopen connections1 have quite an effect on the total occupied memory. Here there's no general recipe. My suggestion would be try to use the smallest possible values that still allow the system to work. Hope it helps, Maurizio -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Fenton, Brian Sent: 01 May 2008 19:30 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Getting a handle on memory usage Hi Titi, I know Gustaf Neumann had a script to find application level leaks. If he doesn't respond I can post it. Brian -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Titi Alailima Sent: 01 May 2008 17:31 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Getting a handle on memory usage This isn't a very helpful response. I'm not even asking about ongoing memory growth and overhead, at least not yet. I want to know what are the parts that we (or at least some of us) _do_ understand and how can we at least calculate those things so that: 1. we even have a sense of how much memory is being eaten up by things we don't know 2. we know at least roughly what sort of impact we can make by adjusting the parts we do know As for these test workloads, can someone enlighten me more as to what they might look like and how they would be used? Depending on what is involved, I or someone I know may be able to figure something out to get us started in the right direction. Titi Ala'ilima Lead Architect MedTouch LLC 1100 Massachusetts Avenue Cambridge, MA 02138 617.621.8670 x309 -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Dossy Shiobara Sent: Wednesday, April 30, 2008 3:10 PM To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Getting a handle on memory usage On 2008.04.30, Titi Alailima [EMAIL PROTECTED] wrote: What are the best ways of figuring out how the memory usage in AOLserver is broken down? [...] There's lots of overheads all over the place. Each Tcl_Obj structure, every Tcl_DString buffer ... the AOLserver driver thread and all its structures used to manage state ... All these little things should (in theory) be inconsequential compared to the larger picture of the application's memory usage, but there's a good chance there are a few lost pointer leaks, etc. -- Dossy Shiobara | [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply
Re: [AOLSERVER] Getting a handle on memory usage
Dear all, If a proper analysis of the memory consumption is of interest to anyone, it could be an idea to use Valgrind/Memcheck: http://valgrind.org/ Anyhow, before going over such detailed and time consuming analysis, I first would try to: 1. use the TCL Standard memory allocator (and not the Zippy one) 2. properly configure the parameters in the 'config.tcl' Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Dossy Shiobara Sent: 01 May 2008 21:11 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Getting a handle on memory usage On 2008.05.01, Maurizio Martignano [EMAIL PROTECTED] wrote: When using Zippy I noticed that the memory occupied by Aolserver (nsd) always grows, without being released. That's probably because we don't invoke madvise() anywhere. Of course, apparently on MacOS X madvise() is broken, and on Win32 we should use VirtualFree(). See this entry about jemalloc where this issue is explored: Perceived jemalloc memory footprint http://www.canonware.com/~ttt/2008/01/perceived-jemalloc-memory-footprint.ht ml I mentioned on the IRC chat that it might be fun to implement a very fine-grained debugging memory allocator implementation that keeps lots of statistics that we can examine and is designed with knowledge that it's debugging an AOLserver process to record the relevant bits of information so we can correlate it back to an activity that the code is performing. I don't know if this is a worthwhile idea, but it was fun to think about for a moment, anyway. -- Dossy Shiobara | [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Fix for ns_tmpnam under Windows
Dear Titi, The proper version of the file, properly compiled and tested is the one attached to this email message. Hope it helps, Maurizio -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Titi Alailima Sent: 05 May 2008 19:03 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Fix for ns_tmpnam under Windows There is a missing variable declaration in this patch for i, the for-loop index. Anyone want to make this fix and commit it? Titi Ala'ilima Lead Architect MedTouch LLC 1100 Massachusetts Avenue Cambridge, MA 02138 617.621.8670 x309 -Original Message- From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On Behalf Of Gustaf Neumann Sent: Thursday, April 24, 2008 3:33 AM To: AOLSERVER@LISTSERV.AOL.COM Subject: [AOLSERVER] Fix for ns_tmpnam under Windows Dear AOLserver community There was a problem with ns_tmpnam, when the aolserver was compiled under windows (esp. Vista). The Tcl commandns_tmpnam is implemented in C using tmpnam(), which exists under WIN32, but behaves differently as on unix counterparts and is practically unusable. The problem is that tmnam() under Windows generates a filename for the root directory of the actual drive (at least under Vista, no permissions under normal conditions) and ignores the TMP environment variable. See some background info form Microsoft: http://msdn2.microsoft.com/en-us/library/hs3e7355(VS.80).aspx I have commited a patch to CVS head to address this problem. http://aolserver.cvs.sourceforge.net/aolserver/aolserver/nsd/tclfile.c? r1=1.25r2=1.26 The patch is local and documents the intended behavior and background in detail. Please crosscheck. best regards -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to [EMAIL PROTECTED] with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. /* * The contents of this file are subject to the AOLserver Public License * Version 1.1 (the License); you may not use this file except in * compliance with the License. You may obtain a copy of the License at * http://aolserver.com/. * * Software distributed under the License is distributed on an AS IS * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See * the License for the specific language governing rights and limitations * under the License. * * The Original Code is AOLserver Code and related documentation * distributed by AOL. * * The Initial Developer of the Original Code is America Online, * Inc. Portions created by AOL are Copyright (C) 1999 America Online, * Inc. All Rights Reserved. * * Alternatively, the contents of this file may be used under the terms * of the GNU General Public License (the GPL), in which case the * provisions of GPL are applicable instead of those above. If you wish * to allow use of your version of this file only under the terms of the * GPL and not to allow others to use your version of this file under the * License, indicate your decision by deleting the provisions above and * replace them with the notice and other provisions required by the GPL. * If you do not delete the provisions above, a recipient may use your * version of this file under either the License or the GPL. */ /* * tclfile.c -- * * Tcl commands that do stuff to the filesystem. */ static const char *RCSID = @(#) $Header: /cvsroot/aolserver/aolserver/nsd/tclfile.c,v 1.24 2005/08/23 21:41:31 jgdavidson Exp $, compiled: __DATE__ __TIME__; #include nsd.h #ifdef _WIN32 #include sys/utime.h #else #include utime.h #endif /* * Structure handling one registered channel for the [ns_chan] command */ typedef struct _NsRegChan { char *name; Tcl_Channel chan; } NsRegChan; static void SpliceChannel(Tcl_Interp *interp, Tcl_Channel chan); static void UnspliceChannel(Tcl_Interp *interp, Tcl_Channel chan); /* *-- * * Ns_TclGetOpenChannel -- * * Return an open channel with an interface similar to the * pre-Tcl7.5 Tcl_GetOpenFile, used throughout AOLserver. * * Results: * TCL_OK or TCL_ERROR. * * Side effects: * The value at chanPtr is updated with a valid open Tcl_Channel. * *-- */ static int GetOpenChannel(Tcl_Interp *interp, Tcl_Obj *obj, int write, int check,
Re: [AOLSERVER] nsopenssl fails to load
Hello Thomas, the variable you seem to be missing Tcl_HashTable NsOpenSSLServers; is declared inside the file sslcontext.c From the errors you're getting I believe you have not included this file in your build. I'm currently working on providing my Win32-OpenACS http://www.friendlybits.com/en/inf_tec_en/win32openacs_en/ distribution with HTTPS/SSL support and up to now I didn't find any problem whatsoever. Of course I'm using makefiles and development tools different from yours, anyhow I believe you should make sure the following files are included in your build: nsopenssl.c ssl.c sslcontext.c tclcmds.c x509.c Hope this helps, Maurizio -Original Message- From: AOLserver Discussion [mailto:aolser...@listserv.aol.com] On Behalf Of Prem Thomas Sent: 16 July 2009 01:28 To: AOLSERVER@LISTSERV.AOL.COM Subject: [AOLSERVER] nsopenssl fails to load I'm having trouble loading nsopenssl with AOLserver 4.5.1. Despite multiple attempts to address the problem, I'm going nowhere. Details below. I'd greatly appreciate advice. --Prem Thomas Details === 64-bit x86 OpenSolaris (Nexenta: SunOS 5.11 kernel with a Ubuntu Hardy 8.04 userland) AOLserver 4.5.1 TCL 8.5.7 OpenSSL 0.9.8k nsopenssl 3.0.b26 GNU Make 3.81 gcc (GCC) 4.2.3 (Ubuntu 4.2.3-2nexenta7) Both OpenSSL and nsopenssl built successfully. When I start AOLserver, I get this fatal error: [12/Jul/2009:23:26:14][1142.1][-main-] Notice: modload: loading '/usr/ local/aolserver/bin/nsopenssl.so' [12/Jul/2009:23:26:14][1142.1][-main-] Warning: modload: could not load /usr/local/aolserver/bin/nsopenssl.so: ld.so.1: nsd: fatal: relocation error: file /usr/local/aolserver/bin/nsopenssl.so: symbol NsOpenSSLServers: referenced symbol not found [12/Jul/2009:23:26:14][1142.1][-main-] Fatal: modload: failed to load module '/usr/local/aolserver/bin/nsopenssl.so' After googling around, and reading this: http://openacs.org/forums/ message-view?message_id=51691 I tried building nsopenssl with the -Bstatic flag: # make install OPENSSL=/usr/local/ssl AOLSERVER=${NS} MODLIBS=- Bstatic -L${OPENSSL}/lib -lssl -lcrypto -L/usr/lib/gcc/i386-pc- solaris2.11/4.2.3 -lgcc Same error. I also tried rebuilding OpenSSL with the no-asm config option: # ./Configure solaris-x86-gcc no-asm Build succeeded and I rebuilt nsopenssl. But I still get the same error. I tried rebuilding nsopenssl with make CC=gcc -fPIC but this produced errors: shlib_target=; if [ -n ]; then \ shlib_target=solaris-shared; \ elif [ -n ]; then \ FIPSLD_CC=gcc -fpic; CC=../fips/fipsld; export CC FIPSLD_CC; \ fi; \ LIBRARIES=-L.. -lssl -L.. -lcrypto ; \ [ x = xlibfips ] LIBRARIES=$LIBRARIES -lfips; \ make -f ../Makefile.shared -e \ CC=${CC} APPNAME=openssl OBJECTS=openssl.o verify.o asn1pars.o req.o dgst.o dh.o dhparam.o enc.o passwd.o gendh.o errstr.o ca.o pkcs7.o crl2p7.o crl.o rsa.o rsautl.o dsa.o dsaparam.o ec.o ecparam.o x509.o genrsa.o gendsa.o s_server.o s_client.o speed.o s_time.o apps.o s_cb.o s_socket.o app_rand.o version.o sess_id.o ciphers.o nseq.o pkcs12.o pkcs8.o spkac.o smime.o rand.o engine.o ocsp.o prime.o cms.o \ LIBDEPS= $LIBRARIES -lsocket -lnsl -ldl \ link_app.${shlib_target} make[2]: Entering directory `/usr/local/src/openssl-0.9.8k/apps' make[2]: pic: No such file or directory make[2]: *** No rule to make target `pic'. Stop. I tried adding extra paths to LD_LIBRARY_PATH, but to no avail. #echo $LD_LIBRARY_PATH /usr/local/src/aolserver45/nsopenssl:/usr/local/pgsql/lib:/usr/local/ ssl/lib:/usr/local/aolserver/lib:/usr/lib/gcc/i386-pc-solaris2.11/4.2.3: Here's a grep of the symbol in the source and aolserver directories: # grep -r NsOpenSSLServers /usr/local/src/aolserver45/nsopenssl Binary file ./nsopenssl.o matches ./sslcontext.c:Tcl_HashTable NsOpenSSLServers; ./sslcontext.c:hPtr = Tcl_FindHashEntry(NsOpenSSLServers, server); ./nsopenssl.c:extern Tcl_HashTable NsOpenSSLServers; ./nsopenssl.c:Tcl_InitHashTable(NsOpenSSLServers, TCL_STRING_KEYS); ./nsopenssl.c:hPtr = Tcl_CreateHashEntry(NsOpenSSLServers, server, new); Binary file ./nsopenssl.so matches Binary file ./libnsopenssl.so matches Binary file ./sslcontext.o matches # grep -r NsOpenSSLServers /usr/local/aolserverBinary file /usr/local/aolserver/bin/nsopenssl.so matches Binary file /usr/local/aolserver/lib/libnsopenssl.so matches I also tried ldd -d (see http://docs.sun.com/app/docs/doc/ 806-1075/6jacsnimr?a=view) #ldd -d /usr/local/aolserver/bin/nsd libnsd.so = /usr/local/aolserver/lib/libnsd.so libnsthread.so =/usr/local/aolserver/lib/ libnsthread.so libtcl8.5.so = /usr/local/aolserver/lib/libtcl8.5.so libdl.so.1 =/lib/libdl.so.1 libsocket.so.1 =/lib/libsocket.so.1 libpthread.so.1 = /lib/libpthread.so.1 libm.so.2
Re: [AOLSERVER] nsopenssl fails to load
Could I have copy of your Makefile? Thanks in advance, Maurizio -Original Message- From: AOLserver Discussion [mailto:aolser...@listserv.aol.com] On Behalf Of Prem Thomas Sent: 17 July 2009 00:41 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] nsopenssl fails to load Maurizio, Thanks for the reply. A Win32 OpenACS! I checked my build directory (below) and it has all the files you mentioned. I think this is a problem with loading and linking on Solaris. drwxr-xr-x 6 root root 26 Jul 15 23:54 . drwxr-xr-x 16 root root 26 Jul 15 23:50 .. drwxr-xr-x 2 root root 6 Jul 15 23:50 CVS -rw-r--r-- 1 root root 40468 Nov 20 2004 ChangeLog -rw-r--r-- 1 root root3981 Nov 20 2004 Makefile -rw-r--r-- 1 root root 15485 Aug 25 2004 README -rw-r--r-- 1 root root2172 Apr 9 2004 TODO drwxr-xr-x 3 root root 6 Jul 15 23:50 ca -rw-r--r-- 1 root root1450 Jun 23 2004 defaults.h drwxr-xr-x 3 root root 3 Jul 15 23:50 docs -rw-r--r-- 1 root root 13877 Jun 16 2004 https.tcl -rwxr-xr-x 1 root root 1312755 Jul 15 23:51 libnsopenssl.so -rw-r--r-- 1 root root8538 Oct 27 2004 nsd.tcl -rw-r--r-- 1 root root 30910 Nov 20 2004 nsopenssl.c -rw-r--r-- 1 root root 7 Mar 28 2006 nsopenssl.h -rw-r--r-- 1 root root 12496 Jul 15 23:51 nsopenssl.o -rwxr-xr-x 1 root root 15067 Jul 15 23:54 nsopenssl.so -rw-r--r-- 1 root root 23439 Mar 28 2006 ssl.c -rw-r--r-- 1 root root8648 Jul 15 23:51 ssl.o -rw-r--r-- 1 root root 51954 Mar 28 2006 sslcontext.c -rw-r--r-- 1 root root 17484 Jul 15 23:51 sslcontext.o -rw-r--r-- 1 root root 56427 Jun 13 2004 tclcmds.c -rw-r--r-- 1 root root 24404 Jul 15 23:51 tclcmds.o drwxr-xr-x 3 root root 18 Jul 15 23:50 tests -rw-r--r-- 1 root root9446 Apr 13 2004 x509.c -rw-r--r-- 1 root root5804 Jul 15 23:51 x509.o -Prem Thomas On Jul 16, 2009, at 8:54 AM, Maurizio Martignano wrote: Hello Thomas, the variable you seem to be missing Tcl_HashTable NsOpenSSLServers; is declared inside the file sslcontext.c From the errors you're getting I believe you have not included this file in your build. I'm currently working on providing my Win32-OpenACS http://www.friendlybits.com/en/inf_tec_en/win32openacs_en/ distribution with HTTPS/SSL support and up to now I didn't find any problem whatsoever. Of course I'm using makefiles and development tools different from yours, anyhow I believe you should make sure the following files are included in your build: nsopenssl.c ssl.c sslcontext.c tclcmds.c x509.c Hope this helps, Maurizio -Original Message- From: AOLserver Discussion [mailto:aolser...@listserv.aol.com] On Behalf Of Prem Thomas Sent: 16 July 2009 01:28 To: AOLSERVER@LISTSERV.AOL.COM Subject: [AOLSERVER] nsopenssl fails to load I'm having trouble loading nsopenssl with AOLserver 4.5.1. Despite multiple attempts to address the problem, I'm going nowhere. Details below. I'd greatly appreciate advice. --Prem Thomas Details === 64-bit x86 OpenSolaris (Nexenta: SunOS 5.11 kernel with a Ubuntu Hardy 8.04 userland) AOLserver 4.5.1 TCL 8.5.7 OpenSSL 0.9.8k nsopenssl 3.0.b26 GNU Make 3.81 gcc (GCC) 4.2.3 (Ubuntu 4.2.3-2nexenta7) Both OpenSSL and nsopenssl built successfully. When I start AOLserver, I get this fatal error: [12/Jul/2009:23:26:14][1142.1][-main-] Notice: modload: loading '/usr/ local/aolserver/bin/nsopenssl.so' [12/Jul/2009:23:26:14][1142.1][-main-] Warning: modload: could not load /usr/local/aolserver/bin/nsopenssl.so: ld.so.1: nsd: fatal: relocation error: file /usr/local/aolserver/bin/nsopenssl.so: symbol NsOpenSSLServers: referenced symbol not found [12/Jul/2009:23:26:14][1142.1][-main-] Fatal: modload: failed to load module '/usr/local/aolserver/bin/nsopenssl.so' After googling around, and reading this: http://openacs.org/forums/ message-view?message_id=51691 I tried building nsopenssl with the -Bstatic flag: # make install OPENSSL=/usr/local/ssl AOLSERVER=${NS} MODLIBS=- Bstatic -L${OPENSSL}/lib -lssl -lcrypto -L/usr/lib/gcc/i386-pc- solaris2.11/4.2.3 -lgcc Same error. I also tried rebuilding OpenSSL with the no-asm config option: # ./Configure solaris-x86-gcc no-asm Build succeeded and I rebuilt nsopenssl. But I still get the same error. I tried rebuilding nsopenssl with make CC=gcc -fPIC but this produced errors: shlib_target=; if [ -n ]; then \ shlib_target=solaris-shared; \ elif [ -n ]; then \ FIPSLD_CC=gcc -fpic; CC=../fips/fipsld; export CC FIPSLD_CC; \ fi; \ LIBRARIES=-L.. -lssl -L.. -lcrypto ; \ [ x = xlibfips ] LIBRARIES=$LIBRARIES -lfips; \ make -f ../Makefile.shared -e \ CC=${CC} APPNAME=openssl OBJECTS=openssl.o verify.o asn1pars.o req.o dgst.o dh.o dhparam.o enc.o passwd.o gendh.o errstr.o ca.o pkcs7.o crl2p7.o crl.o rsa.o
Re: [AOLSERVER] ns_odbc on Win32
Hello all, I could try to add this module to my Win32-OpenACS distribution. Anyhow I would just compile the module. I've no time to test its actual behavior nor to properly support it. It would be a take as is and compile thing. Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:aolser...@listserv.aol.com] On Behalf Of Tom Jackson Sent: 10 September 2009 6:18 PM To: AOLSERVER@listserv.aol.com Subject: Re: [AOLSERVER] ns_odbc on Win32 Hopefully someone with a windows setup will take this up, or maybe a new module can be added without the need to merge with the old version. ODBC on windows is potentially much more useful than on *nix since everything it seems can be an ODBC datasource. Updating AOLserver to include a more complete ODBC/SQL CLI adapter would be a good long term goal for this community, the only real issue is that the current ns_db API works so well, or is easy enough to adapt to, it is not easy to justify the pain for the marginal gain. tom jackson On Thu, Sep 10, 2009 at 3:38 AM, Scott Goodwinsc...@scottg.net wrote: Joe did this work during last year's Tcl conference when he couldn't get ns_odbc to compile on his Windows laptop so he could use it during a tutorial. I'm certain he based the changes on the version of ns_odbc in CVS at that time, and I think the changes were mostly to the build process (Makefile). He did not add or delete any functionality; he just got it to compile with a newer Microsoft compiler. He sent me the changes but his message was promptly buried in the dog pile that is my inbox and I haven't even unpacked and looked at what he's done. Though I could try and merge the changes myself, I have zero Windows dev experience and no Windows systems at home or work to build and test it on (and no interest in moving in that direction). But I figured I'd make the changes available to the list so that others would have the opportunity to do the merge, build, test and commit for the rest of the community that runs AOLserver on Windows. /s. On Sep 10, 2009, at 4:43 AM, Tom Jackson wrote: I wonder what the changes were. This driver is missing several API: select and dml (it has exec which can handle both by itself, so maybe they were removed). I have a more complete driver for unixodbc which includes transaction support, and a shutdown callback. If the changes are not that difficult, maybe I could apply them to this version as well (probably a different file for windows). It may not be possible, I can see some internal API differences. Both drivers seem to be derived from a common source Here's the current version of nsunixodbc: http://www.junom.com/gitweb/gitweb.perl?p=nsunixodbc.git;a=tree;hb=18d7 Unfortunately I have nothing setup on windows. tom jackson On Wed, Sep 9, 2009 at 8:42 AM, Scott Goodwinsc...@scottg.net wrote: Joe Mistachkin did some work on ns_odbc (almost a year ago) to get it to compile with newer Microsoft compilers; I've attached the relevant files if someone with Windows experience wants to diff and merge the changes into the module and check-in to AOLserver CVS. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. /s. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOL Server support on Windows 64 bit.
Dear Balaji, Thanks a lot for your question. I’m the maintainer of the “standard” OpenACS Windows (32) port. http://www.spazioit.com/pages_en/sol_inf_en/win32-openacs_en/ First of all let me state that the Win32-OpenACS 1.19 installation procedure as well as all its executables have been tested and do work on Windows XP, Vista, Windows 7 and Windows Server 2003 and 2008 (both the 32 bit and the 64 bit versions). Of course in the case of Windows 64 architectures they run on WOW (Windows On Windows - the Windows 32 emulator running on top of Windows 64). In several instances, even if the emulation middleware runs in between, Win32-OpenACS runs faster on Windows 64 systems than on Windows 32 systems. Now some few considerations about the future. Somehow all server hardware is shifting from 32 bit to 64 bit architecture... but the software, especially the applications are not following this path at the same pace. For example I haven't yet found a version PostgreSQL for Windows 64 (if you do know something about it please let me know). As you know Win32-OpenACS consists of plenty of different pieces of software. I have made sure that these pieces compile and work correctly on Windows 32. This requires knowing exactly what the compiler and the linker do, changing their parameters as necessary and so on... Making sure that all these pieces of software compile and work correctly on Windows 64 it's not a minor, trivial task. I don't even know if all the involved pieces of code have been developed/written taking into account the (Windows) 64 bit architecture. I was personally thinking about starting to work on a native port to Windows 64 this, towards the second quarter. But given the current economic situation I’m not going to undertake this effort unless explicitly requested (and funded). In any case PostgreSQL may still keep being a Windows 32 application. Before I finish let me repeat that the good point of all this blah/blah is that now, today, Win32-OpenACS installs and runs properly also on Windows 64 systems. Hope it helps, Maurizio From: AOLserver Discussion [mailto:aolser...@listserv.aol.com] On Behalf Of balaji pattabhiraman Sent: 30 June 2010 7:17 PM To: AOLSERVER@LISTSERV.AOL.COM Subject: [AOLSERVER] AOL Server support on Windows 64 bit. Hi All, Is any of the AOL server versions certified to work on 64 bit Windows.?. Was this ever compiled and tested on 64 bit and confirmed the runtime to be working. If so is there any release notes/documentation specifyiong the list of files and changes pertaining to this in AOL source code?. This is critical for us. Any help is appreciated -Balaji -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOL Server support on Windows 64 bit.
Dear Balaji, Limiting the effort to the Aolserver part may simplify the task. As I wrote in my previous mail I was planning to give it a try in the second quarter of this year; but given the current economic situation I’m not going to do that unless requested and funded. For the Aolserver part only the effort could be anything from 3 to 10 working days and the results are not guaranteed. I’m not aware if anyone else has done it or is about to try… If you are interested in my help/support please feel free to contact me. Regards, Maurizio From: AOLserver Discussion [mailto:aolser...@listserv.aol.com] On Behalf Of balaji pattabhiraman Sent: 01 July 2010 10:22 AM To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] AOL Server support on Windows 64 bit. Thanks for you reply. Im interested just in the AOL part (4.5.1), and I want that compiled/working on Windows 64 bit as such as a 64 bit binaries (as 64 bit app). Running 32 bit binaries on 64 bit (WOW) is what we currently do, but that is not good enough. Any idea if anyone including you have attempted such a thing on AOL? On 7/1/2010 12:43 PM, Maurizio Martignano wrote: Dear Balaji, Thanks a lot for your question. I’m the maintainer of the “standard” OpenACS Windows (32) port. http://www.spazioit.com/pages_en/sol_inf_en/win32-openacs_en/ First of all let me state that the Win32-OpenACS 1.19 installation procedure as well as all its executables have been tested and do work on Windows XP, Vista, Windows 7 and Windows Server 2003 and 2008 (both the 32 bit and the 64 bit versions). Of course in the case of Windows 64 architectures they run on WOW (Windows On Windows - the Windows 32 emulator running on top of Windows 64). In several instances, even if the emulation middleware runs in between, Win32-OpenACS runs faster on Windows 64 systems than on Windows 32 systems. Now some few considerations about the future. Somehow all server hardware is shifting from 32 bit to 64 bit architecture... but the software, especially the applications are not following this path at the same pace. For example I haven't yet found a version PostgreSQL for Windows 64 (if you do know something about it please let me know). As you know Win32-OpenACS consists of plenty of different pieces of software. I have made sure that these pieces compile and work correctly on Windows 32. This requires knowing exactly what the compiler and the linker do, changing their parameters as necessary and so on... Making sure that all these pieces of software compile and work correctly on Windows 64 it's not a minor, trivial task. I don't even know if all the involved pieces of code have been developed/written taking into account the (Windows) 64 bit architecture. I was personally thinking about starting to work on a native port to Windows 64 this, towards the second quarter. But given the current economic situation I’m not going to undertake this effort unless explicitly requested (and funded). In any case PostgreSQL may still keep being a Windows 32 application. Before I finish let me repeat that the good point of all this blah/blah is that now, today, Win32-OpenACS installs and runs properly also on Windows 64 systems. Hope it helps, Maurizio From: AOLserver Discussion [mailto:aolser...@listserv.aol.com] On Behalf Of balaji pattabhiraman Sent: 30 June 2010 7:17 PM To: AOLSERVER@LISTSERV.AOL.COM Subject: [AOLSERVER] AOL Server support on Windows 64 bit. Hi All, Is any of the AOL server versions certified to work on 64 bit Windows.?. Was this ever compiled and tested on 64 bit and confirmed the runtime to be working. If so is there any release notes/documentation specifyiong the list of files and changes pertaining to this in AOL source code?. This is critical for us. Any help is appreciated -Balaji -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to mailto:lists...@listserv.aol.com lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- http://www.oracle.com Oracle Balaji Pattabhiraman | Software Engineer Phone: +918041837029 | | | Mobile: +919740200778 Oracle PeopleTools ORACLE India http://www.oracle.com/commitment Green OracleOracle is committed to developing practices and products that help protect the environment -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http
Re: [AOLSERVER] LINK :fatal error LNK1117: syntax error in option 'OPT:NOWIN98' with Visual Studio 2010
Dear all I’m the maintainer of the Windows Port of OpenACS and ]project-open[. All my installers contain a working version of Aolserver 4.5.1, including the the Oracle driver, which has been compiled with Visual Studio 2010. Of course, all the installers contain also the source code (and the used makefiles…). You can find them here: 1. http://www.spazioit.com/pages_en/sol_inf_en/win32-openacs_en/ 2. http://projop.dnsalias.com/documentation/install_windows In case you still have problems, feel free to contact me. Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] Progress in Aolserver
Dear all, I'm the maintainer of the Windows Port of OpenACS. Recently (22/07/2011) I got the CVS HEAD version of Aolserver and compiled it with Microsoft Visual Studio 10 for Windows 32 bit and Windows 64 bit. I did this because I wanted to take profit of the mods introduced by Brian and Gustav to handle the uploading of files (objects) too large. Before that I was using the Aolserver version included in the tar ball aolserver-4.5.1-src.tar.gz dated 2009-02-02. Well. ehm... the version in the tar ball compiles and runs well under Windows. Not so for the version under CVS HEAD: many of the changes introduced have been implemented with a careful eye only for *nix, and not for Windows. By looking at the code I have the feeling that the interest in supporting Windows is fading down. To have things in perspective. please go to the ]project-open[ web site, go to the download section and see how many people have downloaded the Windows version and how many have downloaded any of the other version. I'm not a big organization, I cannot influence you, but I have to say I am very sorry and sad for this. evolution.. Being so small I don't have the energy to look at all your changes and additions and make sure they work properly on Windows. If this trend continues I will be forced to generate a sort of fork and incorporate, add the new changes, the new additions to my code base on a case by case basis, driven by the actual needs. Any opinion? Thanks in advance, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress in Aolserver
Hello all, Some few clarifications, I am afraid I wasn't clear. The compilation in itself was rather simple and went almost ok. The issue was in the behavior in the testing Some of the code, eg, function calls to the networking parts and so on, are working in the tar ball version, but cause the system to hang forever in CVS HEAD version. For the time being, I need to make a delivery rather quickly, I added Brian and Gustaf mods about the upload of large objects to the version in the tar ball, leaving all the other changes out. I might have a look at the other changes, but this is first of all a testing and debugging activity, very time consuming... I will come back to the community when I find something For Jeff, here you can see the numbers: http://sourceforge.net/projects/project-open/files/project-open/V3.5/ Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress in Aolserver
Hi Jim, Once again I'll take ]project-open[ as example. Please look in here: http://sourceforge.net/projects/project-open/files/project-open/V3.5/ How big is the Windows Installer (which installs on both Windows 64 and Windows 32 systems)? How big is the VMware Image? Well in some companies/organizations (believe it or not) Linux is not an accepted platform. Even a VMware appliance with Linux on it won't be accepted because to maintain it the company is obliged to acquire personnel knowledgeable on the system. So if you want to spread the users base of your OpenACS/Aolserver based application, having it running also on Windows may be very helpful. Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Jim Davidson Sent: 03 August 2011 21:21 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Progress in Aolserver Hi, As Dossy mentioned, we spent some time trying to support the win32 port. That included a goofy Tcl-based config thing (which also helped to verify a workable Tcl installation) and a dose of ifdef's and compatibility code. In general, the AOLserver code base is decidedly Unix -- any Win32 specific stuff is made to make Win32 look like Unix (i.e., the thread library, directly reading emulations, etc). I could take a look again but suspect it will take me a day or more just to the muck of Win32 development working again on my Mac. Given the quality of the various virtualization stuff (VMware, Virtual Box, etc.), would it be smarter to just double down on Unix and spend some time with some bundled Unix-in-box for Windows type installs? -Jim On Aug 3, 2011, at 12:30 PM, Dossy Shiobara wrote: I speak only for myself ... I personally have thoughtfully cared for the Windows support in AOLserver -- once upon a time, I had built the Windows binaries that some folks were using. Through discussions I had with Jim Davidson, the new build mechanism for AOLserver 4.5.x was meant to make building AOLserver on Windows easier than it had been in the past. I suspect that any recent changes to AOLserver which cause it to no longer build cleanly on Windows is likely an act of ignorance (i.e., not knowing what will or won't work on Windows) than an act of malice (i.e., not caring about Windows). My guess is that the folks who contribute changes do not have the means to test their changes on a Windows platform. It would be fantastic if the people who use AOLserver on Windows could port the changes such that they also work on Windows, either by replacing changes with more portable code, or providing the necessary platform-specific implementation surrounded by #ifdef __WIN32__ as needed, etc. On 8/3/11 1:19 PM, Maurizio Martignano wrote: By looking at the code I have the feeling that the interest in supporting Windows is fading down. [...] Any opinion? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear all, Id like to provide you with very few examples to explain what I was talking about: These problems manifested themselves in the Win64 version driver.c void NsWaitDriversShutdown(Ns_Time *toPtr) { Driver *drvPtr = firstDrvPtr; int status = NS_OK; while (drvPtr != NULL) { Ns_MutexLock(drvPtr-lock); while (!(drvPtr-flags DRIVER_STOPPED) status == NS_OK) { status = Ns_CondTimedWait(drvPtr-cond, drvPtr-lock, toPtr); } Ns_MutexUnlock(drvPtr-lock); if (status != NS_OK) { Ns_Log(Warning, driver: shutdown timeout: %s, drvPtr-module); } else { Ns_Log(Notice, driver: stopped: %s, drvPtr-module); Ns_ThreadJoin(drvPtr-thread, NULL);ß WAITS FOREVER drvPtr-thread = NULL; } drvPtr = drvPtr-nextPtr; } } And then: static void TriggerDriver(Driver *drvPtr) { if (send(drvPtr-trigger[1], , 1, 0) != 1) { ß ALWAYS ALWAYS RETURNS AN ERROR Ns_Fatal(driver: trigger send() failed: %s, ns_sockstrerror(ns_sockerrno)); } } Ns.h typedef struct Driver { /* * Visible in Ns_Driver. */ void *arg; /* Driver callback data. */ char *server; /* Virtual server name. */ char *module;/* Driver module. */ char*name;/* Driver name, e.g., nssock. */ char*location; /* Location, e.g, http://foo:9090; */ char*address; /* Address in location. */ intsendwait; /* send() I/O timeout. */ intrecvwait;/* recv() I/O timeout. */ int bufsize; /* Conn bufsize (0 for SSL) */ int sndbuf; /* setsockopt() SNDBUF option. */ int rcvbuf; /* setsockopt() RCVBUF option. */ /* * Private to Driver. */ struct Driver *nextPtr; /* Next in list of drivers. */ struct NsServer *servPtr; /* Driver virtual server. */ char *fullname; /* Full name, i.e., server/module. */ int flags; /* Driver state flags. */ Ns_Thread thread;/* Thread id to join on shutdown. */ Ns_Mutex lock; /* Lock to protect lists below. */ Ns_Cond cond;/* Cond to signal reader threads, * driver query, startup, and shutdown. */ inttrigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) Ns_DriverProc *proc; /* Driver callback. */ int opts; /* Driver options. */ etc . I have the feeling that we are not facing Windows specific stuff, but somehow some general errors, mistakes that happen to manifest themselves only on Windows (64). Hope it clarifies the situation, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Don, I went back to my archives This is the situation: 1. the code in CVS had always int trigger[2]; 2. I took the version 4.5.1 from the tar ball dated 2009-02-02 and I did the change SOCKET int trigger[2]; to make it work 3. then I recently took the Aolserver code from CVS Head and forgot to make the above change and so I started to observe problems in the networking code 4. re-inserted the change and everything seems to work... but I will test it and very deeply. 5. I have to disagree with your statement A Unix pipe is just a pair of file descriptors, and a file descriptor in Unix is just an integer. If I look at the code base (and please do the same with me)... check nscp/nscp.c nsd/binder.c, fd.c, listen.c, sock.c, tclfile.c, tclhttp.c, urlopen.c nsext/nsext.c nspd/nspd.c nsproxy/nsproxylib.c nssock/nssock.c nssl/nssl.c Well not always sockets are declared/defines as SOCKET as they should be (and this I am afraid also in Unix) This is a problem of proper data type selection and usage, it is mostly a consistency problem. I would humbly suggest that one of the code maintainers takes a walk on the code base and makes sure that everywhere we need a socket we have a real SOCKET socket and not an int socket. The code will be more consistent and more portable to Windows. Ciao, Maurizio PS: the attached text file shows where the keyword SOCKET is currently used... -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 14:53 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 12:24 AM, Maurizio Martignano wrote: inttrigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) A Unix pipe is just a pair of file descriptors, and a file descriptor in Unix is just an integer. Windows pipes are implemented differently, of course, and the semantics aren't exactly the same, either. Can you tell from CVS who put this code in? I'm sure it was done from ignorance of Windows vs. Unix differences regarding pipes, not intentionally. I have the feeling that we are not facing Windows specific stuff, but somehow some general errors, mistakes that happen to manifest themselves only on Windows (64). Not in this case, no, that's perfectly reasonable Unix code. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. * Socket descriptor or INVALID_SOCKET on error. SOCKET SOCKET sock = INVALID_SOCKET; ./binder.c SOCKET sock; static Sock *SockAccept(SOCKET lsock, Driver *drvPtr); static int Poll(PollData *pdataPtr, SOCKET sock, int events, Ns_Time *timeoutPtr); Ns_QueueWait(Ns_Conn *conn, SOCKET sock, Ns_QueueWaitProc *proc, SOCKET lsock; if (lsock != INVALID_SOCKET) { if (lsock != INVALID_SOCKET) { Poll(PollData *pdataPtr, SOCKET sock, int events, Ns_Time *timeoutPtr) SockAccept(SOCKET lsock, Driver *drvPtr) if (sockPtr-sock == INVALID_SOCKET) { setsockopt(sockPtr-sock, SOL_SOCKET, SO_SNDBUF, setsockopt(sockPtr-sock, SOL_SOCKET, SO_RCVBUF, sockPtr-sock = INVALID_SOCKET; ./driver.c ns_sockpair(SOCKET *socks) SOCKET sock; if (sock == INVALID_SOCKET || if (socks[1] == INVALID_SOCKET || if (socks[0] == INVALID_SOCKET) { ./fd.c SOCKET new, sock; if (sock == INVALID_SOCKET) { if (sock == INVALID_SOCKET) { ListenProc(SOCKET sock, void *arg, int why) SOCKET new; if (new != INVALID_SOCKET) { ./listen.c * 0 or SOCKET_ERROR. ns_socknbclose(SOCKET sock) return SOCKET_ERROR; SOCKET ns_sockdup(SOCKET sock) return INVALID_SOCKET; return (SOCKET) dup; ./nswin32.c static SOCKET SockConnect(char *host, int port, char *lhost, int lport, int async); static SOCKET SockSetup(SOCKET sock); Ns_SockRecv(SOCKET sock, void *buf, int toread, int timeout) Ns_SockSend(SOCKET sock, void *buf, int towrite, int timeout) Ns_SockWait(SOCKET sock, int what, int seconds) Ns_SockWaitEx(SOCKET sock, int what, int ms) SOCKET SOCKET SOCKET sock; if (sock == INVALID_SOCKET) { if (sock != INVALID_SOCKET listen(sock, backlog) != 0) { sock = INVALID_SOCKET; SOCKET Ns_SockAccept(SOCKET lsock, struct sockaddr *saPtr, int *lenPtr) SOCKET sock; if (sock != INVALID_SOCKET) { SOCKET SOCKET SOCKET sock; if (sock != INVALID_SOCKET
Re: [AOLSERVER] Aolserver Progress - Some few examples....
It is not a matter of understanding It is a matter of testing On Windows 64 int trigger[2] doesnt work whereas SOCKET trigger[2] does work. On top of that in several other places SOCKET has been used, so if for no other reason, I suggest one of the code maintainers takes a proper walk on the code base and make sure that all sockets are SOCKET sockets and not int sockets Ciao, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 04 August 2011 15:51 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples It's probably safer to define this as SOCKET, but windows.h says SOCKET is: typedef u_int SOCKET; And: typedef unsigned intu_int; Since Windows is LLP64 and most Unix-like systems are LP64, I don't understand how AOLserver's defining trigger[2] as (int) is the problem -- Windows might complain about some signed/unsigned thing at compile time, but in both cases, (int) is 32 bits. On 8/4/11 3:24 AM, Maurizio Martignano wrote: inttrigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dossy, It is irrelevant... Absolutely irrelevant.. With int trigger[2] static void TriggerDriver(Driver *drvPtr) { if (send(drvPtr-trigger[1], , 1, 0) != 1) { The send doesn't work and always returns error With SOCKET trigger[2]; It DOES Work... Back to your question: The program should be: #include windows.h #include winsock.h int main(int argc, char* argv[]) { printf(sizeof(SOCKET) = %d, sizeof(int) = %d\n, sizeof(SOCKET), sizeof(int)); return 0; } Its output when compiled as Windows 64 is: c:\Itmpcl test.c Microsoft (R) C/C++ Optimizing Compiler Version 16.00.30319.01 for x64 Copyright (C) Microsoft Corporation. All rights reserved. test.c Microsoft (R) Incremental Linker Version 10.00.30319.01 Copyright (C) Microsoft Corporation. All rights reserved. /out:test.exe test.obj c:\Itmptest sizeof(SOCKET) = 8, sizeof(int) = 4 and when compiled with Windows 32 C:\Itmpcl test.c Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86 Copyright (C) Microsoft Corporation. All rights reserved. test.c Microsoft (R) Incremental Linker Version 10.00.30319.01 Copyright (C) Microsoft Corporation. All rights reserved. /out:test.exe test.obj C:\Itmptest sizeof(SOCKET) = 4, sizeof(int) = 4 C:\Itmp I hope you understand my point now (thank you for your educational example which just proves it.) NOW BACK TO US 1. The Aolserver code base inconsistently uses int and SOCKET when declaring /defining variables to be used as Sockets. 2. This inconsistency it is not a Windows/Unix problem - it is just a plain inconsistency 3. We are just lucky that Windows 64 exposed the problem. 4. I do recommend that one of the code maintainers takes a walk on code base and makes all the sockets declarations/definitions consistent. Thank you, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 04 August 2011 16:56 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Win64, can you tell me what sizeof(SOCKET) and sizeof(int) are? Try this simple program: #include windows.h #include winsock2.h int main(int argc, char[] *argv) { printf(sizeof(SOCKET) = %d, sizeof(int) = %d\n, sizeof(SOCKET), sizeof(int)); return 0; } I just learned that on Win64, Microsoft *changed* the definition of SOCKET from u_int to UINT_PTR: /* * The new type to be used in all * instances which refer to sockets. */ #ifdef _WIN64 typedef UINT_PTRSOCKET; #else typedef u_int SOCKET; #endif Which means 64-bit SOCKET on Win64 vs. 32-bit SOCKET on Win32 or most Unix-like systems. ... The short answer is, yes, anywhere we have a variable that contains a handle to a socket, we should use the SOCKET type, not a naked int declaration, for portability. On 8/4/11 10:28 AM, Maurizio Martignano wrote: It is not a matter of understanding It is a matter of testing On Windows 64 int trigger[2] doesn't work whereas SOCKET trigger[2] does work. On top of that in several other places SOCKET has been used, so if for no other reason, I suggest one of the code maintainers takes a proper walk on the code base and make sure that all sockets are SOCKET sockets and not int sockets. -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Don, If you follow the last discussions even Dossy agrees that a SOCKET is not an int on Windows 64 All of this depends on the week type system of C, were types with different names, supposed to be used for different needs are considered equivalent is their size is the same. If we had used Ada none of this would have had happened: types with different names are different no matter what their size is. Anyhow in the base code 95% of the times or even more sockets are declared as SOCKET sockets. Here and there they are declared as int. This is an inconsistency and it should be removed. I do beg the community to do this little change because it is in the benefit and interest of everybody. Hope it helps, Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 18:07 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 7:20 AM, Maurizio Martignano wrote: 5. I have to disagree with your statement A Unix pipe is just a pair of file descriptors, and a file descriptor in Unix is just an integer. Feel free to disagree with the official Linux documentation then: http://www.kernel.org/doc/man-pages/online/pages/man2/pipe.2.html Note int pipefd[2] file descriptors in Unix are integers. Dossy says this is the same in Windows (I wouldn't know), perhaps the problem lies in the semantics, IIRC windows doesn't support anonymous pipes, for instance. (But I could be wrong about that . but I'm not wrong about Unix fds being ints) Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Don In Aolserver source code 95% of more of the times sockets are declared as SOCKET; the other times as int. This is an inconsistency and is a fact. If you wanted to develop only for Unix why did you use SOCKET in some occasions and int in some others? The source code is inconsistent and it just happens to work on Unix because there SOCKET and int have the same size. And this is also a fact. But I believe we should stop here, I admit all the faults you want, but please let's use SOCKET everywhere Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 19:25 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 9:55 AM, Maurizio Martignano wrote: All of this depends on the week type system of C, were types with different names, supposed to be used for different needs are considered equivalent is their size is the same. If we had used Ada none of this would have had happened: types with different names are different no matter what their size is. If we were using Ada a file descriptor in Unix would still be described as an integer. I'm not saying that the Unix code you've uncovered is portable between Unix and Windows. I'm just pointing out that pipes are defined as an array of two integers in Unix, so that the code isn't wrong for Unix, as you originally claimed. I did so hoping it would increase your understanding, i.e. your claim that it appears to be a bug even in Unix is incorrect. If you want to make progress here, just accept that the code is perfectly good Unix code and then figure out how to make the code work for both Unix and Windows, instead of trying to argue incorrectly that the code's not correct for Unix. It's not portable, but it's correct for Unix. Thank you. Anyhow in the base code 95% of the times or even more sockets are declared as SOCKET sockets. Here and there they are declared as int. This is an inconsistency and it should be removed. I do beg the community to do this little change because it is in the benefit and interest of everybody. I'm sure that the community will accept a patch that declares the pipe in a way that makes both Unix and Windows happy if you'll provide one. Meanwhile, quit complaining because I pointed out that, in Unix, int pipefd[2] is the correct declaration for a pipe. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
We are reasoning too much... 1. Compiling the code on Windows 64 made clear there's some inconsistency in the code... 2. This inconsistence is on how sockets are declared: 95% and more of the times as SOCKET and the rest of the times as int 3. On UNIX and WIN32 no problem cause SOCKET and int have the same size 4. On WIN64 there's a problem cause SOCKET and int do not have the same size and the helpful example Dossy made and I compiled on Win32 and Win34 elegantly shows that. 5. We have a good opportunity to clean the code, making it more consistent and more portable... 6. I insist on the need to make this little walk on the code base to make it consistent... We should use everywhere SOCKET. What are we waiting for? :-) :-) :-) -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Jim Davidson Sent: 04 August 2011 20:22 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Hi It's a socket so it can be monitored by select and poll. It should be SOCKET, I think it was in the past. On windows lib-c file handles returned by _open aren't the same as sockets. You can see this in the libc source Microsoft provides. They can't be monitored with select. The ns_sockpair is a pipe-like thing on windows by using socket calls to actually listen and connect on a localhost network socket. Net: seems I does need to change to SOCKET and that should have no effect on unix. Warning: I'm going on memory, not looking at the code. Jim Sent from a phone On Aug 4, 2011, at 11:34 AM, Jeff Rogers dv...@diphi.com wrote: Dossy Shiobara wrote: It's probably safer to define this as SOCKET, but windows.h says SOCKET is: The source comment is misleading, because trigger is set up as a socket pair, not as a pipe. Not sure why it's this way, but there it is. And ns_sockpair is already prototyped as ns_sockpair(SOCKET *socks) so if this is a problem it really should be triggering a compiler warning about it. In any case, since unix defines SOCKET as int, it should make zero difference on the unix side to change the structure definition. -J typedef u_int SOCKET; And: typedef unsigned int u_int; Since Windows is LLP64 and most Unix-like systems are LP64, I don't understand how AOLserver's defining trigger[2] as (int) is the problem -- Windows might complain about some signed/unsigned thing at compile time, but in both cases, (int) is 32 bits. On 8/4/11 3:24 AM, Maurizio Martignano wrote: int trigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/| can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Rusty, I started very politely, gently... Stressing I was seeing that the code base was kind of separating, moving away from Windows support... (I did see how SOCKETwere used). Then I provided the examples Then I stressed the int trigger[2]; Then I made it clear and I am sorry that this was seen as a porting issue and not as an inconsistency... so I stressed again I am sorry if I have been... well as you describe in your mail but at least the point has been made Thank you. Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Rusty Brooks Sent: 04 August 2011 20:55 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Fine, SOCKET should be used everywhere. But also, you're kind of being a dick. Don't do that. Rusty On Aug 4, 2011, at 1:26 PM, Maurizio Martignano wrote: Don In Aolserver source code 95% of more of the times sockets are declared as SOCKET; the other times as int. This is an inconsistency and is a fact. If you wanted to develop only for Unix why did you use SOCKET in some occasions and int in some others? The source code is inconsistent and it just happens to work on Unix because there SOCKET and int have the same size. And this is also a fact. But I believe we should stop here, I admit all the faults you want, but please let's use SOCKET everywhere Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 19:25 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 9:55 AM, Maurizio Martignano wrote: All of this depends on the week type system of C, were types with different names, supposed to be used for different needs are considered equivalent is their size is the same. If we had used Ada none of this would have had happened: types with different names are different no matter what their size is. If we were using Ada a file descriptor in Unix would still be described as an integer. I'm not saying that the Unix code you've uncovered is portable between Unix and Windows. I'm just pointing out that pipes are defined as an array of two integers in Unix, so that the code isn't wrong for Unix, as you originally claimed. I did so hoping it would increase your understanding, i.e. your claim that it appears to be a bug even in Unix is incorrect. If you want to make progress here, just accept that the code is perfectly good Unix code and then figure out how to make the code work for both Unix and Windows, instead of trying to argue incorrectly that the code's not correct for Unix. It's not portable, but it's correct for Unix. Thank you. Anyhow in the base code 95% of the times or even more sockets are declared as SOCKET sockets. Here and there they are declared as int. This is an inconsistency and it should be removed. I do beg the community to do this little change because it is in the benefit and interest of everybody. I'm sure that the community will accept a patch that declares the pipe in a way that makes both Unix and Windows happy if you'll provide one. Meanwhile, quit complaining because I pointed out that, in Unix, int pipefd[2] is the correct declaration for a pipe. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
We are in violent agreement... It was never my intention to raise the discussion to this level. I just observed the code. I may have used tones a bit too strong or too stressing... I never used bad words... I am going to provide a patch that will remove the inconsistencies I tried to explain... It will be very a very limited set of changes but it will make Aolserver more portable. Funny we are ready to fight and willing to kill each other on int trigger[2] versus SOCKET trigger[2] argument BTW: being called a dick is not very pleasant, and the only reason I do not react on this is that I'm interested in making Aolserver portable more than anything else. Thank you for your understanding and kind support. -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Don Baccus Sent: 04 August 2011 21:52 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples On Aug 4, 2011, at 11:26 AM, Maurizio Martignano wrote: If you wanted to develop only for Unix why did you use SOCKET in some occasions and int in some others? Damnit, that was never the point. You said that you thought that the code was incorrect for Unix, as well, a bug lurking that just hadn't caught anyone yet. That's flat out wrong. That declaration is standard Unix, and as I tried to point out previously, the mistake was probably made by someone who wasn't aware that it's not portable to all versions of Windows. End of story. That's all I said. All this other stuff you're ranting about is totally unrelated to my point. As Rusty said, you're being a dick. I don't bother trying to help dicks. So I'll no longer try to help you. Don Baccus http://donb.photo.net http://birdnotes.net http://openacs.org -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
I am doing the scan and I am preparing the patch. I will have also to do the testing as some of my customers do require the Windows 64 version. And I may offer to do this testing also in the future, non on a continuous basis, but every now and then. Ciao and thanks, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Jim Davidson Sent: 05 August 2011 00:43 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Hi, I'm looking at the code now -- definitely needs to be SOCKET in nsd.h. The reason can be seen in ns_sockpair in fd/sock.c where the code for a socket pair is done. It's just a wrapper around Unix socketpair() but has a bunch of extra code to do the loopback-connect thing on Windows. The socks are passed by reference -- this code thinks it's 16 bytes (i.e., 2 8-byte long ints) but it's getting 2 4-bytes from the trigger in the driver code. It's could be overwriting some more of the driver structure -- not clear. Note that ns_pipe != ns_sockpair. An ns_pipe is used to create process pipelines with the lib-c code on Windows. On Windows, it can't be used a s trigger pipe. The fix should be just updating nsd.h with SOCKET. I'd do it but I don't have any of my dev environment setup -- something I was going to finally dig into in a few weeks. Anyone else with commit could handle the code update. Compiling and testing on 64-bit Windows would be more work and require the Visual Studio stuff -- the Express C++ stuff appears to only support 32-bit. BTW: The basic strategy for Windows was to make it look as much like Unix as possible, as lightly as possible. This is why you get some weirdness like the opendir/readdir/closedir emulation code stuffed in the Windows thread library (nsthread/winthread.c). There's also quite a bit of ifdef WIN stuff in the process exec code (nsd/exec.c) and some file management code (nsd/fd.c). The use of trigger pipes is a Unix way of monitoring events -- I think on Windows you could WaitForMultipleHandles with all sorts of things, including network and process sync stuff, but since the goal was to be Unix-like, instead of modifying the event-driven code to eliminate socketpair's for wakeup, this ns_sockpair emulation was used instead. Also, in later versions Windows was being ignored -- you see this in the oversight of SOCKET for the driver trigger we've been talking about and the NsMap code not yet implemented for Windows. I agree a scan of all SOCKETs not being declared are SOCKET's makes good sense. -Jim On Aug 4, 2011, at 8:28 AM, Maurizio Martignano wrote: It is not a matter of understanding It is a matter of testing On Windows 64 int trigger[2] doesnt work whereas SOCKET trigger[2] does work. On top of that in several other places SOCKET has been used, so if for no other reason, I suggest one of the code maintainers takes a proper walk on the code base and make sure that all sockets are SOCKET sockets and not int sockets Ciao, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 04 August 2011 15:51 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples It's probably safer to define this as SOCKET, but windows.h says SOCKET is: typedef u_int SOCKET; And: typedef unsigned intu_int; Since Windows is LLP64 and most Unix-like systems are LP64, I don't understand how AOLserver's defining trigger[2] as (int) is the problem -- Windows might complain about some signed/unsigned thing at compile time, but in both cases, (int) is 32 bits. On 8/4/11 3:24 AM, Maurizio Martignano wrote: inttrigger[2]; /* Wakeup trigger pipe. */ ß Why is this an int when it was a SOCKET (any justification) -- Dossy Shiobara | He realized the fastest way to change do...@panoptic.com | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on. (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Gustav, Thank you so much for you feedback. I just distributed a new mail with an explanation for the patches.. Sorry if it arrives too late.. I will answers your questions here below. Once again, sorry for the bad timing. Ciao, Maurizio From: Gustaf Neumann [mailto:neum...@wu-wien.ac.at] Sent: 05 August 2011 14:23 To: AOLserver Discussion Cc: Maurizio Martignano Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Dear Maurizio and all... i have updated cvs on sourceforge with most of your patches. A few points are questionable (see below). For me, it is still unclear, why 4.5.1 worked for you, but not the head version not. As far i can see, all socket usages were int the same way in 4.5.1, the variable triggers in nsd.h was defined like this at least since 2004. Maybe it was luck that the win version worked in 4.5.1 (different memory layout, different compilers / compiler options, ...) [MM] No.. Just Alzheimer. Me. When I first did the compilation I spotted the problem and I fixed it by changing int trigger[2] into SOCKET trigger[2]. Then I forgot completely about it.. Then the issue showed up again... when I downloaded the code again.. A few comments to the patches (i have omitted these): nsd/nsmain.c +#ifndef _WIN32 Tcl_Finalize(); +#endif [MM] As simple as that: TCL_Finalize never ends on Windows, so it prevents the process/service to stop properly. === If one does no Tcl_Finalize() one introduces a memory leak. [MM] The process/service is about to end anyhow. I believe that mo matter what the poor process/service does all its memory will be released by the OS. So this is not an issue. nsproxy/nsproxy.c -Tcl_FindExecutable(argv[0]); === The call to Tcl_FindExecutable() is required (at least in Tcl 8.5), otherwise tcl will crash (at least under unix like operating systems) [MM] Not my change, please look at my newest email and sorry if I cause you some inconvenience. same situations for the following changes.. Sorry... +#define uint32_t unsigned long +#define uint16_t unsigned short + +typedef void * caddr_t; === These defines should be most probably within an #ifdef static void -FatalExit(char *func) +NSP_FatalExit(char *func) === What's wrong with the static name FatalExit()? -static Proc *firstClosePtr = NULL; +static Proc *firstClosePtr; === What's wrong with the initialization of the static variable? [MM] absolutely nothing.. May be my bad usage of patch, again apologies. I have not updated the nsproxy changes, since these need more work. [MM] Perfect -gustaf neumann On 05.08.11 08:14, Maurizio Martignano wrote: Dear all, here you are with a first patch file and the zip containing the code base I'm using. The nspd module requires more work, but I'm not using it. The files in the zip archive do compile and seems to be working Windows 32 and Windows 64. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Gustav, I understand your concerns about Tcl_Finalize. but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test.. Have Aolserver / nsd running on a big application. observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio From: Gustaf Neumann [mailto:neum...@wu-wien.ac.at] Sent: 05 August 2011 15:52 To: Maurizio Martignano Cc: 'AOLserver Discussion' Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Dear Maurizio, i guess, everything is fine with you with the head version on sourceforge. For me, it is still unclear, why 4.5.1 worked for you, but not the head version not. As far i can see, all socket usages were int the same way in 4.5.1, the variable triggers in nsd.h was defined like this at least since 2004. Maybe it was luck that the win version worked in 4.5.1 (different memory layout, different compilers / compiler options, ...) [MM] No.. Just Alzheimer. Me. When I first did the compilation I spotted the problem and I fixed it by changing int trigger[2] into SOCKET trigger[2]. Then I forgot completely about it.. Then the issue showed up again... when I downloaded the code again.. ok, this explains it. === If one does no Tcl_Finalize() one introduces a memory leak. [MM] The process/service is about to end anyhow. I believe that mo matter what the poor process/service does all its memory will be released by the OS. So this is not an issue. Tcl_finalize shuts down Tcl; it calls the registered exit handlers and then it shuts down various Tcl subsystems. It is certainly not recommended to to omit, but i have not checked the exact consequences for aolserver. If it hangs, it is an indication that there is still something wrong in tcl and/or aolserver. Keep an eye open in this respect. nsproxy/nsproxy.c -Tcl_FindExecutable(argv[0]); === The call to Tcl_FindExecutable() is required (at least in Tcl 8.5), otherwise tcl will crash (at least under unix like operating systems) [MM] Not my change, please look at my newest email and sorry if I cause you some inconvenience. It was included in your first patch (maybe erratic). If this is not needed, fine. All the best, and many thanks for the feedback -gustaf neumann -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Progress in Aolserver
If that is ok, I am willing to take the burden to look at / look after these weirdnesses... Ciao, Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Jim Davidson Sent: 05 August 2011 18:05 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Progress in Aolserver Yup -- seems like it's worth keeping the Win32/Win64 port going. It does muck up the code a bit with ifdef's and there are a few weirdnesses with Windows to work around but I suppose enough effort has been done in the past (if not recently) that it wouldn't be too tough to maintain. -Jim On Aug 3, 2011, at 1:54 PM, Maurizio Martignano wrote: Hi Jim, Once again I'll take ]project-open[ as example. Please look in here: http://sourceforge.net/projects/project-open/files/project-open/V3.5/ How big is the Windows Installer (which installs on both Windows 64 and Windows 32 systems)? How big is the VMware Image? Well in some companies/organizations (believe it or not) Linux is not an accepted platform. Even a VMware appliance with Linux on it won't be accepted because to maintain it the company is obliged to acquire personnel knowledgeable on the system. So if you want to spread the users base of your OpenACS/Aolserver based application, having it running also on Windows may be very helpful. Cheers, Maurizio -Original Message- From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Jim Davidson Sent: 03 August 2011 21:21 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Progress in Aolserver Hi, As Dossy mentioned, we spent some time trying to support the win32 port. That included a goofy Tcl-based config thing (which also helped to verify a workable Tcl installation) and a dose of ifdef's and compatibility code. In general, the AOLserver code base is decidedly Unix -- any Win32 specific stuff is made to make Win32 look like Unix (i.e., the thread library, directly reading emulations, etc). I could take a look again but suspect it will take me a day or more just to the muck of Win32 development working again on my Mac. Given the quality of the various virtualization stuff (VMware, Virtual Box, etc.), would it be smarter to just double down on Unix and spend some time with some bundled Unix-in-box for Windows type installs? -Jim On Aug 3, 2011, at 12:30 PM, Dossy Shiobara wrote: I speak only for myself ... I personally have thoughtfully cared for the Windows support in AOLserver -- once upon a time, I had built the Windows binaries that some folks were using. Through discussions I had with Jim Davidson, the new build mechanism for AOLserver 4.5.x was meant to make building AOLserver on Windows easier than it had been in the past. I suspect that any recent changes to AOLserver which cause it to no longer build cleanly on Windows is likely an act of ignorance (i.e., not knowing what will or won't work on Windows) than an act of malice (i.e., not caring about Windows). My guess is that the folks who contribute changes do not have the means to test their changes on a Windows platform. It would be fantastic if the people who use AOLserver on Windows could port the changes such that they also work on Windows, either by replacing changes with more portable code, or providing the necessary platform-specific implementation surrounded by #ifdef __WIN32__ as needed, etc. On 8/3/11 1:19 PM, Maurizio Martignano wrote: By looking at the code I have the feeling that the interest in supporting Windows is fading down. [...] Any opinion? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Gustav, I understand perfectly than omitting the function is removing the symptom and not the cause, but process/service wise having Tcl_Finalize in that particular place (where the process/service is about to end) or not having it doesn't make any difference. The Operating System will take care of all resources de-allocation. Nevertheless, exactly for the reason you mentioned, that there may be some problems and that these problems may not be present only in that particular piece of code, I'll have a look into the function behavior. I suspect that it might be a matter of the order with which things are de-allocated. but I might be wrong. I'll come back on this. Thank you for your point, I will follow your recommendation. Ciao, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 06 August 2011 10:28 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Maurizio, Tcl_Finalize() is supposed to work, and if it does now work something is still broken in the windows version. Omitting Tcl_Finalize() is removeing the symptom, not the cause. It is not unlikely that something else will have the same problem due to this cause. When Tcl_Finalize() is not run, the registered exit handlers are not executed. How serious this is depends on the exit handlers. You are right, that the memory leak does not matter due to the shutdown. The difference is like between a graceful and an ungraceful shutdown. -gustaf On 05.08.11 16:29, Maurizio Martignano wrote: Dear Gustav, I understand your concerns about Tcl_Finalize. but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test.. Have Aolserver / nsd running on a big application. observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
YES. I do agree that executing or not executing the exit handlers may make a difference.. Just to help me in my troubleshooting can you tell me if and where these handlers are registered. I am digging into Tcl_Finalize.. J Thank you, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 06 August 2011 10:28 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Maurizio, Tcl_Finalize() is supposed to work, and if it does now work something is still broken in the windows version. Omitting Tcl_Finalize() is removeing the symptom, not the cause. It is not unlikely that something else will have the same problem due to this cause. When Tcl_Finalize() is not run, the registered exit handlers are not executed. How serious this is depends on the exit handlers. You are right, that the memory leak does not matter due to the shutdown. The difference is like between a graceful and an ungraceful shutdown. -gustaf On 05.08.11 16:29, Maurizio Martignano wrote: Dear Gustav, I understand your concerns about Tcl_Finalize. but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test.. Have Aolserver / nsd running on a big application. observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
It is me again. Well I noticed that the change I suggested about Tcl_Finalize did not make it into CVS HEAD. If it doesn't go there, I am afraid I will have to anyhow introduce it myself in my distribution. I need to have a working system. With that call still in, the service can't (CANNOT) be stopped gracefully. This is a matter of testing: Take the system, make it run with a real OpenACS based application (how about ]po[, or xowiki..) and see how it works and see how it interacts with the system. Does it start? Does it run? Does it stop properly? For the time being in Windows 64 that function needs to be out. Thanks a lot, Maurizio Thank you, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 06 August 2011 10:28 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Maurizio, Tcl_Finalize() is supposed to work, and if it does now work something is still broken in the windows version. Omitting Tcl_Finalize() is removeing the symptom, not the cause. It is not unlikely that something else will have the same problem due to this cause. When Tcl_Finalize() is not run, the registered exit handlers are not executed. How serious this is depends on the exit handlers. You are right, that the memory leak does not matter due to the shutdown. The difference is like between a graceful and an ungraceful shutdown. -gustaf On 05.08.11 16:29, Maurizio Martignano wrote: Dear Gustav, I understand your concerns about Tcl_Finalize. but it is called just when the process/service is about to end. Once it ends the OS takes charges and releases the process/service resources (memory included). You can make an easy test.. Have Aolserver / nsd running on a big application. observe the OS resources given to the process and released when I finishes. Do this twice: with Tcl_Finalize on and Tcl_Finalize commented out. And see if you can find any difference. Ciao, Maurizio -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Aolserver Progress - Some few examples....
Dear Dossy, Your proposal of your wrapper sounds good to me. Why do not we insert that in the codebase? Till we understand better the issue? Next week I am going to redo some testing also in Win32 and I will let you know.. Thank you very much, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 06 August 2011 17:39 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Could you build AOLserver with debugging symbols and run nsd.exe under a debugger, with Tcl_Finalize *NOT* commented out/removed (i.e., as it currently is in CVS HEAD) and confirm where this 0x4015 exception is actually happening? Not calling Tcl_Finalize at process exit means any callbacks registered with Tcl_CreateExitHandler will not fire. It is absolutely wrong to not call Tcl_Finalize for this reason. If this is causing a problem on Win64 at the moment, as a temporary measure, you could wrap the call to Tcl_Finalize with the appropriate #ifndef: #if !defined(_WIN64) Tcl_Finalize(); #endif // !_WIN64 But, on all other platforms where invoking Tcl_Finalize does work, it should absolutely be done. On 8/6/11 9:35 AM, Maurizio Martignano wrote: Dear Gustav (and all the others) I did some digging. And here are the results.. 1. Tcl_Finalize gets properly called in Windows 64 by tclsh (ok tlcsh85t.exe) at exit time 2. From within Aolserver it doesn't even get actually called, but at the act of calling an exception is generated inside the C/C++ runtime: Faulting application name: nsd.exe, version: 0.0.0.0, time stamp: 0x4e3d1e32 Faulting module name: MSVCR100.dll, version: 10.0.30319.415, time stamp: 0x4d26d15a Exception code: 0x4015 Fault offset: 0x00075fe9 Faulting process id: 0x1114 Faulting application start time: 0x01cc542b82cdba6b Faulting application path: C:\aolserver\bin\nsd.exe Faulting module path: C:\Windows\system32\MSVCR100.dll Report Id: e90de38e-c01e-11e0-9d90-cef6f702c08b 3. Looking at the code in TCL I believe (and here I repeat I BELIEVE) the problem is in the TCL DLL initialization code: tclWin32Dll.c case DLL_PROCESS_DETACH: /* * Protect the call to Tcl_Finalize. The OS could be unloading us from * an exception handler and the state of the stack might be unstable. */ if defined(HAVE_NO_SEH) !defined(_WIN64) __asm__ __volatile__ ( /* * Construct an EXCEPTION_REGISTRATION to protect the call to * Tcl_Finalize */ leal %[registration], %%edx \n\t movl %%fs:0, %%eax\n\t movl %%eax,0x0(%%edx) \n\t /* link */ leal 1f, %%eax\n\t movl %%eax,0x4(%%edx) \n\t /* handler */ movl %%ebp,0x8(%%edx) \n\t /* ebp */ movl %%esp,0xc(%%edx) \n\t /* esp */ movl %[error], 0x10(%%edx) \n\t /* status */ /* * Link the EXCEPTION_REGISTRATION on the chain */ movl %%edx,%%fs:0 \n\t /* * Call Tcl_Finalize */ call _Tcl_Finalize\n\t /* * Come here on a normal exit. Recover the EXCEPTION_REGISTRATION * and store a TCL_OK status */ movl %%fs:0, %%edx\n\t movl %[ok],%%eax\n\t movl %%eax,0x10(%%edx) \n\t jmp2f \n /* * Come here on an exception. Get the EXCEPTION_REGISTRATION that * we previously put on the chain. */ 1:\t movl %%fs:0, %%edx\n\t movl 0x8(%%edx), %%edx\n /* * Come here however we exited. Restore context from the * EXCEPTION_REGISTRATION in case the stack is unbalanced. */ 2:\t movl 0xc(%%edx), %%esp\n\t movl 0x8(%%edx), %%ebp\n\t movl 0x0(%%edx), %%eax\n\t movl %%eax,%%fs:0 \n\t : /* No outputs */ : [registration] m (registration), [ok] i (TCL_OK), [error] i (TCL_ERROR) : %eax, %ebx, %ecx, %edx, %esi, %edi, memory ); #else #ifndef HAVE_NO_SEH __try { #endif Tcl_Finalize(); #ifndef HAVE_NO_SEH } __except (EXCEPTION_EXECUTE_HANDLER) { /* empty handler body. */ } #endif #endif break 4. My personal point of view a. This portion of code is too target specific, it uses assembly, some of these things may become obsolete or wrong when for instance moving
Re: [AOLSERVER] Aolserver Progress - Some few examples.... - Errata Corrige
Hello all I did some tests on Windows 32. Tcl_Finalize prevents the proper stopping of the service also on Windows 32. So the proper mod was and still is: // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Tcl_Finalize(); #endif Tcl_Finalize doesn't work also on the 32 bit code. Ciao, Maurizio Dear Dossy, Your proposal of your wrapper sounds good to me. Why do not we insert that in the codebase? Till we understand better the issue? Next week I am going to redo some testing also in Win32 and I will let you know.. Thank you very much, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Dossy Shiobara Sent: 06 August 2011 17:39 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Could you build AOLserver with debugging symbols and run nsd.exe under a debugger, with Tcl_Finalize *NOT* commented out/removed (i.e., as it currently is in CVS HEAD) and confirm where this 0x4015 exception is actually happening? Not calling Tcl_Finalize at process exit means any callbacks registered with Tcl_CreateExitHandler will not fire. It is absolutely wrong to not call Tcl_Finalize for this reason. If this is causing a problem on Win64 at the moment, as a temporary measure, you could wrap the call to Tcl_Finalize with the appropriate #ifndef: #if !defined(_WIN64) Tcl_Finalize(); #endif // !_WIN64 But, on all other platforms where invoking Tcl_Finalize does work, it should absolutely be done. On 8/6/11 9:35 AM, Maurizio Martignano wrote: Dear Gustav (and all the others) I did some digging. And here are the results.. 1. Tcl_Finalize gets properly called in Windows 64 by tclsh (ok tlcsh85t.exe) at exit time 2. From within Aolserver it doesn't even get actually called, but at the act of calling an exception is generated inside the C/C++ runtime: Faulting application name: nsd.exe, version: 0.0.0.0, time stamp: 0x4e3d1e32 Faulting module name: MSVCR100.dll, version: 10.0.30319.415, time stamp: 0x4d26d15a Exception code: 0x4015 Fault offset: 0x00075fe9 Faulting process id: 0x1114 Faulting application start time: 0x01cc542b82cdba6b Faulting application path: C:\aolserver\bin\nsd.exe Faulting module path: C:\Windows\system32\MSVCR100.dll Report Id: e90de38e-c01e-11e0-9d90-cef6f702c08b 3. Looking at the code in TCL I believe (and here I repeat I BELIEVE) the problem is in the TCL DLL initialization code: tclWin32Dll.c case DLL_PROCESS_DETACH: /* * Protect the call to Tcl_Finalize. The OS could be unloading us from * an exception handler and the state of the stack might be unstable. */ if defined(HAVE_NO_SEH) !defined(_WIN64) __asm__ __volatile__ ( /* * Construct an EXCEPTION_REGISTRATION to protect the call to * Tcl_Finalize */ leal %[registration], %%edx \n\t movl %%fs:0, %%eax\n\t movl %%eax,0x0(%%edx) \n\t /* link */ leal 1f, %%eax\n\t movl %%eax,0x4(%%edx) \n\t /* handler */ movl %%ebp,0x8(%%edx) \n\t /* ebp */ movl %%esp,0xc(%%edx) \n\t /* esp */ movl %[error], 0x10(%%edx) \n\t /* status */ /* * Link the EXCEPTION_REGISTRATION on the chain */ movl %%edx,%%fs:0 \n\t /* * Call Tcl_Finalize */ call _Tcl_Finalize\n\t /* * Come here on a normal exit. Recover the EXCEPTION_REGISTRATION * and store a TCL_OK status */ movl %%fs:0, %%edx\n\t movl %[ok],%%eax\n\t movl %%eax,0x10(%%edx) \n\t jmp2f \n /* * Come here on an exception. Get the EXCEPTION_REGISTRATION that * we previously put on the chain. */ 1:\t movl %%fs:0, %%edx\n\t movl 0x8(%%edx), %%edx\n /* * Come here however we exited. Restore context from the * EXCEPTION_REGISTRATION in case the stack is unbalanced. */ 2:\t movl 0xc(%%edx), %%esp\n\t movl 0x8(%%edx), %%ebp\n\t movl 0x0(%%edx), %%eax\n\t movl %%eax,%%fs:0 \n\t : /* No outputs */ : [registration] m (registration), [ok] i (TCL_OK), [error] i (TCL_ERROR) : %eax, %ebx, %ecx, %edx, %esi, %edi, memory ); #else
Re: [AOLSERVER] Aolserver Progress - Some few examples.... All's well that ends well
Dear Gustav, Thank you. Alls well that ends well Im not sure all the changes I suggested are still in the codebase especially the ones in RED It is not up to Aolserver/nsd include system to define SOCKET as int on Windows. Thank you again, Maurizio include/ns.h 222 ß #define O_BINARY0 // Commented out by M. Martignano on the 05/08/2011 // #define SOCKET int #define INVALID_SOCKET (-1) I removed the define, in Windows (32/64) SOCKET should be defined somewhere else, and not redefined here. include/ns.h 674 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 NS_EXTERN SOCKET Ns_ConnSock(Ns_Conn *conn); Self explanatory. nsd/conn.c - 615 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 SOCKET Ns_ConnSock(Ns_Conn *conn) { Self explanatory nsd/nsd.h - 75 // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 struct pollfd { int fd; short events; short revents; }; #endif struct pollfd is already defined nsd/nsd.h - 312 // int trigger[2] changed into SOCKET trigger[2] by M. Martignano on the 05/08/2011 SOCKET trigger[2]; /* Wakeup trigger socket. */ Well, we all know about this dont we? nsd/nsmain.c 633 // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Tcl_Finalize(); #endif Semantic. Tcl_Finalize never ends on Windows so I removed it. nsd/queue.c 42 // Function added by M. Martignano on the 05/08/2011 #ifdef _WIN32 static double round(double x) { return floor(x + 0.5); } #endif Well believe it or not round does not exist. So I ehm re-implemented it based on floor. nspd/listen.c 80 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 SOCKETsock, new; Self explanatory nspd/pd.h 52 ß #include unistd.h // Conditional compilation added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Well this follows the same reasoning used for ns.h. Anyhow this is the weakest of all my changes and all the module should be carefully looked at. I am personally not using it in my Windows distributions. nsperm/nsperm.c 583 // Conditional compilation added by M. Martignano on the 05/08/2011 #ifndef _WIN32 if (inet_aton(net, ip) == 0 || inet_aton(slash+1, mask) == 0) { #else if (inet_pton(AF_INET,net, ip) == 0 || inet_pton(AF_INET,slash+1, mask) == 0) { #endif Well with Micosoft Visual C 10 inet_aton seems not to be there any longer. Luckily we can still rely on inet_pton. From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 07 August 2011 15:23 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Dear all, i did some more digging/googling in this issue and i share the opinion that - at least for the time being - Tcl_Finalize() could be omitted on windows versions without too much harm. Some background: The Tcl manpage says: Tcl_Finalize is similar to Tcl_Exit except that it does not exit from the current process. It is useful for cleaning up when a process is finished using Tcl but wishes to continue executing, and when Tcl is used in a dynamically loaded extension that is about to be unloaded. On some systems Tcl is automatically notified when it is being unloaded, and it calls Tcl_Finalize internally; on these systems it not necessary for the caller to explicitly call Tcl_Finalize. However, to ensure portability, your code should always invoke Tcl_Finalize when Tcl is being unloaded, to ensure that the code will work on all plat- forms. Tcl_Finalize can be safely called more than once. For aolserver, it is questionable for me why we need Tcl_Finalize() (the primarily purpose for Tcl_Finalize according to its documentation is that the process wants to continue without Tcl), furthermore there seems to be some magic involved, that some systems call Tcl_Finalize() ... automatically (hinting most likely the windows situation with the assembly code). Since finalize tries to unload Tcl, there seems to be some race conditions in this area on windows, at least when there are still are multiple threads around. E.g. [1] says: Because DLL notifications are serialized, entry-point functions should not attempt to communicate with other threads or processes. Deadlocks may occur as a result. Neither aolserver 4.0.10 nor naviserver call Tcl_Finalize(), so i guess we can live with a 4.5.1+ version under windows without it. -gustaf neumann [1] http://msdn.microsoft.com/en-us/library/ms682583.aspx On 06.08.11 16:28, Maurizio Martignano wrote: It is me again Well I noticed that the change I suggested about Tcl_Finalize did not make it into CVS
Re: [AOLSERVER] Aolserver Progress - Some few examples.... All's well that ends well
Dear Gustav, You ARE ABSOLUTELY RIGHT! And I AM DEFINETELY A VICTIM OF ALZHEIMER. APOLOGIES Ciao, Maurizio From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 07 August 2011 19:02 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples All's well that ends well On 07.08.11 17:37, Maurizio Martignano wrote: Dear Gustav, Thank you. Alls well that ends well Im not sure all the changes I suggested are still in the codebase especially the ones in RED The reason, i have not committed theses two suggested changes to the code base is that these are incorrect and would break compilation on the non-windows platforms. Both suggested changes are in #ifdef branches for compilations without _WIN32 set (therefore irrelevant for Win32 and Win64). Please check more carefully first. -gustaf neumann It is not up to Aolserver/nsd include system to define SOCKET as int on Windows. Thank you again, Maurizio include/ns.h 222 ß #define O_BINARY0 // Commented out by M. Martignano on the 05/08/2011 // #define SOCKET int #define INVALID_SOCKET (-1) I removed the define, in Windows (32/64) SOCKET should be defined somewhere else, and not redefined here. include/ns.h 674 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 NS_EXTERN SOCKET Ns_ConnSock(Ns_Conn *conn); Self explanatory. nsd/conn.c - 615 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 SOCKET Ns_ConnSock(Ns_Conn *conn) { Self explanatory nsd/nsd.h - 75 // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 struct pollfd { int fd; short events; short revents; }; #endif struct pollfd is already defined nsd/nsd.h - 312 // int trigger[2] changed into SOCKET trigger[2] by M. Martignano on the 05/08/2011 SOCKET trigger[2]; /* Wakeup trigger socket. */ Well, we all know about this dont we? nsd/nsmain.c 633 // Conditional compilation clause added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Tcl_Finalize(); #endif Semantic. Tcl_Finalize never ends on Windows so I removed it. nsd/queue.c 42 // Function added by M. Martignano on the 05/08/2011 #ifdef _WIN32 static double round(double x) { return floor(x + 0.5); } #endif Well believe it or not round does not exist. So I ehm re-implemented it based on floor. nspd/listen.c 80 // Type changed from int to SOCKET by M. Martignano on the 05/08/2011 SOCKETsock, new; Self explanatory nspd/pd.h 52 ß #include unistd.h // Conditional compilation added by M. Martignano on the 05/08/2011 #ifndef _WIN32 Well this follows the same reasoning used for ns.h. Anyhow this is the weakest of all my changes and all the module should be carefully looked at. I am personally not using it in my Windows distributions. nsperm/nsperm.c 583 // Conditional compilation added by M. Martignano on the 05/08/2011 #ifndef _WIN32 if (inet_aton(net, ip) == 0 || inet_aton(slash+1, mask) == 0) { #else if (inet_pton(AF_INET,net, ip) == 0 || inet_pton(AF_INET,slash+1, mask) == 0) { #endif Well with Micosoft Visual C 10 inet_aton seems not to be there any longer. Luckily we can still rely on inet_pton. From: AOLserver Discussion [mailto:AOLSERVER@LISTSERV.AOL.COM] On Behalf Of Gustaf Neumann Sent: 07 August 2011 15:23 To: AOLSERVER@LISTSERV.AOL.COM Subject: Re: [AOLSERVER] Aolserver Progress - Some few examples Dear all, i did some more digging/googling in this issue and i share the opinion that - at least for the time being - Tcl_Finalize() could be omitted on windows versions without too much harm. Some background: The Tcl manpage says: Tcl_Finalize is similar to Tcl_Exit except that it does not exit from the current process. It is useful for cleaning up when a process is finished using Tcl but wishes to continue executing, and when Tcl is used in a dynamically loaded extension that is about to be unloaded. On some systems Tcl is automatically notified when it is being unloaded, and it calls Tcl_Finalize internally; on these systems it not necessary for the caller to explicitly call Tcl_Finalize. However, to ensure portability, your code should always invoke Tcl_Finalize when Tcl is being unloaded, to ensure that the code will work on all plat- forms. Tcl_Finalize can be safely called more than once. For aolserver, it is questionable for me why we need Tcl_Finalize() (the primarily purpose for Tcl_Finalize according to its documentation is that the process wants to continue without Tcl), furthermore there seems to be some magic involved, that some systems call