Re: [naviserver-devel] test ns_config-7.4.1, Stack around the variable 'filter' was corrupted
On Wed, Mar 15, 2023 at 03:57:40PM -0400, Andrew Piskorski wrote: > On Tue, Mar 14, 2023 at 04:44:21PM -0400, Andrew Piskorski wrote: > > > Debug Error! > > Program: C:\web\ns-fork-pub\naviserver\nsd\libnsd.dll > > Run-Time Check Failure #2 - Stack around the variable 'filter' was > > corrupted. > > > > I never see the usual "all.tcl" test summary output. Maybe nsd is > > hitting the above "Debug Error!" before getting there? > > On Windows, running the "ns_config.test" tests triggers that one. All > the tests through ns_config-7.4.0, pass, then it stops with no further > output. Yep, the next test, ns_config-7.4.1, is sufficient to trigger > the problem all by itself. On the current NaviServer head, that problem has gone away! The ns_config-7.4.1 test now runs fine for me on Windows. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] NaviServer 4.99.29 available
On Thu, Nov 02, 2023 at 03:30:43PM +0100, Gustaf Neumann wrote: > So far, these changes are only available on sourceforge, since i have > lost write access to the repository at bitbucket. Wow. Yes, please remove whatever group members are necessary to restore write access, and let us know when the public NaviServer Git repository is again up to date. (As you say, we can always send you patches via forks in our personal repositories on Bitbucket, if necessary.) -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] nscp segfaults server when missing module/nscp/users
Using the Naviserver head on Linux, the installed "conf/nsd-config.tcl"
and "conf/simple-config.tcl" config fieles both work fine for testing
that a newly compliled server starts up correctly. However, simple-config.tcl
includes a commented-out line loading the nscp module, like so:
#ns_param nscpnscp
If you uncomment that line, then the server crashes every time on startup.
If you ALSO copy this "module/nscp/users" section from nsd-config.tcl,
then the crash goes away and server again starts fine:
ns_section ns/server/default/module/nscp/users {
ns_param user "::"
}
Probably nscp shouldn't segfault if it's missing that section? Or
maybe we should either just add that section into simple-config.tcl,
or remove the commented-out nscp line entirely.
When the server crashes because of the missing nscp configuration, the
backtrace looked like this:
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x7f4a13c2c859 in __GI_abort () at abort.c:79
#2 0x7f4a13ea4288 in Panic () from
/usr/local/pkg/nsd-20230317-1/lib/libnsd.so
#3 0x7f4a13af358d in Tcl_PanicVA (format=,
argList=argList@entry=0x7ffcecfc0b40) at
/home/nobackup/co/tcl/tcl-core-8-6-branch/generic/tclPanic.c:99
#4 0x7f4a13af36ff in Tcl_Panic (format=) at
/home/nobackup/co/tcl/tcl-core-8-6-branch/generic/tclPanic.c:160
#5 0x7f4a13f096c7 in Abort () from
/usr/local/pkg/nsd-20230317-1/lib/libnsd.so
#6
#7 0x7f4a13ebd18d in Ns_SetFindCmp () from
/usr/local/pkg/nsd-20230317-1/lib/libnsd.so
#8 0x7f4a13ebd287 in Ns_SetFind () from
/usr/local/pkg/nsd-20230317-1/lib/libnsd.so
#9 0x7f4a13ebc951 in Ns_SetUpdateSz () from
/usr/local/pkg/nsd-20230317-1/lib/libnsd.so
#10 0x7f4a13e22a6e in LoadUsers () from
/home/local-20.04/pkg/nsd-20230317-1/bin/nscp.so
#11 0x7f4a13e22eb0 in Ns_ModuleInit () from
/home/local-20.04/pkg/nsd-20230317-1/bin/nscp.so
#12 0x7f4a13ea6e61 in Ns_ModuleLoad () from
/usr/local/pkg/nsd-20230317-1/lib/libnsd.so
#13 0x7f4a13ea7141 in NsTclModuleLoadObjCmd () from
/usr/local/pkg/nsd-20230317-1/lib/libnsd.so
--
Andrew Piskorski
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] test ns_config-7.4.1, Stack around the variable 'filter' was corrupted
On Thu, Mar 16, 2023 at 02:36:27PM -0400, Andrew Piskorski wrote:
> > > Debug Error!
> > > Program: C:\web\ns-fork-pub\naviserver\nsd\libnsd.dll
> > > Run-Time Check Failure #2 - Stack around the variable 'filter' was
> > > corrupted.
> I put the same code that the ns_config-7.4.1 test runs into a simple
> *.tcl page like this:
>
> set xx [ns_set array [ns_configsection -filter "unread" ns/testconfig]]
> ns_return 200 {text/plain} "Result: $xx"
I found this helpful article on trying to find the source of stack
corruption via memory access breakpoints:
https://www.timdbg.com/posts/debugger-lies-part-1/#memory-access-breakpoints
Below is my attempt to do that; I got stuck without clear results.
My memory access breakpoint fired, but the stack trace said we were
still in NsTclConfigSectionObjCmd(), seemingly still at the beginning
of that function, with just this cryptic (to me) output:
(41d0.f7c): Break instruction exception - code 8003 (first chance)
libnsd!failwithmessage+0x234:
7ff8`c7101364 cc int 3
I'm not very skilled at using WinDbg, so it's likely I missed or
misinterpreted something.
Here's what I did to get that far:
## I downloaded and installed Windows SDK 10.0.22621:
https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/
## That gave me WinDbg installed here:
C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\windbg.exe
## Build NaviServer on Windows my usual way:
"C:\Program Files (x86)\Microsoft Visual
Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat"
z: & cd Z:\src\web\ns-fork-pub\naviserver
nmake -f Makefile.win32 clean-core clean-mod
nmake -f Makefile.win32 all-core all-mod
# Fix permissions if necessary! e.g.:
# find . -type f \( -name "*.exe" -o -name "*.dll" \) -print | sudo xargs
chmod 775
nmake -f Makefile.win32 _install
nmake -f Makefile.win32 _test
## Start up nsd.exe under WinDbg:
"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\windbg.exe" -g
C:\P\nsd\nsd-fp_2023-03-15-2\bin\nsd.exe -t
Z:/src/web/ns-fork-pub/naviserver/tests/test.nscfg
## Tell WinDbg about my Naviserver code:
.srcpath Z:\src\web\ns-fork-pub\naviserver
.sympath
C:\P\nsd\nsd-fp_2023-03-15-2\lib;C:\P\Tcl-64-8.6.12\lib;SRV*C:\MyLocalSymbols*http://msdl.microsoft.com/download/symbols
!sym noisy
lml
.reload libnsd.dll
.reload nsd.exe
## Set breakpoint 0 on NsTclConfigSectionObjCmd():
bp NsTclConfigSectionObjCmd
g
## Trigger the bug. I like to use Cygwin like so:
wget -O - -q http://localhost:8000/atp-crash.tcl
## Now in WinDbg were are in NsTclConfigSectionObjCmd().
## Show stack:
k
## Show current stack pointer:
dx @$csp
## Set memory access breakpoint 1 on the stack pointer:
ba w 8 @$csp
bl
g
## This is the cryptic output I got:
(41d0.f7c): Break instruction exception - code 8003 (first chance)
libnsd!failwithmessage+0x234:
7ff8`c7101364 cc int 3
## Disable the stack pointer breakpoint:
bd 1
--
Andrew Piskorski
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] test ns_config-7.4.1, Stack around the variable 'filter' was corrupted
On Wed, Mar 15, 2023 at 03:57:40PM -0400, Andrew Piskorski wrote:
> On Tue, Mar 14, 2023 at 04:44:21PM -0400, Andrew Piskorski wrote:
>
> > Debug Error!
> > Program: C:\web\ns-fork-pub\naviserver\nsd\libnsd.dll
> > Run-Time Check Failure #2 - Stack around the variable 'filter' was
> > corrupted.
> On Windows, running the "ns_config.test" tests triggers that one. All
> the tests through ns_config-7.4.0, pass, then it stops with no further
> output. Yep, the next test, ns_config-7.4.1, is sufficient to trigger
> the problem all by itself.
I put the same code that the ns_config-7.4.1 test runs into a simple
*.tcl page like this:
set xx [ns_set array [ns_configsection -filter "unread" ns/testconfig]]
ns_return 200 {text/plain} "Result: $xx"
Hitting that web page, I the ns_return never runs, because nsd.exe has
broken before getting that far.
WinDbg says the stack corruption is happening in NsTclConfigSectionObjCmd()
(in "nsd/tclconf.c"), but it doesn't notice until it gets to the end
of that function. I think most of the work there is in Ns_ParseObjv(),
so maybe that, or something it calls, is the most likely place for
some sort of array bounds overrun to be hiding. I don't know though,
it could be anything that NsTclConfigSectionObjCmd() calls.
I don't know how else to further track down this bug, but if anybody
has further suggestions I'm willing to try them.
--
Andrew Piskorski
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] test ns_config-7.4.1, Stack around the variable 'filter' was corrupted
On Tue, Mar 14, 2023 at 04:44:21PM -0400, Andrew Piskorski wrote: > Debug Error! > Program: C:\web\ns-fork-pub\naviserver\nsd\libnsd.dll > Run-Time Check Failure #2 - Stack around the variable 'filter' was > corrupted. > > I never see the usual "all.tcl" test summary output. Maybe nsd is > hitting the above "Debug Error!" before getting there? On Windows, running the "ns_config.test" tests triggers that one. All the tests through ns_config-7.4.0, pass, then it stops with no further output. Yep, the next test, ns_config-7.4.1, is sufficient to trigger the problem all by itself. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] how to check if nsd has zlib?
On Windows, what's the best (simple?) way to check if my nsd is
actually linked correctly with zlib?
Reason I ask, is on Windows I was always getting warnings like this:
Warning: init server test: compress is enabled, but no zlib support built in
I've now installed zlib-1.2.13, and am playing with the build to try
and make Naviserver use it. That made the, "no zlib support built in"
warning go away, but I'm not sure if I'm linking zlib correctly. I
still get the three "compress" test failures below, but I don't know
if this means my zlib support is completely missing or broken, or some
other problem.
compress-3.3 ns_write streaming, compressed FAILED
Contents of test case:
set b [nstest::http-0.9 -http 1.0 -getbinary 1 -encoding binary
-setheaders {Accept-Encoding gzip} -getheaders {Content-Encoding Vary} GET
/compress]
list {*}[lrange $b 0 2] [llength [lindex $b end]] [lrange [lindex $b end]
end-15 end]
Result was:
200 gzip Accept-Encoding 37 {ff ff 52 48 54 28 49 2d 2e e1 02 00 00 00 ff ff}
Result should have been (exact matching):
200 gzip Accept-Encoding 47 {02 00 00 00 ff ff 03 00 12 13 05 72 0f 00 00 00}
compress-3.3 FAILED
compress-3.4 streaming adp, compressed FAILED
Contents of test case:
set b [nstest::http-0.9 -http 1.0 -getbinary 1 -encoding binary
-setheaders {Accept-Encoding gzip} -getheaders {Content-Encoding Vary} GET
/ns_adp_compress.adp?stream=1]
list {*}[lrange $b 0 2] [llength [lindex $b end]] [lrange [lindex $b end]
end-20 end]
Result was:
200 gzip Accept-Encoding 30 {0a 2a c9 c8 2c 56 00 a2 44 85 92 d4 e2 12 2e 00 00
00 00 ff ff}
Result should have been (exact matching):
200 gzip Accept-Encoding 40 {92 d4 e2 12 2e 00 00 00 00 ff ff 03 00 12 13 05 72
0f 00 00 00}
compress-3.4 FAILED
compress-3.5 ns_write streaming + HTTP 1.1 chunking, compressed FAILED
Contents of test case:
set b [nstest::http-0.9 -http 1.1 -getbinary 1 -encoding binary
-setheaders {Accept-Encoding gzip} -getheaders {Content-Encoding Vary
Transfer-Encoding Content-Length} GET /compress]
list {*}[lrange $b 0 3] [llength [lindex $b end]] [lrange [lindex $b end]
end-30 end]
Result was:
200 gzip Accept-Encoding chunked 44 {2a c9 c8 2c 56 c8 2c 06 00 00 00 ff ff 0a
65 0a 52 48 54 28 49 2d 2e e1 02 00 00 00 ff ff 0a}
Result should have been (exact matching):
200 gzip Accept-Encoding chunked 60 {52 48 54 28 49 2d 2e e1 02 00 00 00 ff ff
0a 61 0a 03 00 12 13 05 72 0f 00 00 00 0a 30 0a 0a}
compress-3.5 FAILED
--
Andrew Piskorski
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Windows test failures, http_chunked
On Tue, Mar 14, 2023 at 04:44:21PM -0400, Andrew Piskorski wrote: > I also see 4 "http_chunked" test failures, below: > http_chunked-1.1 ADP streaming w/chunks to HTTP/1.1 client FAILED > http_chunked-1.1 FAILED > http_chunked-1.3 ADP with longer partly-buffered response and > auto-streaming FAILED > http_chunked-1.3 FAILED > http_chunked-2.1 Tcl streaming w/chunks to HTTP/1.1 client FAILED > http_chunked-2.1 FAILED > http_chunked-2.1.1 Tcl streaming multiple binary buffers w/chunks to > HTTP/1.1 client FAILED > http_chunked-2.1.1 FAILED I am now using this change: https://bitbucket.org/naviserver/naviserver/commits/7e65faade74d4da66dcffd0f7c271d73f748b3a2 commit 7e65faade74d4da66dcffd0f7c271d73f748b3a2 Author: Gustaf Neumann Date: 2023-03-14 10:02:49 +0100 Tue ns_http: Fixed behavior of HEAD request for persistent connections. But despite that change, I still get those same 4 failures on Windows. Output looks the same, the output my nsd is sending ends in "01234\n", but the test is expecting it to be "01234\n0\n". -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] Windows test failures, http_chunked
On Tue, Mar 14, 2023 at 02:54:54PM -0400, Andrew Piskorski wrote:
>> [14/Mar/2023:13:17:45][16420.4aa0][-main:test-] Error: modload:
>> Z:/src/web/ns-fork-pub/naviserver/tests/../nslog/nslog.dll: couldn't load
>> library "Z:/src/web/ns-fork-pub/naviserver/tests/../nslog/nslog.dll":
>> permission denied
>> [14/Mar/2023:13:17:45][16420.4aa0][-main:test-] Fatal: modload: failed to
>> load module 'Z:/src/web/ns-fork-pub/naviserver/tests/../nslog/nslog.dll'
> In my case, trying to run on a network drive (over Samba) seems to be
> causing the "permission denied" problem.
Embarassingly, the problem there was simply that my compiled Windows
*.dll files, located on the Samba drive, lacked execute permission.
Some Samba configurations enforce that and some don't. With that
fixed I can actually look at real test failures.
I am still getting the pop-up I mentioned before:
Debug Error!
Program: C:\web\ns-fork-pub\naviserver\nsd\libnsd.dll
Run-Time Check Failure #2 - Stack around the variable 'filter' was corrupted.
I never see the usual "all.tcl" test summary output. Maybe nsd is
hitting the above "Debug Error!" before getting there?
In the test output, I have lots of "compress" failures, but that's
probably because I don't have zlib installed (yet) on this machine.
I also see 4 "http_chunked" test failures, below:
http_chunked-1.1 ADP streaming w/chunks to HTTP/1.1 client FAILED
Contents of test case:
nstest::http-0.9 -http 1.1 -setheaders {Connection keep-alive} -getbody 1
-getheaders {Transfer-Encoding Connection Content-Length} GET
/http_chunked.adp?stream=1
Result was:
200 chunked keep-alive {} {a
0123456789
5
01234
}
Result should have been (exact matching):
200 chunked keep-alive {} {a
0123456789
5
01234
0
}
http_chunked-1.1 FAILED
http_chunked-1.3 ADP with longer partly-buffered response and
auto-streaming FAILED
Contents of test case:
nstest::http-0.9 -http 1.1 -setheaders {Connection keep-alive} -getheaders
{Transfer-Encoding Connection Content-Length} -getbody t GET
/http_chunked.adp?stream=0=8
Result was:
200 chunked keep-alive {} {a
0123456789
5
01234
}
Result should have been (exact matching):
200 chunked keep-alive {} {a
0123456789
5
01234
0
}
http_chunked-1.3 FAILED
http_chunked-2.1 Tcl streaming w/chunks to HTTP/1.1 client FAILED
Contents of test case:
nstest::http-0.9 -http 1.1 -getheaders {Transfer-Encoding Content-Length}
-getbody 1 GET /tclchunked
Result was:
200 chunked {} {a
0123456789
5
01234
}
Result should have been (exact matching):
200 chunked {} {a
0123456789
5
01234
0
}
http_chunked-2.1 FAILED
http_chunked-2.1.1 Tcl streaming multiple binary buffers w/chunks to
HTTP/1.1 client FAILED
Contents of test case:
nstest::http-0.9 -http 1.1 -getheaders {Transfer-Encoding Content-Length}
-getbody 1 GET /tclchunked
Result was:
200 chunked {} {a
0123456789
5
01234
}
Result should have been (exact matching):
200 chunked {} {a
0123456789
5
01234
0
}
http_chunked-2.1.1 FAILED
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] cannot run Windows tests, mysterious nslog.dll permission denied
On Tue, Mar 14, 2023 at 01:46:34PM -0400, Andrew Piskorski wrote: > nmake -f Makefile.win32 _test_one > > Now, I can't get that to work, and I'm stuck trying to figure out why. In my case, trying to run on a network drive (over Samba) seems to be causing the "permission denied" problem. (I'm still trying to figure out why.) If I simply copy everything to a local Windows disk, then the tests do run. They eventually crash nsd with this popup error message: Debug Error! Program: C:\web\ns-fork-pub\naviserver\nsd\libnsd.dll Run-Time Check Failure #2 - Stack around the variable 'filter' was corrupted. I haven't tried to debug that yet. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] cannot run Windows tests, mysterious nslog.dll permission denied
Back in 2020, we had the regression tests successfully running on Windows, from the command line with these sorts of commands: nmake -f Makefile.win32 _test nmake -f Makefile.win32 _test_one Now, I can't get that to work, and I'm stuck trying to figure out why. Note, I DON'T mean that the tests are failing - they're not. Rather, I currently can't get the test nsd to start up and run the tests at all! When run from nmake like the above, I get no useful output anywhere, just a "return code '0xc022'" message from nmake. If I copy that generated command line and run the it manually at the Windows command prompt, it then DOES write to "tests/nsd.log", where I see errors like this: [14/Mar/2023:13:17:45][16420.4aa0][-main:test-] Error: modload: Z:/src/web/ns-fork-pub/naviserver/tests/../nslog/nslog.dll: couldn't load library "Z:/src/web/ns-fork-pub/naviserver/tests/../nslog/nslog.dll": permission denied [14/Mar/2023:13:17:45][16420.4aa0][-main:test-] Fatal: modload: failed to load module 'Z:/src/web/ns-fork-pub/naviserver/tests/../nslog/nslog.dll' Yes, it says "permission denied", but I have no idea why! AFAICT that nslog.dll' file *IS* readable by everybody in every other context. Note that in that same Windows Command Prompt I can run tclsh and see that the file IS readable: Z:\src\web\ns-fork-pub\naviserver> tclsh % info patchlevel 8.6.12 % file readable Z:/src/web/ns-fork-pub/naviserver/tests/../nslog/nslog.dll 1 % exit I suspect I'm overlooking something. Any debugging suggestions for me? Btw, once I install Naviserver I can start it up using the "simple-config.tcl" just fine, but I don't remember if it's feasible to run the tests that way. In the past I definitely ran them using the build directory like the above, not after installing. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] LC_COLLATE, Windows, nsmain: system configuration mismatch
Since c. 2022-04-10, code at line 614 of nsd/nsmain.c calls Ns_Fatal() with message like this: OpenSSL system configuration mismatch. The provided locale *1 based on the system-wide default locale is not installed on this system. With latest Naviserver code on Windows 10, I was getting that error every time I tried to start Naviserver. In my string "locale *1" above, the "*" symbols were actually weird rectangles, completely illegible. The only readable part was the "1" at the end. I set a System environment variable LC_COLLATE = en_US.UTF-8, and that fixed it, now Naviserver actually starts! I'm confused about why that was necessary, but so far it seems to work. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] aead::encrypt test failures
On Thu, Mar 09, 2023 at 12:27:46PM +0100, Gustaf Neumann wrote: > My first suspicion is the version of OpenSSL in use. OpenSSL is a moving > target. > If i see correctly, there is a version "1.1.1-1ubuntu2.1~18.04.21" in > place for Ubuntu 18.04 will all updates. It this what you are using? Yes, I am. Hm, clearly the best overall approach is for me to upgrade this server, Ubuntu 18.04.4 is very old. In the meantime, how widely used within NaviServer is aead::encrypt? Is it necessary for basic serving of https pages, or just an extra API programmers can optionally use? Do even the latest versions of OpenACS depend on it? (In other words, I'm wondering if these two aead::encrypt test failures actually matter for me.) On that old server, I'm currently using an old version of NaviServer with code from 2020-06-15. It had zero failed tests, and still seems to be working fine. Btw, these are the package versions I see on Ubuntu: Ubuntu 18.04.4: libssl-dev version 1.1.1-1ubuntu2.1~18.04.21 Ubuntu 20.04.1: libssl-dev version 1.1.1f-1ubuntu2.17 Ubuntu 22.04.2: libssl-dev version 3.0.2-0ubuntu1.8 Ubuntu 22.04 stopped shipping OpenSSL 1.1.x entirely, and replaced it with 3.0.2. And it looks like the newer OpenSSL 3.x is NOT included at all in the older 18.04 and 20.04 distributions of Ubuntu. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] aead::encrypt test failures
Building the NaviServer head (latest code from 2023-03-02), I'm getting two "make test" failures, both from aead::encrypt (below). Any advice for me on what the problem might be, or how I should further track it down? (Thanks!) This is on an old Ubuntu 18.04.4 LTS machine, using gcc 8.4.0, and Tcl 8.6.13. My built-from-source Tcl includes nsf 2.4.0, Thread 2.8.9, tdom 0.9.3, and tcllib 1.20. ## Excerpts from "make test" output: [08/Mar/2023:15:23:16][11421.7f2097bfc700][-command-] Notice: SSL_shutdown(33) has failed: error:14094123:SSL routines:ssl3_read_bytes:application data after close notify ns_crypt.test ns_crypto.test aead-1.0 aead::encrypt FAILED Result was: bytes 0 tag 32 Result should have been (exact matching): bytes 22 tag 32 aead-1.0 FAILED aead-1.1 aead::encrypt and decrypt FAILED Test generated error; Return code was: 1 Return code should have been one of: 0 2 aead-1.1 FAILED Tests ended at Wed Mar 08 15:24:13 EST 2023 all.tcl:Total 1998Passed 1971Skipped 25 Failed 2 Sourced 71 Test Files. Files with failing tests: ns_driver.test Number of tests skipped for each constraint: 19 !usingExternalToUtf 2 binaryMismatch 1 copyAliasBug 2 knownBug 1 stress Makefile:236: recipe for target 'test' failed make: *** [test] Error 130 -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Recommended approach to building Naviserver on Windows?
The Tcl version should not really matter, I just install whatever the latest stable ActiveTcl is for Windows. I believe the latest I've used was 8.6.x, which I installed into "C:\P\Tcl-64-8.6\". It should also work to compile Tcl yourself on Windows, but I haven't tried that in many years. For Oracle on Windows, I installed the Oracle "Instant Client" version 19.6.0.0.0 a while back. This was 4 separate zip files: instantclient-basiclite-windows.x64-19.6.0.0.0dbru.zip instantclient-sdk-windows.x64-19.6.0.0.0dbru.zip instantclient-sqlplus-windows.x64-19.6.0.0.0dbru.zip instantclient-tools-windows.x64-19.6.0.0.0dbru.zip To install, I simply unzipped all of those into a tmp directory, which created an "instantclient_19_6" directory. Then I copied the whole "instantclient_19_6" directory into "C:\P\oracle\". The "nsoracle/Makefile.win32" has this line to use it: ORACLE_HOME = C:\P\oracle\instantclient_19_6 For OpenSSL, back in Windows XP days I did compile it on Windows, and as far as I recall that worked, although the compile was VERY slow, and I had to install various support tools first (NASM Assembler, etc.). More recently I've only installed binary OpenSSL builds on Windows. I've used this OpenSSL for Windows: http://www.slproweb.com/products/Win32OpenSSL.html There are also binaries for the older 1.0.2 OpenSSL line, here, but I don't think I ever actually used them: https://indy.fulgan.com/SSL/ See also OpenSSL's own list of Windows Binary Builds: https://wiki.openssl.org/index.php/Binaries Old Win32OpenSSL instructions said to install the Visual C++ 2008 Redistributable Package via its "vcredist_x64.exe" and "vcredist_x86.exe" web installers: x64: http://www.microsoft.com/en-us/download/details.aspx?id=15336 x86: http://www.microsoft.com/en-us/download/details.aspx?id=29 But of course these days should probably be using the 64-bit "Visual C++ 2017 Redistributable" or something even newer instead. (Read their current instuctions.) Around 2019-01, I installed the (at the time) latest "Win64OpenSSL-1_1_1a.exe" and "Win32OpenSSL-1_1_1a.exe" into "C:\P\OpenSSL-Win64\". When it asked my where to "Copy OpenSSL DLLs to:" I always pick "The OpenSSL binaries (/bin) directory", NOT the default "Windows system directory". -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Recommended approach to building Naviserver on Windows?
On Thu, Mar 04, 2021 at 10:24:42AM +, Brian Fenton wrote: > I took a look at the install documentation here > https://bitbucket.org/naviserver/naviserver/src/master/ and I noticed that > there are 3 approaches described for compiling and installing on Windows. > Which one of the 3 is the preferred approach within the community? The only approach I have extensive experience with is using the Microsoft C compiler natively on Windows. That works. I installed "Visual Studio 2019 Community Edition 16.5", and haven't had any reason to look for a newer version yet. In the installer, I checked only the "Desktop development with C++" Workload, and otherwise left everything else set to defaults. Here are my notes to Compile and Install NaviServer on Windows, using the command-line MSVCC compiler: (I believe this info was up-to-date around 2020-07, using the NaviServer head code from c. 2020-06-14.) 1. To compile we need "include/nsversion.h", which is auto-generated by "include/nsversion.h.in", so ou have two choices: Either (bizarrely), run configure on Linux before building on Windows! Surprisingly, this works and is how I did it for a long time. Or if you don't like that, or don't have Linux available, use the "win32-util/configure.tcl" script instead. Actually you don't need to run it yourself at all, as "Makefile.win32" does it for you. 2. If you already built Linux NaviServer in that same directory, run make clean there to remove the Linux object files. (TODO: It would be so much nicer to have separate Build-* subdirectories...) 3. As the instructions in "naviserver/Makefile.win32" say, make sure you edit "include/Makefile.build, changing the "Makefile.module" line to say "Makefile.win32" instead! If you do not you will get cryptic nmake errors. 4. In a Windows Command Prompt window, Build NaviServer like this, for 64-bit: "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat" z: & cd Z:\src\web\ns-fork-myfork\naviserver nmake -f Makefile.win32 clean-core clean-mod all-core all-mod 5. Again from the Command Prompt, run this little script to properly copy all the files to their installed locations: cd Z:\src\web\ns-fork-myfork\naviserver\ tclsh ./win32-util/install-nsd.tcl -i 6. Test plain NaviServer in the foreground, e.g. like so: C:\web\nsd4-myfork-20200411-1\bin\nsd.exe -t C:\web\nsd4-myfork-20200411-1\conf\simple-config.tcl -f Note that by default the "install-nsd.tcl" script above does NOT overwrite the installed binaries, instead it installs to a similar directory alongside it, with a timestamp in the name, e.g., "nsd4-myfork-20040927T190908\". You are free to rename that directory later to whatever you want (because nothing inside it embeds the directory name). You can also use the "install-nsd.tcl" script's "-f" and "-t" switches to control what directory the install script copies from and to, but normally the defaults are what you want. If run without any arguments, "install-nsd.tcl" will show you its defaults. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Sun, Jun 14, 2020 at 02:44:35PM +0200, Gustaf Neumann wrote: > i fixed two more bugs for win64 (see [1]). The most complex > case was handling thread results (threads returning string values). Wow, great news Gustaf, thank you very much. > @Andrew: are the still show-stoppers for you, > which have to be fixed urgently? No, everything for my application is working well now! I've been running your 2020-06-07 time_t Ns_Time fix for a week, that completely fixed the problem with the scheduler thread getting stuck. I just recently upgraded to your 06-14 fixes as well. The other Windows regression test failures don't seem to affect me, but I'll still put some time into some them if/when I come up with any better ideas for how to debug them. And once Ibrahim Tannir gets his nsproxy fixes ready, I can certainly try them. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Mon, Jun 08, 2020 at 05:46:54PM +0200, Gustaf Neumann wrote: > >Assertion failed: tid != NULL, file tclthread.c, line 238 > You might check whether "ns_thread handle" > in a classical setup (e.g. in a ds/shell) thows the same exception. Good idea. I started up NaviServer with the same test.nscfg config file, but using the installed binaries instead of the "nmake -f Makefile.win32 _test" approach. Then I typed "ns_thread handle" at the control port prompt. That threw the exact same exception as before. Under WinDbg it also looks the same, inside Ns_ThreadSelf() wPtr appears to be defined, but threadPtr and wPtr->self are null. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Windows there are still a few compiler warnings that look a little suspicious (below), but I don't see any good way to fix these. cl /W3 /nologo /c /EHsc /MDd /Od /Zi /RTC1 /I "..\include" /I "C:\P\OpenSSL-Win64\include" /I "C:\P\Tcl-64-8.6\include" /D "_WINDOWS" /D "TCL_THREADS=1" /D "FD_SETSIZE=128" /D "_MBCS" /D _CRT_SECURE_NO_WARNINGS /D _CRT_SECURE_NO_DEPRECATE /D "_DEBUG" /c /Foexec.o exec.c exec.c(154): warning C4312: 'type cast': conversion from 'pid_t' to 'HANDLE' of greater size exec.c(371): warning C4311: 'type cast': pointer truncation from 'HANDLE' to 'pid_t' cl /W3 /nologo /c /EHsc /MDd /Od /Zi /RTC1 /I "..\include" /I "C:\P\OpenSSL-Win64\include" /I "C:\P\Tcl-64-8.6\include" /D "_WINDOWS" /D "TCL_THREADS=1" /D "FD_SETSIZE=128" /D "_MBCS" /D _CRT_SECURE_NO_WARNINGS /D _CRT_SECURE_NO_DEPRECATE /D "_DEBUG" /c /Fotls.o tls.c tls.c(228): warning C4244: 'function': conversion from 'SOCKET' to 'int', possible loss of data tls.c(376): warning C4244: 'function': conversion from 'SOCKET' to 'int', possible loss of data cl /W3 /nologo /c /EHsc /MDd /Od /Zi /RTC1 /I "..\include" /I "C:\P\OpenSSL-Win64\include" /I "C:\P\Tcl-64-8.6\include" /D "_WINDOWS" /D "TCL_THREADS=1" /D "FD_SETSIZE=128" /D "_MBCS" /D _CRT_SECURE_NO_WARNINGS /D _CRT_SECURE_NO_DEPRECATE /D "_DEBUG" /c /Fotclcrypto.o tclcrypto.c tclcrypto.c(592): warning C4090: 'initializing': different 'const' qualifiers tclcrypto.c(656): warning C4090: 'initializing': different 'const' qualifiers tclcrypto.c(711): warning C4090: 'initializing': different 'const' qualifiers tclcrypto.c(822): warning C4090: 'initializing': different 'const' qualifiers tclcrypto.c(955): warning C4090: 'initializing': different 'const' qualifiers tclcrypto.c(1011): warning C4090: 'initializing': different 'const' qualifiers tclcrypto.c(1068): warning C4090: 'initializing': different 'const' qualifiers -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Mon, Jun 08, 2020 at 12:04:59PM +0200, Gustaf Neumann wrote: > So, i have modified the code to use "time_t" for the "sec" member, > ... and many of the warnings disappeared. That's a big improvement, thank you, Gustaf! The 22 regression tests below used to fail, but now pass! No change to the other failing tests, nor to the ones that we're currently skipping with the notWin32 constraint. E.g., test ns_thread-2.6 still triggers this: Assertion failed: tid != NULL, file tclthread.c, line 238 ## Gustaf's 2020-06-07 changes fixed these test failures: ns_schedule-2.1 schedule proc: interval FAILED ns_time-1.2ms ns_time incr timeunit float+ms int FAILED ns_time-1.3ms ns_time incr timeunit int+ms int FAILED ns_time-1.3?s ns_time incr timeunit int+ms int FAILED ns_time-1.4-100ms ns_time incr timeunit 100ms int FAILED ns_time-1.4-10ms ns_time incr timeunit 10ms int FAILED ns_time-1.4-1ms ns_time incr timeunit 1ms int FAILED ns_time-1.4-0.1ms ns_time incr timeunit 0.1ms int FAILED ns_time-1.4-0.01ms ns_time incr timeunit 0.01ms int FAILED ns_time-1.4-0.001ms ns_time incr timeunit 0.001ms int FAILED ns_time-format-1.2 ns_time format positive microsecond FAILED ns_time-format-2.1 ns_time format negative second FAILED ns_time-format-2.2 ns_time format negative second with fraction FAILED ns_time-format-2.4 ns_time format negative microsecond FAILED ns_time-format-2.4-0.001ms ns_time format negative microsecond FAILED ns_time-diff-1 ns_time diff simple FAILED ns_time-diff-2 ns_time diff requires adjust FAILED ns_time-diff-3 ns_time diff subtract nothing FAILED ns_time-diff-4 ns_time diff add 1ms FAILED ns_time-diff-5 ns_time diff turn positive to negative FAILED ns_time-diff-6 ns_time diff make negative more negative FAILED ns_time-diff-9 ns_time diff turn negative to positive FAILED -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Thu, Jun 04, 2020 at 04:55:10PM -0400, Andrew Piskorski wrote:
> On Thu, Jun 04, 2020 at 05:26:04PM +0200, Gustaf Neumann wrote:
> > Probably "Ns_ThreadSelf();" does not work under windows (get the
> > id of the current thread). Ns_ThreadSelf() is defined in the OS specific
> > part (winthread.c). The exception is probably coming from
> > test thread-2.3, it looks to me as if the the thread (here the
> > thread running the tests) is not properly initiated under windows.
> Assertion failed: tid != NULL, file tclthread.c, line 238
For debugging, I turned test ns_thread-2.6 back on, and added an
assertion inside Ns_ThreadSelf(), so that it was basically doing this:
void Ns_ThreadSelf(Ns_Thread *threadPtr) {
WinThread *wPtr = TlsGetValue(tlskey);
*threadPtr = (Ns_Thread) wPtr->self;
assert(NULL != *threadPtr);
}
That got it to break into Microsoft's WinDbg debugger there, rather
than later in NsTclThreadObjCmd(). The global "tlskey" seems to be
initialized, and so does the "wPtr" WinThread pointer. But the
"wPtr->self" looks like it's NULL, there is no Ns_Thread structure
stored there!
I see the wPtr WinThread allocation code in DllMain(). That seems to
be working fine.
The wPtr->self Ns_Thread stuff gets set up in NsCreateThread() and
ThreadMain(), but I don't really understand understand what that code
is doing. Is that the place where something is going wrong?
Btw, The tlskey TLS index value looks like it's a 64-bit DWORD
(unsigned integer), not 32-bit. So I think Nsthreads_LibInit() should
be checking for TLS_OUT_OF_INDEXES, not the 0x (decimal
4294967295, the maximum size of a 32-bit DWORD) it's been checking for
since ancient times. That's probably a small bug, but it's not the
cause of the problems here.
WinDbug output:
--
0:001> .frame 7
07 `0062f150 07fe`ed490aae nsthread!Ns_ThreadSelf+0x89
[Z:\src\web\ns-fork-pub\naviserver\nsthread\winthread.c @ 848]
0:001> dt wPtr
Local var @ 0x62f170 Type WinThread*
0x`004706d0
+0x000 nextPtr : (null)
+0x008 wakeupPtr: (null)
+0x010 self : (null)
+0x018 event: 0x`015c Void
+0x020 condwait : 0n0
+0x028 slots: [100] (null)
0:001> dt threadPtr
Local var @ 0x62f190 Type Ns_Thread_**
0x`0062f208
-> (null)
0:001> ? tlskey
Evaluate expression: 8791677299988 = 07fe`f8cd6d14
0:001> .formats tlskey
Evaluate expression:
Hex: 07fe`f8cd6d14
Decimal: 8791677299988
Octal: 000177737063266424
Binary: 0111 1110 1000 11001101 01101101
00010100
Chars: ..m.
Time:Thu Jan 11 00:12:47.729 1601 (UTC - 4:00)
Float: low -3.33323e+034 high 2.86706e-042
Double: 4.34367e-311
0:001> dt -v tlskey
Got address 07fef8cd6d14 for symbol
nsthread!tlskey
7
0:001> kb
: Call Site
: ucrtbased!issue_debug_notification+0x45
[minkernel\crts\ucrt\src\appcrt\internal\report_runtime_error.cpp @ 28]
: ucrtbased!__acrt_report_runtime_error+0x13
[minkernel\crts\ucrt\src\appcrt\internal\report_runtime_error.cpp @ 154]
: ucrtbased!abort+0x1d [minkernel\crts\ucrt\src\appcrt\startup\abort.cpp @ 61]
: ucrtbased!common_assert_to_stderr_direct+0xe5
[minkernel\crts\ucrt\src\appcrt\startup\assert.cpp @ 161]
: ucrtbased!common_assert_to_stderr+0x27
[minkernel\crts\ucrt\src\appcrt\startup\assert.cpp @ 179]
: ucrtbased!common_assert+0x68
[minkernel\crts\ucrt\src\appcrt\startup\assert.cpp @ 420]
: ucrtbased!_wassert+0x2f [minkernel\crts\ucrt\src\appcrt\startup\assert.cpp @
444]
: nsthread!Ns_ThreadSelf+0x89
[Z:\src\web\ns-fork-pub\naviserver\nsthread\winthread.c @ 848]
: libnsd!NsTclThreadObjCmd+0x42e
[Z:\src\web\ns-fork-pub\naviserver\nsd\tclthread.c @ 238]
: tcl86t!TclNRRunCallbacks+0x63
: tcl86t!Tcl_EvalEx+0x9dd
: tcl86t!Tcl_FSEvalFileEx+0x223
: tcl86t!Tcl_MainEx+0x4be
: libnsd!CmdThread+0x6e [Z:\src\web\ns-fork-pub\naviserver\nsd\nsmain.c @ 1333]
: nsthread!NsThreadMain+0x77
[Z:\src\web\ns-fork-pub\naviserver\nsthread\thread.c @ 236]
: nsthread!ThreadMain+0x6c
[Z:\src\web\ns-fork-pub\naviserver\nsthread\winthread.c @ 880]
: ucrtbased!thread_start+0x9c
[minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 97]
: kernel32!BaseThreadInitThunk+0xd
: ntdll!RtlUserThreadStart+0x1d
--
--
Andrew Piskorski
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Thu, Jun 04, 2020 at 04:55:10PM -0400, Andrew Piskorski wrote: > Yes, with your new change, when running ns_thread.test on Windows I > now always get this: > > Assertion failed: tid != NULL, file tclthread.c, line 238 A bunch of different tests seem to trigger that assertion failure. However, it does seem to be the only thing in the tests that causes crashes, which is good. In my latest code here, I used the new "notWin32" tcltest contstraint to turn off all the tests that tend to trigger that assertion: https://bitbucket.org/apiskors/naviserver/commits/ That let's me run the rest of the regression tests to completion, with the summary results below. Is there someplace I should upload or attach the full test output? It's about 5k lines and 3 megabytes. Tests ended at Fri Jun 05 13:32:03 EDT 2020 all.tcl:Total 1569Passed 1376Skipped 39 Failed 154 Sourced 70 Test Files. Files with failing tests: encoding.test http.test http_byteranges.test http_chunked.test http_keep.test ns_adp_compress.test ns_base64.test ns_driver.test ns_hostbyaddr.test ns_httptime.test ns_info.test ns_log.test ns_proxy.test ns_schedule.test ns_time.test ns_urlencode.test tclconnio.test tclresp.test Number of tests skipped for each constraint: 2 binaryMismatch 5 curl 2 knownBug 1 notDarwin 28 notWin32 1 stress -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Fri, May 15, 2020 at 10:37:15AM +0200, Gustaf Neumann wrote: > On 15.05.20 09:21, Andrew Piskorski wrote: > > Previously on Windows I was running NaviServer code from > > c. 2019-07 and an ancient Microsoft compiler from 2010; the problem > > did NOT happen then. > Can you try with the released version 4.99.17 (2018-11-04) > with your new Windows environment? It can be tricky to find an old version of the NaviServer code that builds correctly on Windows. I did successfully build these two older points in the code: commit c31d3a0c4ef60b79c542cacbdc66c9cb53428faa Author: Gustaf Neumann Date: 2019-06-18 20:43:41 +0200 Tue fix prototype of Ns_SockListenCallback in in ns.h (many thanks to Maurizio Martignano) commit 83e8c50a38a6986f3c0468b69e8ef3abd68f926e Author: Gustaf Neumann Date: 2020-01-17 21:03:31 +0100 Fri improve spelling For each of those, first I did a "git checkout VERSION" to the commit version number above. Then I copied the latest makefiles and tests on top of the old code like so: cp -p $NEW_DIR/Makefile.win32 . cp -p $NEW_DIR/include/Makefile.* include/ cp -pr $NEW_DIR/win32-util . cp -pr $NEW_DIR/tests . With that, those two older codebases both compiled on Windows. However, when I then ran the latest regression tests, NaviServer crashed with: Run-Time Check Failure #2 - Stack around the variable 'spoolLimit' was corrupted. (Press Retry to debug the application) That terminated the tests early, of course. It looks like about 93 tests passed and 42 failed before the testing NaviServer crashed. Many of the failed tests did look like the same ones failing on the latest head code. The variable spoolLimit only appears in "nsd/tclhttp.c", so presumably one of the later commits fixed a bug there. But at that point I gave up trying to test the older code. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Thu, Jun 04, 2020 at 05:26:04PM +0200, Gustaf Neumann wrote: > >Assertion failed: (addr != ((void *)0)), file tclobj.c, line 325 > Probably "Ns_ThreadSelf();" does not work under windows (get the > id of the current thread). Ns_ThreadSelf() is defined in the OS specific > part (winthread.c). The exception is probably coming from > test thread-2.3, it looks to me as if the the thread (here the > thread running the tests) is not properly initiated under windows. > > i have added one more assert, to make it easier to pinpoint the > problem. Yes, with your new change, when running ns_thread.test on Windows I now always get this: Assertion failed: tid != NULL, file tclthread.c, line 238 -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
Is nsproxy supposed to work correctly on Windows? Its test framework wants to use test-nsproxy.sh to set LD_LIBRARY_PATH, which of course can't work on Windows. But as I work around that, when I run just the ns_proxy.test tests, I get this error: Assertion failed: sec >= 0, file time.c, line 344 Which gives this stacktrace when run under the WinDbg debugger: : nsthread!Ns_IncrTime+0x6c [naviserver\nsthread\time.c @ 344] : nsproxy!Ns_ModuleInit+0x7a76 : nsthread!NsThreadMain+0x77 [naviserver\nsthread\thread.c @ 236] : nsthread!ThreadMain+0x6c [naviserver\nsthread\winthread.c @ 874] : ucrtbased!thread_start+0x9c [minkernel\crts\ucrt\src\appcrt\startup\thread.cpp @ 97] : kernel32!BaseThreadInitThunk+0xd : ntdll!RtlUserThreadStart+0x21 Weirdly, that stacktrace seems like it must be missing some intermediate function calls, because nsproxy's Ns_ModuleInit() definitely never calls Ns_IncrTime() DIRECTLY. So I'm not sure what's going on there either. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Mon, Jun 01, 2020 at 11:08:48AM -0400, Andrew Piskorski wrote:
> ## Last test that runs on Windows, it locks up forever:
>
> http_persistent.test
> ns_sockioctl failed: no such file or directory
> while executing
> "ns_socknread $s"
> (procedure "client_readable" line 2)
For now I simply moved the entire http_persistent.test file out of the
way, so the server skips those tests. With that, test server got
further, but eventually crashed with what looks like a null pointer
dereference, here:
[03/Jun/2020:14:25:28][4844.2b10][-conn:test:default:1:229-] Notice: inside
the filter 3.4
ns_serverpath.test
ns_set.test
ns_sha1.test
ns_sls.test
ns_striphtml.test
ns_thread.test
[03/Jun/2020:14:25:28][4844.13bc][-tcl-nsthread:7-] Notice: update
interpreter to epoch 1, trace none, time 0.219973 secs
Assertion failed: (addr != ((void *)0)), file tclobj.c, line 325
[03/Jun/2020:14:25:32][4844.1dd8][-tcl-nsthread:8-] Notice: update
interpreter to epoch 1, trace none, time 3.902536 secs
The stack trace looked like this:
ucrtbased.dll!07feedad41cf()Unknown
libnsd.dll!Ns_TclSetAddrObj(Tcl_Obj * objPtr, const char * type, void * addr)
Line 325 C
libnsd.dll!NsTclThreadObjCmd(void * clientData, Tcl_Interp * interp, int
objc, Tcl_Obj * const * objv) Line 239 C
[External Code]
libnsd.dll!CmdThread(void * arg) Line 1333 C
nsthread.dll!NsThreadMain(void * arg) Line 236 C
nsthread.dll!ThreadMain(void * arg) Line 874C
So that was inside Ns_TclSetAddrObj(), probably in the
"NS_NONNULL_ASSERT(addr != NULL);" line. It was called from
NsTclThreadObjCmd(), in "case THandleIdx", line 238 in tclthread.c.
That presumably came from a Tcl "ns_thread handle" call, and there's
only one of those in the test suite, "test ns_thread-2.6" on line 70
of "ns_thread.test". But I don't understand why that would throw a
null pointer exception!
Prior to that crash, various other interesting test failures cropped
up, including both "ns_listencallback-1.0" and "ns_schedule-2.1" below.
The ns_schedule-2.1 failure certainly sounds related to my original
problem of the scheduler thread getting stuck, but there's enough else
going on here that don't have any idea where the real source of the
problem might be.
ns_listencallback-1.0 register FAILED
Contents of test case:
set localhost [expr {[ns_info ipv6] ? "::1" : "127.0.0.1"}]
ns_log notice "open sockent on $localhost 7227"
set fds [ns_sockopen $localhost 7227]
lassign $fds rfd wfd
set size 0
if {[gets $rfd line] == -1} {
ns_log error "got no data"
} else {
incr size [string length $line]
puts $wfd "How are you?"
flush $wfd
gets $rfd line
incr size [string length $line]
}
return [list size $size]
Result was:
size 0
Result should have been (exact matching):
size 46
ns_listencallback-1.0 FAILED
ns_schedule-2.1 schedule proc: interval FAILED
Contents of test case:
set id [ns_schedule_proc 1s {nsv_lappend . . ns_schedule-2.1}]
ns_sleep 2.5s
ns_unschedule_proc $id
nsv_get . .
Result was:
ns_schedule-2.1
Result should have been (glob matching):
ns_schedule-2.1 ns_schedule-2.1*
ns_schedule-2.1 FAILED
--
Andrew Piskorski
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Mon, Jun 01, 2020 at 11:08:48AM -0400, Andrew Piskorski wrote:
> encoding-1.1 Send body with ns_return and charset utf-8 FAILED
> errorInfo: select failed: no such file or directory
> invoked from within
> "nstest::http-0.9 -encoding utf-8 -getbody 1 -getheaders {Content-Type
> Content-Length} GET "/encoding""
There are 7 different versions of the encoding.* page present. If I
start up the test server and then ask it for the FULL URL of any one
of those files, like "encoding.utf2iso_adp", it works fine! But if I
just ask for "encoding" without the extension it fails.
So hitting this URL works fine:
http://localhost:8000/encoding.utf_adp
But this fails with 404 Not Found:
http://localhost:8000/encoding
I see that test.nscfg has what look like appropriate "ns/mimetypes"
and "ns/encodings" sections, and of course that same config file works
fine on Linux. So what could be going wrong on my Windows box to
break the mapping of "/encoding" to "/encoding.utf_adp"?
--
Andrew Piskorski
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] scheduler thread getting stuck
On Fri, May 15, 2020 at 10:37:15AM +0200, Gustaf Neumann wrote:
> does the regression test run ok?
Good question. Unfortunately, I'd never run the regression tests on
Windows before. I now have them set up to run, however, I get LOTS of
failures, and I don't know if these are real problems with NaviServer,
or something wrong with my testing setup. Either way, I would like to
track it down so I can rely on running these same regression tests on
Windows as on Linux.
Are the tests in "naviserver/tests/all.tcl" supposed to work correctly
on Windows too? Is anyone else successfully running these tests there?
On Windows, I always get immediate failures due to return codes of 1.
The first such failure is "encoding-1.1", output shown below.
More concerning, is that once it gets to "http_persistent.test", the
whole NaviServer process locks up and never gets any farther. So any
tests that come AFTER that one are not being run at all. I've left
the test NaviServer running overnight just to be sure, and after that
point there's never any more output until I hit Ctrl-c to shut it down.
So far I've tested the nearly latest NaviServer head code on Windows 7
(no Windows 10 yet), with both the old 2010 and newer 2019 Microsoft
compilers. The behavior of the regression tests appears identical in
both cases. I have not yet tested older versions of NaviServer. On
Linux these tests all run fine, of course.
On Windows, I can invoke "tests/all.tcl" either before or after
installing NaviServer. Test behavior appears to be the same in both
cases. Before installing, I start the tests like this:
nmake -f Makefile.win32 _test
For that to work, you need these small patches to Makefile.win32:
https://bitbucket.org/apiskors/naviserver/commits/7d7e245f8451419de3ac9b1d6202e5f26c883fdd
## First test to fail on Windows:
encoding-1.1 Send body with ns_return and charset utf-8 FAILED
Contents of test case:
nstest::http-0.9 -encoding utf-8 -getbody 1 -getheaders {Content-Type
Content-Length} GET "/encoding"
Test generated error; Return code was: 1
Return code should have been one of: 0 2
errorInfo: select failed: no such file or directory
invoked from within
"nstest::http-0.9 -encoding utf-8 -getbody 1 -getheaders {Content-Type
Content-Length} GET "/encoding""
("uplevel" body line 2)
invoked from within
"uplevel 1 $script"
errorCode: NONE
encoding-1.1 FAILED
## Last test that runs on Windows, it locks up forever:
http_persistent.test
ns_sockioctl failed: no such file or directory
while executing
"ns_socknread $s"
(procedure "client_readable" line 2)
invoked from within
"client_readable 1000 $s"
(procedure "tcltest::client_receive" line 2)
invoked from within
"tcltest::client_receive sock05DA6CE0"
--
Andrew Piskorski
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] scheduler thread getting stuck
Recently I started seeing some weird behavior from NaviServer that I've never seen before. From time to time, it looks like the scheduler thread is getting stuck and not running anything, often for hours at a time. Then, on rare occasions, it will inexplicably come unstuck and go back to normal. I never see any "scheduled proc took too long" sorts of warnings in the log, so I don't THINK I'm running Tcl code that's tying up the scheduler thread. So far it seems more like the scheduler itself silently just does nothing. Interestingly, when the scheduler comes unstuck, any "missed" ns_schedule_daily jobs suddenly run all at once, in one second! In my case that includes the ns_logroll I schedule daily at 23:59. Just yesterday I saw it finally come unstuck and roll the serer log at 16:45 the next day! So far I've ONLY seen this strange behavior on Windows 7, where I recently upgraded to newer NaviServer code and a newer Microsoft 2019 Visual Studio Community Edition compiler. I suspect the problem doesn't happen on Linux at all, but I haven't checked for that thoroughly. Previously on Windows I was running NaviServer code from c. 2019-07 and an ancient Microsoft compiler from 2010; the problem did NOT happen then. Do you have any advice on how I should go about trying to debug this? E.g., are there commands I should run to check exactly what the scheduler thread is doing at any given time? I've started reading "nsd/tclsched.c" and "nsd/sched.c", but I've never looked at this code before and have no idea what might be wrong. Btw, the sched.c source says it's based on the 1988 paper below, but just how useful is it for understanding things? Those old conference proceedings seem hard to find outside of university libraries. The USENIX website itself says they only have content back to 1993! (Thanks for any help and advice!) https://www.usenix.org/legacy/publications/bibliography/byAuthor.html Author: Ronald E. Barkley Author: T. Paul Lee Title: A Heap-based Callout Implementation to Meet Real-Time Needs Pages: 213-222 Publisher: USENIX Proceedings: USENIX Conference Proceedings Date: Summer 1988 Location: San Francisco Institution: AT Information Systems The table of contents (nothing else) is online here: http://www.gbv.de/dms/tib-ub-hannover/303823399.pdf -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] git clone missing recent naviserver commits?
On Fri, Apr 10, 2020 at 02:19:50PM -0400, Andrew Piskorski wrote: > https://bitbucket.org/naviserver/naviserver/commits/ > > The weird thing is, after doing the commands below to update my clone, > the MOST recent commit I see is still only from 2020-04-05, Gustaf's > "add debug entries to 404 returns". The three more recent commits on > the website are apparently NOT in my local repository at all! Still had the same problem, so I discarded my local repository and cloned again from scratch. I'm still not sure what the problem was, but I notice that this new repository includes my BitBucket username in the URL, while the old one did not. AFAIK that should not matter, but perhaps it did somehow. $ git remote -v origin https://apisk...@bitbucket.org/naviserver/naviserver.git (fetch) origin https://apisk...@bitbucket.org/naviserver/naviserver.git (push) -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] git clone missing recent naviserver commits?
Last week, I cloned the BitBucket naviserver git repository. Now, I see that it has some new commits, so I want to pull them into my local clone: https://bitbucket.org/naviserver/naviserver/commits/ The weird thing is, after doing the commands below to update my clone, the MOST recent commit I see is still only from 2020-04-05, Gustaf's "add debug entries to 404 returns". The three more recent commits on the website are apparently NOT in my local repository at all! Am I doing something wrong with Git, or is this some sort of bug on BitBucket's end? $ git remote -v origin https://bitbucket.org/naviserver/naviserver.git (fetch) origin https://bitbucket.org/naviserver/naviserver.git (push) $ git fetch --all Fetching origin $ git log --full-history -n 2 '--date=format:%Y-%m-%d %T %z %a' commit 56492e5643367c57a2ea76128ea8b1d86e1be999 (HEAD -> master) Author: Gustaf Neumann Date: 2020-04-05 21:20:29 +0200 Sun add debug entries to 404 returns commit d5a4faaf07dc54e12afbac8e40b1c36575c64f62 Author: Gustaf Neumann Date: 2020-04-05 21:19:49 +0200 Sun change version number to 4.99.20a -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Mercurial at Bitbucket is ending. What next?
Can someone with write access also convert the AOLserver Mercurial repositories on BitBucket to Git, so we still have that stuff for historical reference, once BitBucket deletes all the Mercurial repositories? https://bitbucket.org/aolserver/aolserver-40x/src/default/ https://bitbucket.org/aolserver/aolserver/src/default/ Or is there a better repository for the old AOLserver code somewhere else? I see some stuff on GitHub, but I'm not sure how that compares to the BitBucket repositories. https://github.com/aolserver/aolserver https://github.com/aolserver/aolserver.github.com https://github.com/aolserver/nsoracle https://aolserver.github.io/2010/04/aolserver-on-github/ https://github.com/OldsSourcesBackups/AolWebServer -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] small bugs break Windows build
On Tue, Jul 16, 2019 at 02:51:06PM -0400, Andrew Piskorski wrote: > There are two small recent bugs that break compilation on Windows. One more: In "naviserver/include/ns.h" on line 2874, the declaration of Ns_SockaddrParseIPMask() is missing NS_EXTERN, which breaks linking for nsperm, like so: nsperm.o : error LNK2019: unresolved external symbol Ns_SockaddrParseIPMask referenced in function AddUserObjCmd nsperm.dll : fatal error LNK1120: 1 unresolved externals Adding "NS_EXTERN" fixes the problem. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] small bugs break Windows build
There are two small recent bugs that break compilation on Windows.
First, this 2019-07-16 commit has a simple typo in the Windows-only
code, "masedkBits" instead of "maskedBits":
https://bitbucket.org/naviserver/naviserver/commits/65a47eea3818e515f235cebc59a385cea6e5cf06
- New feature: context filter for urlspace
Second, the 2019-07-04 change to using "static inline bool" on the
Retry function causes the errors below. Simply removing the "inline"
fixes the problem. I don't know if there's a better way.
https://bitbucket.org/naviserver/naviserver/commits/5a97670cb105330c645ff25ebe03c3eef8851799
Define Retry as pure inline function
sock.c(76) : error C2054: expected '(' to follow 'inline'
sock.c(76) : error C2085: 'Retry' : not in formal parameter list
sock.c(78) : error C2085: 'CloseLater' : not in formal parameter list
sock.c(101) : error C2082: redefinition of formal parameter 'inline'
sock.c(101) : error C2146: syntax error : missing ',' before identifier ''
sock.c(101) : error C2146: syntax error : missing ',' before identifier ''
sock.c(101) : error C2143: syntax error : missing ';' before '('
sock.c(101) : error C2059: syntax error : ')'
sock.c(355) : warning C4013: 'Retry' undefined; assuming extern returning int
sock.c(665) : warning C4133: 'function' : incompatible types - from 'int *' to
'char *'
sock.c(1583) : warning C4133: 'function' : incompatible types - from 'int *' to
'char *'
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio
10.0\VC\Bin\amd64\cl.EXE"' : return code '0x2'
Stop.
--
Andrew Piskorski
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] head http.test failures
Using the latest NaviServer head (last commit 2019-07-09) on Linux
(Ubuntu 16.04.2 LTS) and Tcl 8.6.9 (built from source), I'm getting
'make test' failures that I don't understand, primarily in http.test.
Can anyone advice, please? Brief failure messages are below, full
test output is attached.
Tests ended at Mon Jul 15 04:23:22 EDT 2019
all.tcl:Total 1391Passed 1380Skipped 8 Failed 3
Sourced 69 Test Files.
Files with failing tests: http.test ns_server.test
http-3.3new limits: too many headers FAILED
Contents of test case:
nstest::http -http 1.1 -getbody 1 -setheaders [split [string repeat xy
1024] ""] GET /limits
Result was:
ns_http failed: recv failed
Result should have been (exact matching):
414
http-3.3new FAILED
http-5.5 check encoding ns_conn content POST, binary FAILED
Contents of test case:
set string "Testing "
nstest::http -getbody 1 -setheaders {content-type text/html} POST /post
$string
Result was:
200 {utf-8 A??ATesting Z??Z}
Result should have been (exact matching):
200 {utf-8 A??ATesting Z??Z}
http-5.5 FAILED
ns_server-2.5 basic operation FAILED
Result was:
10
Result should have been (exact matching):
9
ns_server-2.5 FAILED
--
Andrew Piskorski
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsthread'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsthread'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsd'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsd'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nssock'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nssock'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nscgi'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nscgi'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nscp'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nscp'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nslog'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nslog'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsperm'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsperm'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsdb'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsdb'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsdbtest'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsdbtest'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsssl'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsssl'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsproxy'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsproxy'
LD_LIBRARY_PATH="/usr/local/src/web/ns-head-pub/naviserver/nsd:/usr/local/src/web/ns-head-pub/naviserver/nsthread:/usr/local/src/web/ns-head-pub/naviserver/nsdb:/usr/local/src/web/ns-head-pub/naviserver/nsproxy:$LD_LIBRARY_PATH"
DYLD_LIBRARY_PATH="/usr/local/src/web/ns-head-pub/naviserver/nsd:/usr/local/src/web/ns-head-pub/naviserver/nsthread:/usr/local/src/web/ns-head-pub/naviserver/nsdb:/usr/local/src/web/ns-head-pub/naviserver/nsproxy:$DYLD_LIBRARY_PATH"
./nsd/nsd -u root -c -d -t
/usr/local/src/web/ns-head-pub/naviserver/tests/test.nscfg
/usr/local/src/web/ns-head-pub/naviserver/tests/all.tcl
[15/Jul/2019:04:21:46][1991.7feae6dfc700][-main-] Notice: OpenSSL 1.0.2g 1 Mar
2016 initialized
[15/Jul/2019:04:21:46][1991.7feae6dfc700][-main-] Notice: try ::1 8000
[15/Jul/2019:04:21:46][1991.7feae6dfc700][-main-] Notice: configure LOOPBACK
::1
[15/Jul/2019:04:21:46][1991.7feae6dfc700][-main-] Notice: configure LISTENURL
http://[::1]:8000
[15/Jul/2019:04:21:46][1991.7feae6dfc700][-main-] Notice: nsmain: enable
progress statistics for uploads >= 1 bytes
[15/Jul/2019:04:21:46
Re: [naviserver-devel] Tcl nonblocking file IO in NaviServer?
On Thu, Jun 06, 2019 at 09:23:12PM +0200, Gustaf Neumann wrote: > The main question is, why are you interested in async writes? My original use case wasn't clear? I have one thread busy processing data and sending it various places. I want to log some of that data to a bunch of files, probably 10 to 100 different small files at any given time. Each write is only a small amount of data, but for easier use later I'd rather keep the data organized into separate files, rather than e.g. dump it all into one great big file. The easiest thing to do is call Tcl's open, puts, close every single time I need to write data, but that's blocking, and I'm worried that it could easily degrade the latency of my main processing thread. I don't necessarily need async IO to the files, but I WOULD like to get the latency of writing the files out of the main thread. If I create my own writer thread for this, it could do the writes synchronously (blocking), that would still accomplish my main goal of getting the file write latency out of my main thread. However, (particularly given my small writes to many files), clearly asynchronous file IO should be more efficient, and scale better, so I'd like to understand how to do that. The ease of simply calling "fconfigure" on a file handle is attractive, but I don't yet understand what else I'd need to do to make that seemingly simple approach actually work in the NaviServer context. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Tcl nonblocking file IO in NaviServer?
On Thu, Jun 06, 2019 at 07:22:08PM +0200, Zoran Vasiljevic wrote: > There are also writer threads that spool files in a dedicated thread thus > offloading the connection threads. I see AsyncWriterThread() in "naviserver/nsd/driver.c", but I don't really understand the code, nor how it could be useful to my situation. I thought that stuff was only for NaviServer's own log files; is it actually for general purpose use? -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Tcl nonblocking file IO in NaviServer?
On Thu, Jun 06, 2019 at 05:23:38PM +0200, Zoran Vasiljevic wrote: > > From inside NaviServer, I'd like to repeatedly write data to some > > files in the background, > I assume you want to consume data from remote? No, the data isn't being uploaded by clients, it's received or generated server-side. Think a streaming data feed from some other server. The nssock module isn't involved at all, so I don't think there's any way for me to use its spoolerthreads or maxupload feature here. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] Tcl nonblocking file IO in NaviServer?
>From inside NaviServer, I'd like to repeatedly write data to some files in the background, while letting the thread that schedules the writes go on to do other work. The obvious way to do that might seem be simply calling Tcl's "open" on each file and then doing: fconfigure $fd -blocking 0 However, the Tcl docs clearly say that: http://www.tcl.tk/man/tcl8.6/TclCmd/puts.htm The application must use the Tcl event loop for nonblocking output to work; otherwise Tcl never finds out that the file or device is ready for more output data. I believe most threads in NaviServer never enter the Tcl event loop, and therefore Tcl nonblocking IO won't work at all. Is that correct? Is there another simple approach I should use from NaviServer instead, perhaps something analogous to the "background delivery" (ns_writer) scheme for writing to network sockets? Or would I need to roll my own? For rolling my own, what approach and APIs would make the most sense to use? So far I'm guessing something like this: 1. Temporarily store the data to be written in an nsv somewhere, and have a special write-to-file thread running in the background, which reads from the nsv. 2. The write-to-file thread can itself use Tcl noblocking IO to do the actual writes, and enter the Tcl event loop whenever it's idle. 3. Send a signal to the write-to-file thread telling it, "Wake up, you have work to do." But how should I do step 3 above? Normally I'd use ns_cond (ns_event), but that can't work if my write-to-file thread is waiting in the Tcl event loop, instead of waiting on a conditional variable, right? So what's the right way to send a signal to a thread that's using the Tcl event loop? Should I be using "thread::send" for that? Or is there some other or better approach? (I don't currently have the Tcl Threads Extension built for my NaviServer.) Hm, if I need to use thread::send anyway, maybe I can (should?) simply include all the data in each message, and skip queuing it temporarily in an nsv? https://www.tcl.tk/man/tcl8.6/ThreadCmd/thread.htm#M17 Thanks in advance for your advice! -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] store C pointers in Nsv vs. Tcl_HashTable?
On Fri, Feb 01, 2019 at 05:28:50PM +0100, Zoran Vasiljevic wrote: > OTOH, if you use ns:addr object, you need not take care about > locking and all the other trivia at all. It is the "rest" of > the system that needs to do that. > > Some of the stuff in "nsd/tclobj.c" seems be primarily concerned with > > the "address object type" (IP addresses?), but I get the sense that > It's (memory-)addresses == pointers. Ah! And you've already written the code to correctly convert between the pointer and string representations, so the ns:addr object approach is also a way to work around the "Nsvs only store strings" limitation. Thanks you, Zoran. I'm not sure which way I'll go in my implementation, but the choices available to me are making more sense now. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Ns_VarGet vs. NsTclNsvGetObjCmd
On Fri, Feb 01, 2019 at 04:25:20PM +0100, Zoran Vasiljevic wrote: > > I'm a little confused by (e.g.) Ns_VarGet(), though. Why is its first > > arg a string for the "server"? What does that mean, and where am I > > supposed to get it from? > > This is the identification of the Naviserver's virtual server > that is implicitly "known" to the Tcl code. The nsv_XXX is > thus bound to a virtual server since your Tcl code executes > within a specific virtual server. But on the C-level you need > to specify the virtual server explicitly because your C code > may operate for all known virtual servers or just bunch of them. Ok. But where do I get the server string from, in C? Or since I'm not using virtual servers anyway, how do I say "all of them, process-wide"? -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] store C pointers in Nsv vs. Tcl_HashTable?
On Fri, Feb 01, 2019 at 11:44:45AM +0100, Zoran Vasiljevic wrote: > The only safe way to share data between threads > in Tcl is over strings. One can go to some extent > and try making Tcl objects shareable but for that > you need to know the object internals which is a > great PITA and does not always work well, except for > a limited set of object types. What if you don't care much about Tcl objects? A C pointer is already easily shareable across threads (you just need the proper mutex locking when using it), the trick is finding it again using some handle from Tcl. For simplicity, should I just completely ignore the whole Tcl_Obj machinery when doing that? Or SHOULD I (always?) make my handle into a Tcl_Obj? If so, can you go into a bit more detail on WHY that's a better approach? Even despite the fact that this Tcl_Obj thing is now more difficult to share across threads? If I take an ordinary sort of Tcl string or integer and hand it to my C code, that works fine as the key for an nsv or Tcl_HashTable lookup. Of course, the code doesn't "know" that that particular string is supposed to be a handle, but there's nothing stopping me from simply using it as one in the C code, and throwing an error if I can't find a hash entry for it. I suspect that the above is NOT the recommended approach, but I don't fully understand why. It seems "simpler" to me... But maybe that's only because I don't yet understand the Tcl_Obj integration approach, and what benefits it provides. On Fri, Feb 01, 2019 at 10:29:47AM +0100, Zoran Vasiljevic wrote: > You might want to look into: > >nsd/tclobj.c Hm, that has Ns_TclSetTwoPtrValue(), Ns_TclGetOpaqueFromObj(), and other black magic (or so it seems to me). Are there docs on these C APIs anywhere? Some of the stuff in "nsd/tclobj.c" seems be primarily concerned with the "address object type" (IP addresses?), but I get the sense that much of that code is much more general purpose than that. I see that "nsd/tcltime.c" uses the tclobj.c functions to implement an Ns_Time Tcl_Obj type, but I definitely don't understand how or why yet! -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Ns_VarGet vs. NsTclNsvGetObjCmd
On Fri, Feb 01, 2019 at 10:22:46AM +0100, Zoran Vasiljevic wrote: > Andrew Piskorski wrote: > > > I know that the Tcl nsv_get command is implemented by > > NsTclNsvGetObjCmd(). But in the NaviServer C code, what is > > Ns_VarGet() for? I don't see it used anywhere. Is Ns_VarGet() a C > > interface to exact same nsv_get functionality, or is it for something > > completely different? > > The Ns_VarXXX is a programatic interface (from C) > to the nsv (Naviserver Shared Variables) storage. So they do the exact same thing, and are completely interoperable? Meaning I can have C and Tcl talk to each other via Ns_Var* in C and nsv_* in Tcl? Great! I'm a little confused by (e.g.) Ns_VarGet(), though. Why is its first arg a string for the "server"? What does that mean, and where am I supposed to get it from? Also, why are NsTclNsvGetObjCmd() and Ns_VarGet() implemented independently, rather than NsTclNsvGetObjCmd() calling Ns_VarGet()? I assume there's some good reason for this, but I'd like to understand what it is... -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] store C pointers in Nsv vs. Tcl_HashTable?
I'm using a C library that allocates some opaque structure, and returns a pointer to it. I'd like to save that pointer in a safe process-wide place, generate my own unique integer key (easy), and then return that simple key as a handle to my Tcl code. I also need to be able to find and use the C pointer from multiple threads. In the past I've done that sort of thing with my own Tcl_HashTable in C, but those are a bit low level, not that friendly to deal with. I'd rather use nsv_set, nsv_get, etc.! And I already know how to call NsTclNsvGetObjCmd() etc. from C. But the Nsv functions all operate ONLY on strings... I suppose I should be able to (very carefully) convert my pointer value to a string and back each time, but that seems awfully kludgy. Is there a better way? Something that lets me store the binary pointer values as is, without string conversions? The Tcl_HashTable C code DOES let me do that. In general, what's the recommended way for Tcl and/or NaviServer programmers to allocate handles to C-level pointers, and then access those handles and pointers from more than one Tcl thread? Can anyone point me to examples of code doing it the right way? Is Tcl_HashTable the right tool to use, or is some nicer Nsv-like interface available? (Nsv itself is implemented using Tcl_HashTable, of course.) Does Zoran's Thread tsv package have any additional functionality to help with this, beyond what NaviServer's built-in nsv commands can do? I found these wiki pages, but these (handles as values) approaches seem to basically just be fancier versions of what I've done before with Tcl_HashTable entries, and don't address use from multiple threads at all: https://wiki.tcl-lang.org/page/Tcl+Handles https://wiki.tcl-lang.org/page/Creating+and+Using+Tcl+Handles+in+C+Extensions (Thanks in advance for your help and advice!) -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] Ns_VarGet vs. NsTclNsvGetObjCmd
I know that the Tcl nsv_get command is implemented by NsTclNsvGetObjCmd(). But in the NaviServer C code, what is Ns_VarGet() for? I don't see it used anywhere. Is Ns_VarGet() a C interface to exact same nsv_get functionality, or is it for something completely different? -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_log output garbled on Windows
On Tue, Jan 29, 2019 at 12:02:23AM +0100, Gustaf Neumann wrote: > In case, the name is for unknown reason not initialized, the change [1] > should help. > https://bitbucket.org/naviserver/naviserver/commits/a1a07e7f1397e71809c74ca3a4bf93649affd8b2 That didn't make any difference. > The function Ns_ThreadId() returns uintptr_t, which is printed with > format-code PRIxPTR, maybe here is something > wrong. There is some indication [2], that the C99 prefix "ll" does not > work in all versions of MSC. > https://stackoverflow.com/questions/18107426/printf-format-for-unsigned-int64-on-windows Ah ha, thanks! I'm building NaviServer 32-bit on Windows. The garbled output was due to using that "PRIxPTR" macro to format the ThreadId. If I replace that format specifier with a either plain "%u" or "%I32u", then everything works fine! Hm, Ns_ThreadId() returns type uintptr_t, so why does LogToDString() use PRIxPTR in the first place? Shouldn't it use PRIuPTR, with a "u" for unsigned? I think so, so I switched to that. Here's the fix I'm using now: https://bitbucket.org/apiskors/naviserver/commits/16e6658319ae9595e7d874e68086e482f2f74f85 Fix garbled log output on Windows, PRIuPTR now gives I64u or I32u there. And here's a pull request that includes that, plus a couple other (very minor) fixes: https://bitbucket.org/naviserver/naviserver/pull-requests/20/fix-garbled-log-output-on-windows/diff -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] ns_log output garbled on Windows
On Windows, ns_log output is garbled. Below is some of the output from running "nsd.exe -f" in a Windows Command Prompt window. The 4th printed field, immediately after the severity, always has some garbled junk, which is then followed by readable text. This happens regardless of what NaviServer config file I use, including the simple-config.tcl that ships with NaviSever. This never happened with old c. 2014 versions of NaviServer on Windows. Btw, the log output differs between Windows and Linux in other minor but suspicious ways! Linux says "Notice:" as I expect, while Windows instead says "[Notice]" for the severity. The 3rd Linux field, "[-main-]", is completely missing on Windows! Likely that's related somehow? If Ns_ThreadGetName() returns NULL or empty string there, could that mess up the formatting of the rest of the line? The code that prints the log lines appears to be Ns_VALog() in "naviserver/nsd/log.c", either the Ns_DStringVPrintf() there or (more likely) the LogFlush() call. There are several code paths, and I do not know why one is relevant here, but maybe the string formatting actually happens in LogToDString(). ## Windows -f log output: [25/Jan/2019:16:10:43][12164.1cc047006a0][Notice] \???0: No support for OpenSSL compiled in [25/Jan/2019:16:10:43][12164.1cc047006a0][Notice] ?0: nsmain: NaviServer/4.99 (NaviServer 4.99.17) starting [25/Jan/2019:16:10:43][12164.1cc047006a0][Notice] ??0: nsmain: Tcl version: 8.5.16 [25/Jan/2019:16:10:43][12164.1cc047006a0][Notice] h???0: pool default: queueLength 90 low water 9 high water 72 [25/Jan/2019:16:10:43][12164.1cc047006a0][Notice] L??0: nsd/init.tcl[default]: booting virtual server: Tcl system encoding: "utf-8" [25/Jan/2019:16:10:43][12164.1cc047006a0][Notice] ?0: modload: loading module nscp from file nscp [25/Jan/2019:16:10:43][12164.1cc047006a0][Notice] T???0: nscp[default]: listening on [::1]:2080 [25/Jan/2019:16:10:43][12164.1cc047006a0][Warning] (??0: nscp[default]: no authorized users [25/Jan/2019:16:10:43][12164.1cc047006a0][Notice] ?0: modload: loading module nslog from file nslog ## Linux -f log output: [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Notice: OpenSSL 1.0.2g 1 Mar 2016 initialized [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Notice: nsmain: NaviServer/4.99.17 (7460924484c2 default tip) starting [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Notice: nsmain: security info: uid=1010, euid=1010, gid=501, egid=501 [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Notice: nsmain: Tcl version: 8.6.5 [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Notice: nsmain: max files: soft limit 1048576, hard limit 1048576 [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Warning: nsmain: rl_cur (1048576) > FD_SETSIZE (1024), select() calls should not be used [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Notice: pool default: queueLength 90 low water 9 high water 72 [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Notice: nsd/init.tcl[default]: booting virtual server: Tcl system encoding: "utf-8" [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Notice: modload: loading module nscp from file nscp [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Notice: nscp[default]: listening on [::1]:2080 [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Warning: nscp[default]: no authorized users [28/Jan/2019:10:33:20][28016.7fb29de89700][-main-] Notice: modload: loading module nslog from file nslog -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Windows nsd build breaks on tclhttp.c SSL_get_cipher
On Tue, Jan 22, 2019 at 10:09:26AM +0100, Gustaf Neumann wrote: > One thing, which is different with?? SSL_get_version() and > SSL_get_cipher() to > other SSL* function called from tclhttp.c is that these calls are not > guarded by "HAVE_OPENSSL_EVP_H" (but should probably). I added that additional ifdef, and now linking nsd.exe works! I think actually using nsssl on Windows is still broken, but now NaviServer builds and appears to run the simple-config.tcl ok. These fixes are here: https://bitbucket.org/naviserver/naviserver/pull-requests/19/windows-build-works-now/diff -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Windows nsd build breaks on tclhttp.c SSL_get_cipher
On Tue, Jan 22, 2019 at 10:09:26AM +0100, Gustaf Neumann wrote: > After merging it, i had to revert a change, otherwise i got a > compilation error: > > ../include/Makefile.module:88: *** Recursive variable `CLEAN' references > itself (eventually).?? Stop. Sorry about that one, Gustaf. That "simple" change fixed Windows, but I forgot to also test it on Linux. Unfortunately Gnu make and Microsoft nmake are only compatible for very simple makefiles. The other NaviServer core module makefiles are so simple that they do work in both versions, but appending to the CLEAN variable does not. I'll see if I can come up with some workaround. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Windows nsd build breaks on tclhttp.c SSL_get_cipher
On Thu, Jan 17, 2019 at 10:10:35PM +0100, Gustaf Neumann wrote: > Maurizio has built binaries of NaviServer 4.99.17 with openssl-1.1.1a [2]. I sure could use some hints on how he built NaviServer on Windows! I guess he's not reading this, so I'll try contacting him by email. > Are you sure, you comipile and link against OpenSSL 1.1.1a? I'm fairly sure I'm linking against OpenSSL 1.1.1a, because the link error about SSL_get_version went away. The mystery is why I still get the SSL_get_cipher link error. I suspect that the compile may NOT be correct, because of those "SSL_get_cipher undefined; assuming extern returning int" warnings. But nothing I tried seemed to fix that. Btw, nsssl.dll now appears to build fine! It's just libnsd.dll and nsd.exe that fail to link. (Perhaps because SSL_get_cipher is not used in nsssl/nsssl.c, it's only in nsd/tclhttp.c.) My recent Windows build fixes are now here in this pull request: https://bitbucket.org/naviserver/naviserver/pull-requests/18/windows-build-fixes-and-attempted-fixes/diff That includes some working fixes, plus my attempts (failures) to get nsd.exe to build. Weirdly, I could not add Gustaf as a reviewer, as the reviewer look-up box refuses to find him! (Same thing for Zoran.) Hopefully that's some sort of temporary bitbucket.org bug. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_striphtml test failures?
On Thu, Jan 17, 2019 at 11:11:08PM +0100, Gustaf Neumann wrote:
> I've now replaced the UTF-8 string literals by
> hexadecimal-escape-sequences on bitbucket [1].
> [1]
> https://bitbucket.org/naviserver/naviserver/commits/5c915e2412de64d0ee9e5c85a9fd46c59bc64ea4
I tried it, no difference, still the same 8 ns_striphtml test failures
as before. After that, I also rebuilt with "-finput-charset=UTF-8"
added to CFLAGS_EXTRA in include/Makefile.global; that also made no
difference.
However, next I built Tcl 8.6.9 from source myself, and to my
surprise, THAT fixed the ns_striphtml test failures! For some reason
NaviServer now skipped the 4 nsf tests, but hopefully that doesn't
matter, since they passed before.
## Newer from-source Tcl:
$ /usr/local/pkg/tcl-8.6.9-20190118/bin/tclsh8.6
% info patchlevel
8.6.9
## Older Ubuntu Tcl, ns_striphtml tests fail:
$ /usr/bin/tclsh8.6
% info patchlevel
8.6.5
--
Andrew Piskorski
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsthread'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsthread'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsd'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsd'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nssock'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nssock'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nscgi'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nscgi'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nscp'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nscp'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nslog'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nslog'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsperm'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsperm'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsdb'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsdb'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsdbtest'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsdbtest'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsssl'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsssl'
make[1]: Entering directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsproxy'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory
'/net/pinky/home/local-16.04/src/web/ns-head-pub/naviserver/nsproxy'
LD_LIBRARY_PATH="/usr/local/src/web/ns-head-pub/naviserver/nsd:/usr/local/src/web/ns-head-pub/naviserver/nsthread:/usr/local/src/web/ns-head-pub/naviserver/nsdb:/usr/local/src/web/ns-head-pub/naviserver/nsproxy:$LD_LIBRARY_PATH"
DYLD_LIBRARY_PATH="/usr/local/src/web/ns-head-pub/naviserver/nsd:/usr/local/src/web/ns-head-pub/naviserver/nsthread:/usr/local/src/web/ns-head-pub/naviserver/nsdb:/usr/local/src/web/ns-head-pub/naviserver/nsproxy:$DYLD_LIBRARY_PATH"
./nsd/nsd -u root -c -d -t
/usr/local/src/web/ns-head-pub/naviserver/tests/test.nscfg
/usr/local/src/web/ns-head-pub/naviserver/tests/all.tcl
[18/Jan/2019:16:19:25][39532.7fbf701ec700][-main-] Notice: OpenSSL 1.0.2g 1
Mar 2016 initialized
[18/Jan/2019:16:19:25][39532.7fbf701ec700][-main-] Notice: try ::1 8000
[18/Jan/2019:16:19:25][39532.7fbf701ec700][-main-] Notice: configure LOOPBACK
::1
[18/Jan/2019:16:19:25][39532.7fbf701ec700][-main-] Notice: configure LISTENURL
http://[::1]:8000
[18/Jan/2019:16:19:25][39532.7fbf701ec700][-main-] Notice: nsmain: enable
progress statistics for uploads >= 1 bytes
[18/Jan/2019:16:19:25][39532.7fbf701ec700][-main-] Notice: nsmain:
NaviServer/4.99.17 (fd186ea6d7d2+ default tip) starting
[18/Jan/2019:16:19:25][39532.7fbf701ec700][-main-] Notice: nsmain: security
info: uid=1010, euid=1010, gid=501, egid=501
[18/Jan/2019:16:19:25][39532.7fbf701ec700][-main-] Notice: nsmain: Tcl version:
8.6.9
[18/Jan/2019:16:19:25][39532.7fbf701ec700][-
[naviserver-devel] Windows nsd build breaks on tclhttp.c SSL_get_cipher
: 1 unresolved externals NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\Bin\link.exe"' : return code '0x460' Stop. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] TclOO in NaviServer, How to use?
NaviServer (and OpenACS) include integration with NSF/XOTcl, so that's what I'd try for object oriented work in NaviServer, not TclOO. Btw, my understanding is that NSF/XOTcl is strictly more capable than TclOO, and has certainly been in real-world use much longer, so it'd be more first choice to try anyway. The better NaviServer integration just makes the choice more obvious. -- Andrew Piskorski <a...@piskorski.com> -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] does NsDbLogSql() do anything?
NsDbLogSql() is defined in naviserver/nsdb/dbinit.c, and sometimes calls Ns_Log(Ns_LogSqlDebug). But at least with the nsoracle driver, NsDbLogSql() apparently NEVER gets called at all! (I temporarily put in Ns_Log calls to be sure.) Is it supposed to? After learning about NaviServer's "ns_logctl severity Debug(sql) on" approach to SQL logging, I fixed nsoracle so that SQL logging actually works: https://bitbucket.org/naviserver/nsoracle/commits/cf5986c3b2e6ecd5cf9970d4c7ad37f667fc4c62 http://openacs.org/forums/message-view?message_id=5374519 https://bitbucket.org/naviserver/naviserver/src/tip/openacs-config.tcl?fileviewer=file-view-default But I was surprised that the Ns_log statements that make the SQL logging happen, are apparently all issued directly by the Oracle driver. The generic nsdb layer and NsDbLogSql() don't seem to be involved at all. Is that normal for NaviServer database drivers, or is there something funny about nsoracle? -- Andrew Piskorski <a...@piskorski.com> -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] tests need update: cookie-4.5, cookie-4.7
I updated my NaviServer build from the c. 2017-08-14 head to today's
latest code, and now the two cookie tests below "fail". However, it
looks like this is not a real problem, the tests just need to be
updated to accept the new "no such cookie" string introduced in
changeset 4935:a71c34edd12, today on 2017-08-21.
cookie-4.5 get missing cookie FAILED
Contents of test case:
nstest::http -getbody 1 GET /cookie-4.5
Result was:
200 {no such cookie}
Result should have been (exact matching):
200 {no matching cookie}
cookie-4.5 FAILED
cookie-4.7 get missing cookie FAILED
Contents of test case:
nstest::http -getbody 1 -getheaders {Set-Cookie} GET /cookie-4.7
Result was:
200 {foo="foovalue"; HttpOnly} {no such cookie}
Result should have been (exact matching):
200 {foo="foovalue"; HttpOnly} {no matching cookie}
cookie-4.7 FAILED
--
Andrew Piskorski <a...@piskorski.com>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] test failures: ns_cache-12.1, hotp-1, totp-1
I installed nsf 2.1.0 from source, and that fixed the failing hotp-1 and totp-1 tests. -- Andrew Piskorski <a...@piskorski.com> -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] --enable-rpath
On Thu, Mar 05, 2015 at 03:01:10PM +, David Osborne wrote:
> I've been struggling to get --enable-rpath to work in our build (on Debian
> Wheezy). Up to now I'd always been patching Makefile.global.in during the
> Debian build to add -Wl,-rpath manually.
That was the same problem I had back in 2014. I see now that Gustaf
committed a clean fix a month later (f7808bea8567, below). It seems
to work well, both "--enable-rpath" (which is what I need) and
"--disable-rpath" are obeyed. Thank you both for figuring that out!
$ hg -v log -p -r f7808bea8567 configure.ac
changeset: 3548:f7808bea8567
user:Gustaf Neumann <neum...@wu-wien.ac.at>
date:Mon Apr 06 15:50:46 2015 +0200
files: configure.ac
description:
- Improved rpath handling in configure.ac for Linux distros,
where TCL_CC_SEARCH_FLAGS and TCL_LD_SEARCH_FLAGS are set
empty, like e.g.Debian (Many thanks to David Osborne for the recommendation)
diff -r 4230345c76e4 -r f7808bea8567 configure.ac
--- a/configure.ac Mon Apr 06 15:06:15 2015 +0200
+++ b/configure.ac Mon Apr 06 15:50:46 2015 +0200
@@ -133,8 +133,8 @@
LDSO="\$(LDLIB)"
CCRPATHS="\$(CCRPATH)"
LDRPATHS="\$(LDRPATH)"
-CCRFLAG=$TCL_CC_SEARCH_FLAGS
-LDRFLAG=$TCL_LD_SEARCH_FLAGS
+CCRFLAG=$CC_SEARCH_FLAGS
+LDRFLAG=$LD_SEARCH_FLAGS
if test "$CCRFLAG" = "" ; then
CCRPATH=
fi
--
Andrew Piskorski <a...@piskorski.com>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] test failures: ns_cache-12.1, hotp-1, totp-1
On the NaviServer head, (on Ubuntu 16.04.2 LTS, Linux 4.4.0-31,
x86_64), "make test" gives me the three errors below. Should I be
worried about these? Are these known problems, or is something broken
with my build?
Thanks!
ns_cache-12.1 transaction rollback FAILED
Result was:
content-length 3 content-length 4 1 1 2 2 3 k2 {k1 k2 k3 k4} 1 2 2 k1 1
Result should have been (exact matching):
1 1 2 2 3 k2 {k1 k2 k3 k4} 1 2 2 k1 1
ns_cache-12.1 FAILED
hotp-1 Test for HOTP based on testvector in Appendix D from RFC 4226
https://www.ietf.org/rfc/rfc4226.txt FAILED
Test generated error; Return code was: 1
Return code should have been one of: 0 2
hotp-1 FAILED
totp-1 Test for HOTP based on testvector from Appendix B of RFC 6238
https://tools.ietf.org/html/rfc6238 FAILED
Test generated error; Return code was: 1
Return code should have been one of: 0 2
totp-1 FAILED
--
Andrew Piskorski <a...@piskorski.com>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] Windows compilers, 2013 vs. 2010, etc.
On Fri, Nov 14, 2014 at 12:00:47PM +0100, Maurizio Martignano wrote: Subject: Re: [naviserver-devel] more logging problems on Windows Dear Andrew, I would like to draw your attention to the availability of Visual Studio 2013 Community Edition. http://msdn.microsoft.com/en-us/visual-studio-community-vs.aspx So the new thing here seems to be that with this latest version, the free Visual Studio GUI now includes both 32 and 64 bit support. In older versions with the 2010 and/or 2012 compilers, the GUI was only integrated with the 32-bit compiler. The 64-bit compilers were ALSO free, in the separate Windows SDK 7.1 for Windows 7 package, but only command line, no GUI. The 2010 compilers can build for versions of Windows as old as Windows XP, while it looks like the VS 2013 Community Edition supports ONLY Windows 7 and newer. That doesn't matter to me as I'm building solely for Windows 7 anyway, but it might to some. If I was starting from scratch, I'd probably go with the 2013 stuff, but I don't currently see any reason to upgrade from the older compiler I'm currently using. -- Andrew Piskorski a...@piskorski.com -- Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=157005751iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] more logging problems on Windows
On Thu, Nov 06, 2014 at 03:03:18PM +0100, Gustaf Neumann wrote:
The seemingly same write() operation, which refuses to write to
the access.log in driver.c works fine when called directly
in nslog.c.
https://bitbucket.org/naviserver/naviserver/commits/4cb76f20a8464fde98ac3147c6184a90442a2808
Wow, weird. I tried turning your new fix on/off, and confirm the
behavior you see. Yet as you said, the new code you added to
NsAsyncWrite() is the same as the old/normal code in NsAsyncWrite()!
So what could the difference possibly be? Since we're not actually
using the async writer thread feature at all, it's even the same
thread calling _write() in both cases. LogTrace() is in nslog.dll
while NsAsyncWrite() is in libnsd.dll, but I don't see how/why that
could possibly matter.
Microsoft's docs for _write() and _open() don't mention anything
suspicious:
http://msdn.microsoft.com/en-us/library/1570wh78%28v=vs.100%29.aspx
http://msdn.microsoft.com/en-us/library/z0kc8e3z%28v=vs.100%29.aspx
WinDbg thinks the source file used for Microsoft's _write() was:
f:\dd\vctools\crt_bld\self_x86\crt\src\write.c
Naturally I don't have that exact path, but I do have this, which
seems to be the same thing:
c:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\crt\src\write.c
For binary files, _write() is a reasonably simple wrapper around
WriteFile(), but it's surprisingly complicated for text-mode files.
My access log seems to always be file descriptor 3, so in WinDbg I set
a breakpoint like this:
bu NsAsyncWrite .if (fd = 0n3) {} .else {gc}
Naviserver opens its access log in text mode; that's the default, and
by stepping through in WinDbg I do see it enter some of the text-mode
sections in _write_nolock().
As a little experiment, in LogOpen() I added a _O_BINARY flag, so it
opened the access log like this:
fd = open(logPtr-file, O_APPEND|O_WRONLY|O_CREAT|_O_BINARY, 0644);
That made no difference to the behavior. (I did NOT run that under
WinDbg to confirm, but presumably it was in binary mode like I asked.)
So this lost writes bug doesn't seem to depend on text vs. binary
mode, it's the same for both.
So, I have no idea. The only ways I can imagine to try further to
track this down would be:
1. See if the bug is replicable in a standalone, non-Naviserver C
program. Maybe with two different DLLs involved.
2. Or, in NsAsyncWrite, do not use _write(), instead call WriteFile()
directly and see what happens.
That sounds like a lot of work though, much more than this really
warrants. I'm really glad you found a work-around, Gustaf! I suggest
we just leave it as is, but add some comments about why it's there.
--
Andrew Piskorski a...@piskorski.com
--
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] 'nsv_unset myarray' fails to unset the array
On Thu, Oct 30, 2014 at 06:45:20PM +0100, Gustaf Neumann wrote: I agree that the array should be gone after the call of nsv_unset a. The behavior is now fixed in the tip branch on bitbucket. Excellent, thank you, Gustaf! I earlier tried some simpler changes in my sandbox (essentially the approach used in AOLserver 4.0.10), which crashed the server. Your fix definitely works, but is more complicated than I expected. When you get a chance, can you say a bit more about how it works? E.g., what is the clientData for exactly, and why do some functions need to use it but others do not? Before calling Tcl_DeleteHashTable() to remove the nsv array, you do something interesting with BucketIndex() and an extra round of locking. Is that due to some new feature that AOLserver did not have, maybe the mutex timings? -- Andrew Piskorski a...@piskorski.com -- ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] log rolled every time Naviserver shuts down
On Sun, Nov 02, 2014 at 01:38:37PM +0100, Gustaf Neumann wrote: I've changed the code by using distinct bit-patterns for signal detection which should address the problem, but have just checked that the changes compile correctly under windows. That seems to fix the excess rolling problem on Windows, thanks! However, i've not located the code, where SIGHUP is handled on the windows side. Andrew, how do you send SIGHUP to nsd? The Windows SIGHUP handling is in NsHandleSignals() in nsd/nswin32.c, which you just fixed, right? Hm, how come NsSendSignal() only supports NS_SIGTERM, NS_SIGINT, and NS_SIGHUP, with no support for NS_SIGQUIT and NS_SIGPIPE ? Andrew, how do you send SIGHUP to nsd? In the distant past on either Linux or Solaris, I think I used an external shell script to send an external SIGHUP to the nsd process, but I never do that anymore. On Windows, the only way I've ever done it is by explicitly calling ns_logroll from the nsd process itself, typically in a scheduled procedure. In nsd/log.c, LogOpen() uses Ns_RegisterAtSignal() with Ns_LogRoll() for handling an incoming SIGHUP. But, the ns_logroll Tcl command simply explicitly calls Ns_LogRoll(), so I don't think it exercises the signal-handling machinery at all. Btw, I just got a copy of this book and read some of it: http://www.amazon.com/gp/product/0321657748/ Windows System Programming, 4th Edition; by Johnson M. Hart published Feb. 2010; ISBN-13: 978-0321657749 It seems pretty good, and has some useful compare/contrast discussion of Windows vs. POSIX thread synchronization APIs. Instead of Unix signals, Windows uses Console Control Handlers or Exceptions (depending on what you're trying to do). Naviserver uses a Console Control Handler. -- Andrew Piskorski a...@piskorski.com -- ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] more logging problems on Windows
On Sun, Nov 02, 2014 at 10:27:21PM -0500, Andrew Piskorski wrote: Two, calling ns_logroll crashed Naviserver with this error: Debug Assertion Failed! f:\dd\vctools\crt_bld\self_x86\crt\src\write.c Line: 68 Expression: (_osfile(fh) FOPEN) Once the nsd debug symbols were working, that one turned out to be an easy fix. All it took was moving the call to Ns_Log() AFTER LogOpen() rather than before it. Done here: https://bitbucket.org/apiskors/naviserver/commits/8d3c79974f084d074f44a9169be75bfa2c5f86bd -- Andrew Piskorski a...@piskorski.com -- ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] log rolled every time Naviserver shuts down
At least on Windows (I have checked this behavior on Linux), when I start Naviserver in a Command Shell and then shut it down by hitting Ctrl-c, it rolls both the error and access logs (e.g. renaming myserver-error.log to myserver-error.log.000). That's strange, and not what I expect or want. I want to roll logs only at certain times of day, not every time I restart the server. That roll-on-shutdown behavior goes away if I set these parameters: ns_section ns/parameters ns_param LogRoll false ns_section ns/server/my_server/module/nslog ns_param RollOnSignal false But I expect that will interfere with scheduled log rolling as well. I WOULD like to roll the logs on SIGHUP, just not every time the server shuts down. Any idea why the two seem to be conflated? Is Windows actually sending a HUP signal when I hit Ctrl-c? (I don't THINK so.) Or is Navisever somehow misinterpreting whatever Windows does as SIGHUP? This is a very minor problem but I do wonder just what is going on there. (Thanks!) -- Andrew Piskorski a...@piskorski.com -- ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Windows Service startup fails
On Thu, Oct 23, 2014 at 12:45:36PM +0200, Gustaf Neumann wrote: By reading the code for windows services, i see that the corresponding thread is named -service-. I've extended the sanity check to include this value. if this is not sufficient, please report, what the returned threadName is. With your change, it still triggered the assertion. I added this code, and when run as a windows service, the logged threadName is empty string: char *threadName = Ns_ThreadGetName(); Ns_Log(Notice, CreateConnThread: threadName is: '%s', threadName); Does an empty string thread name indicate a problem somewhere else, or is it ok and just something special about Windows Services? The good news is, with that the assertion turned off completely, Naviserver now appears to run ok as a Windows Service, yay! -- Andrew Piskorski a...@piskorski.com -- ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Windows Service startup fails
On Wed, Oct 22, 2014 at 09:46:11AM +0200, Gustaf Neumann wrote:
sorry, I've added this nonnull constraint, which was too eager. I missed
that invocation path via windows. This is fixed in the bitbucket tip
Well, last night I tried allowing a NULL initProc there, but it then
failed this assertion on line 1624 of nsd/queue.c, in CreateConnThread():
{ char *threadName = Ns_ThreadGetName();
assert(strncmp(-driver:, threadName, 8) == 0
|| strncmp(-main-, threadName, 6) == 0
|| strncmp(-spooler, threadName, 8) == 0
);
}
Is this a separate problem, not a consequence of skipping some sort of
needed initialization because the initProc from ServiceMain() was NULL?
--
Andrew Piskorski a...@piskorski.com
--
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Windows Service startup fails
On Mon, Oct 20, 2014 at 10:12:47PM -0400, Andrew Piskorski wrote: Fatal: nsmain: no such server 'w7-1-prod' But, that is the correct server name from my config file, and it works fine when I'm not running as a Windows Service! I'm confused, what could cause it to fail ONLY when running as a Windows Service? Ok, after further debugging, that particular problem was my fault, but there are others. In my config file, I decide whether to set up the dev or prod server based on a regexp of the full path of the config file (from ns_info config). And that path now has backslashes, not the forward slashes expected by my regexp (which worked in the past). So likely I installed this Naviserver service with a command line using backslashes, while for AOLserver I used forward slashes. Whatever, clearly that was a bug in my config file, it needs to be robust to that sort of thing. Now with that fixed, the service startup gets farther, and instead fails in NsInitServer with: Assertion failed! File: server.c Line: 220 Expression: initProc != NULL That confuses me. because NsInitServer is only called from two places, both of them right next to each other in Ns_Main. So that assertion must mean that the initProc being passed into Ns_Main is NULL, which is only called from main, like so: return Ns_Main(argc, argv, ServerInit); But where does that ServerInit thing come from? And why is it NULL when running as a Windows Service, but ok when Naviserver is started interactively? -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Windows Service startup fails
On Tue, Oct 21, 2014 at 03:22:59PM -0400, Andrew Piskorski wrote: return Ns_Main(argc, argv, ServerInit); But where does that ServerInit thing come from? And why is it NULL when running as a Windows Service, but ok when Naviserver is started interactively? Ah, I see: ServiceMain() explicitly passes in a NULL initProc: Ns_Main((int)argc, argv, NULL); So, is it supposed to be passing in something else, or is it ok for that to be NULL and the assertion is inappropriate in this case? Where are these initProc pointers supposed to come from? Are we missing a call to NsRegisterServerInit() somewhere? Is there a document somewhere explaining the design of Naviserver's virtual servers, what they're for and how the code underneath sets them up? Maybe in an older email? So far I found this thread, which gives some hints but not a full explanation: http://www.mail-archive.com/naviserver-devel%40lists.sourceforge.net/msg01707.html Re: [naviserver-devel] Explanation of last changes Vlad Seryakov Thu, 07 Sep 2006 12:38:46 -0700 -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_cond is broken on Naviserver
On Sat, Oct 18, 2014 at 01:56:10PM +0200, Gustaf Neumann wrote: Dear Andrew, i have fixed the problem in the head branch. Your test is working for me, at least under Mac OS X and Linux. Excellent, thank you, Gustaf! I confirm that ns_cond now works on both Linux and Windows. -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] ns_cond vs Tcl event loop?
Although the C version of ns_cond is used in various places in Naviserver itself, clearly very few people actually use the Tcl ns_cond command. I have ns_cond working nicely for my needs and so don't plan to change anytime soon, but I do wonder: Is there a higher-level, more Tcl'ish way to accomplish the same thing (waking up another thread) as ns_cond these days? Perhaps something with the Tcl event loop? Or some other form of message passing? I expect that ns_cond will be more efficient, since it a thin layer over the POSIX C API, but it is tricky enough to use correctly that a more user-friendly alternative could be nice when doing general-purpose multi-threaded programming in Tcl. Of course, part of the beauty of Naviserver and AOLserver is that you rarely need to DO general-purpose multi-threaded programming, because they're already set up for the typical concurrency needs of a web server. But it is very nice that they (and the Tcl Threads Extension) have the tools available when you need to do something different. -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] sometimes crashes on shutdown
On Sat, Oct 18, 2014 at 01:54:22PM +0200, Gustaf Neumann wrote: My understanding is that it happens due to varying deletion orders during shutdown, probably influenced by the traversal order in hash tables. Hard to debug, but for practical applications without much value. Good to know. Also, I've read that Tcl 8.6 has simplified its process exit code, so with luck this will get easier to debug in the future. Btw, is anybody running Naviserver with Tcl 8.6 yet? I was going to try it (on Windows), but noticed that unlike the earlier versions, ActiveTcl 8.6 ships with almost no add-on packages, not even tcllib. -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Windows Service startup fails
On Mon, Oct 20, 2014 at 04:07:09PM -0400, Andrew Piskorski wrote: Nothing at all gets written to the log file, so I'm not sure how to debug this. But presumably that means something is dying very early, perhaps in NsConnectService()? Briefly, I managed to the nsd Windows Service to start under the WinDbg debugger. There, it clearly emits a few ns_log lines I have in my config file, and then says: Fatal: nsmain: no such server 'w7-1-prod' But, that is the correct server name from my config file, and it works fine when I'm not running as a Windows Service! I'm confused, what could cause it to fail ONLY when running as a Windows Service? Additional gory detail: After much googling, I found and followed the (rather tedious) instructions in both of these pages: http://support.microsoft.com/kb/824344 http://ww2.ntrglobal.com/support/why-do-i-get-error-message-regarding-interactive-services That let me attach the WinDbg debugger to the Service. The key bits to set that up were: - Run C:\Program Files\Debugging Tools for Windows (x64)\gflags.exe, configure nsd.exe to use this debugger: C:\Program Files\Debugging Tools for Windows (x64)\windbg.exe - Run regedit, and increase the millisecond counter for HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\ServicesPipeTimeout to an hour or so. - In Services, for my NaviServer service, under Log On, Local System account, TURN ON the Allow service to interact with desktop check box. - Reboot Windows. - Manually start up the Interactive Service Detection service BEFORE trying to start the NaviServer service. With that, when I start the service an icon starts blinking on the bottom Windows tool bar. Clicking on it pops up a special dialog box, Interactive Services Detection, and clicking View the Message then takes me to a weird separate Windows desktop (under a different user, but no icons or start menu). There the nsd.exe service is running under WinDbg, and this is the only place I ever see the Fatal: nsmain: no such server message. After the no such server message prints out, it seems nsd.exe has already exited, because the backtrace in WinDbg says I am in function: wow64cpu!TurboDispatchJumpAddressEnd Which ultimately was called by: ntdll!LdrInitializeThunk If I set a breakpoint on Ns_Fatal or Ns_SetFind, I can get it to stop with actual Naviserver code on the stack. WinDbg's ~ (threads) and | (process) commands show 5 threads, only the first one of which seems to be doing much of anything. Oddly, the Ns_Main function does NOT appear on the stack trace, only main, which can't be correct because the only thing main does is call Ns_Main. I can't seem to get WinDbg to examine most of the interesting Ns_Main or NsConnectService code. Possibly because: Stack unwind information not available. Following frames may be wrong. That I don't understand, because I compiled with /Zi complete debugging information, and have all the *.pdb files in my lib directory, d which I put onto WinDbg's symbol path and told it to reload. Odlly, WinDbg knows the correct line number for nsd!main in nsd/main.c, but claims it does not have any symbol info for Ns_SetFind. I can get it to stop at a Ns_SetFind breakpoint, but can't seem to see anything of use. -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] ns_cond is broken on Naviserver
I tried to bisect, but getting old versions of Naviserver to actually build and run tests is extraordinarily frustrating. I gave up. I see various places where Naviserver itself uses the Ns_Cond* C functions, which makes me think they likely work. Also, the Unix and Windows C implementations are in two different files. Since the problems I see with Tcl ns_cond are the same on Linux and Windows, perhaps the problem is in nsd/tclthread.c, which is cross-platform. Most of NsTclCondObjCmd() dates from 2005-10-21, when Stephen Deasey pretty much completely rewrote it. Maybe that's a likely place for bugs? I wasn't able to build or test the code from around then though, so it's just an arbitrary guess on my part. -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] sometimes crashes on shutdown
Btw, simply running all the tests with make test sometimes crashes Naviserver while it is shutting down; see below. Or just running this one test (which is fast) several times works pretty well for triggering a crash. Other tests do it too though, not just this one: make test TCLTESTARGS='-file ns_nsv.test' [18/Oct/2014:05:27:59][18593.2ac57b648240][-main-] Notice: nsmain: NaviServer/4.99.6 stopping [18/Oct/2014:05:27:59][18593.2ac57b648240][-main-] Notice: [driver:nssock]: stopping [18/Oct/2014:05:27:59][18593.2ac57b648240][-main-] Notice: server [test]: stopping [18/Oct/2014:05:27:59][18593.2ac57b648240][-main-] Notice: server [testvhost]: stopping [18/Oct/2014:05:27:59][18593.2ac57b648240][-main-] Notice: server [testvhost2]: stopping [18/Oct/2014:05:27:59][18593.2ac5a4200700][-driver:nssock-] Notice: exiting [18/Oct/2014:05:27:59][18593.2ac588c10700][-conn:test:1] Notice: exiting: shutdown pending [18/Oct/2014:05:27:59][18593.2ac588e11700][-conn:testvhost:0] Notice: exiting: shutdown pending [18/Oct/2014:05:27:59][18593.2ac589012700][-conn:testvhost2:0] Notice: exiting: shutdown pending [18/Oct/2014:05:27:59][18593.2ac588a0f700][-conn:test:0] Notice: exiting: shutdown pending [18/Oct/2014:05:27:59][18593.2ac58880e700][-conn:test:emergency:0] Notice: exiting: shutdown pending called Tcl_FindHashEntry on deleted table Aborted (core dumped) make: *** [test] Error 134 Using host libthread_db library /lib/x86_64-linux-gnu/libthread_db.so.1. Core was generated by `./nsd/nsd -u root -c -d -t /usr/local/web/ns-fork/nv-tcl85-linux/tests/test.nsc'. Program terminated with signal 6, Aborted. (gdb) bt #0 0x2ac57a22d425 in __GI_raise (sig=optimized out) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x2ac57a230b8b in __GI_abort () at abort.c:91 #2 0x2ac57acb634e in Tcl_PanicVA () from /usr/lib/libtcl8.5.so.0 #3 0x2ac57acb640c in Tcl_Panic () from /usr/lib/libtcl8.5.so.0 #4 0x2ac57ac8e079 in ?? () from /usr/lib/libtcl8.5.so.0 #5 0x2ac57acd12fa in ?? () from /usr/lib/libtcl8.5.so.0 #6 0x2ac57accff5e in Tcl_GetThreadData () from /usr/lib/libtcl8.5.so.0 #7 0x2ac57acb3cfa in TclFreeObj () from /usr/lib/libtcl8.5.so.0 #8 0x2ac57acb5d45 in ?? () from /usr/lib/libtcl8.5.so.0 #9 0x2ac57ac8db82 in Tcl_DeleteHashTable () from /usr/lib/libtcl8.5.so.0 #10 0x2ac57ac730b8 in ?? () from /usr/lib/libtcl8.5.so.0 #11 0x2ac57acb3d59 in TclFreeObj () from /usr/lib/libtcl8.5.so.0 #12 0x2ac57acb222d in ?? () from /usr/lib/libtcl8.5.so.0 #13 0x2ac57ac3b995 in Tcl_DeleteCommandFromToken () from /usr/lib/libtcl8.5.so.0 #14 0x2ac57acac0dc in TclTeardownNamespace () from /usr/lib/libtcl8.5.so.0 #15 0x2ac57ac3a3dc in ?? () from /usr/lib/libtcl8.5.so.0 #16 0x2ac579f9062e in DeleteInterps (arg=0x20be470) at tclinit.c:1861 #17 0x2ac57aa0c814 in NsCleanupTls (slots=0x31ac220) at tls.c:186 #18 0x2ac57aa0d16a in CleanupTls (arg=optimized out) at pthread.c:816 #19 0x2ac57b135c83 in __nptl_deallocate_tsd () at pthread_create.c:156 #20 0x2ac57b135ea8 in start_thread (arg=0x2ac58880e700) at pthread_create.c:315 #21 0x2ac57a2eb3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #22 0x in ?? () (gdb) -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] nsthreadtest fails on both Linux and Windows
On Thu, Oct 16, 2014 at 09:53:15AM +0200, Gustaf Neumann wrote: Am 16.10.14 08:14, schrieb Gustaf Neumann: i can look into the issue, ... nevermind, fixed on tip. The problem was missing tcl initialization (required in Tcl 8.5) So it just needed a call to Tcl_FindExecutable()! The Tcl docs aren't very clear on that, but the comments in the Tcl source code are very clear that you must call it. Good to know. Gustaf, I just merged in all your fixes, and now nsthreadtest runs to completion for me on both Linux and Windows, and with Tcl 8.4 and 8.5. So nsthreadtest looks good now. https://bitbucket.org/naviserver/naviserver/commits/72b9fbac08d60479328ded73afd668faa993da70 http://tcl.activestate.com/man/tcl8.5/TclLib/FindExec.htm tcl-core-8-5-branch/win/tclWin32Dll.c * The three NULL function pointers will only be set when * Tcl_FindExecutable is called. If you don't ever call that function, the * application will crash whenever WinTcl tries to call functions through * these null pointers. That is not a bug in Tcl - Tcl_FindExecutable is * mandatory in recent Tcl releases. -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] nsthreadtest fails on both Linux and Windows
On Thu, Oct 16, 2014 at 08:14:09AM +0200, Gustaf Neumann wrote: i can look into the issue, but i am right now quite busy. For what purpose do you need nsthreadtest? is out of curiosity, do you want to check, if everything works, or is there an application need? I only care about nsthreadtest in order to debug the Naviserver code. Now nsthreadtest is working on all the combinations of platform and versionf of Tcl I tried, great. Unfortunately, nsd itself has some problems. In my very limited testing, nsd linked with Tcl 8.5 appears to run mostly ok on both Linux and Windows. However, at least on Windows, it has buggy behavior - ns_cond silently does nothing. I have no idea why. I haven't tested yet whether Linux has the same problem. Linked with Tcl 8.4, nsd immediately crashes on startup, on both Linux (backtrace below) and Windows. This surprises me... Any ideas? Does this suggest some sort of incorrect linking in my build, or a bug somewhere in the Naviserver code? Has anybody actually been using Tcl 8.4 with Naviserver lately? Staring it up with nsd -c threw a whole bunch Tcl errors because the namespace ensemble command only exists in Tcl 8.5, not 8.4. I fixed that here: https://bitbucket.org/apiskors/naviserver/history-node/bfbb7d1dcfa5/tcl/nstrace.tcl warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library /lib/x86_64-linux-gnu/libthread_db.so.1. Core was generated by `/usr/local/nsd-t84-atp-1/bin/nsd -f -t /usr/local/nsd-t84-atp-1/conf/nsd-config'. Program terminated with signal 6, Aborted. #0 0x7fc8e754f425 in __GI_raise (sig=optimized out) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 0x7fc8e754f425 in __GI_raise (sig=optimized out) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x7fc8e7552b8b in __GI_abort () at abort.c:91 #2 0x7fc8e791c4d9 in Panic (fmt=optimized out) at log.c:616 #3 0x7fc8e6c7129d in Tcl_PanicVA () from /usr/lib/libtcl8.4.so.0 #4 0x7fc8e6c71409 in Tcl_Panic () from /usr/lib/libtcl8.4.so.0 #5 signal handler called #6 0x7fc8e6c54b89 in Tcl_CreateHashEntry () from /usr/lib/libtcl8.4.so.0 #7 0x7fc8e7934382 in Ns_TclDestroyInterp (interp=0x1453730) at tclinit.c:521 #8 0x7fc8e790b3ac in NsConfigEval ( config=0x146b160 #set, ' ' repeats 13 times, home, ' ' repeats 16 times, /usr/local/ns\nset, ' ' repeats 13 times, home, ' ' repeats 16 times, [file dirname [file dirname [info nameofexecutable]]]\n\nns_section \ns/server/default/modules\\nns_param ..., argc=4, argv=optimized out, optind=4) at config.c:717 #9 0x7fc8e791e8ac in Ns_Main (argc=4, argv=0x7fff4d205348, initProc=0x400750 ServerInit) at nsmain.c:449 #10 0x7fc8e753a76d in __libc_start_main (main=0x400660 main, argc=4, ubp_av=0x7fff4d205348, init=optimized out, fini=optimized out, rtld_fini=optimized out, stack_end=0x7fff4d205338) at libc-start.c:226 #11 0x00400695 in _start () (gdb) -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] nsthreadtest fails on both Linux and Windows
On Wed, Oct 15, 2014 at 10:24:19PM +0100, Stephen wrote: You have a known good revision: aolserver-4.0.10, a known bad: tip, and an automated test to distinguish between the two states: nsthreadtest. Try 'hg bisect ...' to figure out where the error was introduced. Good idea. The automatic bisect didn't give me the answer, but it did get me started down a useful path. The culprit with nsthreadtest seems to be, Tcl 8.5! I typically generate and run the configure script like this: AUTOCONF=autoconf2.50 ./autogen.sh --prefix=$inst_dir --enable-symbols --enable-rpath --with-tcl=/usr/lib/tcl8.5 --with-zlib=/usr/lib Turns out, it doesn't matter what version of the C code I use, and it doesn't matter if I run autoconf from the tip or use the ancient configure script from 2005 (which was deleted on 2005-10-09 in rev 1345:f6af29cbe4fb). All that seems to matter, is that nsthreadtest ALWAYS fails when I configured with this: --with-tcl=/usr/lib/tcl8.5 And if I instead use Tcl 8.4 like this it works fine: --with-tcl=/usr/lib/tcl8.4 I can imagine several possible causes for this: - A bug in Tcl 8.5. - A bug in how Naviserver uses Tcl 8.5. - Some side effect of the configure process. - A broken Tcl 8.5 on my machine(s). Note that I see the very same sort of nsthreadtest failures on Windows 7. I never run configure or autoconf on Windows. Unfortunately there is some crosstalk with Linux, because the Windows build uses some subset of the stuff (via NFS) generated by Linux configure. (That is horribly ugly, but hasn't seemed to break anything so far and I haven't gone back and tried to fix it yet.) On Linux I am using the stock Tcl 8.4 and 8.5 included with Ubuntu 10.04.4 LTS. On Windows 7 I am using ActiveTcl 8.4 and 8.5 (32-bit), both installed into the same C:\P\tcl-32\ directory tree. So, what could possibly be wrong here that shows itself by nsthreadtest passing with Tcl 8.4, but crashing with 8.5? Next, I rebuilt my 32-bit Naviserver on Windows with ActiveTcl 8.4, and its nsthreadtest now passes too! So Linux and Windows are consistent, 8.4 nsthreadtest works, 8.5 fails. Unfortunately, Linux and Windows are consistent in another way - on both platforms, my new Tcl 8.4 Naviserver build fails in at least two ways: One, running Naviserver with the sample conf/nsd-config.tcl file dumps core. On Linux that happens while running in NsConfigEval(); the backtrace is below. Two, running nsd -c throws a Tcl backtrace: bad option ensemble: must be children, code, current, delete, eval, exists, export, forget, import, inscope, origin, parent, qualifiers, tail, or which while executing namespace ensemble exists $cmd invoked from within nstrace::statescript I will try to debug these new 8.4 failures more tomorrow. # Linux, Tcl 8.4, nsthreadtest passes but Naviserver fails like this: warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library /lib/x86_64-linux-gnu/libthread_db.so.1. Core was generated by `/usr/local/nsd-t84-atp-1/bin/nsd -f -t /usr/local/nsd-t84-atp-1/conf/nsd-config'. Program terminated with signal 6, Aborted. #0 0x7fb77715d425 in __GI_raise (sig=optimized out) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 0x7fb77715d425 in __GI_raise (sig=optimized out) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x7fb777160b8b in __GI_abort () at abort.c:91 #2 0x7fb77752a4d9 in Panic (fmt=optimized out) at log.c:616 #3 0x7fb77687f29d in Tcl_PanicVA () from /usr/lib/libtcl8.4.so.0 #4 0x7fb77687f409 in Tcl_Panic () from /usr/lib/libtcl8.4.so.0 #5 signal handler called #6 0x7fb776862b89 in Tcl_CreateHashEntry () from /usr/lib/libtcl8.4.so.0 #7 0x7fb777542382 in Ns_TclDestroyInterp (interp=0x1497730) at tclinit.c:521 #8 0x7fb7775193ac in NsConfigEval ( config=0x14af160 #set, ' ' repeats 13 times, home, ' ' repeats 16 times, /usr/local/ns\nset, ' ' repeats 13 times, home, ' ' repeats 16 times, [file dirname [file dirname [info nameofexec\ utable]]]\n\nns_section \ns/server/default/modules\\nns_param ..., argc=4, argv=optimized out, optind=4) at config.c:717 #9 0x7fb77752c8ac in Ns_Main (argc=4, argv=0x7fff38339a18, initProc=0x400750 ServerInit) at nsmain.c:449 #10 0x7fb77714876d in __libc_start_main (main=0x400660 main, argc=4, ubp_av=0x7fff38339a18, init=optimized out, fini=optimized out, rtld_fini=optimized out, stack_end=0x7fff38339a08) at libc-start.c:226 #11 0x00400695 in _start () (gdb) -- Andrew Piskorski a...@piskorski.com -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push
[naviserver-devel] 32-bit Naviserver fails on Windows
Well, I belatedly realized that I need to build a 32-bit Naviserver on my 64-bit Windows 7 box. (The old proprietary library I currently need to link against is only available in 32-bit. I hope to port to a newer library, but first I need to get my old code actually working with Naviserver on this box.) So I configured my command prompt build with /x86 rather than /x64, like so: %ProgramFiles%\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd /Debug /x86 /win7 nmake -f Makefile.win32 all-core My branch includes better build support for 32-bit (although still not yet good enough): https://bitbucket.org/apiskors/naviserver/commits/all My new 32-bit Naviserver can't even run nsd -h! It dies immediately with: The application was unable to start correctly (0xc07b). Click OK to close the application. Looks like it dies before invoke any Naviserver or Tcl code at all, so I think either I've linked Naviserver wrong, or something is broken on my Windows box. I verified that all my compiled Naviserver code is indeed 32-bit with this: dumpbin /headers *.exe *.dll | findstr machine Oddly, Dependency Walker thinks that ALL of the external DLLs nsd.exe is using are 64-bit! That would be very bad, but I'm not sure if it's actually true. Reports on the web say that Dependency Walker can incorrectly report such problems because it doesn't ignore 64-bit libraries the sam way Windows itself does. http://www.dependencywalker.com/ Naviserver's include/Makefile.win32 links against these Windows libraries: kernel32.lib advapi32.lib ws2_32.lib user32.lib Do I need to do something special to find the 32-bit versions of those? -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://p.sf.net/sfu/Zoho ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Wed, Oct 08, 2014 at 08:05:42AM +0200, Maurizio Martignano wrote: I do agree with your remarks in your mail, I had a look at the Visual Studio Project Files you use, the only relevant differences I can see are the I do not use any Visual Studio Project Files for Naviserver. The ones in Mercurial are old and I have no idea if they work or not. The Naviserver Windows build I've been working on is controlled by the makefiles, a combination of the two Windows-only Makefile.win32 for nmake, and the individual module-specific makefiles which work in both Gnu make and Microsoft nmake. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] warning, shift count undefined behavior
On Wed, Oct 08, 2014 at 08:14:14AM +0200, Maurizio Martignano wrote: Dear Andrew, Yes, it is a potential issue. http://sonarsrv.spazioit.com:9000/dashboard/index?id=my%3Anaviserver%3Ansd%2 Neat! Hm, it uses SonarQube, which I'd never heard of before. What static analyzer is ultimately being used to generate that info? I couldn't seem to save a direct link to the shift/cast issue in nsd/sockfile.c, but it is pretty easy to browse to it. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Wed, Oct 08, 2014 at 06:50:35PM +0200, Maurizio Martignano wrote: Dear Andrew and Gustaf, In due time, when you both are confident on the status of Naviserver on Windows, I would try it inside my Windows-OpenACS distribution. That sounds excellent to me. My own use of Naviserver on Windows will be in a rather narrow application (not OpenACS; we use Linux for that), so it would be great to see it tested on a wider range of workloads. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Fri, Oct 03, 2014 at 04:16:41PM +0200, Maurizio Martignano wrote: Dear Andrew, These are the basic options I would use for a Windows 64 bit, AMD64: COPTS = /EHsc /W3 /nologo /c DEFS = $(DEFS) /D _WINDOWS /D TCL_THREADS=1 /D WIN32 /D _WIN32 \ /D FD_SETSIZE=128 /D NO_CONST=1 /D _MBCS # /D _USE_32BIT_TIME_T LOPTS = /NOLOGO /SUBSYSTEM:CONSOLE /OPT:NOREF /OPT:NOICF Did you comment out your _USE_32BIT_TIME_T there? What does /D _WIN32 do? What C run-time library are you using? I've been able to find Microsoft docs on the various compiler and linker flags, but NOT anything clear on what all the magic /D preprocessor definitions actually do. The nmake files I got from Ibrahim (at Archiware) use the /MTd compiler flag, which should give me the debug version of the libcmtd.lib CRT, as explained here: http://msdn.microsoft.com/en-us/library/abx4dbyh%28v=vs.100%29.aspx http://msdn.microsoft.com/en-us/library/2kzt1wy3%28v=vs.100%29.aspx The other main option is /MDd for msvcrtd.lib, but I don't know which is the default when you pass no flags at all. Most of our other build options are the same. However, besides the ones I mentioned above, I am also using these extra flags that you do not have: In COPTS: /Zi /RTC1 In DEFS: /D _CRT_SECURE_NO_WARNINGS /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _DEBUG In LOPTS: /DEBUG AFAICT those all just enable additional debugging and/or run-time checks, and so should be pretty safe. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Mon, Oct 06, 2014 at 05:18:08PM -0400, Andrew Piskorski wrote: On Sun, Oct 05, 2014 at 09:37:39PM +0200, Maurizio Martignano wrote: Did you use the define _USE_32BIT_TIME_T yes or not? I haven't tried using _USE_32BIT_TIME_T yet, but I think using it would be INCORRECT on Windows-64. If I do pass the /D _USE_32BIT_TIME_T flag, I get this: C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\INCLUDE\crtdefs.h(463) : fatal error C1189: #error : You cannot use 32-bit time_t (_USE_32BIT_TIME_T) with _WIN64 So that seems pretty clear, I must NOT set that flag. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Tue, Oct 07, 2014 at 04:09:02PM -0400, Andrew Piskorski wrote:
C:\ C:\web\nsd-atp\bin\nsd -c
[384.e14][-main-] Notice: nsmain: NaviServer/4.99 starting
[384.e14][-main-] Fatal: nsthreads: localtime_s failed in ns_localtime: win32
err: 22
In ns_localtime(), I changed this call:
errNum = localtime_s(tlsPtr-ltbuf, clock);
to instead do this:
struct tm foo;
errNum = localtime_s(foo, clock);
And yet it still fails in exactly the same way, with EINVAL = 22.
This makes me think there is nothing wrong with the tlsPtr code or
data, rather something is seriously broken in my build. (But I have
no idea what.)
Possibly of interest, is that _localtime64_s() fails in exactly the
same way. _localtime32_s() apparently does not trigger EINVAL, but
instead fails on the same debug assertion, where the tm_mday value is
out of range (and most likely -1 just like before, although I did not
confirm that).
I increased my compiler warning level from /W1 to /W3, and now I see
LOTS of type warnings, e.g.:
log.c(689) : warning C4133: 'function' : incompatible types - from 'long *'
to 'const time_t *'
nssock.c(377) : warning C4133: 'function' : incompatible types - from 'int *'
to 'const char *'
binder.c(324) : warning C4244: 'function' : conversion from 'SOCKET' to
'int', possible loss of data
driver.c(3888) : warning C4244: '=' : conversion from 'SOCKET' to 'int',
possible loss of data
driver.c(2356) : warning C4244: 'function' : conversion from 'ssize_t' to
'unsigned int', possible loss of data
driver.c(3475) : warning C4244: '=' : conversion from 'ssize_t' to 'off_t',
possible loss of data
driver.c(3351) : warning C4244: 'function' : conversion from 'Tcl_WideInt' to
'long', possible loss of data
driver.c(2346) : warning C4267: 'function' : conversion from 'size_t' to
'int', possible loss of data
driver.c(3354) : warning C4267: 'function' : conversion from 'size_t' to
'unsigned int', possible loss of data
tclhttp.c(371) : warning C4244: '=' : conversion from '__int64' to 'int',
possible loss of data
That first one in log.c looks mildly suspicious. But, on a 64-bit
machine like this long and time_t are both 64-bit integers, so I
don't see the problem. Why are they incompatible?
http://msdn.microsoft.com/en-us/library/323b6b3k%28v=vs.100%29.aspx
However, in include/nsthread.h, why is Ns_Time defined like this?:
typedef struct Ns_Time {
longsec;
longusec;
} Ns_Time;
Shouldn't the type of sec be time_t rather than long?
--
Andrew Piskorski a...@piskorski.com
--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] warning, shift count undefined behavior
Hm, this warning sounds kind of scary: sockfile.c(351) : warning C4293: '' : shift count negative or too big, undefined behavior That's in the Windows-only implementation of pread(), in nsd/sockfile.c: overlapped.Offset = (DWORD)offset; overlapped.OffsetHigh = ((DWORD)offset 32); I don't know if it's a real problem or how to fix it. Some related docs: http://msdn.microsoft.com/en-us/library/cx0bb1cy.aspx http://msdn.microsoft.com/en-us/library/windows/desktop/ms684342%28v=vs.85%29.aspx -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] modules/tcl/ is empty, tcl/ is full
On Sun, Oct 05, 2014 at 08:01:13PM +0200, Gustaf Neumann wrote: Am 02.10.14 23:40, schrieb Andrew Piskorski: Do we actually need the modules/tcl/ directory for anything now? yes, we need it for installing site specific tcl library files. Ah, so presumably Naviserver's moving of its own files from modules/tcl/ into tcl/ was in order to keep the Navisever and site-specific tcl files separate. Ok, that makes sense. -- Andrew Piskorski a...@piskorski.com -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Sun, Oct 05, 2014 at 09:31:31PM +0200, Gustaf Neumann wrote: On busy systems, Ns_GetTime() is one of the most frequent calls, used e.g. for mutex timings, so this makes a difference. Yes, so it's better for performance reasons, like I said. You may have written that small paragraph of code from scratch, but it is still true that inside the ifdef you added, the code calling gettimeofday() is essentially identical to what AOLserver used for many years. This is all fine, I see no problems with that code. What i did was to extend configure to look, if there is gettimeofday() Actually, and strangely, my Naviserver on Windows appeared to NEVER call gettimeofday() there, regardless of whether HAVE_GETTIMEOFDAY was true or false! I tried it both ways. I can't explain that; perhaps I was merely confused. But what is much more important to me, is that running the final portable Tcl-dependent code at the end definitely triggered a serious bug, making Naviserver unusable. b) When Tcl_GetTime() is not working on your system than you have a broken Tcl installation. Does this happen with your self-compiled tcl version? I do not know whether Tcl_GetTime() is working correctly in ActiveTcl 8.5 on Windows 7 64-bit or not. I will try something to verify that. Right now all I can say for sure is that funny indirect way that Naviserver calls Tcl_GetTime() in Ns_GetTime() fails horribly on my Windows build. So far I've never compiled Tcl on Windows 7 myself. I may try that soon to help with the debugging, but so far I've been exclusing using ActiveTcl 8.5. I strongly suspect that ActiveTcl is compiled correctly, but perhaps I am somehow linking against it incorrectly, or using incompatible build options, or something like that. http://msdn.microsoft.com/en-us/library/a442x3ye.aspx The *tp and *clock values look like a correct time_t count of seconds since the Unix epoch. So perhaps something is wrong with the only other argument ther to localtime_s(): tlsPtr-ltbuf The page you are citing contains the error conditions, when EINVAL is returned. The first argument can't be NULL, so clock must be invalid according to this information. No, the clock value is not invalid, I already checked that and it is a correct count of seconds since the Unix epoch. The problem must lie either in the other argument, or in something entirely separate and out of band (e.g., build options). on this issue. Have you tried the suggestion of Maurizio concerning USE_32BIT_TIME_T? I have not tried using it yet, but maybe I should. Previously, I wasn't sure if Maurizio was recommending it or warning against it. (I see now that he recommends using it.) My previous experience with AOLserver 4.0.x on Windows was with 32-bit Windows, which would have natively used 32-bit time_t. So perhaps forcing my 64-bit build to use 32-bit time_t would be useful. -- Andrew Piskorski a...@piskorski.com -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Mon, Oct 06, 2014 at 10:16:31AM -0700, Jeff Rogers wrote: If there is not significant performance advantage to the platform-specific optimized version of Ns_GetTime over the platform-independent Tcl_GetTime, is it worth it to keep the optimized version? It is very much worth it to me, at least for now, because: One, I don't understand how the invocation of the platform-independent Tcl_GetTime even works. There is never any explicit call to Tcl_GetTime nor any other Tcl function, and Tcl_GetTime does not appear on the call stack. The so-callled optimized code is easier to understand. Two, the current platform-independent approach locks up in my Windows build, while the old Windows-specific code that AOLserver used for many years does not. I do not know why this is, but I certainly will use the old Windows-specific code until I figure it out. -- Andrew Piskorski a...@piskorski.com -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Sun, Oct 05, 2014 at 09:37:39PM +0200, Maurizio Martignano wrote:
Did you use the define _USE_32BIT_TIME_T yes or not?
I believe you should use it.
I haven't tried using _USE_32BIT_TIME_T yet, but I think using it
would be INCORRECT on Windows-64.
Tcl 8.5.16 does this in win/tclWinPort.h:
#ifndef _WIN64
/* See [Bug 3354324]: file mtime sets wrong time */
# define _USE_32BIT_TIME_T
#endif
So it does NOT set _USE_32BIT_TIME_T when _WIN64 is defined.
I assume _WIN64 is always defined for 64-bit Windows.
Relevant discussion appears to be here:
http://sourceforge.net/p/tcl/bugs/4845/
http://sourceforge.net/p/tcl/bugs/5115/
http://sourceforge.net/p/mingw/bugs/1973/
It looks to me like the horrible confusion about what should be done
is only for ** 32-bit ** Windows systems, especially when trying to
support multiple old and new versions of 32-bit Windows.
In contrast, 64-bit Windows seems straightforward. And I am on 64-bit
Windows now! So I think I should simply NEVER set _USE_32BIT_TIME_T.
Does that sound right?
What does Tcl actually do, in practice, on 64-bit Linux and Window?
Here's a simple test, which shows that Tcl 8.5.x does indeed use a
64-bit time_t in both those cases, and that the integer values are
exactly the same on Windows and Linux, as they should be. That
corresponds to my understanding of the code above, that Tcl NEVER sets
_USE_32BIT_TIME_T on 64-bit Windows.
# On Windows 7 64-bit, running ActiveTcl 8.5:
% info patchlevel
8.5.15
% info nameofexecutable
C:/P/Tcl85/bin/tclsh85.exe
% info sharedlibextension
.dll
% set in_l [list {1970-01-01 00:00:00 GMT} {2038-12-01 00:00:00 GMT}
{-12-01 00:00:00 GMT}]
{1970-01-01 00:00:00 GMT} {2038-12-01 00:00:00 GMT} {-12-01 00:00:00 GMT}
% foreach ss $in_l { set tt [clock scan $ss] ; puts $tt - [clock format $tt
-gmt 1] }
0 - Thu Jan 01 00:00:00 GMT 1970
2174774400 - Wed Dec 01 00:00:00 GMT 2038
253399622400 - Wed Dec 01 00:00:00 GMT
% exit
# On Linux 64-bit, Ubuntu 12.04.3 LTS:
$ /usr/bin/tclsh8.5
% info patchlevel
8.5.11
% info nameofexecutable
/usr/bin/tclsh8.5
% info sharedlibextension
.so
% set in_l [list {1970-01-01 00:00:00 GMT} {2038-12-01 00:00:00 GMT}
{-12-01 00:00:00 GMT}]
{1970-01-01 00:00:00 GMT} {2038-12-01 00:00:00 GMT} {-12-01 00:00:00 GMT}
% foreach ss $in_l { set tt [clock scan $ss] ; puts $tt - [clock format $tt
-gmt 1] }
0 - Thu Jan 01 00:00:00 GMT 1970
2174774400 - Wed Dec 01 00:00:00 GMT 2038
253399622400 - Wed Dec 01 00:00:00 GMT
% exit
--
Andrew Piskorski a...@piskorski.com
--
Slashdot TV. Videos for Nerds. Stuff that Matters.
http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Mon, Oct 06, 2014 at 11:31:27PM +0200, Gustaf Neumann wrote: I've merged in your changes to the main branch. In order to keep your change history, i merged the changes rather than copying the files, but had at the end the problem to get rid the branch clean, since mercurial likes to keep branches around. Is my clean branch helpful to you, or should I get rid of it? I created that clean branch with that idea that you'd prefer to merge from it rather than from my default branch. But I'm new to Mercurial and not entirely sure how useful that really is. What I wanted is to have my clean branch be almost exactly the same as my default, but with any Andy-specific hacks removed. So I branched clean, and then removed my rpath hack. (As we've previously discussed, I need that rpath hack to work on Linux, but it's clearly the wrong fix. I added it only because I don't understand where the true bug is coming from.) In Mercurial, it doesn't seem that easy to keep two branches SLIGHTLY different from each other. Apparently the recommended way to cherry pick changes is with hg graft, but I did not try that, as it didn't seem like what I really wanted in this case. I'll see how it goes as I do more merges across my two branches, assuming I keep the clean branch around. -- Andrew Piskorski a...@piskorski.com -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Mon, Oct 06, 2014 at 11:33:25PM +0100, Stephen wrote: Here are some build errors while cross-compiling with mingw using this docker image: https://bitbucket.org/groks/build-naviserver-mingw/src/tip/naviserver/ The docker image sounds useful. However, building with Mingw on Windows would use the Unix makefiles. I've never tested that build combination and doubt that anyone else has done so recently either. Right now I'm focused on just getting Windows builds to work with nmake and MSVC. Mingw can come later, especially since it (at least currently) and MSVC are tied to different build systems. It would be nice to eventually adopt a single build system that would work well with all combinations of platforms and compilers, but discussing that seems somewhat premature. Btw, the pages below on Tcl-based replacements for make and autoconf are interesting, especially in that it should be feasible to have **Tcl itself** self-host on such tools, while avoiding the chicken-and-egg problem: http://wiki.tcl.tk/27561 http://wiki.tcl.tk/27197 http://wiki.tcl.tk/12593 But I'd want get the opinion of the Tcl core team and other experts before going very far down such a path, either for Naviserver or in my own code. -- Andrew Piskorski a...@piskorski.com -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] were all AOLserver 4.0.x bugfixes merged?
When exactly did Naviserver fork from AOLserver, and have all the relevent bugfixes made to AOLserver since then been merged into Naviserver? (I see a few that look like they haven't been...) Although I don't see any official mention of it, from comparing the Naviserver and aolserver-40x repositories on Bitbucket it sure looks like Naviserver forked from the aolserver-4.0.10 release, and that this was the last commit made by anyone PRIOR to forking: changeset: 1095:e9c50ae686b5 user:Dossy Shiobara do...@panoptic.com date:Tue Jan 18 21:10:07 2005 + description: Tag aolserver-4.0.10 -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] use nsconfig.tcl to compile on Windows?
On Fri, Oct 03, 2014 at 05:20:48PM +0200, Maurizio Martignano wrote: Subject: Re: Naviserver hangs on Windows The Windows-OpenACS distribution which I make available here (http://www.spazioit.com/pages_en/sol_inf_en/windows-openacs_en/) is based on AOLServer 4.5.2, contains the sources, is compiled with Visual Studio 2013 and runs on Windows 64. Indeed, the first thing I noticed is that Maurizio's Windows build seems to use nsconfig.tcl in place of the Unix configure/autoconf, which Jim Davidson added back in 2005. Naviserver folks, can you comment on what you think of that approach? Does Naviserver not include it solely because it wasn't included in the AOLserver 4.0.10 branch that Naviserver forked from? https://bitbucket.org/aolserver/aolserver/src/2aa0f24395ae0a42fd590f3635e4fdf2c7eefdfd/nsconfig.tcl?at=default andy@milo:/home/nobackup/co/nsd-aol/aolserver-head-hg$ hg log nsconfig.tcl configure.tcl changeset: 1370:76895d8bf843 user:Jim Davidson jgdavid...@aol.com date:Thu Aug 18 21:48:21 2005 +0100 summary: Renamed configure script configure.tcl changeset: 1367:9c88ea73dcad user:Jim Davidson jgdavid...@aol.com date:Wed Aug 17 23:55:57 2005 +0100 summary: Updates to new build tools to support Unix changeset: 1360:697679717350 user:Jim Davidson jgdavid...@aol.com date:Wed Aug 17 22:18:46 2005 +0100 summary: New platform-indepedent build support -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] Naviserver hangs on Windows
+0xa0 [f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dllcrt0.c @ 330] `0027f2a0 `7761b108 nsthread!_DllMainCRTStartup+0x31 [f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dllcrt0.c @ 294] `0027f2d0 `77621e5a ntdll!LdrpRunInitializeRoutines+0x1fe `0027f4a0 `77621937 ntdll!LdrpInitializeProcess+0x1b8a `0027f990 `7760c34e ntdll! ?? ::FNODOBFM::`string'+0x28ff0 `0027fa00 ` ntdll!LdrInitializeThunk+0xe -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Fri, Oct 03, 2014 at 03:21:02PM +0200, Maurizio Martignano wrote: Dear Andrew, Did you happen to use this define _USE_32BIT_TIME_T? No, I did not. Should I? Does Tcl use that? Grepping my source files, I see this: ./include/nsconfig.h:276:#define SIZEOF_TIME_T 8 That nsconfig.h was created on Linux, but my Windows build uses it too. Perhaps that is the problem? But how the heck can I generate a correct version of nsconfig.h for my Windows 7 system? -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Hm, the old AOLserver 4.0.7 version of Ns_GetTime() had a Windows
ifdef which is entirely missing from Naviserver! (See below.)
I found two relevant commits in Mercurial, from 2009 and 2012, below.
From those logs, it looks like what happened is that in 2009, Zoran
removed the platform specific C code and instead made Naviserver call
Tcl_GetTime (somehow). Then in 2012, Gustaf added back the old
Unix-specific gettimeofday() implementation for performance reasons.
So should we just add back the old Windows-specific C implementation
too? Or is it better to use Zoran's way of invoking Tcl_GetTime? Do
we even know for sure if that method really is platform-independent
as Zoran thought it was? E.g., did Zoran or someone else at Archiware
test it on Windows?
Part of Ns_GetTime() from ancient AOLserver 4.0.7 sources:
#ifdef _WIN32
/*
* Number of 100 nanosecond units from 1/1/1601 to 1/1/1970
*/
#define EPOCH_BIAS 1164447360i64
union {
unsigned __int64i;
FILETIMEs;
} ft;
GetSystemTimeAsFileTime(ft.s);
timePtr-sec = (time_t)((ft.i - EPOCH_BIAS) / 1000i64);
timePtr-usec =(long)((ft.i / 10i64) % 100i64);
#else
#endif
changeset: 2435:16508ab4b3de
user:Gustaf Neumann neum...@wu-wien.ac.at
date:Fri Nov 02 01:54:38 2012 +0100
files: configure.in include/nsconfig.h.in nsthread/time.c
description:
use gettimeofday() if available instead of the rount trip to tcl
changeset: 2009:4bf1ee6ebf84
user:Zoran Vasiljevic z...@archiware.com
date:Sat Sep 29 14:14:03 2007 +0100
files: ChangeLog nsproxy/nsproxylib.c nsthread/time.c
description:
Ns_GetTime now relies on Tcl_GetTime to be platform-independent
--
Andrew Piskorski a...@piskorski.com
--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
[naviserver-devel] modules/tcl/ is empty, tcl/ is full
AOLserver used to install sendmail.tcl and various other *.tcl files into a modules/tcl/ subdirectory. The stock Naviserver Unix Makefile still creates modules/tcl/, but it remains empty because sendmail.tcl and the other files are instead installed into tcl/, one level higher in the directory tree. Why was that change made, and was it intentional? Do we actually need the modules/tcl/ directory for anything now? Given Gustaf's 2014-01-07 change below, I guess we do, but only for backwards compatibility? changeset: 2714:3a7c1c9fe0e0 user:Gustaf Neumann neum...@wu-wien.ac.at date:Tue Jan 07 19:18:36 2014 +0100 files: Makefile description: - build directory modues/tcl during install, since default configs point to it -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] NS_EXTERN set wrong on Windows?
On Wed, Sep 17, 2014 at 06:50:08PM -0400, Andrew Piskorski wrote: include/ns.h has this: #define NS_EXTERN extern NS_EXPORT But I believe that is NOT relevent here because master.c, thread.h, and nsthread.h (correctly) do NOT include ns.h. Instead, I think it is the separate definition of NS_EXTERN in nsthread.h that matters when building nsthread. Gustaf, on 9/27 you changed nsthread/reentrant.c to import ns.h. It never did that before, and now it imports ns.h before thread.h and nsthread.h. It looks like you did that in order to call Ns_Log in reentrant.c. However, I suspect that ALSO changed the definition of NS_EXTERN to the one in ns.h rather than nsthread.h. Was that an unintentional? Defining NS_EXTERN in two separate places like that is kind of confusing. Can/should we merge them into one somehow? I bring this up because after merging all your latest changes into my fork, the nsthread Windows build (which worked before) now fails like this: Creating library nsthread.lib and object nsthread.exp reentrant.o : error LNK2019: unresolved external symbol Ns_Log referenced in function ns_asctime nsthread.dll : fatal error LNK1120: 1 unresolved externals changeset: 2884:950a8cebabb8 user:Gustaf Neumann neum...@wu-wien.ac.at date:Sat Sep 27 22:17:55 2014 +0200 files: include/nsthread.h nsd/exec.c nsthread/reentrant.c description: - setting WINVERSION to Vista to allow usage of InetNtop() - make code more robust against previous defines - use the right pid_t type for win - add shutdown flags - use *_s function in win-specific reentrant code -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] NS_EXTERN set wrong on Windows?
On Wed, Oct 01, 2014 at 03:12:34PM -0400, Andrew Piskorski wrote: Creating library nsthread.lib and object nsthread.exp reentrant.o : error LNK2019: unresolved external symbol Ns_Log referenced in function ns_asctime nsthread.dll : fatal error LNK1120: 1 unresolved externals Well, I fixed that by simply commenting out that new Ns_Log call, here: https://bitbucket.org/apiskors/naviserver/commits/f4c8f273833539b5d48ec1b73b594a9d961fe788 I still worry a bit about the two separate definitions of NS_EXTERN and which one is actually getting used, but so far the change to include ns.h from reentrant.c does NOT seem to have broken anything. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] no mkstemp() on Windows, use TclpCreateTempFile()?
On Fri, Sep 26, 2014 at 08:55:37PM +0200, Maurizio Martignano wrote: Dear Andrew and Gustaf, I believe this is the function you are looking for: http://msdn.microsoft.com/en-us/library/ms235413.aspx No, it is not. mktemp and friends merely generate a file name, while mkstemp also opens the file and returns a file handle. So yes, you could use mktemp as part of an implementation of mkstemp, but they do are not replacements for each other. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
