Re: [naviserver-devel] Error on startup

[Maksym Zinchenko]

Dear Gustaf, my naviserver and modules are compiled and installed in the
docker container, using the same image, so I assume there's no problem with
different versions of tcl. This error happens when I'm trying to start a
container, after a few retries it works fine.

On Tue, Mar 19, 2024 at 8:21 AM Gustaf Neumann (sslmail) 
wrote:

> The error indicates inconsistent usage of memory allocation/deallocation
> functions. Typical reasons are inconsistent usage of library functions
> (e.g. different tcl versions during compilation and runtime of nsd or some
> of the c modules) or inconsistent usage of memory allocator options (e.g.
> malloc libraries in use).
>
> One can obtain details what happened via debugging options during
> compilation.
>
> -gn
>
> > On 18.03.2024, at 18:39, Maksym Zinchenko  wrote:
> >
> > Hello, what can cause this errors, how can I get more information about,
> debug:
> >
> > Fatal: alloc: invalid block: 0x7fd5285e5670: 0 0
> > Fatal: received fatal signal 11
> >
> > Thank you
> > ___
> > naviserver-devel mailing list
> > naviserver-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Error on startup

[Maksym Zinchenko]

Hello, what can cause this errors, how can I get more information about,
debug:

Fatal: alloc: invalid block: 0x7fd5285e5670: 0 0
Fatal: received fatal signal 11

Thank you
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] (no subject)

[Maksym Zinchenko]

I managed to make it work. It's a complex issue. With cloudflare not
supporting certificates, host.docker.internal not working well on linux,
using naviserver ip in docker config will not redirect to the correct named
server etc. Thank you all for your help

On Wed, Mar 6, 2024 at 4:34 PM Gustaf Neumann  wrote:

> Dear Maksym,
>
> in order to help you, I tried to mimic your configuration:
>
> - nginx running in a docker container (listening on port 1000)
> - nsd s1.com running in a docker container (listening on port 8081)
> - nsd s2.com running on the docker host (listening on port 8082, no
> container involved)
> - s1 and s2 have info.tcl installed
>
> On the docker host, i made the following tests
> 1) request to s1.com via containerized nginx
> 2) request to s2.com via containerized nginx
> 3) direct request to s2.com
>
> All requests are made via curl.
> To me, everything looks ok. I still do not understand, what you see as a
> problem.
>
> When you make a request to nginx with a host name neither s1.com or s2.com
> ,
> the first entry is chosen, and the request goes to s1.com. This is nginx
> behavior.
>
> all the best
>
> -g
>
>
>  case 1 
> % curl -H "Host: s1.com" http://localhost:1000/info.tcl
> Host: s1.com
> X-Real-IP: 192.168.65.1
> X-Forwarded-For: 192.168.65.1
> X-Forwarded-Proto: http
> X-Name: s1.com
> Connection: close
> User-Agent: curl/8.6.0
> Accept: */*
>
> ns_conn host: s1.com
>
>
>  case 2 
> % curl -H "Host: s2.com" http://localhost:1000/info.tcl
> Host: s2.com
> X-Real-IP: 192.168.65.1
> X-Forwarded-For: 192.168.65.1
> X-Forwarded-Proto: http
> X-Name: s2.com
> Connection: close
> User-Agent: curl/8.6.0
> Accept: */*
>
> ns_conn host: s2.com
>
>
>  case 3 
> % curl  -H "Host: s2.com" http://localhost:8082/info.tcl
> Host: s2.com
> User-Agent: curl/8.6.0
> Accept: */*
>
> ns_conn host: s2.com
>
>
> info.tcl:
> ###
> ns_return 200 text/plain [subst [ns_trim -delimiter | {
> |[join [lmap {key value} [ns_set array [ns_conn headers]] {set _ " 
> <http://localhost:8082/info.tclHost:s2.comUser-Agent:curl/8.6.0Accept:*/*ns_connhost:s2.cominfo.tcl:#%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23%23ns_return200text/plain[subst[ns_trim-delimiter%7C%7B%7C[join[lmap%7Bkeyvalue%7D[ns_setarray[ns_connheaders]]%7Bset_>$key:
>  $value\n"}] ""]
> |
> |ns_conn host: [ns_conn host]
> }]]
> ###
>
>
> nginx configuration:
> ###
> server {
> listen 80;
> server_name s1.com;
>
> location / {
> proxy_pass http://nsd:8081/;
> proxy_set_header Host $host;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header X-Forwarded-Proto $scheme;
> proxy_set_header X-Name s1.com;
> }
> }
>
> server {
> listen 80;
> server_name s2.com;
>
> location / {
> proxy_pass http://host.docker.internal:8082/;
> proxy_set_header Host $host;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header X-Forwarded-Proto $scheme;
> proxy_set_header X-Name s2.com;
> }
> }
> ###
>
> docker-compose.yaml
> ###
> version: '3'
>
> services:
>   nginx:
> image: nginx:alpine
> ports:
>   - 1000:80
> volumes:
>   - /opt/docker/nginx/conf.d:/etc/nginx/conf.d
> depends_on:
>   - nsd
>
>   nsd:
> image: gustafn/naviserver:latest-alpine
> container_name: nsd
> restart: unless-stopped
> command: /usr/local/ns/bin/nsd -f -t /usr/local/ns/conf/nsd-config.tcl -u 
> nsadmin -g nsadmin
> ports:
>   - 8081:8081
> environment:
>   - TZ=${TZ:-Europe/Vienna}
>   - nsd_httpport=8081
>   - nsd_httpsport=
> ###
>
>
> On 05.03.24 20:25, Maksym Zinchenko wrote:
>
> Hello, no right now i have single nsd with many subdomains clients,
> without docker.
>
&g

Re: [naviserver-devel] (no subject)

[Maksym Zinchenko]

Hello, no right now i have single nsd with many subdomains clients, without
docker.

For example: sub1.domain.com, sub2.domain.com etc. What i want is to move
each one of this subdomains to a separeted docker container. But I need to
move them one by one, because I still testing my application on docker, so
I want to move for example sub1 today test it, next time sub2 test it and
so on.

While im working on one subdomain i still want to serve other subdomains.

So my idea was to move host's nsd to other ports 8080 and 8443 and install
nginx reverse proxy, so i can redirect some subdomains to host's nsd and
some to dockerized nsd.

Thanj you

On Tue, 5 Mar 2024, 18:08 Gustaf Neumann (sslmail), 
wrote:

> Hi Maxsym,
>
> if i understand correctly, you are sending from the docker host requests
> to a single dockerized nginx instance, that forwards these requests to a
> single dockerized nsd backend instance. The nginx instance distinguishes
> the incoming requests to the same port based on the host header field. And
> the same should happen as well on the backend (nsd).
>
> Is this a correct understanding of your setup?
>
> This is somewhat unusual, since typically one uses nginx to forward
> requests to multiple backend instances. So i am not sure, why you are doing
> this. I am not an nginx expert, but my suspicion is that one does not need
> 2 nginx “server” definitions for this.
>
> The problem that you are seeing is that on the backend, you see always the
> same host header field, which is used for virtual hosting. Correct?
>
> Maybe install for your backend servers the following script e.g. under the
> name “info.tcl” and show the result of “curl -H ….” requests, and what you
> are expecting.
>
> all the best
> -g
>
> ns_return 200 text/plain [subst [ns_trim -delimiter | {
> |[join [lmap {key value} [ns_set array [ns_conn headers]] {set _
> "$key: $value\n"}] ""]
> |
>
> |ns_conn host: [ns_conn host]
>
> |ns_conn peer: [ns_conn peeraddr]
> |ns_conn peer -source configured: [ns_conn peeraddr -source
> configured]
> |ns_conn peer -source direct: [ns_conn peeraddr -source direct]
>
> |ns_conn peer -source forwarded: [ns_conn peeraddr -source forwarded]
>
> }]]
>
>
>
> On 04.03.2024, at 11:45, Maksym Zinchenko  wrote:
>
> I have a Naviserver with a couple of virtual web servers listening on a
> single ip:port. I want to move all those virtual servers to Docker
> containers gradually. So I'm trying to install Nginx Reverse Proxy on my
> Docker container and configure it to redirect requests to my host
> Naviserver.
>
> I've changed my Naviserver config to listen to 8080 and 8443 ports on
> docker host gateway IP:
>
> /opt/ns/bin/nsd -w -u nsadmin -t /opt/ns/conf/dz_nsd.tcl -b
>> 172.17.0.1:8080,172.17.0.1:8443
>
>
> In my Nginx config I have 2 listeners one for dev and dummy1 subdomains:
>
>  server {
>> listen 80;
>> server_name dev.daidze.org;
>>
>> location / {
>> proxy_set_header X-Real-IP $remote_addr;
>> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>> proxy_set_header Host $http_host;
>> proxy_set_header X-Forwarded-Proto $scheme;
>> proxy_pass http://172.17.0.1:8080/;
>> }
>> }
>>
>> server {
>> listen 443 ssl;
>> server_name dev.daidze.org;
>> ssl_certificate /opt/ns/modules/nsssl/fullchain.pem;
>> ssl_certificate_key  /opt/ns/modules/nsssl/privkey.pem;
>> ssl_prefer_server_ciphers on;
>>
>> location / {
>> proxy_set_header X-Real-IP $remote_addr;
>> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>> proxy_set_header Host $http_host;
>> proxy_set_header X-Forwarded-Proto $scheme;
>> proxy_pass https://172.17.0.1:8443/;
>> }
>> }
>>
>> server {
>> listen 80;
>> server_name dummy1.daidze.org;
>>
>> location / {
>> proxy_set_header X-Real-IP $remote_addr;
>> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>> proxy_set_header Host $http_host;
>> proxy_set_header X-Forwarded-Proto $scheme;
>> proxy_pass http://172.17.0.1:8080/;
>> }
>> }
>>
>> server {
>> listen 443 ssl;
>> server_name dummy1.daidze.org;
>> ssl_certificate /opt/ns/modules/nsssl/fullchain.pem;
>> ssl_certificate_key  /opt/ns/modules/nsssl/privkey.pem;
>> ssl_prefer_server_ciphers on;
>>
>> location / {
>> proxy_set_header X-Real-IP $remote_addr;
>> proxy_set_header 

Re: [naviserver-devel] (no subject)

[Maksym Zinchenko]

I think it's still smth with Naviserver. When there is no domain name on
request, even if I send a Host, the default server will always respond.
For example:

curl -v -H "Host: dummy1.daidze.com" http://172.17.0.1:8080
> *   Trying 172.17.0.1:8080...
> * Connected to 172.17.0.1 (172.17.0.1) port 8080 (#0)
> > GET / HTTP/1.1
> > Host: dummy1.daidze.com
> > User-Agent: curl/7.81.0
> > Accept: */*
> >
> * Mark bundle as not supporting multiuse
> < HTTP/1.1 200 OK
> < Server: NaviServer/5.0.0a
> < Date: Tue, 05 Mar 2024 14:14:46 GMT
> < Content-Type: text/html; charset=utf-8
> < Content-Length: 94
> < Connection: keep-alive
> <
> dev server
> Host: dummy1.daidze.com 
> User-Agent: curl/7.81.0 
> Accept: */* 
> * Connection #0 to host 172.17.0.1 left intact


On Tue, Mar 5, 2024 at 12:39 PM Georg Lehner  wrote:

> Hello Maksym,
>
> we can see two requests, the lines starting with "Ns_SockAccept" and
> ending with "[0] end of job". I singled them out from your message below.
>
> However, we do not know the request you sent in each case.
>
> What we can see:  they are only https requests, since only nsssl is
> involved, and both of them go to the default virtual server: the lines
> after "driver:nssl" have "conn:dev.daidze.org:default:" in them.
>
> Please:
>
> - do the tests again with http requests.
>
> - check on your own if the results show a different picture
>
> - check the nginx logs, to see if the requests are really done to
> different hosts. If needed add logging to nginx to see the backend used for
> the proxy requests
>
> Regards,
>   Georg
>
>
> On 3/5/24 14:19, Maksym Zinchenko wrote:
>
> Ive added debug, here is my log:
>
> ...
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: Ns_SockAccept returns sock 6, err NONE [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: nssock(6): option TCP_NODELAY activated [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: SSL_read(6) ERROR_WANT_READ got:0 [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: ### SSL_read(6) return:0 sockState:80 [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: Ns_SockAccept returns sock -1, err Resource temporarily
>> unavailable [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: SSL_read(6) ERROR_WANT_READ got:0 [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: ### SSL_read(6) return:0 sockState:80 [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: SSL_read(6) got:418 [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: ### SSL_read(6) return:418 sockState:01 [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: ### Request SetUrl calls Ns_UrlPathDecode '/' [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug:  ### decoded path '/' [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: === length < avail (length 0, avail 0) tfd 0 tfile (nil)
>> chunkStartOff 0 [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: driver sets  reqPtr->content (len 0) to '' [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7fead19fd640][-driver:nsssl:0-] [0m
>> [0;34mDebug: [0] dequeue thread connPtr 0x55d4804627f0 idle 1 state 4
>> create 0 [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7feadab59640][-conn:dev.daidze.org:default:0:0-]
>> [0m [0;34mDebug: ns:interptrace[dev.daidze.org]: allocate ns:tcltrace
>> ns_init [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7feadab59640][-conn:dev.daidze.org:default:0:0-]
>> [0m [0;39mNotice: ns_perm_load: /opt/ns/modules/nsperm/passwd: /
>> ns_perm_adduser [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7feadab59640][-conn:dev.daidze.org:default:0:0-]
>> [0m [0;34mDebug: --- ns_perm_adduser /opt/ns/modules/nsperm/passwd / {{""}
>> {} {} {} {} {} {}} [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7feadab59640][-conn:dev.daidze.org:default:0:0-]
>> [0m [0;34mDebug: --- ns_perm_adduser /opt/ns/modules/nsperm/passwd /
>> {nsadmin ubphBI5lDzSYc {} {} {} {} {}} [0m
>> [1;32m[05/Mar/2024:12:16:35][189217.7feadab59640][-conn:dev.daidze.org:default:0:0-]
>> [0m [0;34mDebug: --- 

Re: [naviserver-devel] (no subject)

[Maksym Zinchenko]

dding virtual host entry for host <
> dummy1.daidze.org:8080> location: http://dummy1.daidze.org:8080 mapped to
> server: dummy1.daidze.org ctx (nil) [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb12619840][-main:dummy1.daidze.org-]
> [0m [0;39mNotice: nsssl:0: adding virtual host entry for host <
> dev.daidze.org:8443> location: https://dev.daidze.org:8443 mapped to
> server: dev.daidze.org ctx (nil) [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb12619840][-main:dummy1.daidze.org-]
> [0m [0;39mNotice: nsssl:0: adding virtual host entry for host <
> dummy2.daidze.org:8443> location: https://dummy2.daidze.org:8443 mapped
> to server: dummy2.daidze.org ctx (nil) [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb12619840][-main:dummy1.daidze.org-]
> [0m [0;39mNotice: nsssl:0: adding virtual host entry for host <
> dummy1.daidze.org:8443> location: https://dummy1.daidze.org:8443 mapped
> to server: dummy1.daidze.org ctx (nil) [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb08dd1640][-driver:nssock:0-] [0m
> [0;39mNotice: starting [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb08dd1640][-driver:nssock:0-] [0m
> [0;39mNotice: nssock:0: listening on [172.17.0.1]:8080 [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb08dd1640][-driver:nssock:0-] [0m
> [0;39mNotice: driver: accepting connections [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb03fff640][-driver:nsssl:0-] [0m
> [0;39mNotice: starting [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb03fff640][-driver:nsssl:0-] [0m
> [0;39mNotice: nsssl:0: listening on [172.17.0.1]:8443 [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb03fff640][-driver:nsssl:0-] [0m
> [0;39mNotice: driver: accepting connections [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb12619840][-main:dummy1.daidze.org-]
> [0m [0;39mNotice: nsmain: NaviServer/5.0.0a (tar-5.0.0a) running [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb12619840][-main:dummy1.daidze.org-]
> [0m [0;39mNotice: nsmain: security info: uid=1000, euid=1000, gid=1000,
> egid=1000 [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb122c6640][-sched-] [0m
> [0;39mNotice: sched: starting [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb037fe640][-writer0-] [0m
> [0;39mNotice: writer0: accepting connections [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb095d2640][-conn:dummy2.daidze.org:default:0:0-]
> [0m [0;39mNotice: start update interpreter dummy2.daidze.org to epoch 1,
> concurrent 3 [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb11a33640][-conn:dev.daidze.org:default:0:0-]
> [0m [0;39mNotice: start update interpreter dev.daidze.org to epoch 1,
> concurrent 3 [0m
> [1;32m[05/Mar/2024:11:26:06][188117.7fdb09dd3640][-conn:dummy1.daidze.org:default:0:0-]
> [0m [0;39mNotice: start update interpreter dummy1.daidze.org to epoch 1,
> concurrent 3 [0m
> [1;32m[05/Mar/2024:11:26:07][188117.7fdb095d2640][-conn:dummy2.daidze.org:default:0:0-]
> [0m [0;39mNotice: update interpreter dummy2.daidze.org to epoch 1 done,
> trace none, time 0.379366 secs concurrent 3 [0m
> [1;32m[05/Mar/2024:11:26:07][188117.7fdb095d2640][-conn:dummy2.daidze.org:default:0:0-]
> [0m [0;39mNotice: thread initialized (0.414374 secs) [0m
> [1;32m[05/Mar/2024:11:26:07][188117.7fdb11a33640][-conn:dev.daidze.org:default:0:0-]
> [0m [0;39mNotice: update interpreter dev.daidze.org to epoch 1 done,
> trace none, time 0.372765 secs concurrent 3 [0m
> [1;32m[05/Mar/2024:11:26:07][188117.7fdb11a33640][-conn:dev.daidze.org:default:0:0-]
> [0m [0;39mNotice: thread initialized (0.410192 secs) [0m
> [1;32m[05/Mar/2024:11:26:07][188117.7fdb09dd3640][-conn:dummy1.daidze.org:default:0:0-]
> [0m [0;39mNotice: update interpreter dummy1.daidze.org to epoch 1 done,
> trace none, time 0.370452 secs concurrent 3 [0m
> [1;32m[05/Mar/2024:11:26:07][188117.7fdb09dd3640][-conn:dummy1.daidze.org:default:0:0-]
> [0m [0;39mNotice: thread initialized (0.407226 secs) [0m
> [1;32m[05/Mar/2024:11:26:14][188117.7fdb122c6640][-sched-] [0m
> [0;39mNotice: start update interpreter dev.daidze.org to epoch 1,
> concurrent 1 [0m
> [1;32m[05/Mar/2024:11:26:14][188117.7fdb122c6640][-sched-] [0m
> [0;39mNotice: update interpreter dev.daidze.org to epoch 1 done, trace
> none, time 0.175986 secs concurrent 1 [0m
> [1;32m[05/Mar/2024:11:26:15][188117.7fdb122c6640][-sched-] [0m
> [0;39mNotice: start update interpreter dummy2.daidze.org to epoch 1,
> concurrent 1 [0m
> [1;32m[05/Mar/2024:11:26:15][188117.7fdb122c6640][-sched-] [0m
> [0;39mNotice: update interpreter dummy2.daidze.org to epoch 1 done, trace
> none, time 0.130161 secs concurrent 1 [0m
> [1;32m[05/Mar/2024:11:26:16][188117.7fdb122c6640][-sched-] [0m
> [0;39mNotice: start update interpreter dummy1.daidze.org to epoch 1,
> concurrent 1 [0m
> [1;32m[05/Mar/2024:11:26:16][188117.7fdb122c6640][-sched-] [0m
> [0;39mNotice: update interpreter dummy1.daidze.or

[naviserver-devel] (no subject)

[Maksym Zinchenko]

I have a Naviserver with a couple of virtual web servers listening on a
single ip:port. I want to move all those virtual servers to Docker
containers gradually. So I'm trying to install Nginx Reverse Proxy on my
Docker container and configure it to redirect requests to my host
Naviserver.

I've changed my Naviserver config to listen to 8080 and 8443 ports on
docker host gateway IP:

/opt/ns/bin/nsd -w -u nsadmin -t /opt/ns/conf/dz_nsd.tcl -b 172.17.0.1:8080,
> 172.17.0.1:8443


In my Nginx config I have 2 listeners one for dev and dummy1 subdomains:

 server {
> listen 80;
> server_name dev.daidze.org;
>
> location / {
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header Host $http_host;
> proxy_set_header X-Forwarded-Proto $scheme;
> proxy_pass http://172.17.0.1:8080/;
> }
> }
>
> server {
> listen 443 ssl;
> server_name dev.daidze.org;
> ssl_certificate /opt/ns/modules/nsssl/fullchain.pem;
> ssl_certificate_key  /opt/ns/modules/nsssl/privkey.pem;
> ssl_prefer_server_ciphers on;
>
> location / {
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header Host $http_host;
> proxy_set_header X-Forwarded-Proto $scheme;
> proxy_pass https://172.17.0.1:8443/;
> }
> }
>
> server {
> listen 80;
> server_name dummy1.daidze.org;
>
> location / {
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header Host $http_host;
> proxy_set_header X-Forwarded-Proto $scheme;
> proxy_pass http://172.17.0.1:8080/;
> }
> }
>
> server {
> listen 443 ssl;
> server_name dummy1.daidze.org;
> ssl_certificate /opt/ns/modules/nsssl/fullchain.pem;
> ssl_certificate_key  /opt/ns/modules/nsssl/privkey.pem;
> ssl_prefer_server_ciphers on;
>
> location / {
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header Host $http_host;
> proxy_set_header X-Forwarded-Proto $scheme;
> proxy_pass https://172.17.0.1:8443/;
> }
> }


I'm using proxy_set_header Host $http_host; to redirect the headers real
Host parameter to my host Naviserver installation, but I'm getting response
only from the dev Virtual server. It doesn't matter what url I request
dev.daidze.org or dummy1.daidze.org. Here an example of request to
https://dummy1.daidze.org/ :

dev server
> X-Real-IP: 172.64.238.37
> X-Forwarded-For: 165.90.99.154, 172.64.238.37
> Host: dummy1.daidze.org
> X-Forwarded-Proto: https
> Connection: close
> accept-encoding: gzip, br
> CF-RAY: 85f13d442cd66671-MAD
> CF-Visitor: {"scheme":"https"}
> user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0)
> Gecko/20100101 Firefox/123.0
> accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
>
> accept-language: en-US,en;q=0.5
> upgrade-insecure-requests: 1
> sec-fetch-dest: document
> sec-fetch-mode: navigate
> sec-fetch-site: none
> sec-fetch-user: ?1
> sec-gpc: 1
> pragma: no-cache
> cache-control: no-cache
> CF-Connecting-IP: 165.90.99.154
> CDN-Loop: cloudflare
> CF-IPCountry: CV


What am I doing wrong? According to Naviserver docs redirection is done
based on the content of the *host* header field. Right? So this should work.

Thank you,
Maksym
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] NX and Naviserver init

[Maksym Zinchenko]

Hi Gustaf, thank you for your explanation and link.

On Fri, Sep 15, 2023 at 9:19 AM Gustaf Neumann  wrote:

> Dear Maksym,
> On 15.09.23 01:09, Maksym Zinchenko wrote:
>
> Hello, I've been struggling with a problem for a few days now. First I
> thought it's something with my understanding of NX, but I think its has to
> do something with Naviserver.
>
> actually, the problem is not with NaviServer, but it is rooted in the
> XOTcl/NX serializer. When a NaviServer blueprint is created, NaviServer
> performs "::Serializer all" to serialize the current workspace. Actually,
> it does not serialize the full workspace, since the workspace contains as
> well the NX internal class definitions. Therefore, the serializer has
> "ignore patterns", including everything from, e.g., the "::ns"f namespace
> [1].
>
> When objects are created with "new", these are named by default
> "::nsf::__#1", "::nsf::__#2". These names match the ignore pattern, and
> therefore, these names are excluded from the blueprint.
>
> If you create the objects in your code with the "-childof" flag (see
> below), you can place these outside the "::nsf" namespace, and everything
> should be fine
>
> Hope this helps
>
> -g
>
>
> [1]
> https://github.com/gustafn/nsf/blob/master/library/serialize/serializer.tcl#L868
>
>
> nx::Class create B {
> :method init {} {
> ns_log notice "Im class [self]"
> }
> }
>
> nx::Class create C {
> :method init {} {
> ns_log notice "Im class [self]"
> }
> }
>
> nx::Class create A {
> :property -accessor public inBObj:object
> :property -accessor public inCObj:object
>
> :method init {} {
> set :inBObj [B new -childof [::nsf::current object]]
> set :inCObj [C new -childof [::nsf::current object]]
> }
>
> :public method myObjs {} {
> ns_log notice  "My B obj is: [${:inBObj} info name]"
> ns_log notice  "My C obj is: [${:inCObj} info name]"
> }
> }
>
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] NX and Naviserver init

[Maksym Zinchenko]

Hello, I've been struggling with a problem for a few days now. First I
thought it's something with my understanding of NX, but I think its has to
do something with Naviserver.
I have a global library named "oodz" in my tcl folder, insideI have some
subfolders with my NX classes. I have an init.tcl file inside. According to
documentation this "init.tcl" file executed first. First thing I do is
looping through my subfolders and sourcing my NX Classes. It works fine as
expected. Next Im creating some instances of classes I need inside
"init.tcl"
I made a simple example, to show: I have an NX class like that:

nx::Class create b {
> :method init {} {
> :say
> }
>
> :public method say {} {
> puts "Im class B"
> }
> }
>
> nx::Class create c {
> :method init {} {
> :say
> }
>
> :public method say {} {
> puts "Im class C"
> }
> }
>
> nx::Class create a {
> :property -accessor public inBObj:object
> :property -accessor public inCObj:object
>
> :method init {} {
> set :inBObj [b new]
> set :inCObj [c new]
> }
>
> :public method myObjs {} {
> puts "My B obj is: [${:inBObj} info name]"
> puts "My C obj is: [${:inCObj} info name]"
> }
> }
>

In my "init.tcl" I have line: a create aObj

Here's what's happenin:
1. I can see in the server log that aObj was created, I see line:
Im class B, Im class C

2. If I open nsshell console, and run aObj cget:
aObj cget -inBObj
::nsf::__#3
aObj cget -inCObj
::nsf::__#4

3. But when I try to run myObjs, I get an error:
aObj myObjs
invalid command name "::nsf::__#3"

The same code works fine in tcl shell, is it related to Naviserver? Or am I
doing something wrong with NX?

Thank you
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Some questions about Docker images for Naviserver

[Maksym Zinchenko]

Hello, I'm using Docker Naviserver in production, but it's not a big one. I
can recommend it. In my case I'm using only Naviserver, not OpenACS. I have
some images on docker hub under siqsuruq/naviserver, it's not official, but
they work for me. They are based on a custom ubuntu image I made with Tcl.
They are kinda big, about 300 mb but it works for me. About S6 and non S6,
difference is only on type of init system used in the image, but i'm not
pro at this question. The S6 version of Docker images uses an init system
called s6-overlay, the non-S6 version of Docker images typically use a
different init system, such as Systemd or SysVinit. In my case I'm using
Postgres only, at host not in a Docker container. it looks like its working
faster this way.
So, in general, you can use a Docker container with Naviserver without any
problem.

Maksym

On Fri, 31 Mar 2023, 10:44 Brian Fenton, 
wrote:

> Hello all
>
> We're exploring reviving the Linux version of our Naviserver product. As
> part of this we'd like to investigate the feasibility of using Docker,
> initially for ease of development, and potentially later for production
> systems. I have a couple of questions that would be good to get the
> community's responses to.
>
> 1. Is there anyone using Docker Naviserver in production? If so would you
> recommend it, and are there any peculiarities that we should know about?
>
> 2. There are quite a few Naviserver images on on https://hub.docker.com,
> but I don't see an official one. It seems from posts on OpenACS.org
> that Vlad V's build has
> had a lot of community input, and seems to be the *de facto *build. Would
> it be worth trying to get that version tagged as an official Naviserver
> Docker  build to
> give newcomers clarity? I must admit to some confusion regarding the
> difference between the S6 and the non-S6 versions - should we as a
> community pick one as the official one?
>
> 3. For those of us using Oracle, what would be involved in getting Oracle
> driver support added as a build/run option? I'd be happy to help with this
> in any way I can.
>
> thanks in advance
> Brian
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Ns_conn location questions

[Maksym Zinchenko]

Thank you, I realised that it comes from my config  < docker.
Now I pass --hostname=${HOSTNAME} to docker run and set it up in
nsd-config.tcl as


set server [ns_info hostname]
ns_section ns/module/nssock/servers {
ns_param default${server}
}

About the second question, I didn't notice this before :) Thank you
for your information.


On Sun, Jan 29, 2023 at 3:02 PM Gustaf Neumann  wrote:

> > My first question is where this localhost comes from?
>
> I would think, this comes from your configuration file and/of from the
> request.
>
> If one starts e.g. with the sample configuration file nsd-config.tcl, one
> sees entries like:
>
> [29/Jan/2023:16:35:24][54720.100490580][-main:default-] Notice: nssock:0: 
> adding virtual host entry for host  location: 
> http://localhost:8080 mapped to server: default ctx 0x0
> [29/Jan/2023:16:35:24][54720.100490580][-main:default-] Notice: nssock:0: 
> adding virtual host entry for host  location: 
> http://MacBook-Pro-6.local:8080 mapped to server: default ctx 0x0
>
> These log-entries are coming e.g. from the following section
>
> ns_section ns/module/nssock/servers {
> ns_param defaultlocalhost
> ns_param default[ns_info hostname]
> }
>
> from the configuration file that define the mapping of hostnames for the
> "default" server. When there are incoming requests following HTTP/1.1,
> these come with a "Host:" header field, which might be in your case as well
> "localhost".
>
>
> > When I try to run ns_conn location in nsshell it gives me error
> bad_option "location".
>
> This is a know limitation (see first line of "Current shortcomings" on
> https://bitbucket.org/naviserver/nsshell/src/main/
>
> The reason for this is that nsshell communicates with a "kernel" thread in
> the background that keeps the state of your nsshell session (e.g. to be
> able to obtain the variable value for later requests, if one types "set x
> 1" in nsshell). Since this kernel runs in the background, it is no
> connection thread, and has no connection information available.
>
> Theoretically, one could stretch the limits of nsshell further, but it is
> quite hard to make the background job look completely like the a connection
> thread.
>
> -gn
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Ns_conn location questions

[Maksym Zinchenko]

Hello, I have some questions about ns_conn location.
According to manual "The location is determined via the following means:

1. if *ns_locationproc
*
is configured, its result is returned.
2. if virtual hosting is enabled, and the "Host:" header field is provided
and valid, it returns its content.
3. If everything above fails, it is determined by virtual hosts mapping
table (as defined in the "ns/module/nssock/servers" or
"ns/module/nsssl/servers" section in the configuration file).
4. If everything above fails, and a connection is open, it is determined by
the current socket address.
5. If everything above fails, it is determined by configuration values of
the driver."

First and second is not my case, because i'm not using virtual hosting, in
my config i have part like that:

ns_section "ns/module/nsssl" {
ns_param defaultserver $hostname
ns_param address $ip_addr
ns_param port $ssl_port
ns_param hostnamens_section "ns/module/nsssl" {
ns_param defaultserver $hostname
ns_param certificate /opt/ns/modules/nsssl/daidze.pem
ns_param address $ip_addr
ns_param port $ssl_port
ns_param hostname $hostname
...
}

So I'm assuming i'm gonna get $hostname:$ssl_port when i run [ns_conn
location]
I'm running Naviserver inside Docker container, and instead getting
localhost.

My first question is where this localhost comes from?

And second question when I try to run ns_conn location in nsshell it gives
me error bad_option "location". Actually it gives this error when I try to
run any subcommand of ns_conn inside nsshell.

Thank you
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] nsshell

[Maksym Zinchenko]

Hello, I'm having trouble after update naviserver with shell, it shows

>
> <% # Get hostname without protocol (i.e. "localhost") set host [lindex
> [split [ns_conn location] "/"] 2] set wsConfigPath "ns/server/[ns_info
> server]/module/websocket/shell" set shConfigPath "ns/server/[ns_info
> server]/module/nsshell" # Get WebSocket Protocol (i.e. "ws") set wsProtocol
> [expr {[ns_conn protocol] eq "http" ? "ws" : "wss"}] # Get Shell URL from
> module (i.e. "nsshell") set shellURL [ns_config $wsConfigPath urls] set
> heartBeat [ns_config $shConfigPath kernel_heartbeat 4] # Generate Base URL
> (i.e. "localhost/shell") set baseURL [string trimright $host/$shellURL] #
> Generate WebSocket URL (i.e. ws://localhost/shell/connect) set wsUri
> $wsProtocol://$baseURL/connect # Generate kernelId # Combine uuid with
> connection id and encrpyt with sha-1 set kernelID [ns_sha1 "[ns_uuid]
> [ns_conn id]"] # Remember ns_conn settings from the start of this shell #
> in a nsv variable for this kernel. foreach subcommand { acceptedcompression
> auth authpassword authuser contentfile contentlength contentsentlength
> driver files flags form headers host id isconnected location method
> outputheaders peeraddr peerport pool port protocol query partialtimes
> request server sock start timeout url urlc urlv version zipaccepted } {
> nsv_set shell_conn $kernelID,$subcommand [ns_conn $subcommand] } # # Check,
> if we are connected via the WebSocket interface or via XHR # if
> {${shellURL} ne "" && [string match ${shellURL}* [ns_conn url]]} { ns_log
> notice "nsshell uses WebSocket interface" # # Flag WebSocket usage by
> setting xhrURL empty. # set xhrURL "" # If kernelID is specified on URL,
> change kernelId to the one # specified in the URL if {"kernel" in [ns_conn
> urlv]} { set kernelID [string trim [lindex [ns_conn urlv] end]] # KernelID
> cannot empty if {$kernelID eq ""} { ns_returnredirect [ns_conn
> location]/$shellURL } } else { # If kernel isn't specified on URL, redirect
> with the generated kernelId ns_returnredirect [ns_conn
> location]/$shellURL/kernel/$kernelID } } else { ns_log notice "nsshell uses
> XHR interface" # # Flag XHR usage by setting wsUri empty. # set wsUri ""
> set xhrURL [ns_conn url] # If kernel is specified on URL, change kernelId
> to the URL one set query_kernelID [ns_queryget kernelID ""] if
> {$query_kernelID eq ""} { ns_returnredirect "[ns_conn
> url]?kernelID=$kernelID" } else { set kernelID $query_kernelID } } %>
>

In the browser window
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] ns_returnfile return original file name

[Maksym Zinchenko]

Thank you all, that's exactly what I needed.

On Mon, Jul 18, 2022 at 1:00 PM Wolfgang Winkler via naviserver-devel <
naviserver-devel@lists.sourceforge.net> wrote:

> Hi!
>
> We use
>
> ns_set update [ns_conn outputheaders] Content-Disposition "attachment;
> filename=\"${filename}\""
>
> for downloading and
>
> ns_set update [ns_conn outputheaders] Content-Disposition
> "filename=\"${filename}\""
>
> for viewing files
>
>
> Am 15.07.22 um 16:50 schrieb Maksym Zinchenko:
>
> Hello, I have a question about how to return the original file name. For
> example:
>
> ns_register_proc GET /dev/rtrn_file ::dev::rtrn_file
> proc rtrn_file {args} {
> set f [file join /tmp test.csv]
> ns_returnfile 200 [ns_guesstype "$f"] $f
> }
>
> When I do GET request  I'm getting "rtrn_file.csv" instead of "test.csv"
>
> Maybe I'm doing it wrong.
>
> Thank you.
>
>
> ___
> naviserver-devel mailing 
> listnaviserver-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
> --
>
> *Wolfgang Winkler*
> Geschäftsführung
> wolfgang.wink...@digital-concepts.com
> mobil +43.699.19971172
>
> dc:*büro*
> digital concepts Novak Winkler OG
> Software & Design
> Landstraße 68, 5. Stock, 4020 Linz
> www.digital-concepts.com
> tel +43.732.997117.72
> tel +43.699.1997117.2
>
> Firmenbuchnummer: 192003h
> Firmenbuchgericht: Landesgericht Linz
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] ns_returnfile return original file name

[Maksym Zinchenko]

Hello, I have a question about how to return the original file name. For
example:

ns_register_proc GET /dev/rtrn_file ::dev::rtrn_file
proc rtrn_file {args} {
set f [file join /tmp test.csv]
ns_returnfile 200 [ns_guesstype "$f"] $f
}

When I do GET request  I'm getting "rtrn_file.csv" instead of "test.csv"

Maybe I'm doing it wrong.

Thank you.
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Buffer overflow detected

[Maksym Zinchenko]

Thank you very much for explaining Gustaf, at least I know now that
somewhere in my Tcl code something is wrong. I'm using only 2 C
modules/packages.

On Fri, Nov 5, 2021 at 7:37 AM Gustaf Neumann  wrote:

> Dear Maksym,
>
> These kind of errors should never happen, but when this happen, this can
> come from NaviServer or Tcl or some C modules/package loaded.
>
> The last time i saw this kind of crash in a NaviServer environment, it
> was triggered from Tcl, where a select() was tried in a situation where
> more than 1024 file descriptors were open (stay away from async Tcl I/O
> operations and use NaviServer built-in features ... e.g. ns_http instead
> of tcllib http). The number of open file descriptors of NaviServer can
> be monitored e.g. by the munin-plugins [2,3].
>
> To debug this situation,
> - make sure to compile Tcl and NaviServer with debugging symbols (i.e.
> with compiler flag "-g" [1]),
> - make sure, core dumps are enabled, and when you have a core,
> - use "gdb /usr/local/ns/bin/nsd YOURCORE" to see exactly, where this
> happened.
>
> all the best
>
> -g
>
> [1] https://openacs.org/forums/message-view?message_id=5537675
> [2] https://github.com/gustafn/munin-plugins-ns
> [3]
>
> https://openacs.org/munin/localdomain/localhost.localdomain/naviserver_openacs_lsof.html
>
> On 04.11.21 21:17, Maksym Zinchenko wrote:
> > Hi every now and then my server crashes with the last message in the
> > log file *** Buffer overflow detected *** I would really appreciate if
> > someone can point me in the right direction of debugging and figure
> > out what's going on.
> > Thank you
> > Maksym Zinchenko
>
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Buffer overflow detected

[Maksym Zinchenko]

Hi every now and then my server crashes with the last message in the log
file *** Buffer overflow detected *** I would really appreciate if someone
can point me in the right direction of debugging and figure out what's
going on.
Thank you
Maksym Zinchenko
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Snit error

[Maksym Zinchenko]

Hi, I'm getting this kind of error in my log file. What do they mean:

: Error: can't read "selfns": no such variable
:can't read "selfns": no such variable
:while executing
:"list upvar 1 ${selfns}::$varname $varname"
:(procedure "::snit::RT.variable" line 5)
:invoked from within
:"variable Snit_typemethodInfo"
:(in namespace eval "::snit::window" script line 2)
:invoked from within
:"namespace eval ::snit::window {
: variable Snit_typemethodInfo
:array set Snit_typemethodInfo {info {0 {::snit::RT.typemethod.info %t}
{}} create {0 {::..."
:invoked from within
:"ns_ictl update"
:(procedure "ns_cleanup" line 8)
:invoked from within
:"ns_cleanup"
:while executing callback
:ns:tcltrace ns_cleanup
:(context: trace proc)
:(context: trace deallocate) line 1

Thank you
Maksym Zinchenko
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Fatal: received fatal signal 11

[Maksym Zinchenko]

Hello everyone, 2 latest commits are not letting me start naviserver.
It just crashes without any log with message Fatal: received fatal signal
11.

This one works fine: e11e014

Cheers. Maksym Zinchenko
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] ns_cache question

[Maksym Zinchenko]

 Hello, I have some question about ns_cache. Im creating cache with
command: ns_cache_create -timeout 1800 -expires 1800 max 5MB When I do
ns_cache_exists max right away, it shows 1 If i wait a little bit, lets say
5 min, not 30 min as difined (1800 are seconds right?) it shows 0, cache
doesnt exists, if i run command again it shows 1 again. I dont know whats
going on, may be you can advise me. Thank you
Maksym Zinchenko
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] ns_charsets

[Maksym Zinchenko]

Thank you for clarifications Gustaf, i'll check what's going on with my
code.

On Sun, Jul 11, 2021 at 5:00 PM Gustaf Neumann  wrote:

>
> On 11.07.21 14:14, Maksym Zinchenko wrote:
>
> Hello,I'm having trouble with encodings, or maybe I don't understand
> something.
> My first question is: when I run *"ns_charsets*" command it will not
> return UTF-8 encoding in the list, why and what does it mean?
>
> Dear Maksym,
>
> "ns_charsets" is a code that was not touched since a very long time. In
> essence, it provides a mapping between "official" charset names and
> tcl-encodings. In general charsets can have multiple names for the the same
> thing (preferred MIME name, official name, aliases)
>
> For example the preferred mime charset "iso-8859-1" is mapped to the
> Tcl_Encoding named "iso8859-1". If a charset name is not returned via
> "ns_charsets", it is used literally. So, the charset "utf-8" is mapped the
> the Tcl_encoding with the same name. The charset mapping can be extended
> via the configuration file (section "ns/charsets") so in essence,
> "ns_charset" just returns entries which require special mappings.
>
> When i look at the IANA page, i see several entries there, which are not
> in the naviserver default table. So, one should check, what's feasible to
> be added.
>
> The Second problem is: I'm trying to *"ns_return"* XML UTF-8 data from my
> RESTFull API responder.
> ns_return 200 "application/xml; charset=utf-8" [dict get $response data]
>
> But firefox complains about "XML Parsing Error: not well-formed" it will
> get stuck at portuguese characters, Response header in firefox dev console
> shows: "Content-Type
> application/xml; charset=utf-8", what I'm doing wrong?
>
> Probably, the "data" element of dict "response" contains binary data (a
> Tcl byte array).
> I can see nothing wrong in NaviServer, check below for a minimal test
> setup.
>
> all the best
>
> -gn
>
> ==
>  xml-responder.tcl
> ns_return 200 "application/xml; charset=utf-8" 
> Motörhead\n
> ==
>
> Testing:
>
> $ curl -k -i  https://localhost:8443/xml-responder.tcl
> HTTP/1.1 200 OK
> Server: NaviServer/4.99.21
> Date: Sun, 11 Jul 2021 17:51:50 GMT
> Content-Type: application/xml; charset=utf-8
> Content-Length: 37
> Connection: keep-alive
> X-Content-Type-Options: nosniff
> X-XSS-Protection: 1; mode=block
> Referrer-Policy: strict-origin
>
> Motörhead
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] ns_charsets

[Maksym Zinchenko]

Hello,I'm having trouble with encodings, or maybe I don't understand
something.
My first question is: when I run *"ns_charsets*" command it will not return
UTF-8 encoding in the list, why and what does it mean?
The Second problem is: I'm trying to *"ns_return"* XML UTF-8 data from my
RESTFull API responder.
ns_return 200 "application/xml; charset=utf-8" [dict get $response data]

But firefox complains about "XML Parsing Error: not well-formed" it will
get stuck at portuguese characters, Response header in firefox dev console
shows: "Content-Type
application/xml; charset=utf-8", what I'm doing wrong?
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] ns_getcontent UTF-8

[Maksym Zinchenko]

Hello everyone, I have a strange issue with UTF-8. Im sending AJAX request
to Naviserver with defined UTF-8, this is part in my JS script.

function send_POST(url, postedData, callbackf) {
$.ajaxSetup({
contentType: "application/json; charset=UTF-8"
});

I can see in console that value really UTF-8 encoded:

add_category_modal{ "name": "Colchões" }

On server Im using "ns_getcontent":

if {[set ct [ns_getcontent -as_file false]] ne ""} {
puts $ct
}

And it returns not UTF-8 content

May be its a bug?

Thank you
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Content-Type application/json request

[Maksym Zinchenko]

Thank you. that's exactly what I need

On Fri, Apr 30, 2021 at 6:42 PM Gustaf Neumann  wrote:

> Dear Maksym,
>
> one can access the content of a PUT/POST request via "ns_getcontent".
> Just get the content and parse content e.g. into a dict.
>
> -gn
>
> 
> if {[ns_conn method] in {PUT POST}
> && [ns_set iget [ns_conn headers] Content-Type] eq "application/json"
> } {
> package req json
>
> set dict [json::json2dict [ns_getcontent -as_file false]]
> ns_return 200 text/plain $dict\n
>
> } else {
> ad_return_complaint 1 "unsupported HTTP method: [ns_conn method]"
> }
> 
>
> Test:
> curl --header "Content-Type: application/json" --request POST --data 
> '{"username":"xyz","password":"xyz"}' http://localhost:8100/json-receiver.tcl
>
>
>
>
> On 30.04.21 19:14, Maksym Zinchenko wrote:
>
> Hello, maybe it's obvious but I'm a little bit stuck in here.
> When I'm doing POST requests to Naviserver with Content-Type:
> application/x-www-form-urlencoded, I can use [ns_conn files] [ns_getform]
> to access data.
> How do I access Content-Type: application/json requests? For example if
> I'm sending JSON like that:
> {
> "action": "list",
> "path": "/public_html"
> }
> How do I get those values? What about Content-Type: application/xml or
> text/plain?
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Content-Type application/json request

[Maksym Zinchenko]

Hello, maybe it's obvious but I'm a little bit stuck in here.
When I'm doing POST requests to Naviserver with Content-Type:
application/x-www-form-urlencoded, I can use [ns_conn files] [ns_getform]
to access data.
How do I access Content-Type: application/json requests? For example if I'm
sending JSON like that:
{
"action": "list",
"path": "/public_html"
}
How do I get those values? What about Content-Type: application/xml or
text/plain?
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Recommended approach to building Naviserver on Windows?

[Maksym Zinchenko]

As I remember naviserver needs zlib also, to support compression. I had
some issues with that too, I used https://github.com/kiyolee/zlib-win-build
this guy project to make it working.
Cheers

On Thu, Mar 4, 2021 at 4:05 PM Maksym Zinchenko  wrote:

> I did something like:
> Compile Tcl 8.6.11
> --
>
> 1. Download sources and extract to D:\NSWIN\SRC\tcl8.6.11
> 2. Run Visual Studio "x64 Native Tools Command Prompt for VS 2019" as
> Administrator
> 3. Change directory to D:\NSWIN\SRC\tcl8.6.11\win
> 4. nmake -f makefile.vc INSTALLDIR=C:\NS\Tcl8.6.11
> 5. nmake -f makefile.vc install INSTALLDIR=C:\NS\Tcl8.6.11
>
> Compile Tk 8.6.11
> --
> 1. Download sources and extract to D:\NSWIN\SRC\tk8.6.11
> 2. Run Visual Studio "x64 Native Tools Command Prompt for VS 2019" as
> Administrator
> 3. Change directory to D:\NSWIN\SRC\tk8.6.11\win
> 4. nmake -f makefile.vc INSTALLDIR=C:\NS\Tcl8.6.11
> 5. nmake -f makefile.vc install INSTALLDIR=C:\NS\Tcl8.6.11
>
> OpenSSL
> --
> 1. Download sources and extract to D:\NSWIN\SRC\openssl-master
> 2. Download and install Strawberry Perl (dont use ActivePerl, something
> wrong with that)
> 3. Download and install NASM
> 4. Add Perl and NASM to the windows Path in Windows System Properties,
> reboot to be sure :)
> 5. Run Visual Studio "x64 Native Tools Command Prompt for VS 2019" as
> Administrator
> 6. Change directory to D:\NSWIN\SRC\openssl-master
> 7. Install some Perl dependencies:
> a) cpan -i Text::Template
> b) cpan -i Test::More
> 8. perl Configure VC-WIN64A --prefix=C:\NS\OpenSSL
> --openssldir=C:\NS\OpenSSL\SSL
> 9. nmake (and go grab some coffee)
> 10. nmake test (More Coffee!!)
> 11. nmake install_sw
>
> After that downloaded naviserver source, edit Makefile.win32,
> include\Makefile.win32, changed path to tcl and openssl and compiled with
> nmake
>
> On Thu, Mar 4, 2021 at 2:56 PM Andrew Piskorski  wrote:
>
>>
>> The Tcl version should not really matter, I just install whatever the
>> latest stable ActiveTcl is for Windows.  I believe the latest I've
>> used was 8.6.x, which I installed into "C:\P\Tcl-64-8.6\".  It should
>> also work to compile Tcl yourself on Windows, but I haven't tried that
>> in many years.
>>
>>
>> For Oracle on Windows, I installed the Oracle "Instant Client" version
>> 19.6.0.0.0 a while back.  This was 4 separate zip files:
>>
>>   instantclient-basiclite-windows.x64-19.6.0.0.0dbru.zip
>>   instantclient-sdk-windows.x64-19.6.0.0.0dbru.zip
>>   instantclient-sqlplus-windows.x64-19.6.0.0.0dbru.zip
>>   instantclient-tools-windows.x64-19.6.0.0.0dbru.zip
>>
>> To install, I simply unzipped all of those into a tmp directory, which
>> created an "instantclient_19_6" directory.  Then I copied the whole
>> "instantclient_19_6" directory into "C:\P\oracle\".
>> The "nsoracle/Makefile.win32" has this line to use it:
>>
>>   ORACLE_HOME = C:\P\oracle\instantclient_19_6
>>
>>
>> For OpenSSL, back in Windows XP days I did compile it on Windows, and
>> as far as I recall that worked, although the compile was VERY slow,
>> and I had to install various support tools first (NASM Assembler,
>> etc.).  More recently I've only installed binary OpenSSL builds on
>> Windows.
>>
>> I've used this OpenSSL for Windows:
>>
>>   http://www.slproweb.com/products/Win32OpenSSL.html
>>
>> There are also binaries for the older 1.0.2 OpenSSL line, here, but I
>> don't think I ever actually used them:
>>
>>   https://indy.fulgan.com/SSL/
>>
>> See also OpenSSL's own list of Windows Binary Builds:
>>
>>   https://wiki.openssl.org/index.php/Binaries
>>
>> Old Win32OpenSSL instructions said to install the Visual C++ 2008
>> Redistributable Package via its "vcredist_x64.exe" and
>> "vcredist_x86.exe" web installers:
>>
>>   x64: http://www.microsoft.com/en-us/download/details.aspx?id=15336
>>   x86: http://www.microsoft.com/en-us/download/details.aspx?id=29
>>
>> But of course these days should probably be using the 64-bit "Visual
>> C++ 2017 Redistributable" or something even newer instead.  (Read
>> their current instuctions.)
>>
>> Around 2019-01, I installed the (at the time) latest
>> "Win64OpenSSL-1_1_1a.exe" and "Win32OpenSSL-1_1_1a.exe" into
>> "C:\P\OpenSSL-Win64\".  When it asked my where to "Copy OpenSSL DLLs
>> to:" I always pick "The OpenSSL binaries (/bin) directory", NOT the
>> default "Windows system directory".
>>
>> --
>> Andrew Piskorski 
>>
>>
>> ___
>> naviserver-devel mailing list
>> naviserver-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>>
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Recommended approach to building Naviserver on Windows?

[Maksym Zinchenko]

I did something like:
Compile Tcl 8.6.11
--

1. Download sources and extract to D:\NSWIN\SRC\tcl8.6.11
2. Run Visual Studio "x64 Native Tools Command Prompt for VS 2019" as
Administrator
3. Change directory to D:\NSWIN\SRC\tcl8.6.11\win
4. nmake -f makefile.vc INSTALLDIR=C:\NS\Tcl8.6.11
5. nmake -f makefile.vc install INSTALLDIR=C:\NS\Tcl8.6.11

Compile Tk 8.6.11
--
1. Download sources and extract to D:\NSWIN\SRC\tk8.6.11
2. Run Visual Studio "x64 Native Tools Command Prompt for VS 2019" as
Administrator
3. Change directory to D:\NSWIN\SRC\tk8.6.11\win
4. nmake -f makefile.vc INSTALLDIR=C:\NS\Tcl8.6.11
5. nmake -f makefile.vc install INSTALLDIR=C:\NS\Tcl8.6.11

OpenSSL
--
1. Download sources and extract to D:\NSWIN\SRC\openssl-master
2. Download and install Strawberry Perl (dont use ActivePerl, something
wrong with that)
3. Download and install NASM
4. Add Perl and NASM to the windows Path in Windows System Properties,
reboot to be sure :)
5. Run Visual Studio "x64 Native Tools Command Prompt for VS 2019" as
Administrator
6. Change directory to D:\NSWIN\SRC\openssl-master
7. Install some Perl dependencies:
a) cpan -i Text::Template
b) cpan -i Test::More
8. perl Configure VC-WIN64A --prefix=C:\NS\OpenSSL
--openssldir=C:\NS\OpenSSL\SSL
9. nmake (and go grab some coffee)
10. nmake test (More Coffee!!)
11. nmake install_sw

After that downloaded naviserver source, edit Makefile.win32,
include\Makefile.win32, changed path to tcl and openssl and compiled with
nmake

On Thu, Mar 4, 2021 at 2:56 PM Andrew Piskorski  wrote:

>
> The Tcl version should not really matter, I just install whatever the
> latest stable ActiveTcl is for Windows.  I believe the latest I've
> used was 8.6.x, which I installed into "C:\P\Tcl-64-8.6\".  It should
> also work to compile Tcl yourself on Windows, but I haven't tried that
> in many years.
>
>
> For Oracle on Windows, I installed the Oracle "Instant Client" version
> 19.6.0.0.0 a while back.  This was 4 separate zip files:
>
>   instantclient-basiclite-windows.x64-19.6.0.0.0dbru.zip
>   instantclient-sdk-windows.x64-19.6.0.0.0dbru.zip
>   instantclient-sqlplus-windows.x64-19.6.0.0.0dbru.zip
>   instantclient-tools-windows.x64-19.6.0.0.0dbru.zip
>
> To install, I simply unzipped all of those into a tmp directory, which
> created an "instantclient_19_6" directory.  Then I copied the whole
> "instantclient_19_6" directory into "C:\P\oracle\".
> The "nsoracle/Makefile.win32" has this line to use it:
>
>   ORACLE_HOME = C:\P\oracle\instantclient_19_6
>
>
> For OpenSSL, back in Windows XP days I did compile it on Windows, and
> as far as I recall that worked, although the compile was VERY slow,
> and I had to install various support tools first (NASM Assembler,
> etc.).  More recently I've only installed binary OpenSSL builds on
> Windows.
>
> I've used this OpenSSL for Windows:
>
>   http://www.slproweb.com/products/Win32OpenSSL.html
>
> There are also binaries for the older 1.0.2 OpenSSL line, here, but I
> don't think I ever actually used them:
>
>   https://indy.fulgan.com/SSL/
>
> See also OpenSSL's own list of Windows Binary Builds:
>
>   https://wiki.openssl.org/index.php/Binaries
>
> Old Win32OpenSSL instructions said to install the Visual C++ 2008
> Redistributable Package via its "vcredist_x64.exe" and
> "vcredist_x86.exe" web installers:
>
>   x64: http://www.microsoft.com/en-us/download/details.aspx?id=15336
>   x86: http://www.microsoft.com/en-us/download/details.aspx?id=29
>
> But of course these days should probably be using the 64-bit "Visual
> C++ 2017 Redistributable" or something even newer instead.  (Read
> their current instuctions.)
>
> Around 2019-01, I installed the (at the time) latest
> "Win64OpenSSL-1_1_1a.exe" and "Win32OpenSSL-1_1_1a.exe" into
> "C:\P\OpenSSL-Win64\".  When it asked my where to "Copy OpenSSL DLLs
> to:" I always pick "The OpenSSL binaries (/bin) directory", NOT the
> default "Windows system directory".
>
> --
> Andrew Piskorski 
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Recommended approach to building Naviserver on Windows?

[Maksym Zinchenko]

Hello. I've managed to compile Naviserver 4.99.20 64bits on Windows 10 with
Visual Studio 2019 (Express), used 8.6 Tcl sources and PostgreSQL, had some
problems with OpenSSL 1.1.1, but I'm not really good at it. But it's doable.
Cheers
Maksym

On Thu, Mar 4, 2021 at 9:57 AM Brian Fenton  wrote:

> Hi
>
> we are looking to start doing our own Windows builds of Naviserver. We
> have been fortunate to have used Maurizio's great work in the past, but our
> management now see it as a priority that we do our own builds in-house.
> I see that a lot of good work has been done recently by the community on
> the Windows front. I just have a few questions to help us get started as we
> don't have much experience of this kind of thing, and there seem to be
> multiple ways of doing some of the steps.
>
> I took a look at the install documentation here
> https://bitbucket.org/naviserver/naviserver/src/master/ and I
> noticed that there are 3 approaches described for compiling and installing
> on Windows.  
> Which one of the 3 is the preferred approach within the community?
> Which TCL version is currently recommended of the two versions currently
> available (8.6.11 and 8.7a3 from
> https://www.tcl-lang.org/software/tcltk/download.html )?
> Are there Naviserver-specific instructions for configuring, building and
> installing TCL on Windows (apart from the standard instructions here
> https://www.tcl-lang.org/doc/howto/compile.html#win)?
> Are there Windows-specific instructions for compiling the latest Oracle
> driver? I see https://bitbucket.org/naviserver/nsoracle/src/master/ has
> recent commits, but the documentation still references AOLserver so it's
> not clear how up to date that is.
> What is the recommended version of OpenSSL (or LibreSSL?) for Naviserver
> running on Windows (seems to be quite a few choices for OpenSSL here
> https://wiki.openssl.org/index.php/Binaries )? Are there Naviserver
> -specific instructions for getting either of these running on Windows too?
>
> thanks
> Brian
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] ns_quotehtml

[Maksym Zinchenko]

Thanks Gustaf, I've seen it already in the release changelog announcement.

As always amazing job.

On Sun, Jan 17, 2021, 21:06 Gustaf Neumann  wrote:

> I forgot to mention here: the updated version of "nsshell" performs now
> proper html-quoting, such that confusing result interpretations won't
> happen anymore.
>
> all the best -gn
> On 10.01.21 11:40, Maksym Zinchenko wrote:
>
> :) Sorry false alert, my bad. Due to lack of coffee and some good sleep, I
> was testing some procs in "nsshell", and of course browser rendering
> escaped characters back to normal, and my zombie brain couldn't process
> why. It works fine, at least it has a regression test now ;)
>
> On Sat, Jan 9, 2021 at 5:44 PM Gustaf Neumann  wrote:
>
>>
>> On 09.01.21 19:01, Maksym Zinchenko wrote:
>>
>> Hello everyone, may someone confirm that ns_quotehtml works as it should,
>> because on my system its not:
>>
>> Command: ns_quotehtml {}
>> Result: 
>>
>> What is your system? It works for me. What is your OS?
>> Interestingly, ns_quotehtml was not covered by the regression tests.
>>
>> -g
>>
>>
>> % ./nsd/nsd -c
>> [09/Jan/2021:19:36:16][93167.10d7eb5c0][-main:conf-] Notice: OpenSSL 1.1.1i  
>> 8 Dec 2020 initialized
>> [09/Jan/2021:19:36:16][93167.10d7eb5c0][-main:conf-] Notice: nsmain: 
>> NaviServer/4.99.20rc2 (c407fafe7b3f+) starting
>> ...
>> %
>> % ns_quotehtml {}
>> span class=foo
>>
>>
>> ___
>> naviserver-devel mailing list
>> naviserver-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>>
>
>
> ___
> naviserver-devel mailing 
> listnaviserver-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] SNI virtual hosting

[Maksym Zinchenko]

Hello, all config files are TCL scripts, so I have in my main "conf" folder
a folder called "servers".
My main nsd configuration file like this:

~~~tcl
...
set servers [glob -directory [file join /opt/ns/conf/ servers] *]
...
ns_section  "ns/module/nsssl" {
ns_paramdefaultserver   
$hostname
ns_paramcertificate 
/opt/ns/modules/nsssl/main.pem
ns_paramaddress 
$ip_addr
ns_paramport
$ssl_port
ns_paramhostname
$hostname
ns_paramciphers 

"ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!RC4"
ns_paramprotocols   
"!SSLv2:!SSLv3"
ns_paramverify  
0
ns_paramextraheaders {
Strict-Transport-Security "max-age=31536000; 
includeSubDomains"
X-Frame-Options SAMEORIGIN
X-Content-Type-Options nosniff
}
ns_parammaxinput
[expr {$max_file_upload_mb * 1024 * 1024}]
ns_paramrecvwait
[expr {$max_file_upload_min * 60}]
}
...
ns_section  "ns/servers" {
foreach server $servers {
ns_param [file tail $server] "[file tail $server]"
}
}

ns_section  "ns/module/nsssl/servers" {
foreach server $servers {
ns_param [file tail $server] [file tail $server]
}
}

ns_section  "ns/module/nssock/servers" {
foreach server $servers {
ns_param [file tail $server] [file tail $server]
}
}

...
~~~
For each server I want to add, I create a config file inside servers
folder, and name it as my domain or subdomain. For example:
myfirstserver.com, secondserver.com


~~~tcl
set server "myfirstserver.com"
set db_name$server
...
ns_section  "ns/server/${server}/module/nsssl" {
ns_paramcertificate 
${homedir}/modules/nsssl/${server}.pem
ns_paramciphersuites
"TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256"
}
...
~~~

Basically I'm just cloning one file and setting up domain name in the
beginning of it. My certificates have the same name as domain they are
used for

So when Naviserver starts it will read my main config file, read
config files from the "server" directory with certificates, database
connection settings etc.

So it's really easy to setup new virtual server: clone one of
conf/server files, change name, setup server var, get and save
certificate with the same name as server, reboot nsd and ready to go.

Cheers


On Sun, Jan 17, 2021 at 10:30 AM Jeff R  wrote:

> Hi all,
>
> Looking at the docs and recent list postings Naviserver supports virtual
> hosting over ssl using SNI by defining multiple servers in the config and
> specifying the certificate for each server.  The downside of this is that
> in order to add a new host you need to modify the config, rather than just
> adding new certificates.
>
> I'm thinking it would allow a simpler config to support having a scheme
> matching the host name to the certificate file name implicitly, rather than
> explicitly.   That is, instead of looking for the certificate configured
> for a matching nsssl server, look for a file matching
> "directory/${host}.pem"
>
> This could be mostly accomplished by having the config file create the
> sections at runtime (i.e., at config load time) by scanning the desired
> directory, but doing so would still require the server to be restarted if a
> new certificate was added.
>
> A config supporting this might look like
>
> ns_section ns/module/nsssl {
>ns_param port  8433
>ns_param defaultserver s1
>ns_param certificate   /usr/local/ns/modules/nsssl/server.pem
>ns_param sni_certificate   /usr/local/ns/modules/nsssl/%host%.pem
> }
> with whatever placeholder syntax makes sense for "host" there.   ($host
> obviously can't work)
>
> I don't know how this approach would work for wildcard certificates -
> maybe it would need to look for "domain" and then "*.domain", or if looking
> for "foo.domain.xyz" if could try "domain.xzy" if it exists on the
> assumption it's a wildcard?
>
> This is not critical for me as a letsencrypt SAN certificate will probably
> work fine, but I like to keep my options open :)
>
> Cheers,
> -J
>
> 

Re: [naviserver-devel] ns_quotehtml

[Maksym Zinchenko]

:) Sorry false alert, my bad. Due to lack of coffee and some good sleep, I
was testing some procs in "nsshell", and of course browser rendering
escaped characters back to normal, and my zombie brain couldn't process
why. It works fine, at least it has a regression test now ;)

On Sat, Jan 9, 2021 at 5:44 PM Gustaf Neumann  wrote:

>
> On 09.01.21 19:01, Maksym Zinchenko wrote:
>
> Hello everyone, may someone confirm that ns_quotehtml works as it should,
> because on my system its not:
>
> Command: ns_quotehtml {}
> Result: 
>
> What is your system? It works for me. What is your OS?
> Interestingly, ns_quotehtml was not covered by the regression tests.
>
> -g
>
>
> % ./nsd/nsd -c
> [09/Jan/2021:19:36:16][93167.10d7eb5c0][-main:conf-] Notice: OpenSSL 1.1.1i  
> 8 Dec 2020 initialized
> [09/Jan/2021:19:36:16][93167.10d7eb5c0][-main:conf-] Notice: nsmain: 
> NaviServer/4.99.20rc2 (c407fafe7b3f+) starting
> ...
> %
> % ns_quotehtml {}
> span class=foo
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] ns_quotehtml

[Maksym Zinchenko]

Hello everyone, may someone confirm that ns_quotehtml works as it should,
because on my system its not:

Command: ns_quotehtml {}
Result: 

Thank you in advance
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] ns_conn auth

[Maksym Zinchenko]

Gustaf, thank you very much.
You are amazing as always.

I thought "Authorization" is always kind of key -> pair values, guess I was
wrong.
I think most of them are so.

Thank you.

On Sat, Nov 21, 2020 at 7:05 PM Gustaf Neumann  wrote:

> Dear Maksym,
>
> [ns_conn auth] handles Digest authentication (i've never used it), but
> "Bearer" is not handled.
> The fields in the "Authorization" request header field are not always
> structured the same way,
> so NaviServer tries it interprete it based on the first word. The known
> types are enumerated
> in the C code.
>
> I've added a sensible result for the "Bearer" Authorization to bitbucket.
>
> all the best -g
>
>
>
> On 21.11.20 17:59, Maksym Zinchenko wrote:
>
> Hello, hope you are doing well.
> I have a little question. Maybe a bug, would be so kind to help me out.
> According to documentation [ns_conn auth] should return all authorization
> headers as a ns_set.
> It works fine when using "Basic Authentication scheme", i've seen in the
> source code it should work with "Digest"
> But when im trying to use "Bearer Authentication scheme" or "Custom" it
> returns empty ns_set, but i can see Authentication part in [ns_conn
> headers].
>
> P.S.
> Im using https://reqbin.com/ to test my API request.
>
> Thank you
>
>
> ___
> naviserver-devel mailing 
> listnaviserver-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] ns_conn auth

[Maksym Zinchenko]

Hello Iuri, I think you misunderstood my question.
My problem is not to set but to get "ns_set"

I'm using this:
https://naviserver.sourceforge.io/n/naviserver/files/ns_conn.html#4


When client sends request to my server i want to get authentication method
using [ *ns_conn auth* ]
Here is simple example code:
...
ns_register_proc GET /test_api ::api::test_api

proc test_api {args} {
set url [file split [ns_conn url]]
set api_version [lindex $url 2]
set resource [lindex $url 3]
set method [ns_conn method]
set auth [ns_conn auth]

puts "AUTHORIZATION: [ns_set array $auth]"
puts "HEADERS: [ns_set array [ns_conn headers]]"
ns_respond -status 200 -type "application/txt" -string "OK"
}
...

This is what i get when use basic, looks good:

...

AUTHORIZATION: AuthMethod Basic Password admin Username admin
HEADERS: Host dev.testdomain.org Accept */* Accept-Encoding {deflate,
gzip} Authorization {Basic YWRtaW46YWRtaW4=} User-Agent {Mozilla/5.0
(Windows NT 10.0; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0}
X-Real-IP
xxx.90.xxx.244

...

This is what I get using bearer:

...
AUTHORIZATION:

HEADERS: Host dev.testdomain.org Accept */* Accept-Encoding {deflate,
gzip} Authorization {Bearer } User-Agent {Mozilla/5.0
(Windows NT 10.0; Win64; x64; rv:83.0) Gecko/20100101 Firefox/83.0}
X-Real-IP xxx.90.xxx.244


...

I was expecting [*ns_conn auth*] to return something like: AuthMethod
bearer Token 








On Sat, Nov 21, 2020 at 4:50 PM Iuri de Araujo Sampaio 
wrote:

> Hello Maksym,
> One can add any custom variable (i.e. Authorization “Bearer …") in the
> response headers as in:
>
> …
> set headers [ns_set new]
> ns_set put $headers "Authorization" "Bearer $token"
>  …
> ns_respond -status $status -type "application/json" -headers $headers
> -string $response
> ad_script_abort
>
>
> That chunk is part of a JWT/login authentication API method. However,
> that’s very basic, and probably I haven’t understood the actual problem
> that you are facing.
>
> Let us know,
>
> Best wishes,
> I
>
>
>
> On Rab. II 6, 1442 AH, at 13:59, Maksym Zinchenko 
> wrote:
>
> Hello, hope you are doing well.
> I have a little question. Maybe a bug, would be so kind to help me out.
> According to documentation [ns_conn auth] should return all authorization
> headers as a ns_set.
> It works fine when using "Basic Authentication scheme", i've seen in the
> source code it should work with "Digest"
> But when im trying to use "Bearer Authentication scheme" or "Custom" it
> returns empty ns_set, but i can see Authentication part in [ns_conn
> headers].
>
> P.S.
> Im using https://reqbin.com/ to test my API request.
>
> Thank you
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] ns_conn auth

[Maksym Zinchenko]

Hello, hope you are doing well.
I have a little question. Maybe a bug, would be so kind to help me out.
According to documentation [ns_conn auth] should return all authorization
headers as a ns_set.
It works fine when using "Basic Authentication scheme", i've seen in the
source code it should work with "Digest"
But when im trying to use "Bearer Authentication scheme" or "Custom" it
returns empty ns_set, but i can see Authentication part in [ns_conn
headers].

P.S.
Im using https://reqbin.com/ to test my API request.

Thank you
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] nsstats.tcl

[Maksym Zinchenko]

Hello, I've updated Naviserver from the latest commit in bitbucket and
nsstats.tcl stopped authenticating me with plain authentication. I've
checked and noticed that "ns_conn authpassword" and "ns_conn *authuser*"
return empty values. Is it a bug ?
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] CMD exec, CMD 1row,CMD select

[Maksym Zinchenko]

Ok, but now its giving me compilation error in nsd:

/usr/bin/ld: libnsd.so: undefined reference to `NsTclCryptoScryptObjCmd'

What i'm doing wrong?

On Sun, Sep 6, 2020 at 11:26 AM Gustaf Neumann  wrote:

> On 06.09.20 12:00, Maksym Zinchenko wrote:
> > Hello, I have a lot of lines in my logs like:
> this was escaped debug output on the (unreleasaed) master branch. this
> is removed by now.
>
> Many thanks for noting this
>
> -g
>
>
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] CMD exec, CMD 1row,CMD select

[Maksym Zinchenko]

Hello, I have a lot of lines in my logs like:

CMD exec:  expire> (none)

CMD select:  (string)

CMD 1row:  (none)

I've searched my code, but it looks like it's not mine :) Where do I switch
off this sql logging?

Thanks
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] TDBC question

[Maksym Zinchenko]

Hello Thorpe, Im not sure if this question is related to Naviserver. It
looks like you have more application logic error or smth.
As I understood from your email its 2 lines you get from DB or smth,
shopping_cart_id looks identical  ( 186467 186467  --- may be it is a
typo), but try to check your code, may be use dict, for example when you
run query get resul as a dict in format "column_name" "value" etc.
Im pretty sure it has nothing to do with TDBC or Naviserver.

Maksym Zinchenko

On Sat, Aug 15, 2020 at 7:54 PM THORPE MAYES via naviserver-devel <
naviserver-devel@lists.sourceforge.net> wrote:

> HI,
>
> I am using TDBC to connect to a postgresql database.
>
> I am getting strange behavior with a tcl foreach statement.
>
> Here is the list I am stepping through:
>
> LIST: {186467 2225 1 122 771 {6 inch Ham Sub} 1 5.75 t {Plain no cheese}
> t White t Chips f {} {}} {186467 2225 2 13 1192 {Smoothie with chips} 1
> 5.50 t strawberry f {} f {} f {} {}}
>
> The first element in each sub-list in the list is the shopping_cart_id
>
> When I move through the list I assign values to variables using lassign
>
> Every once and a while the variables are not assigned values. For example,
> I get an error saying that shopping_cart_id does not exist when I try to
> use in in the foreach loop.
>
> I have never had this issue before.
>
> The major difference in this project is that I am using TDBC to connect to
> the database.
>
> Is there a problem with TDBC that might cause this behavior?
>
> Thank you.
>
> Thorpe
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Host based virtual hosts

[Maksym Zinchenko]

Thank you, very much. I'm going to implement it right now.

On Wed, Jul 8, 2020 at 2:29 PM Gustaf Neumann  wrote:

> Dear Maksym,
>
>  > So, for the time being, we have no such support in NaviServer.
>
> With the commit [1], i've added the first draft for server-side SNI
> to NaviServer. It is not fully finished, but sufficiently stable
> (already running on  OpenACS.org) and should provide the
> basic server-side support to supply and use per-virtual-server
> certificates (see [1] for a configuration example). If possible
> test this with your configuration.
>
> The known open items are listed below.
>
> all the best
>
> -g
>
> TODOs:
> - add support for OCSP with SNI
> - add support for Diffie Hellman with SNI
> - add "ciphersuites" to ns_http
> - add documentation for virtual hosting with nsssl
>
> [1]
>
> https://bitbucket.org/naviserver/naviserver/commits/4662b12c3c548db4b2d0ed6625c66260662d2089
>
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Host based virtual hosts

[Maksym Zinchenko]

OMG, Gustaf you are amazing, I just saw last commit with SNI (Server Name
Indication (SNI.
Thank you

On Sat, Jul 4, 2020 at 7:55 PM Gustaf Neumann  wrote:

>
> On 04.07.20 21:08, Maksym Zinchenko wrote:
> > So now im facing this issues:
> > 1) Some of the domains are not mine and belong to some clients, they
> > might have their own certificates they bought.
> > 2) Also I don't know how many clients I will have in the future, so
> > everytime I add a new domain I need to regenerate multi-domain SAN
> > cert (they are really expensive and I can't afford it)
>
> i see, i am having probably too much the letsencrypt world in mind.
>
> > I thought it was more simple like:
> > 1) Load nsssl globally
> > 2) Define "defaultserver" and certificate for this "defaultserver"
> > 3) Define different certificates for each domain in ns_section
> > "ns/server/${server}/module/nsssl
> You want probably all these virtual servers listening on the same port
> (if not, just load multiple drivers for different ports).
>
> With the single port, there is a chicken-egg problem: the right certificate
> is needed at the time the connection is opened, and the virtual server can
> be only detected while reading the request header.
>
> This is a well known problem, for which the SNI TLS extension was invented
> (a hostname that can be used for identifying the certificate is passed
> during the TLS handshake as well). Currently, NaviServer supports SNI only
> at the client side (in ns_http), but not at the server side. It is not
> overly
> complicated to implemented this, but not trivial either, and will take
> some effort.
>
> So, for the time being, we have no such support in NaviServer.
>
> all the best
> -gn
>
> [1] https://en.wikipedia.org/wiki/Server_Name_Indication
>
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Host based virtual hosts

[Maksym Zinchenko]

 Dear Gustaf, my main problem is certificate. Loading the drivers globally
works. When I access domains they are pointing and working correctly.
The problem is nsssl when loaded globally must have "ns_param certificate",
I misunderstood this parameter. I thought its like "ns_param
defaultserver", like this "default certificate" will be used to connect to
defaultserver, not like "this certificate" will be used to connect to all
servers.

So now im facing this issues:
1) Some of the domains are not mine and belong to some clients, they might
have their own certificates they bought.
2) Also I don't know how many clients I will have in the future, so
everytime I add a new domain I need to regenerate multi-domain SAN cert
(they are really expensive and I can't afford it)

I thought it was more simple like:
1) Load nsssl globally
2) Define "defaultserver" and certificate for this "defaultserver"
3) Define different certificates for each domain in ns_section
"ns/server/${server}/module/nsssl

I hope my explanation makes sense.

Cheers, Maksym


On Sat, Jul 4, 2020 at 4:32 PM Gustaf Neumann  wrote:

> Dear Maksym,
>
> If you are using virtual host, you should load the drivers globally
> and - when connect via https - use multi-domain SAN certificates
> (as produced e.g. by [1]).
>
> openacs.org is a configuration using (a) multiple names for the
> same server (such as .org, .net, with and without "www." prefix),
> and (b) uses virtual hosts to redirect requests to different servers
> (e.g. https://dotlrn.org/, https://cvs.openacs.org/, http://xotcl.org/)
>
> it has essentially two global drivers, where both drivers
> have multiple IP addresses assigned:
>
>set address "137.208.116.31 2001:628:404:74::31"
>
> The essential part of the log file is below (pasted together).
> Below this is the definition of the xotcl.org website.
>
> Does this help?
>
> -gn
>
>
> ==
> set server  "openacs.org"
> # ...
>
> ns_section ns/module/nsssl/servers
> ns_param$server $server
> ns_param$server www.openacs.net
> ns_param$server www.openacs.org
> ns_param$server www.openacs.com
> ns_param$server openacs.net
> ns_param$server openacs.com
> ns_param$server openacs.wu.ac.at
> ns_param$server openacs.wu-wien.ac.at
> ns_param$server smtp.openacs.org
> ns_paramdotlrn  dotlrn.org
> ns_paramdotlrn  dotlrn.net
> ns_paramdotlrn  dotlrn.com
> ns_paramdotlrn  www.dotlrn.com
> ns_paramdotlrn  www.dotlrn.org
> ns_paramcvs fisheye.openacs.org
> ns_paramcvs cvs.openacs.org
>
> ns_section ns/module/nssock/servers
> ns_param$server $server
> ns_param$server www.openacs.net
> ns_param$server www.openacs.org
> ns_param$server www.openacs.com
> ns_param$server openacs.net
> ns_param$server openacs.com
> ns_param$server openacs.wu.ac.at
> ns_param$server openacs.wu-wien.ac.at
> ns_param$server smtp.openacs.org
> ns_paramdotlrn  dotlrn.org
> ns_paramdotlrn  dotlrn.net
> ns_paramdotlrn  dotlrn.com
> ns_paramdotlrn  www.dotlrn.com
> ns_paramdotlrn  www.dotlrn.org
> ns_paramdotlrn  www.dotlrn.net
> ns_paramcvs fisheye.openacs.org
> ns_paramcvs cvs.openacs.org
> ns_paramxotcl   xotcl.org
> ns_paramxotcl   www.xotcl.org
> ==
>
>
> ==
> 
> # XOTcl.org
> 
> ns_section "ns/servers"
> ns_param xotcl "Old XOTcl Web Site"
>
> ns_section ns/module/nssock/servers
> ns_param xotcl xotcl.org
> ns_param xotcl www.xotcl.org
>
> ns_section ns/server/xotcl
> ns_param minthreads 2
>
> ns_section "ns/server/xotcl/tcl"
> ns_param library /home/xotcl/www.xotcl.org/naviserver
> ==
>
>
> [1] https://bitbucket.org/naviserver/letsencrypt/src/maste

[naviserver-devel] Host based virtual hosts

[Maksym Zinchenko]

Hello, I'm trying to set up host based virtual hosts with naviserver but I
can't understand it easily or correctly. so what i'm trying to accomplish:
I have one big dedicated server, and i'm moving all my VPS's to it.
So Ive created the main config file /opt/ns/conf/dz_nsd.tcl,
subdirectory/opt/ns/conf/ servers.
In dz_nsd.tcl I have this lines:
>set servers [glob -directory [file join /opt/ns/conf/ servers] *]
>ns_section "ns/servers" {
>foreach server $servers {
>ns_param [file tail $server] "[file tail $server]"
>}
>}

>ns_section "ns/module/nsssl/servers" {
>foreach server $servers {
>ns_param [file tail $server] [file tail $server]
>}
>}

>ns_section "ns/module/nssock/servers" {
>foreach server $servers {
>ns_param [file tail $server] [file tail $server]
>}
>}

In the server directory I have files in format: domain_name.com,
domain_name.org, domain_name.net. Basically its 1 template file where I
have defined pageroot, servername, db pools and other configurations
specific to this server.
According to manual nsssl module is loaded per server, so i'm doing it in
my files in server directory:
>ns_section ns/server/${server}/modules {
>ns_param  nssslnsssl.so
>}
And loading a certificate:
>ns_section ns/server/${server}/module/nsssl {
>ns_param   certificate   /opt/ns/modules/nsssl/${server}.pem
>
>}

What I'm trying to accomplish with this:
1) If i need a new server, just copy my server template, change name to
domain_name, set server var inside to domain name and drop it in server
directory.
2) Get a certificate for domain place it in modules/nsssl/${domain_name}.pem
3) Restart naviserver and voilà!

The problem is when I'm trying to access my domains they all go to the same
domain. Doesn't matter what im trying to access domain_name.com,
domain_name.org, domain_name.net all goes to domain_name.com

Would be so kind to tell me what I'm doing wrong?
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] using rwlocks for nsv variables

[Maksym Zinchenko]

Hello,first of all thank you for great work.
If I understood correctly on compile time we need to provide a
configuration option to compile NaviServer with mutex or rwlocks, so option
b sounds good and more than enough.
Default is rwlocks.
Correct me if I'm wrong.


On Tue, Jun 30, 2020 at 7:56 PM Zoran Vasiljevic  wrote:

> Excellent work as always!
> I'd go for the b. option.
>
> Am 30.06.2020 um 22:39 schrieb Gustaf Neumann :
>
> 
>
> Dear all,
>
> some of you might have noticed the recent changes in the NaviServer
> repository concerning rwlocks. rwlocks support multiple concurrent readers
> and behave like mutex locks in the case of writers. rwlocks can improve
> concurrency especially in applications with many parallel threads and high
> loads, but these are only better in cases, where there are more readers
> than writers.
>
> The current version of NaviServer uses rwlocks e.g. for URLspace and for
> nsv variables. Here are some statistics of these locks collected on a
> real-world application (openacs.org):
>
> Name  Locks   BusyReadWrite   Write %
> nsv:7:openacs.org 33.14M  4   33.13M  8.41K   0.03%
> nsv:6:openacs.org 16.85M  249 16.4M   453.88K 2.69%
> nsv:3:openacs.org 15.09M  3   15.04M  46.88K  0.31%
> nsv:2:openacs.org 10.26M  5   10.23M  38.17K  0.37%
> nsv:5:openacs.org 9.98M   0   9.98M   1.57K   0.02%
> ns:rw:urlspace:4  4.45M   0   4.45M   86  0.00%
>
>
> As one can see, the vast majority of operations are read operations, where
> typically less than one percent are write operations. One can see as well
> the very little number of busy operations, where a lock has to be blocked.
> The same site with the same traffic reaches about 2K busy locks for nsv:6
> (instead of 4) and about 3.5K busy locks for nsv:6 (instead of 249) on the
> same number of locks. The improvements on the busy locks are significantly
> higher on sites with more connection threads and more traffic.
>
> However, on some other applications, the write ratio might be different,
> and there might not be always only improvements, as the test below show.
>
> In our little test, we create 20 background threads busily reading and
> writing from nsv variables, while we are measuring of a foreground task
> hammering on the very same nsv variables (with different mixes of
> background tasks and writer percentages).
>
> The first chart shows results under macOS, relative to the same task using
> mutex locks instead of rwlocks:
>
> 
>
> In the first three columns, we see the performance without background
> traffic (i.e. without concurrency). The performance is measured relative to
> the same application using mutex locks (lower numbers are better). We see
> that  without concurrency, the rwlocks lead to better results (runtime
> improvement about 25%). The next three columns show results with 20
> background tasks, just reading busily from the nsv variables. The version
> with rwlocks is faster by a factor of 10 in the best case (just foreground
> and background readers). But as one can see, the performance benefit
> reduces when more write operations are performed. The last three columns
> show the situation, when we have 50% read and write operations in the
> background.
>
> When we consider these values in combination with the statistics of
> openacs.org, we see that we are there essentially in the sweep spot where
> rwlocks shine (with practically no writers).
>
> The situation on Linux is different with less positive results (macOS
> seems to have a very good implementation of POSIX rwlocks). While in the
> best cases, the improvement on Linux is just a factor of 5 better than with
> mutex locks, in the worst cases, the performance is nearly 3 times worse.
>
> 
>
> One should also notice that these values were achieved with a conservative
> setting of rwlocks, which prevents writer starvation (the results would
> have been better without these).
>
> Since on sites like openacs.org (and even more on our high traffic sites
> of our e-learning environment), we are always in range of the first 5 bars,
> so using rwlocks is a big improvement. The improvement will be even higher
> for situations, where rwlocks are protecting areas where more computing
> intense operations are performed, or when more cores are available, etc.
>
> However, there might be NaviServer applications with nsvs out there, for
> which the change of using rwlocks for nsv variables might lead to a reduced
> performance. So we have in general the following options for the
> forthcoming release:
>
> a) hardwire nsvs to rwlocks
> b) make it a compile-time decision to choose between rwlocks and mutex
> locks for nsvs
> c) provide a configuration variable in the config file to choose between
> rwlocks and mutex locks for nsvs at startup
> d) provide a runtime API for creating nsv arrays with rwlock or mutex
>
> Currently, we have essentially (b) in the repository. (c) seems feasible
> 

Re: [naviserver-devel] What to do when [ns_base64encode] doesn't encrypt properly after the 63rd char?

[Maksym Zinchenko]

As I understood, you want to send data to your REST API in url, so you cant
use ns_base64encodin.
Because its base64 not base64url.
May be some of this will be helpful:
https://wiki.tcl-lang.org/page/base64
https://wiki.tcl-lang.org/page/base64url

Chilkat:
https://www.example-code.com/tcl/base64url_encoding.asp

On Mon, May 25, 2020 at 12:08 AM Maksym Zinchenko 
wrote:

> What do you mean its not encoded correctly?
>
> ns_base64decode
> eydpc3MnOiAnMjY5NzM0MTAwMDAxMDInLCAnYXVkJzogJ2l1cml4LmNvbS9SRVNUJywgJ3N1Yic
> 6ICc3MDQnLCAnaWF0JzogMTU5MDM2NTY1MH0=
> {'iss': '2697341102', 'aud': 'iurix.com/REST', 'sub': '704', 'iat':
> 1590365650}
>
>
> On Sun, May 24, 2020 at 11:56 PM Maksym Zinchenko 
> wrote:
>
>> sourcecode/nsd/uuencode.c maybe
>>
>> On Sun, May 24, 2020 at 11:23 PM Iuri de Araujo Sampaio 
>> wrote:
>>
>>> Hi there,
>>>
>>> It looks like *[ns_base64encode]* generates a wrong payload when
>>> message size contains more than 63 characters.
>>>
>>>
>>> https://tools.ietf.org/html/rfc4648#section-5
>>>
>>> An alternative alphabet has been suggested that would use "~" as the
>>>63rd character.
>>>
>>>
>>>
>>> (1) Payload, which generates a proper encryption.
>>>
>>> set payload [ns_base64encode "{'sub': '$user(user_id)', 'it':
>>> [ns_time]}”]
>>>
>>> eydzdWInOiAnNzA0JywgJ2lhdCc6IDE1OTAzNjU2ODF9
>>>
>>> (2) Payload, which generates a wrong encryption
>>>
>>> set payload [ns_base64encode "{'iss': '2697341102', 'aud': '
>>> iurix.com/REST', 'sub': '$user(user_id)', 'iat': [ns_time]}”]
>>>
>>> eydpc3MnOiAnMjY5NzM0MTAwMDAxMDInLCAnYXVkJzogJ2l1cml4LmNvbS9S
>>> RVNUJywgJ3N1Yic6ICc3MDQnLCAnaWF0JzogMTU5MDM2NTY1MH0=
>>>
>>>
>>> item (2) is not encoded correctly using base64url (
>>> https://tools.ietf.org/html/rfc4648#section-5). Note that padding ("=")
>>> must be omitted as per https://tools.ietf.org/html/rfc7515#section-2
>>>
>>> How would I use [ns_base64encoding], if i decide to add more content in
>>> the payload message? Is it possible?
>>>
>>>
>>> I was trying find where ns_base64encoding is implemented, within
>>> Naviserver source code. But I haven’t found yet.
>>> Then I’d be able to go deeper into the problem to identify the actual
>>> cause and try to write a solution to it.
>>>
>>> Does anyone know where is the implementation of [ns_base64encode] ?
>>>
>>> Best wishes,
>>> I
>>> ___
>>> naviserver-devel mailing list
>>> naviserver-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>>>
>>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] What to do when [ns_base64encode] doesn't encrypt properly after the 63rd char?

[Maksym Zinchenko]

What do you mean its not encoded correctly?

ns_base64decode
eydpc3MnOiAnMjY5NzM0MTAwMDAxMDInLCAnYXVkJzogJ2l1cml4LmNvbS9SRVNUJywgJ3N1Yic
6ICc3MDQnLCAnaWF0JzogMTU5MDM2NTY1MH0=
{'iss': '2697341102', 'aud': 'iurix.com/REST', 'sub': '704', 'iat':
1590365650}


On Sun, May 24, 2020 at 11:56 PM Maksym Zinchenko 
wrote:

> sourcecode/nsd/uuencode.c maybe
>
> On Sun, May 24, 2020 at 11:23 PM Iuri de Araujo Sampaio 
> wrote:
>
>> Hi there,
>>
>> It looks like *[ns_base64encode]* generates a wrong payload when message
>> size contains more than 63 characters.
>>
>>
>> https://tools.ietf.org/html/rfc4648#section-5
>>
>> An alternative alphabet has been suggested that would use "~" as the
>>63rd character.
>>
>>
>>
>> (1) Payload, which generates a proper encryption.
>>
>> set payload [ns_base64encode "{'sub': '$user(user_id)', 'it':
>> [ns_time]}”]
>>
>> eydzdWInOiAnNzA0JywgJ2lhdCc6IDE1OTAzNjU2ODF9
>>
>> (2) Payload, which generates a wrong encryption
>>
>> set payload [ns_base64encode "{'iss': '2697341102', 'aud': '
>> iurix.com/REST', 'sub': '$user(user_id)', 'iat': [ns_time]}”]
>>
>> eydpc3MnOiAnMjY5NzM0MTAwMDAxMDInLCAnYXVkJzogJ2l1cml4LmNvbS9S
>> RVNUJywgJ3N1Yic6ICc3MDQnLCAnaWF0JzogMTU5MDM2NTY1MH0=
>>
>>
>> item (2) is not encoded correctly using base64url (
>> https://tools.ietf.org/html/rfc4648#section-5). Note that padding ("=")
>> must be omitted as per https://tools.ietf.org/html/rfc7515#section-2
>>
>> How would I use [ns_base64encoding], if i decide to add more content in
>> the payload message? Is it possible?
>>
>>
>> I was trying find where ns_base64encoding is implemented, within
>> Naviserver source code. But I haven’t found yet.
>> Then I’d be able to go deeper into the problem to identify the actual
>> cause and try to write a solution to it.
>>
>> Does anyone know where is the implementation of [ns_base64encode] ?
>>
>> Best wishes,
>> I
>> ___
>> naviserver-devel mailing list
>> naviserver-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>>
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] What to do when [ns_base64encode] doesn't encrypt properly after the 63rd char?

[Maksym Zinchenko]

sourcecode/nsd/uuencode.c maybe

On Sun, May 24, 2020 at 11:23 PM Iuri de Araujo Sampaio 
wrote:

> Hi there,
>
> It looks like *[ns_base64encode]* generates a wrong payload when message
> size contains more than 63 characters.
>
>
> https://tools.ietf.org/html/rfc4648#section-5
>
> An alternative alphabet has been suggested that would use "~" as the
>63rd character.
>
>
>
> (1) Payload, which generates a proper encryption.
>
> set payload [ns_base64encode "{'sub': '$user(user_id)', 'it': [ns_time]}”]
>
> eydzdWInOiAnNzA0JywgJ2lhdCc6IDE1OTAzNjU2ODF9
>
> (2) Payload, which generates a wrong encryption
>
> set payload [ns_base64encode "{'iss': '2697341102', 'aud': '
> iurix.com/REST', 'sub': '$user(user_id)', 'iat': [ns_time]}”]
>
> eydpc3MnOiAnMjY5NzM0MTAwMDAxMDInLCAnYXVkJzogJ2l1cml4LmNvbS9S
> RVNUJywgJ3N1Yic6ICc3MDQnLCAnaWF0JzogMTU5MDM2NTY1MH0=
>
>
> item (2) is not encoded correctly using base64url (
> https://tools.ietf.org/html/rfc4648#section-5). Note that padding ("=")
> must be omitted as per https://tools.ietf.org/html/rfc7515#section-2
>
> How would I use [ns_base64encoding], if i decide to add more content in
> the payload message? Is it possible?
>
>
> I was trying find where ns_base64encoding is implemented, within
> Naviserver source code. But I haven’t found yet.
> Then I’d be able to go deeper into the problem to identify the actual
> cause and try to write a solution to it.
>
> Does anyone know where is the implementation of [ns_base64encode] ?
>
> Best wishes,
> I
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Registering/Allowing request method HTTP PUT

[Maksym Zinchenko]

0][-main-] Notice: nsssl: version
> 2.1 loaded, based on OpenSSL 1.1.0l  10 Sep 2019
> [24/May/2020:18:30:06][30098.7efbff6ea700][-main-] Notice: modload:
> loading module libthread from file /usr/local/ns/lib/thread2.8.2/
> libthread2.8.2.so
> [24/May/2020:18:30:06][30098.7efbff6ea700][-main-] Notice: nx::serializer
> version 2.3
> [24/May/2020:18:30:06][30098.7efbff6ea700][-main-] Notice: XOTcl 2.3
> loaded featuring: memcount 0 profile 0 memtrace 0 assertions 1 dtrace 0
> development 0
> [24/May/2020:18:30:06][30098.7efbff6ea700][-main-] Notice: Using ns_cache
> based on NX 2.3
> [24/May/2020:18:30:06][30098.7efbff6ea700][-main-] Notice: adp[iurix]:
> mapped {GET HEAD POST} /*.adp
> [24/May/2020:18:30:06][30098.7efbff6ea700][-main-] Notice: Loading
> OpenACS, rooted at /var/www/iurix
> [24/May/2020:18:30:06][30098.7efbff6ea700][-main-] Notice: Sourcing
> /var/www/iurix/packages/acs-bootstrap-installer/bootstrap.tcl
> [24/May/2020:18:30:06][30098.7efbff6ea700][-main-] Notice: bootstrap
> begin encoding utf-8
> [24/May/2020:18:30:06][30098.7efbff6ea700][-main-] Notice: Bootstrap:
> sourcing
> /var/www/iurix/packages/acs-bootstrap-installer/tcl/00-proc-procs.tcl
> [24/May/2020:18:30:06][30098.7efbff6ea700][-main-] Notice: Bootstrap:
> sourcing
> /var/www/iurix/packages/acs-bootstrap-installer/tcl/10-utilities-procs.tcl
> [24/
>
>
>
> On May 24, 2020, at 14:23, Maksym Zinchenko  wrote:
>
> Hello, Ive tested ns_register_tcl with my installation and its working
> fine:
> In my index.adp:
> ns_register_tcl PUT /REST/ test/test.tcl
>
> My test.tcl:
> ns_log Notice "Running TCL script users/edit"
>
> In log:
>
>  
> [0;32m[24/May/2020:16:16:47][26138.7f62510c4700][-conn:***:default:0:12-]
>   [0m [0;39mNotice: Running TCL script users/edit [0m
>
> Check your logs if naviserver reads /tcl/http-handlers-init.tcl. Your path to 
> scripts etc. Its something with your installation
> Cmpts.: Maksym Zinchenko
>
>
> On Sat, May 23, 2020 at 7:22 PM  wrote:
>
>> Hi there,
>> In attempt to register PUT in my NS instalation  ( ie. HTTP PUT method),
>> I added the following line, within  /tcl/http-handlers-init.tcl library :
>>
>> ns_register_tcl -options {stream stricterror} -- PUT /REST/*.tcl
>>
>> I also try the simplest call:  ns_register_tcl PUT /REST/*.tcl
>>
>> Then, within /REST/ directory, there's a TCL script edit.tclm and the
>> first line is a simple log call, for testing purposes.
>>
>> ns_log Notice "Running TCL script users/edit"
>>
>> However, it still returns "Method not allowed:
>>
>> Reference:
>> https://naviserver.sourceforge.io/n/naviserver/files/ns_register.html
>>
>>
>> Furthermore, I've tested another variation of ns_register, which works
>> just fine
>>
>> ns_register_proc PUT /REST/* ::my_proc
>>
>> What am I missing ?
>>
>>
>> Best wishes,
>> I
>> ___
>> naviserver-devel mailing list
>> naviserver-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Registering/Allowing request method HTTP PUT

[Maksym Zinchenko]

Hello, Ive tested ns_register_tcl with my installation and its working fine:
In my index.adp:
ns_register_tcl PUT /REST/ test/test.tcl

My test.tcl:
ns_log Notice "Running TCL script users/edit"

In log:

 
[0;32m[24/May/2020:16:16:47][26138.7f62510c4700][-conn:***:default:0:12-]
 [0m [0;39mNotice: Running TCL script users/edit [0m

Check your logs if naviserver reads /tcl/http-handlers-init.tcl. Your
path to scripts etc. Its something with your installation
Cmpts.: Maksym Zinchenko


On Sat, May 23, 2020 at 7:22 PM  wrote:

> Hi there,
> In attempt to register PUT in my NS instalation  ( ie. HTTP PUT method), I
> added the following line, within  /tcl/http-handlers-init.tcl library :
>
> ns_register_tcl -options {stream stricterror} -- PUT /REST/*.tcl
>
> I also try the simplest call:  ns_register_tcl PUT /REST/*.tcl
>
> Then, within /REST/ directory, there's a TCL script edit.tclm and the
> first line is a simple log call, for testing purposes.
>
> ns_log Notice "Running TCL script users/edit"
>
> However, it still returns "Method not allowed:
>
> Reference:
> https://naviserver.sourceforge.io/n/naviserver/files/ns_register.html
>
>
> Furthermore, I've tested another variation of ns_register, which works
> just fine
>
> ns_register_proc PUT /REST/* ::my_proc
>
> What am I missing ?
>
>
> Best wishes,
> I
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Multiple file input

[Maksym Zinchenko]

Hello, I have an html5 form with multiple file input. Command
ns_getformfile returns only one tmp file, may be changing it to something
like that will be better?

proc ns_getformfile {name} {
set form [ns_getform]
if {[ns_set find $form $name.tmpfile] > -1} {
return [ns_querygetall $name.tmpfile]
# return [ns_set get $form $name.tmpfile]
}
}

So command will return a list of tmp files
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Bug in bitbucket-install.tcl?

[Maksym Zinchenko]

I think few weeks ago all repositories have been migrated, because
Bitbucket ended support of Mercurial, so there is no tip branch.

On Wed, Mar 11, 2020 at 2:59 PM Christopher Chase 
wrote:

> Hi All,
>
> I've just started experimenting with NaviServer after getting into Tcl.
> I've successfully compiled and installed the server on my Mac, but I've run
> into issues when attempting to install "nsstats" via the default page.
>
> Upon clicking the link to install nsstats (
> http://localhost:8080/bitbucket-install.tcl?file=nsstats.tcl) a blank
> screen appears.
>
> Following through the logs led me to the bitbucket-install.tcl file. This
> script attempts to download things using "tip.tar.gz", e.g. "
> https://bitbucket.org/naviserver/nsstats/get/tip.tar.gz; although no such
> file exists. I've found that modifying "tip" to be "master" works as
> expected.
>
> The line in question:
> https://bitbucket.org/naviserver/naviserver/src/0c9ad59d91d48bf6f214b081bdfec94291fc2cdd/bitbucket-install.tcl#lines-5
>
> Thanks,
> Chris
> ___
> naviserver-devel mailing list
> naviserver-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Naviserver Error: Command called outside of schema context

[Maksym Zinchenko]

Hello everybody, im getting this error after updating to tDOM 0.9.2 from
http://tdom.org/index.html/info/7ecc6a8e29f80b6d
I have latest compiled NaviServer from bitbucket

:Command called outside of schema context
:while executing
:"namespace export"
:(in namespace eval "::tdom::schema" script line 1)
:invoked from within
:"namespace eval $nsp [list namespace export]"
:(procedure "_serializensp" line 48)
:invoked from within
:"_serializensp $n"
:(procedure "nstrace::statescript" line 68)
:invoked from within
:"nstrace::statescript"
:invoked from within
:"if {$use_trace_inits} {
:
:#
:# ns_eval --
:#
:# Used to eval Tcl code which is then
:# known in all threads.
:#
:
:proc ns_eval {c..."
:(file "/opt/ns/bin/init.tcl" line 420)
:(context: init server)

Once again thank you for your amazing software.
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Naviserver 4.99.19

[Maksym Zinchenko]

Hello everyone, I have a question about latest "Naviserver".

After updating to 4.99.19, I have 2 process running, one from root and
another my "Naviserver" user nsadmin for example:

 ps -aux | grep  nsd

root  1441  0.0  0.0  14272  2960 ?Ss   13:48   0:00
/opt/ns/bin/nsd -w -u nsadmin -t /opt/ns/conf/dz_nsd.tcl -b 0.0.0.0:80,
0.0.0.0:443

nsadmin   1442  0.0  0.1 565044 65456 ?Sl   13:48   0:04
/opt/ns/bin/nsd -w -u nsadmin -t /opt/ns/conf/dz_nsd.tcl -b 0.0.0.0:80,
0.0.0.0:443

root  2607  0.0  0.0   6640   980 pts/1S+   15:53   0:00 grep
--color=auto nsd



Is it normal behavior? Or I did something wrong?

One more time thank you for your great work.
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Naviserver crashing while loading chilkat library

[Maksym Zinchenko]

Hello, merry Christmas.
Im trying to load chilkat (https://www.chilkatsoft.com/tcl.asp) extention
to my naviserver, from global tcl libraries folder. Basically I have a
simple tcl file in tcllib folder (/opt/ns/tcl) which says load
/usr/share/tcltk/chilkat/chilkat.so, but its crashing my server at start up
without even saying why in logs. Ive tried to load it from adp page and its
working.
Any idea why i cant load it globally, i want use it across all virtual
servers im gonna run, and dont want to load it one by one.

Thank you
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] MinGW Compile error

[Maksym Zinchenko]

Hello, I'm trying to compile Naviserver 4.99.6 with MinGw/Msys but getting
an error while run make command:


c:/mingw/bin/../lib/gcc/mingw32/4.8.1/../../../../mingw32/bin/ld.exe:
cannot find -ltcl86
collect2.exe: error: ld returned 1 exit status
make[1]:*** [libnsthread.dll] Error 1


I'm running Windows7 32-bit, I have compiled and installed Tcl8.6.3 and
Tk8.6.3 with MinGW under C:/Tcl8.6.3

Sorry for lame question I'm not very good with computers and compiling
things under Windows.


Thank you.
--
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration  more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151iu=/4140/ostg.clktrk___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel