Dear friends of NaviServer,
The updates version of NaviServer contains now improved
support for handling large files. One can now specify to
"ns_http" the parameters "-spoolize" and "-file". When the
retrieved content is larger than spoolsize, then it is
spooled to a temp file and the file name is reported back
to the variable denoted by file (same pattern is used for
other return values as well). This addition makes it
practical to retrieve substantially large files with ns_http
(and ns_ssl) as well without bloating the memory. The
retrieved files can be conveniently be further delivered
via the writer threads. I've made the same changes to
ns_ssl and took the opportunity to made ns_ssl working
with asynchronous I/O, to refactor the code, etc.
With these changes, one can use NaviServer with
reverse-proxy functionality for http and https backends.
The sample script below supports
- delivery from upstream http and https server
(spooling of large content)
- delivery of local content (optionally compressed)
- obtain statistics via nsstats from the proxy server
The current implementation supports upstream only
HTTP/1.0 (due to the different socket management in Ns_Task)
and has no support for delivering upstream packages
directly to the client. Since upstreams connections are
quite fast and due to spooling this is many applications already
useful. On my local machine, it uses with 50 threads
just 150 MB after startup (due to the tiny blue-print).
The sample script proxy.tcl uses "nsf::proc"
(from http://next-scripting.org) to allow for nice
configuration per filter rule. If there is no nsf install,
one can replace it by a simple "proc" and do the
configuration differently...
Best regards
-gustaf neumann
PS: the script requires NaviServer 4.99.6
/usr/local/ns/bin/nsd -f -t /usr/local/proxy-https/etc/proxy-conf-https.tcl
/usr/local/proxy-https/etc/proxy-conf-https.tcl
/usr/local/proxy-https/pages/img/DotLrnLogo.gif
/usr/local/proxy-https/pages/proxy-admin/nsstats.tcl
/usr/local/proxy-https/tcl/proxy.tcl
tcl/proxy.tcl
===================================================================================
# Tiny Reverse Proxy
#
# - deliver from upstream http and https server (spooling of large content)
# - delivery of local content (optionally compressed)
# - obtain statistics via nsstats from the proxy server
#
# -Gustaf Neumann (June 2013)
#
#
# To get feedback from the proxy, we want to execute e.g. nsstats.tcl,
# which is provided in the proxy-admin directory
#
ns_register_tcl GET /proxy-admin/*.tcl
#
# Deliver images from the local files
#
ns_register_filter preauth GET /img/* proxy::local
ns_register_filter preauth GET * proxy::upstream -target https://localhost:8443
ns_register_filter preauth POST * proxy::upstream -target
https://localhost:8443 {-timeout 5:0}
namespace eval ::proxy {
#
# Local handler (deliver from disk)
#
# Serve the requested content from the server page directory. If
# the client accepts compressed content, the file is served from a
# compressed file. If the compressed file does not exist, or is
# stale, the content is compressed on the fly.
#
nsf::proc local { what } {
set fn [ns_server pagedir]/[ns_conn url]
if {[file readable $fn]} {
set mime [ns_guesstype $fn]
if {[ns_conn zipaccepted] && [ns_set iget [ns_conn headers] Range]
eq ""} {
if {![file readable $fn.gz] || [file mtime $fn] > [file mtime
$fn.gz]} {
# compress on the fly (once)
exec gzip -9 < $fn > $fn.gz
}
if {[file readable $fn.gz]} {
set fn $fn.gz
ns_set put [ns_conn outputheaders] Vary Accept-Encoding
ns_set put [ns_conn outputheaders] Content-Encoding gzip
}
}
ns_respond -file $fn
} else {
ns_returnnotfound
}
return filter_break
}
#
# Upstream handler (deliver from a different server)
#
# Serve the requested file from an upstream server, which might be
# an http or https server. NaviServer acts as a reverse proxy
# server. Files larger than a certain size (spoolsize) are
# spooled to a temp file and are delivered asynchronously via the
# writer threads if configured. Note that we can specify for every
# filter rule different parameters (e.g. different timeouts).
#
nsf::proc upstream { what -target {-timeout 10:0} {-spoolsize 10000}} {
if {[string match /proxy-admin/*tcl [ns_conn url]]} {
return filter_ok
}
#
# Assemble URL
#
set url $target[ns_conn url]
set query [ns_conn query]
if {$query ne ""} {append url ?$query}
set content [ns_conn content]
#
# Update Headers
#
set queryHeaders [ns_conn headers]
ns_set update $queryHeaders X-Forwarded-For [ns_conn peeraddr]
set http_cmd [expr {[string match https://* $target] ? "ns_ssl" :
"ns_http"}]
#
# Build query for the upstream server
#
set cmd $http_cmd
lappend cmd queue \
-method [ns_conn method] \
-headers $queryHeaders \
-timeout $timeout
if {$content ne ""} {lappend cmd -body $content}
lappend cmd $url
set handle [{*}$cmd]
#
# Build reply headers
#
set replyHeaders [ns_set create]
ns_set update $replyHeaders X-Processed Naviserver
#
# Wait for results
#
$http_cmd wait -result R -headers $replyHeaders -status S -spoolsize
$spoolsize -file F $handle
if {[info exists F]} {
ns_log notice "Spooled [file size $F] bytes to $F"
ns_respond -status $S -headers $replyHeaders -file $F
file delete $F
} else {
ns_respond -status $S -headers $replyHeaders -binary $R
}
return filter_break
}
}
===================================================================================
etc/proxy-conf-https.tcl
===================================================================================
#
# Configuration for https reverse proxy
# (upstream https)
#
ns_section ns/server/default
#ns_param minthreads 30
#ns_param maxthreads 30
ns_section "ns/server/default/modules"
ns_param nssock nssock.so
ns_param nssssl nsssl.so
ns_section "ns/server/default/module/nssock"
ns_param port 9002
ns_param writerthreads 10
ns_section "ns/server/default/module/nssssl"
ns_param port 9443
ns_param certificate /usr/local/oacs-head/openacs-4/etc/server.pem
ns_param ciphers
"ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
#
# Place for local file deliveries
#
ns_section ns/server/default/fastpath
ns_param serverdir /usr/local/proxy-https/
#
# Place for Tcl libraries to load for proxy
#
ns_section ns/server/default/tcl
ns_param library /usr/local/proxy-https/tcl
===================================================================================
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:
Build for Windows Store.
http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
