Roxen WebServer 5.0.381 is now available from http://download.roxen.com/.
Note: This is a major release, and as such there are more structural
changes than usual which have larger effects on compatibility.
Wherever feasible, compatibility is kept through the compatibility
level setting (found under the Settings tab for each site). There
are however several incompatibilities that are not overcome that
way. All compatibility and upgrade considerations are detailed in
the sections labelled COMPAT NOTE below.
WebServer-specific changes
Core improvements:
o Moved to Pike version 7.8, which among many other things enables
full use of 64-bit hardware on most OS:es. Roxen 5.0 does not run on
any earlier Pike version.
COMPAT NOTE: There are a number of incompatibilities between Pike
7.4 and 7.8, but that is of no concern if you do not have your own
custom modules. Roxen module developers should take a look at the
Pike release notes for 7.6 and 7.8. Most incompatibilities are
mitigated by running in 7.4 compatibility mode, which is enabled by
simply putting "#pike 7.4" at the top of each pike (and pmod) file.
o Moved to MySQL version 5.0. Roxen no longer works with an earlier
version. By default it does not accept 5.1 or any later version
either, since that combination is unsupported. That check can
however be disabled with a define ALLOW_UNSUPPORTED_MYSQL.
COMPAT NOTE: There are many MySQL compatibility considerations, but
unless you have your own databases or tables in the Roxen MySQL you
need not bother very much with them. Otherwise, please consult the
upgrade notes in the MySQL manual for further details, for instance
to get information on how to update your query syntax. Note that
there might be actions you need to do on your MySQL data before
upgrading.
On upgrade, Roxen automatically updates the privilege tables in
MySQL to cover all new privileges, similar to what the mysql_upgrade
script does. Read-only access in the DB interface now translates to
Select_priv and Execute_priv, and read/write access gives
Select_priv, Insert_priv, Update_priv, Delete_priv, Create_priv,
Drop_priv, References_priv, Index_priv, Alter_priv,
Create_tmp_table_priv, Lock_tables_priv, Create_view_priv,
Show_view_priv, Create_routine_priv, Alter_routine_priv, and
Execute_priv.
o It is now easier to make Roxen use another MySQL installation,
either one separately installed or one that comes with the OS
distribution. Paths to the MySQL installation directory and the most
important executables are specified in the file mysql-location.txt
in the Roxen server directory.
o Databases: Introduced charset support for database connections. The
server can now keep track of the charset a database uses for queries
and returned text, and automatically do conversions to the Unicode
representation used internally in RXML etc. This currently works for
MySQL and Postgres (using the new pgsql driver). (Later 4.5 releases
also had this feature to a limited degree.)
o Databases: Fixed security issues where arbitrary databases in the
Roxen server could be reached through the SQL tags. The SQL tags
module now has a setting Allowed databases that specifies which
databases may be accessed through the SQL tags. This setting is also
used by some other modules that access databases, e.g. <emit
source="timerange" query="...">.
COMPAT NOTE: Database access is disabled by default, meaning that
essentially all SQL tags that don't use the default database will
not work until the Allowed databases setting has been configured to
local needs. The module logs connection attempts to blocked
databases in the event log, so it is possible to see that way which
ones that need to be let through.
o Added new start up script for Roxen CMS on OS X. Handles start, stop
and restart.
o IPv6 support. Roxen can now bind IPv6 ports, initiate connections to
other IPv6 servers, etcetera. (4.5 also had this to a limited
degree, if run on Pike 7.6 instead of the included 7.4.)
o SNMP (Simple Network Management Protocol) support. This allows
monitoring of the Roxen server through SNMP v1 or v2c (but not v3).
It is enabled simply by registrering another port with protocol snmp
for a site configuration. Each site has its own SNMP prefix, so
several sites can share the same SMTP port.
o Removed the old argcache system so that the new one is used by
default (i.e. ENABLE_NEW_ARGCACHE is no longer necessary). This
primarily means that links to auto-generated images are handled in a
more robust way, especially in replicated setups.
COMPAT NOTE: There is no compatibility fallback to read argcache
entries from the old database. An upgrade instead depends on that
the image cache remains intact for long enough so that old images
can be served without requiring their argcache entries.
COMPAT NOTE: The old arguments table in the local database is not
dropped automatically. Administrators are adviced to do that to free
up space.
o The server now automatically runs a consistency check of the MySQL
tables at startup.
o Experimental support for gzip compression of http responses. Enabled
by the define HTTP_COMPRESSION.
o The protocol cache can now continue to deliver stale cache entries
while new results are generated in the background. This can mitigate
very long response times and server choking when a cache further
back is being invalidated (typically the persistent disk cache in
the CMS). (This was also available as an experimental feature in
4.5rel4 when ENABLE_SPCI was defined.)
o Handling of incoming and outgoing charsets has been improved and
primarily works better in UTF-8 mode. UTF-8 is also the default
charset for form and query variables, with fallback to ISO-8859-1 if
UTF-8 decode fails. That should lessen the need for
<roxen-automatic-charset-variable> in forms (although it still is
useful to make forms work better for users with really old
browsers).
o The size calculations for entries in the various RAM caches (seen
under Tasks/Status/Cache status in the Administration Interface) are
now much more accurate.
o Added support for software packages. This is a system to be able to
install complex module distributions without putting everything in
the Roxen module path.
o COMPAT NOTE: Roxen is no longer distributed with its own copies of
fonts that may override bitmap versions shipped to customers, since
the FreeType renderings are different enough to affect spacing etc.
o COMPAT NOTE: If a port is opened for more than one server
configuration, the fallback in case there is no site with the
"Default site" flag set in the server has changed slightly: Now the
configuration with the least specific port URL is used, while in
earlier versions the most specific port URL was chosen. This is a
fringe case that should only affect badly configured servers.
Administration interface improvements:
o New patch management system, usable both to apply patches from Roxen
Internet Software and to manage your own local changes. It is
accessible from Tasks/Maintenance/Patch management, and there is
also a command line tool bin/rxnpatch in the server directory.
o The database browser under the DBs tab has been overhauled and is
now much faster and has more features. Among other things there are
tools to optimize or repair tables, and the permission matrix is on
a separate subtab.
o The Tasks/Debug Information/Resolve Path dialogue can now send form
variables and cookies to simulate sessions and to aid in debugging
input forms. The logging has also been extended to provide more
details from various facilities.
o Added a debug tool that dumps all threads whenever any handler
thread or background job has been running for a set number of
seconds. It's configurable from Globals/Logging and can be disabled
with the define NO_SLOW_REQ_BT.
o Support IDN hostnames for port bindings.
o Added experimental support for scheduled database backups. It is
only enabled if ENABLE_DB_BACKUPS is defined, and it then adds a new
tag DBs/Backup schedules.
RXML core improvements:
o The RXML type system has been extended with array and mapping types,
and it is now fully deployed in various RXML tags through type
attributes and type context sensitivity. See the new RXML Type
System chapter in the Web Developer manual for all the details.
o Cleaned up handling of the RXML nil value (RXML.nil):
1. Do not allow an RXML variable to be set to RXML.nil. That
deletes the variable instead.
2. <emit source="sql" ...> maps SQL NULL to a null value instead
of RXML.nil.
3. <if variable="var.foo"> is false both for undefined variables
(i.e. RXML.nil) and the SQL null values.
4. A new test <if variable-exists="var.foo"> is added to test
whether a variable is defined or not, i.e. it is true for null
values but not for undefined variables.
5. <if sizeof=...> evaluates to zero both for undefined and null
variables, for compatibility.
6. <copy-scope> no longer copies undefined variables.
7. <emit source="values"> and <insert> sources variables and
scopes no longer list undefined variables.
COMPAT NOTE: Items 2, 6 and 7 are only activated if the compat level
is 5.0 or higher (items 3 and 5 don't affect compatibility since
null values didn't exist earlier).
o Added an <emit> attribute filter-exclude which does the opposite of
filter.
o Added encodings utf16, utf16be, utf16le and hex which can be used in
the encoding spec for variable entities, e.g. &var.x:hex;.
o Accessing the cookie scope no longer implicitly disables the
protocol cache. The protocol cache instead starts to vary on the
cookie value.
COMPAT NOTE: The old behavior meant that any RXML accessing cookies
implicitly disabled the protocol cache. That means the new behavior
can introduce overcaching side-effects in old code, even if the
protocol cache entries are cookie specific. Therefore the
cache-disabling behavior is kept if the compat level is 4.5 or
earlier.
o The truth value (page.last-true) is now always set to false when an
RXML error is caught.
COMPAT NOTE: This is a change that might have compat implications.
It's only enabled on compat level 5.0.
o The formatting of the online tag documentation has been improved and
shows the structure more clearly.
RXML tag improvements:
o Added a <value> tag to do type casting and to build compound values
like arrays and mappings.
o Added a <substring> tag that can pick out parts of strings in a
number of ways.
o Added a <range> tag which is similar to <substring> but operates on
arrays instead.
o <insert source="variables"> is made context sensitive so that it
returns the scope mapping as-is in an array or mapping context.
o The type handling in the <set> and <append> tags has been cleaned
up.
COMPAT NOTE: These changes have compatibility effects in some cases.
Their old behavior is retained on the 4.5 compat level.
In particular, the <append> tag behaves differently with arrays now.
If you get errors in code that appends string elements to arrays,
then you probably want to add type="text/*" to the <append> tag.
o Extended the <set expr=...> expressions to allow various set
operations on arrays and mappings. The docs for it is also adequate
now.
o SQL tags: The charset handling has been changed to fit the design
principle that strings are always unencoded Unicode internally in
RXML. It therefore not only controls the connection charset, but
perhaps more importantly it also encodes queries and decodes results
with that charset. That makes it useful also with databases without
charset support in the client library.
COMPAT NOTE: This change of the charset handling is incompatible.
The old behavior is retained on the 4.5 compat level.
o Additional RXML tags: Added two tags <dirname> and <basename> to
pick out the respective parts of a path.
o Additional RXML tags: Added an <xml-rpc-call> tag to make simple
synchronous xml-rpc calls. It is enabled by the same option that
enables synchronous <insert href>.
o Additional RXML tags: Made it possible to pass data content in
<insert href> POST requests.
o Image converter and GXML modules: A filename attribute has been
added to the <cimg> and GXML tags to append a filename to the
autogenerated URL. The GXML module has also been blessed with the
option to add image extensions to the URLs, like <cimg> already
could do.
o Graphics tags: Several of the attributes that these tags accept
refer to files that are used for purposes such as textures or
backgrounds. We now properly register timestamps to detect changes
for some attributes that weren't handled earlier. If any of these
resources are protected the RXML parser will now answer with a HTTP
Auth Required response to the browser to force authentication
instead of generating and caching an incorrect image.
o Business graphics: Introduced a color-scheme attribute to the
<diagram> tag to autogenerate colors for data.
o E-mail module: Improved error handling in the <email> tag and added
a new attribute error-variable. Also added an attribute
envelope-from to set the envelope sender address.
o Added type attribute to <redirect> to make it possible to do e.g.
permanent redirects.
o Added attribute http-time to <date>.
o Added new mode safe-utf8 to <recode from>. When provided, silently
ignore any illegal UTF-8 sequences.
o Added a showvar attribute to the <debug> tag to be able to print out
the value of a variable without conversion in an unambiguous format
(useful to figure out charset conversion issues, for instance). The
<debug> tag also works in any type context.
o COMPAT NOTE: A bug has been fixed in the <contents> tag, used within
<define>, when both the value-of and result-set attributes were used
at the same time. The fix might have compatibility implications in
code that tried to work around the bug, but the old behavior was too
quirky to keep compatibility with on the 4.5 compat level.
o COMPAT NOTE: A type problem that could cause extra entity quoting in
<strlen> has been fixed and hence wrong length reports. Old code
might possibly depend on the broken behavior, so it is kept on 4.5
compat level.
Module improvements:
o The Yahoo! User Interface Library: This is a new module can be used
to make the YUI available to web applications on the site. It
supports several YUI versions simultaneously and it is easy to add
another by downloading the tar distribution and letting the module
unpack it.
o Path info support: Added path limit option.
o Redirect module: Added permanent keyword to send permanent (301)
redirects instead of temporary (302).
o RXML parser module: Added an option to censor potentially sensitive
user authentication data from requests before RXML evaluation
starts.
o CGI interface now exports REQUEST_URI, REDIRECT_URL and
REDIRECT_STATUS.
o CGI scripting support: Added a setting to be able to run CGI scripts
in a chrooted environment.
o CGI scripting support: Added a setting to disable the Roxen extended
environment variables.
o Javascript support: Don't output empty <script> tags. Marginally
smarter quoting.
Minor improvements:
o Administration interface: Updated the default logging format to the
Combined Log Format which extends Common Log Format with two fields.
The proposed format with extended usage info is changed accordingly.
It also uses $ip-number instead of $host by default, to avoid the
DNS overhead.
o Administration interface: The Add modules page now shows the site
name, to avoid adding modules to the wrong site by mistake.
o Administration interface: A bit nicer sort order for threads in
Tasks/Debug Information/Thread backtrace, e.g. the backend thread is
always listed first.
o Administration interface: Improved display of Unix sockets in
Tasks/Debug Information/Open files.
o Administration interface: The binary distribution identifier is now
shown at the bottom of the pages.
o Core: Optimizations in the protocol cache, and in the vary callback
system in particular.
o Core: The image cache now allows protocol caching of authenticated
images, which is made possible by the new arg cache implementation.
o Core: Speed up accepting of connections.
o Core: Improved the p-code codec to handle references to arbitrary
Pike modules.
o Core: The sample start script in the tools directory now has a
"status" command to query whether the Roxen instance is running or
not. It can also be configured to handle several instances by
keeping their configuration directories in a special "configuration
collection" directory.
o Core: Added an experimental mode where the RAM cache retention
policy is based on the time to create the entries. This is enabled
by the define TIME_BASED_CACHE.
o Core: Log a warning if a background job takes more than one minute.
o Core: Roxen modules are now always loaded in the same order.
o Core: There is now a bat file bin\mysql_client_nt.bat to make it
simpler to start a MySQL command line client against Roxen's MySQL
process on Windows.
o RXML tags: The <nocache> tag now properly disables protocol and
client caching, just like the <cache> tag does by default. It has
also gotten the attributes enable-client-cache and
enable-protocol-cache that works like in the <cache> tag, to control
this behavior.
o RXML tags: <remove-cookie> no longer requires the cookie to exist.
Enjoy!
--
Henrik Grubbström [email protected]
Roxen Internet Software AB
