>Number: 3541
>Category: mod_negotiation
>Synopsis: mod_negotiation major cleanup
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: apache
>State: open
>Class: change-request
>Submitter-Id: apache
>Arrival-Date: Wed Dec 16 09:20:00 PST 1998
>Last-Modified:
>Originator: [EMAIL PROTECTED]
>Organization:
apache
>Release: 1.3.3
>Environment:
Cleanup patch developed using Linux/GCC
>Description:
The following patch (against Apache 1.3.3) represents a major cleanup
of mod_negotiation. The main goals of the patch are:
- making the module compliant with the HTTP/1.1 proposed standard
(rfc2068) and supporting everything in the upcoming HTTP/1.1
revision (draft-ietf-http-v11-spec-rev-06.txt).
- improved cache correctness (i.e. eliminating cases where 1.1
caches could end up returning the wrong variant to a client)
- providing support for transparent content negotiation (rfc2296
and rfc2296) and, indirectly, client-driven negotiation, by
updating the existing experimental implementation of transparent
content negotiation
- cleanups of things like outdated comments, misleading structure
member names, duplicate code, documentation of problems which
have no clear or easy fix, to provide solid basis for further
development
Non-goals: the patch
- does not change the Apache server-side negotiation algorithm
- does not change the negotiation on content encoding
- does not add new module directives
The patch incorporates a fix for a Vary bug posted on the new-httpd
list in <[EMAIL PROTECTED]>
It should also address PR1987.
Affected files:
src/include/http_protocol.h
src/include/httpd.h
src/main/http_protocol.c
src/main/http_request.c
src/modules/standard/mod_negotiation.c
Detailed change log:
- cleanups to mod_negotiation comments and code structure
- made compliant with HTTP/1.1 proposed standard (rfc2068) and added
support for everything in the upcoming HTTP/1.1
revision (draft-ietf-http-v11-spec-rev-06.txt).
- language tag matching also handles tags which more than 2
levels like x-y-z
- empty Accept, Accept-Language, Accept-Charset headers are
processed correctly, previously an empty header would make all
values acceptable in stead of un-acceptable.
- fixed bug in Vary according to
<[EMAIL PROTECTED]>
- changed Vary construction to take 'mxb=' support into account
- allowed for q values in Accept-Encoding
- added support for transparent content negotiation (rfc2295 and
rfc2296) (though we do not implement all features in these drafts,
e.g. no feature negotiation).
- lots of changes/bug fixes/cleanups in the existing experimental
implementation
- implemented 'structured entity tags' for better cache correctness
(structured entity tags ensure that caches which can deal with Vary
will (eventually) be updated if the set of variants on the server
is changed)
- this involved adding a vlist_validator element to request_rec
- this involved adding the ap_make_etag() function to the global API
- modified guessing of charsets used by Apache negotiation algorithm
to guess 'no charset' if the variant is not a text/* type
- added code so sort multiviews variants into a canonical order so that
negotiation results are consistent across backup/restores and
mirrors
- removed possibility of a type map file resolving to another type map
file as its best variant
- fixed typo at the 506 response in http_protocol.c status_lines[]
declaration
- documented various problems in mod_negotiation which were found but
not solved (by adding XXX: comments).
[In case the bug database corrupts actual patch file, I have made a
copy available at http://home.cern.ch/~kholtman/apache/negotiate_patch
]
Note: I can't guarantee being able to respond to e-mail during the CERN xmas
break from 19 dec 1998 to 3 jan 1999.
Koen.
>How-To-Repeat:
>Fix:
diff -r -u ../apache_1.3.3/src/include/http_protocol.h
src/include/http_protocol.h
--- ../apache_1.3.3/src/include/http_protocol.h Sun Aug 9 16:33:10 1998
+++ src/include/http_protocol.h Sun Dec 6 16:25:10 1998
@@ -115,6 +115,7 @@
API_EXPORT(int) ap_set_content_length(request_rec *r, long length);
API_EXPORT(int) ap_set_keepalive(request_rec *r);
API_EXPORT(time_t) ap_rationalize_mtime(request_rec *r, time_t mtime);
+API_EXPORT(char *) ap_make_etag(request_rec *r, int force_weak);
API_EXPORT(void) ap_set_etag(request_rec *r);
API_EXPORT(void) ap_set_last_modified(request_rec *r);
API_EXPORT(int) ap_meets_conditions(request_rec *r);
diff -r -u ../apache_1.3.3/src/include/httpd.h src/include/httpd.h
--- ../apache_1.3.3/src/include/httpd.h Wed Oct 7 10:19:06 1998
+++ src/include/httpd.h Sat Nov 21 22:50:38 1998
@@ -759,6 +759,7 @@
* binary compatibility for some other reason.
*/
unsigned expecting_100; /* is client waiting for a 100 response? */
+ char *vlist_validator; /* variant list validator (if negotiated) */
};
diff -r -u ../apache_1.3.3/src/main/http_protocol.c src/main/http_protocol.c
--- ../apache_1.3.3/src/main/http_protocol.c Tue Oct 6 20:06:09 1998
+++ src/main/http_protocol.c Tue Dec 15 22:17:53 1998
@@ -469,7 +469,7 @@
* could be modified again in as short an interval. We rationalize the
* modification time we're given to keep it from being in the future.
*/
-API_EXPORT(void) ap_set_etag(request_rec *r)
+API_EXPORT(char *) ap_make_etag(request_rec *r, int force_weak)
{
char *etag;
char *weak;
@@ -487,7 +487,7 @@
* would be incorrect.
*/
- weak = (r->request_time - r->mtime > 1) ? "" : "W/";
+ weak = ((r->request_time - r->mtime > 1)&&!force_weak) ? "" : "W/";
if (r->finfo.st_mode != 0) {
etag = ap_psprintf(r->pool,
@@ -501,6 +501,45 @@
(unsigned long) r->mtime);
}
+ return etag;
+}
+
+
+API_EXPORT(void) ap_set_etag(request_rec *r)
+{
+ char *etag;
+ char *variant_etag,*vlv;
+ int vlv_weak;
+
+ if (!r->vlist_validator) {
+ etag = ap_make_etag(r,0);
+ }
+ else {
+ /* If we have a variant list validator (vlv) due to the
+ response being negotiated, then we create a structured
+ entity tag which merges the variant etag with the variant
+ list validator (vlv). This merging makes revalidation
+ somewhat safer, ensures that caches which can deal with
+ Vary will (eventually) be updated if the set of variants is
+ chanced, and is also a protocol requirement for transparent
+ content negotiation. */
+
+ /* if the variant list validator is weak, we make the whole
+ structured etag weak. If we would not, then clients could
+ have problems merging range responses if we have different
+ variants with the same non-globally-unique strong etag. */
+
+ vlv=r->vlist_validator;
+ vlv_weak = vlv[0]=='W';
+
+ variant_etag = ap_make_etag(r,vlv_weak);
+
+ /* merge variant_etag and vlv into a structured etag */
+ variant_etag[strlen(variant_etag)-1]=0;
+ if(vlv_weak) vlv+=3; else vlv++;
+ etag = ap_pstrcat(r->pool, variant_etag , ";" , vlv , NULL);
+ }
+
ap_table_setn(r->headers_out, "ETag", etag);
}
@@ -1110,7 +1149,7 @@
"503 Service Temporarily Unavailable",
"504 Gateway Time-out",
"505 HTTP Version Not Supported",
- "506 Variant Also Negotiates"
+ "506 Variant Also Negotiates",
"507 unused",
"508 unused",
"509 unused",
@@ -2307,9 +2346,11 @@
ap_bputs("response from an upstream server.<P>\015\012", fd);
break;
case VARIANT_ALSO_VARIES:
- ap_bvputs(fd, "A variant for the requested entity ",
- ap_escape_html(r->pool, r->uri), " is itself a ",
- "transparently negotiable resource.<P>\n", NULL);
+ ap_bvputs(fd, "A variant for the requested entity ",
+ ap_escape_html(r->pool, r->uri), " is itself a ",
+ "negotiable resource.<P>\n",
+ "This is a configuration problem in the requested ",
+ "entity.\n", NULL);
break;
case HTTP_REQUEST_TIME_OUT:
ap_bputs("I'm tired of waiting for your request.\n", fd);
diff -r -u ../apache_1.3.3/src/main/http_request.c src/main/http_request.c
--- ../apache_1.3.3/src/main/http_request.c Tue Oct 6 20:06:09 1998
+++ src/main/http_request.c Sat Nov 21 22:23:55 1998
@@ -1250,6 +1250,7 @@
new->no_cache = r->no_cache;
new->no_local_copy = r->no_local_copy;
new->read_length = r->read_length; /* We can only read it once */
+ new->vlist_validator = r->vlist_validator;
ap_table_setn(new->subprocess_env, "REDIRECT_STATUS",
ap_psprintf(r->pool, "%d", r->status));
diff -r -u ../apache_1.3.3/src/modules/standard/mod_negotiation.c
src/modules/standard/mod_negotiation.c
--- ../apache_1.3.3/src/modules/standard/mod_negotiation.c Thu Aug 6
19:31:00 1998
+++ src/modules/standard/mod_negotiation.c Tue Dec 15 22:16:53 1998
@@ -65,19 +65,11 @@
#include "httpd.h"
#include "http_config.h"
#include "http_request.h"
+#include "http_protocol.h"
#include "http_core.h"
#include "http_log.h"
#include "util_script.h"
-/* define TCN_02 to allow for Holtman I-D transparent negotiation.
- * This file currently implements the draft-02, except for
- * anything to do with features and cache-control (max-age etc)
- *
- * Since the draft is just that, and we don't yet implement
- * everything, regard the transparent negotiation stuff as experimental.
- */
-/*#define TCN_02 */
-
/* Commands --- configuring document caching on a per (virtual?)
* server basis...
*/
@@ -157,7 +149,7 @@
*/
typedef struct accept_rec {
- char *type_name; /* MUST be lowercase */
+ char *name; /* MUST be lowercase */
float quality;
float max_bytes;
float level;
@@ -174,11 +166,19 @@
* if the client actually accepts this media type at that
* level (and *not* if it got in on a wildcard). See level_cmp
* below.
+ * mime_stars -- initialized to zero. Set to the number of stars
+ * present in the best matching Accept header element.
+ * 1 for star/star, 2 for type/star and 3 for
+ * type/subtype.
+ *
+ * definite -- initialized to 1. Set to 0 if there is a match which
+ * makes the variant non-definite according to the rules
+ * in rfc2296.
*/
typedef struct var_rec {
request_rec *sub_req; /* May be NULL (is, for map files) */
- char *type_name; /* MUST be lowercase */
+ char *mime_type; /* MUST be lowercase */
char *file_name;
const char *content_encoding;
array_header *content_languages; /* list of languages for this variant
*/
@@ -187,18 +187,18 @@
/* The next five items give the quality values for the dimensions
* of negotiation for this variant. They are obtained from the
- * appropriate header lines, except for accept_type_quality, which
+ * appropriate header lines, except for source_quality, which
* is obtained from the variant itself (the 'qs' parameter value
- * from the variant's mime-type). Apart from type_quality,
+ * from the variant's mime-type). Apart from source_quality,
* these values are set when we find the quality for each variant
- * (see best_match()). type_quality is set from the 'qs' parameter
+ * (see best_match()). source_quality is set from the 'qs' parameter
* of the variant description or mime type: see set_mime_fields().
*/
float lang_quality; /* quality of this variant's language */
- int encoding_quality; /* ditto encoding (1 or 0 only) */
+ float encoding_quality; /* ditto encoding */
float charset_quality; /* ditto charset */
- float accept_type_quality; /* ditto media type */
- float type_quality; /* quality of source for this type */
+ float mime_type_quality; /* ditto media type */
+ float source_quality; /* source quality for this variant */
/* Now some special values */
float level; /* Auxiliary to content-type... */
@@ -226,19 +226,24 @@
int accept_q; /* 1 if an Accept item has a q= param */
float default_lang_quality; /* fiddle lang q for variants with no lang */
-
+ /* the array pointers below are NULL if the corresponding accept
+ headers are not present */
array_header *accepts; /* accept_recs */
- int have_accept_header; /* 1 if Accept-Header present */
array_header *accept_encodings; /* accept_recs */
array_header *accept_charsets; /* accept_recs */
array_header *accept_langs; /* accept_recs */
+
array_header *avail_vars; /* available variants */
int count_multiviews_variants; /* number of variants found on disk */
- int ua_can_negotiate; /* 1 if ua can do transparent negotiate */
- int use_transparent_neg; /* 1 if we are using transparent neg */
- int short_accept_headers; /* 1 if ua does trans neg & sent short accpt */
+ int is_transparent; /* 1 if this resource is trans. negtotiable */
+
+ int dont_fiddle_headers; /* 1 if we may not fiddle with accept hdrs */
+ int ua_supports_trans; /* 1 if ua supports trans negotiation */
+ int send_alternates; /* 1 if we want to send an Alternates header */
+ int may_choose; /* 1 if we may choose a variant for the client */
+ int use_rvsa; /* 1 if we must use RVSA/1.0 negotiation algo */
} negotiation_state;
/* A few functions to manipulate var_recs.
@@ -248,7 +253,7 @@
static void clean_var_rec(var_rec *mime_info)
{
mime_info->sub_req = NULL;
- mime_info->type_name = "";
+ mime_info->mime_type = "";
mime_info->file_name = "";
mime_info->content_encoding = NULL;
mime_info->content_languages = NULL;
@@ -264,10 +269,11 @@
mime_info->definite = 1;
mime_info->charset_quality = 1.0f;
- mime_info->type_quality = 0.0f;
- mime_info->encoding_quality = 1;
+ mime_info->encoding_quality = 1.0f;
mime_info->lang_quality = 1.0f;
- mime_info->accept_type_quality = 1.0f;
+ mime_info->mime_type_quality = 1.0f;
+ mime_info->source_quality = 0.0f;
+
}
/* Initializing the relevant fields of a variant record from the
@@ -276,16 +282,33 @@
static void set_mime_fields(var_rec *var, accept_rec *mime_info)
{
- var->type_name = mime_info->type_name;
- var->type_quality = mime_info->quality;
+ var->mime_type = mime_info->name;
+ var->source_quality = mime_info->quality;
var->level = mime_info->level;
var->content_charset = mime_info->charset;
- var->is_pseudo_html = (!strcmp(var->type_name, "text/html")
- || !strcmp(var->type_name, INCLUDES_MAGIC_TYPE)
- || !strcmp(var->type_name, INCLUDES_MAGIC_TYPE3));
+ var->is_pseudo_html = (!strcmp(var->mime_type, "text/html")
+ || !strcmp(var->mime_type, INCLUDES_MAGIC_TYPE)
+ || !strcmp(var->mime_type, INCLUDES_MAGIC_TYPE3));
}
+/* Create a variant list validator in r using info from vlistr. */
+
+static void set_vlist_validator(request_rec *r,request_rec *vlistr)
+{
+ /* Calculating the variant list validator is similar to
+ calculating an etag for the source of the variant list
+ information, so wer use ap_set_etag(). Note that this
+ validator can be 'weak' in extreme case. */
+
+ ap_update_mtime (vlistr, vlistr->finfo.st_mtime);
+ r->vlist_validator = ap_make_etag(vlistr,0);
+
+ /* ap_set_etag will later take r->vlist_validator into account
+ when creating the etag header */
+}
+
+
/*****************************************************************
*
* Parsing (lists of) media types and their parameters, as seen in
@@ -315,8 +338,8 @@
* in the CERN server code? I must be missing something).
*/
- result->type_name = ap_get_token(p, &accept_line, 0);
- ap_str_tolower(result->type_name); /* You want case-insensitive,
+ result->name = ap_get_token(p, &accept_line, 0);
+ ap_str_tolower(result->name); /* You want case-insensitive,
* you'll *get* case-insensitive.
*/
@@ -324,13 +347,13 @@
* *explicitly* says something else.
*/
- if (!strcmp(result->type_name, "text/html") && (result->level == 0.0)) {
+ if (!strcmp(result->name, "text/html") && (result->level == 0.0)) {
result->level = 2.0f;
}
- else if (!strcmp(result->type_name, INCLUDES_MAGIC_TYPE)) {
+ else if (!strcmp(result->name, INCLUDES_MAGIC_TYPE)) {
result->level = 2.0f;
}
- else if (!strcmp(result->type_name, INCLUDES_MAGIC_TYPE3)) {
+ else if (!strcmp(result->name, INCLUDES_MAGIC_TYPE3)) {
result->level = 3.0f;
}
@@ -405,18 +428,19 @@
* basic structure of a list of items of the format
* name; q=N; charset=TEXT
*
- * where q is only valid in Accept, Accept-Charset and Accept-Languages,
- * and charset is only valid in Accept.
+ * where charset is only valid in Accept.
*/
static array_header *do_header_line(pool *p, const char *accept_line)
{
- array_header *accept_recs = ap_make_array(p, 40, sizeof(accept_rec));
+ array_header *accept_recs;
if (!accept_line) {
- return accept_recs;
+ return NULL;
}
+ accept_recs = ap_make_array(p, 40, sizeof(accept_rec));
+
while (*accept_line) {
accept_rec *new = (accept_rec *) ap_push_array(accept_recs);
accept_line = get_entry(p, new, accept_line);
@@ -469,88 +493,165 @@
new->accepts = do_header_line(r->pool, ap_table_get(hdrs, "Accept"));
- hdr = ap_table_get(hdrs, "Accept-encoding");
- if (hdr) {
- new->have_accept_header = 1;
+ /* calculate new->accept_q value */
+ if (new->accepts) {
+ elts = (accept_rec *) new->accepts->elts;
+
+ for (i = 0; i < new->accepts->nelts; ++i) {
+ if (elts[i].quality < 1.0) {
+ new->accept_q = 1;
+ }
+ }
}
- new->accept_encodings = do_header_line(r->pool, hdr);
+ new->accept_encodings = do_header_line(r->pool, ap_table_get(hdrs,
+ "Accept-encoding"));
new->accept_langs = do_header_line(r->pool,
ap_table_get(hdrs, "Accept-language"));
new->accept_charsets = do_header_line(r->pool,
ap_table_get(hdrs,
"Accept-charset"));
new->avail_vars = ap_make_array(r->pool, 40, sizeof(var_rec));
-#ifdef TCN_02
- if (ap_table_get(r->headers_in, "Negotiate")) {
- /* Negotiate: header tells us UA does transparent negotiation
- * We have to decide whether we want to ... for now, yes,
- * we do */
-
- new->ua_can_negotiate = 1;
- if (r->method_number == M_GET) {
- new->use_transparent_neg = 1; /* should be configurable */
- }
-
- /* Check for 'Short Accept', ie either no Accept: header,
- * or just "Accept: * / *" */
- if (new->accepts->nelts == 0 ||
- (new->accepts->nelts == 1 &&
- (!strcmp(((accept_rec *) new->accepts->elts)[0].type_name,
- "*/*")))) {
- /* Using short accept header */
- new->short_accept_headers = 1;
- }
- }
-#endif
+ return new;
+}
- if (!new->use_transparent_neg) {
- /* Now we check for q-values. If they're all 1.0, we assume the
- * client is "broken", and we are allowed to fiddle with the
- * values later. Otherwise, we leave them alone.
- */
- elts = (accept_rec *) new->accepts->elts;
+static void parse_negotiate_header(request_rec *r, negotiation_state *neg) {
- for (i = 0; i < new->accepts->nelts; ++i) {
- if (elts[i].quality < 1.0) {
- new->accept_q = 1;
- }
- }
- }
- else {
- new->accept_q = 1;
+ const char *negotiate = ap_table_get(r->headers_in, "Negotiate");
+
+ if (negotiate) {
+ /* Negotiate: header tells us UA does transparent negotiation */
+
+ /* sending Alternates on non-transparent resources is allowed,
+ * and may even be useful, but we don't for now, also
+ * because it could clash with an Alternates header set by
+ * a sub- or super- request on a transparent resource.
+ */
+
+ while(*negotiate) {
+ char *tok = ap_get_token(neg->pool, &negotiate, 1);
+ char *cp;
+
+ for (cp = tok; (*cp && !ap_isspace(*cp) && *cp != '='); ++cp) {
+ *cp = ap_tolower(*cp);
+ }
+ *cp=0;
+
+ if (strcmp(tok,"trans")==0||
+ strcmp(tok,"vlist")==0||
+ strcmp(tok,"guess-small")==0||
+ ap_isdigit(tok[0])||
+ strcmp(tok,"*")==0) {
+
+ /* The user agent supports transparent negotiaton */
+ neg->ua_supports_trans = 1;
+
+ /* Send-alternates could be configurable, but note
+ that it must be 1 if we have 'vlist' in the
+ negotiate header. */
+ neg->send_alternates = 1;
+
+ if (strcmp(tok,"1.0")==0) {
+ /* we may use the RVSA/1.0 algorithm, configure for it */
+ neg->may_choose = 1;
+ neg->use_rvsa = 1;
+ neg->dont_fiddle_headers = 1;
+ }
+
+ if (strcmp(tok,"*")==0) {
+ /* we may use any variant selection algorithm, configure
+ to use the Apache algorithm */
+ neg->may_choose = 1;
+
+ /* We disable header fiddles on the assumption that a
+ client sending Negotiate knows how to send correct
+ headers which don't need fiddling */
+ neg->dont_fiddle_headers = 1;
+ }
+
+ }
+
+ if (*negotiate) negotiate++; /* skip over , */
+ }
}
- return new;
+ if (!neg->ua_supports_trans) {
+ /* User agent does not support transparent negotiation,
+ configure to do server-driven negotiation with the Apache
+ algorithm */
+ neg->may_choose = 1;
+
+ /* To save network bandwidth, we do not configure to send an
+ Alternates header to the user agent in this case. User
+ agents which want an Alternates header for agent-driven
+ negotiation will have to request it by sending an
+ appropriate Negotiate header. */
+ }
+
+#if NEG_DEBUG
+ fprintf(stderr, "dont_fiddle_headers=%d use_rvsa=%d ua_supports_trans=%d "
+ "send_alternates=%d, may_choose=%d\n",
+ neg->dont_fiddle_headers, neg->use_rvsa,
+ neg->ua_supports_trans, neg->send_alternates, neg->may_choose);
+#endif
+
}
/* Sometimes clients will give us no Accept info at all; this routine sets
* up the standard default for that case, and also arranges for us to be
* willing to run a CGI script if we find one. (In fact, we set up to
* dramatically prefer CGI scripts in cases where that's appropriate,
- * e.g., POST).
+ * e.g., POST).
*/
-static void maybe_add_default_encodings(negotiation_state *neg, int
prefer_scripts)
+/* XXX: This function should really be called
+ maybe_add_default_accepts, it has nothing to do with encodings. */
+
+static void maybe_add_default_encodings(negotiation_state *neg,
+ int prefer_scripts)
{
- accept_rec *new_accept = (accept_rec *) ap_push_array(neg->accepts);
+ accept_rec *new_accept;
- new_accept->type_name = CGI_MAGIC_TYPE;
- new_accept->quality = prefer_scripts ? 1e-20f : 1e20f;
- new_accept->level = 0.0f;
- new_accept->max_bytes = 0.0f;
+ if (!neg->accepts) {
+ neg->accepts = ap_make_array(neg->pool, 40, sizeof(accept_rec));
- if (neg->accepts->nelts > 1) {
- return;
- }
+ new_accept = (accept_rec *) ap_push_array(neg->accepts);
+
+ new_accept->name = "*/*";
+ new_accept->quality = 1.0f;
+ new_accept->level = 0.0f;
+ new_accept->max_bytes = 0.0f;
+ }
new_accept = (accept_rec *) ap_push_array(neg->accepts);
- new_accept->type_name = "*/*";
- new_accept->quality = 1.0f;
+ new_accept->name = CGI_MAGIC_TYPE;
+ if (neg->use_rvsa) {
+ new_accept->quality = 0;
+ }
+ else {
+ /* XXX: Eek! This is the wrong way around, we set the script
+ quality to 1e-20 if we _prefer_ scripts. There is also a
+ bug in handle_multi() however: it always mis-calculates the
+ prefer_scripts argument as 1, so that we end up with
+ new_accept->quality = 1e-20 in all cases, which causes the
+ Apache negotiation algorithm to ignore any scripts if
+ present. handle_map_file() always supplies
+ prefer_scripts==0, leading to an incorrect script quality
+ of 1e20f in all cases, but this would not hurt unless one
+ uses CGI_MAGIC_TYPE in a map file.
+
+ I guess nobody is attempting to use this `prefer scripts if
+ the request is scripty' functionality, else they would have
+ found out that never works and filed a bug report. Maybe
+ we should just ax this feature? -kh */
+
+ new_accept->quality = prefer_scripts ? 1e-20f : 1e20f;
+ }
new_accept->level = 0.0f;
new_accept->max_bytes = 0.0f;
+
}
/*****************************************************************
@@ -707,6 +808,7 @@
char buffer[MAX_STRING_LEN];
enum header_state hstate;
struct var_rec mime_info;
+ int has_content;
/* We are not using multiviews */
neg->count_multiviews_variants = 0;
@@ -719,6 +821,7 @@
}
clean_var_rec(&mime_info);
+ has_content = 0;
do {
hstate = get_header_line(buffer, MAX_STRING_LEN, map);
@@ -732,7 +835,7 @@
}
strip_paren_comments(body1);
- body=body1;
+ body = body1;
if (!strncmp(buffer, "uri:", 4)) {
mime_info.file_name = ap_get_token(neg->pool, &body, 0);
@@ -742,39 +845,72 @@
get_entry(neg->pool, &accept_info, body);
set_mime_fields(&mime_info, &accept_info);
+ has_content = 1;
}
else if (!strncmp(buffer, "content-length:", 15)) {
mime_info.bytes = atof(body);
+ has_content = 1;
}
else if (!strncmp(buffer, "content-language:", 17)) {
mime_info.content_languages = do_languages_line(neg->pool,
&body);
+ has_content = 1;
}
else if (!strncmp(buffer, "content-encoding:", 17)) {
mime_info.content_encoding = ap_get_token(neg->pool, &body, 0);
+ has_content = 1;
}
else if (!strncmp(buffer, "description:", 12)) {
- mime_info.description = ap_get_token(neg->pool, &body, 0);
+ /* XXX: The possibility to set a description is
+ currently not documented. */
+ char *desc = ap_pstrdup(neg->pool, body);
+ char *cp;
+
+ for (cp = desc; *cp; ++cp) {
+ if (*cp=='\n') *cp=' ';
+ }
+ if (cp>desc) *(cp-1)=0;
+ mime_info.description = desc;
}
}
else {
- if (mime_info.type_quality > 0 && *mime_info.file_name) {
+ if (*mime_info.file_name && has_content) {
void *new_var = ap_push_array(neg->avail_vars);
memcpy(new_var, (void *) &mime_info, sizeof(var_rec));
}
clean_var_rec(&mime_info);
+ has_content = 0;
}
} while (hstate != header_eof);
ap_pfclose(neg->pool, map);
+
+ set_vlist_validator(r, rr);
+
return OK;
}
+
+/* Sort function used by read_types_multi. */
+static int variantsortf(var_rec *a, var_rec *b) {
+
+ /* First key is the source quality, sort in descending order. */
+ /* XXX: note that we currently implement no method of setting the
+ source quality for multiviews variants, so we are always comparing
+ 1.0 to 1.0 for now */
+ if (a->source_quality<b->source_quality) return 1;
+ if (a->source_quality>b->source_quality) return -1;
+
+ /* Second key is the variant name */
+ return strcmp(a->file_name, b->file_name);
+}
+
/*****************************************************************
*
- * Same, except we use a filtered directory listing as the map...
+ * Same as read_type_map, except we use a filtered directory listing
+ * as the map...
*/
static int read_types_multi(negotiation_state *neg)
@@ -883,13 +1019,27 @@
}
ap_pclosedir(neg->pool, dirp);
+
+ set_vlist_validator(r, r);
+
+ /* Sort the variants into a canonical order. The negotiation
+ result sometimes depends on the order of the variants. By
+ sorting the variants into a canonical order, rather than using
+ the order in which readdir() happens to return them, we ensure
+ that the negotiation result will be consistent over filesystem
+ backup/restores and over all mirror sites.
+ */
+
+ qsort((void *) neg->avail_vars->elts, neg->avail_vars->nelts,
+ sizeof(var_rec), (int (*)(const void *, const void *)) variantsortf);
+
return OK;
}
/*****************************************************************
* And now for the code you've been waiting for... actually
- * finding a match to the client's requirements.
+ * finding a match to the client's requirements.
*/
/* Matching MIME types ... the star/star and foo/star commenting conventions
@@ -906,8 +1056,8 @@
static int mime_match(accept_rec *accept_r, var_rec *avail)
{
- char *accept_type = accept_r->type_name;
- char *avail_type = avail->type_name;
+ char *accept_type = accept_r->name;
+ char *avail_type = avail->mime_type;
int len = strlen(accept_type);
if (accept_type[0] == '*') { /* Anything matches star/star */
@@ -970,9 +1120,13 @@
return 0;
}
- if (!var1->is_pseudo_html && strcmp(var1->type_name, var2->type_name)) {
+ if (!var1->is_pseudo_html && strcmp(var1->mime_type, var2->mime_type)) {
return 0;
}
+ /* The result of the above if statements is that, if we get to
+ * here, both variants have the same mime_type or both are
+ * pseudo-html.
+ */
/* Take highest level that matched, if either did match. */
@@ -1008,12 +1162,12 @@
* directive order.
*
* When we do the variant checking for best variant, we use language
- * quality first, and if a tie, language_index next (this only
- * applies when _not_ using the network algorithm). If using
- * network algorithm, lang_index is never used.
+ * quality first, and if a tie, language_index next (this only applies
+ * when _not_ using the RVSA/1.0 algorithm). If using the RVSA/1.0
+ * algorithm, lang_index is never used.
*
* set_language_quality() calls find_lang_index() and find_default_index()
- * to set lang_index.
+ * to set lang_index.
*/
static int find_lang_index(array_header *accept_langs, char *lang)
@@ -1021,14 +1175,14 @@
accept_rec *accs;
int i;
- if (!lang) {
+ if (!lang || !accept_langs) {
return -1;
}
accs = (accept_rec *) accept_langs->elts;
for (i = 0; i < accept_langs->nelts; ++i) {
- if (!strncmp(lang, accs[i].type_name, strlen(accs[i].type_name))) {
+ if (!strncmp(lang, accs[i].name, strlen(accs[i].name))) {
return i;
}
}
@@ -1076,8 +1230,8 @@
* are acceptable. The default q value set here is assigned to variants
* with no language type in set_language_quality().
*
- * Note that if using the transparent negotiation network algorythm,
- * we don't use this fiddle.
+ * Note that if using the RFVA/1.0 algorithm, we don't use this
+ * fiddle.
*/
static void set_default_lang_quality(negotiation_state *neg)
@@ -1085,7 +1239,7 @@
var_rec *avail_recs = (var_rec *) neg->avail_vars->elts;
int j;
- if (!neg->use_transparent_neg) {
+ if (!neg->dont_fiddle_headers) {
for (j = 0; j < neg->avail_vars->nelts; ++j) {
var_rec *variant = &avail_recs[j];
if (variant->content_languages &&
@@ -1133,17 +1287,19 @@
static void set_language_quality(negotiation_state *neg, var_rec *variant)
{
int i;
- int naccept = neg->accept_langs->nelts;
+ int naccept;
int idx;
neg_dir_config *conf = NULL;
char *firstlang;
- if (naccept == 0) {
+ if (!neg->accept_langs) {
conf = (neg_dir_config *) ap_get_module_config(neg->r->per_dir_config,
&negotiation_module);
+ } else {
+ naccept = neg->accept_langs->nelts;
}
- if (naccept == 0 && (!variant->content_languages ||
+ if (!neg->accept_langs && (!variant->content_languages ||
!variant->content_languages->nelts)) {
return; /* no accept-language and no variant lang */
}
@@ -1151,116 +1307,149 @@
if (!variant->content_languages || !variant->content_languages->nelts) {
/* This variant has no content-language, so use the default
* quality factor for variants with no content-language
- * (previously set by set_default_lang_quality()). */
- variant->lang_quality = neg->default_lang_quality;
-
- if (naccept == 0) {
- return; /* no accept-language items */
- }
-
- }
- else if (naccept) {
- /* Variant has one (or more) languages, and we have one (or more)
- * language ranges on the Accept-Language header. Look for
- * the best match. We do this by going through each language
- * on the variant description looking for a match on the
- * Accept-Language header. The best match is the longest matching
- * language on the header. The final result is the best q value
- * from all the languages on the variant description.
- */
- int j;
- float fiddle_q = 0.0f;
- accept_rec *accs = (accept_rec *) neg->accept_langs->elts;
- accept_rec *best = NULL, *star = NULL;
- char *p;
-
- for (j = 0; j < variant->content_languages->nelts; ++j) {
- char *lang; /* language from variant description */
- accept_rec *bestthistag = NULL;
- int prefixlen = 0;
- int longest_lang_range_len = 0;
- int len;
-
- /* lang is the variant's language-tag, which is the one
- * we are allowed to use the prefix of in HTTP/1.1
- */
- lang = ((char **) (variant->content_languages->elts))[j];
- p = strchr(lang, '-'); /* find prefix part (if any) */
- if (p) {
- prefixlen = p - lang;
- }
-
- /* now find the best (i.e. longest) matching Accept-Language
- * header language. We put the best match for this tag in
- * bestthistag. We cannot update the overall best (based on
- * q value) because the best match for this tag is the longest
- * language item on the accept header, not necessarily the
- * highest q.
- */
- for (i = 0; i < neg->accept_langs->nelts; ++i) {
- if (!strcmp(accs[i].type_name, "*")) {
- if (!star) {
- star = &accs[i];
- }
- continue;
- }
-
- /* Find language. We match if either the variant language
- * tag exactly matches, or the prefix of the tag up to the
- * '-' character matches the whole of the language in the
- * Accept-Language header. We only use this accept-language
- * item as the best match for the current tag if it
- * is longer than the previous best match */
- if ((!strcmp(lang, accs[i].type_name) ||
- (prefixlen &&
- !strncmp(lang, accs[i].type_name, prefixlen) &&
- (accs[i].type_name[prefixlen] == '\0'))) &&
- ((len = strlen(accs[i].type_name)) >
- longest_lang_range_len)) {
- longest_lang_range_len = len;
- bestthistag = &accs[i];
- }
-
- if (!bestthistag) {
- /* The next bit is a fiddle. Some browsers might be
- * configured to send more specific language ranges
- * than desirable. For example, an Accept-Language of
- * en-US should never match variants with languages en
- * or en-GB. But US English speakers might pick en-US
- * as their language choice. So this fiddle checks if
- * the language range has a prefix, and if so, it
- * matches variants which match that prefix with a
- * priority of 0.001. So a request for en-US would
- * match variants of types en and en-GB, but at much
- * lower priority than matches of en-US directly, or
- * of any other language listed on the Accept-Language
- * header
- */
- if ((p = strchr(accs[i].type_name, '-'))) {
- int plen = p - accs[i].type_name;
- if (!strncmp(lang, accs[i].type_name, plen)) {
- fiddle_q = 0.001f;
- }
- }
- }
- }
- /* Finished looking at Accept-Language headers, the best
- * (longest) match is in bestthistag, or NULL if no match
- */
- if (!best ||
- (bestthistag && bestthistag->quality > best->quality)) {
- best = bestthistag;
- }
- }
-
- variant->lang_quality = best
- ? best->quality
- : (star ? star->quality : fiddle_q);
+ * (previously set by set_default_lang_quality()).
+ * Leave the factor alone (it remains at 1.0) when we may not fiddle
+ * with the headers.
+ */
+
+ if (!neg->dont_fiddle_headers)
+ variant->lang_quality = neg->default_lang_quality;
+
+ if (!neg->accept_langs) {
+ return; /* no accept-language header */
+ }
+
+ } else {
+ /* Variant has one (or more) languages. Look for the best
+ * match. We do this by going through each language on the
+ * variant description looking for a match on the
+ * Accept-Language header. The best match is the longest
+ * matching language on the header. The final result is the
+ * best q value from all the languages on the variant
+ * description. */
+
+ if (!neg->accept_langs) {
+ /* no accept-language header, this makes the variant
+ indefinite */
+ variant->definite = 0;
+
+ } else {
+ /* There is an accept-language with 0 or more items */
+ int j;
+ float fiddle_q = 0.0f;
+ accept_rec *accs = (accept_rec *) neg->accept_langs->elts;
+ accept_rec *best = NULL, *star = NULL;
+ int any_match_on_star = 0;
+
+ for (j = 0; j < variant->content_languages->nelts; ++j) {
+ char *lang; /* language from variant description */
+ accept_rec *bestthistag = NULL;
+ int longest_lang_range_len = 0;
+ int alen;
+ char *p;
+
+ /* lang is the variant's language-tag, which is the one
+ * we are allowed to use the prefix of in HTTP/1.1
+ */
+ lang = ((char **) (variant->content_languages->elts))[j];
+
+ /* now find the best (i.e. longest) matching
+ * Accept-Language header language. We put the best match
+ * for this tag in bestthistag. We cannot update the
+ * overall best (based on q value) because the best match
+ * for this tag is the longest language item on the accept
+ * header, not necessarily the highest q.
+ */
+ for (i = 0; i < neg->accept_langs->nelts; ++i) {
+ if (!strcmp(accs[i].name, "*")) {
+ if (!star) {
+ star = &accs[i];
+ }
+ continue;
+ }
+ /* Find language. We match if either the variant
+ * language tag exactly matches the language range
+ * from the accept header, or a prefix of the variant
+ * language tag up to a '-' character matches the
+ * whole of the language range in the Accept-Language
+ * header. Note that HTTP/1.x allows any number of
+ * '-' characters in a tag or range, currently only
+ * tags with zero or one '-' characters are defined
+ * for general use (see rfc1766).
+ *
+ * We only use language range in the Accept-Language
+ * header the best match for the variant language tag
+ * if it is longer than the previous best match.
+ */
+
+ alen = strlen(accs[i].name);
+
+ if ((strlen(lang)>=alen) &&
+ !strncmp(lang, accs[i].name, alen) &&
+ ((lang[alen]==0) || (lang[alen]=='-')) ) {
+
+ if (alen>longest_lang_range_len) {
+ longest_lang_range_len = alen;
+ bestthistag = &accs[i];
+ }
+ }
+
+ if (!bestthistag && !neg->dont_fiddle_headers) {
+ /* The next bit is a fiddle. Some browsers might
+ * be configured to send more specific language
+ * ranges than desirable. For example, an
+ * Accept-Language of en-US should never match
+ * variants with languages en or en-GB. But US
+ * English speakers might pick en-US as their
+ * language choice. So this fiddle checks if the
+ * language range has a prefix, and if so, it
+ * matches variants which match that prefix with a
+ * priority of 0.001. So a request for en-US would
+ * match variants of types en and en-GB, but at
+ * much lower priority than matches of en-US
+ * directly, or of any other language listed on
+ * the Accept-Language header. Note that this
+ * fiddle does not handle multi-level prefixes. */
+ if ((p = strchr(accs[i].name, '-'))) {
+ int plen = p - accs[i].name;
+ if (!strncmp(lang, accs[i].name, plen)) {
+ fiddle_q = 0.001f;
+ }
+ }
+ }
+ }
+ /* Finished looking at Accept-Language headers, the best
+ * (longest) match is in bestthistag, or NULL if no match
+ */
+ if (!best ||
+ (bestthistag && bestthistag->quality > best->quality)) {
+ best = bestthistag;
+ }
+
+ /* See if the tag matches on a * in the Accept-Language
+ * header. If so, record this fact for later use
+ */
+ if (!bestthistag && star) any_match_on_star=1;
+ }
+
+ /* If one of the language tags of the variant mached on *, we
+ * need to see if its q is better than that of any non-* match
+ * on any other tag of the variant. If so the * match takes
+ * precedence and the overall match is not definite. */
+ if ( any_match_on_star &&
+ ((best && star->quality > best->quality) ||
+ (!best)) ) {
+ best = star;
+ variant->definite = 0;
+ }
+
+ variant->lang_quality = best ? best->quality : fiddle_q;
+ }
}
/* Now set the old lang_index field. Since this is old
* stuff anyway, don't both with handling multiple languages
- * per variant, just use the first one assigned to it
+ * per variant, just use the first one assigned to it
*/
idx = 0;
if (variant->content_languages && variant->content_languages->nelts) {
@@ -1269,7 +1458,7 @@
else {
firstlang = "";
}
- if (naccept == 0) { /* Client doesn't care */
+ if (!neg->accept_langs) { /* Client doesn't care */
idx = find_default_index(conf, firstlang);
}
else { /* Client has Accept-Language */
@@ -1307,29 +1496,36 @@
/* For a given variant, find the best matching Accept: header
* and assign the Accept: header's quality value to the
- * accept_type_quality field of the variant, for later use in
+ * mime_type_quality field of the variant, for later use in
* determining the best matching variant.
*/
static void set_accept_quality(negotiation_state *neg, var_rec *variant)
{
int i;
- accept_rec *accept_recs = (accept_rec *) neg->accepts->elts;
+ accept_rec *accept_recs;
float q = 0.0f;
int q_definite = 1;
/* if no Accept: header, leave quality alone (will
- * remain at the default value of 1) */
- if (!neg->accepts || neg->accepts->nelts == 0) {
+ * remain at the default value of 1)
+ *
+ * XXX: This if is currently never true because of the effect of
+ * maybe_add_default_encodings().
+ */
+ if (!neg->accepts) {
+ if (variant->mime_type && *variant->mime_type) variant->definite = 0;
return;
}
+ accept_recs = (accept_rec *) neg->accepts->elts;
+
/*
* Go through each of the ranges on the Accept: header,
* looking for the 'best' match with this variant's
* content-type. We use the best match's quality
* value (from the Accept: header) for this variant's
- * accept_type_quality field.
+ * mime_type_quality field.
*
* The best match is determined like this:
* type/type is better than type/ * is better than * / *
@@ -1354,22 +1550,25 @@
}
}
- /* Check maxbytes -- not in HTTP/1.1 or Holtman */
+ /* Check maxbytes -- not in HTTP/1.1 or TCN */
if (type->max_bytes > 0
&& (find_content_length(neg, variant) > type->max_bytes)) {
continue;
}
- /* If we are allowed to mess with the q-values,
+ /* If we are allowed to mess with the q-values
+ * and have no explicit q= pararameters in the accept header,
* make wildcards very low, so we have a low chance
* of ending up with them if there's something better.
*/
- if (!neg->accept_q && variant->mime_stars == 1) {
+ if (!neg->dont_fiddle_headers && !neg->accept_q &&
+ variant->mime_stars == 1) {
q = 0.01f;
}
- else if (!neg->accept_q && variant->mime_stars == 2) {
+ else if (!neg->dont_fiddle_headers && !neg->accept_q &&
+ variant->mime_stars == 2) {
q = 0.02f;
}
else {
@@ -1378,11 +1577,9 @@
q_definite = (variant->mime_stars == 3);
}
- variant->accept_type_quality = q;
+ variant->mime_type_quality = q;
variant->definite = variant->definite && q_definite;
- /* if the _best_ quality we got for this variant was 0.0,
- * eliminate it now */
}
/* For a given variant, find the 'q' value of the charset given
@@ -1393,17 +1590,40 @@
static void set_charset_quality(negotiation_state *neg, var_rec *variant)
{
int i;
- accept_rec *accept_recs = (accept_rec *) neg->accept_charsets->elts;
+ accept_rec *accept_recs;
char *charset = variant->content_charset;
accept_rec *star = NULL;
/* if no Accept-Charset: header, leave quality alone (will
* remain at the default value of 1) */
- if (!neg->accept_charsets || neg->accept_charsets->nelts == 0) {
+ if (!neg->accept_charsets) {
+ if (charset && *charset) variant->definite = 0;
return;
}
+ accept_recs = (accept_rec *) neg->accept_charsets->elts;
+
if (charset == NULL || !*charset) {
+ /* Charset of variant not known */
+
+ /* if not a text / * type, leave quality alone */
+ if (!(!strncmp(variant->mime_type, "text/",5)
+ || !strcmp(variant->mime_type, INCLUDES_MAGIC_TYPE)
+ || !strcmp(variant->mime_type, INCLUDES_MAGIC_TYPE3)
+ ))
+ return;
+
+ /* Don't go guessing if we are in strict header mode,
+ e.g. when running the rvsa, as any guess won't be reflected
+ in the variant list or content-location headers */
+ if (neg->dont_fiddle_headers) return;
+
+ /* Guess that charset is iso-8859-1.
+ * XXX: Is it documented that Apache does this?
+ * There are cases where guessing wrong could badly impact the
+ * negotiation result.
+ */
+
charset = "iso-8859-1";
}
@@ -1416,17 +1636,18 @@
accept_rec *type = &accept_recs[i];
- if (!strcmp(type->type_name, charset)) {
+ if (!strcmp(type->name, charset)) {
variant->charset_quality = type->quality;
return;
}
- else if (strcmp(type->type_name, "*") == 0) {
+ else if (strcmp(type->name, "*") == 0) {
star = type;
}
}
/* No explicit match */
if (star) {
variant->charset_quality = star->quality;
+ variant->definite = 0;
return;
}
/* If this variant is in charset iso-8859-1, the default is 1.0 */
@@ -1438,11 +1659,6 @@
}
}
-/* For a given variant, find the best matching Accept: header
- * and assign the Accept: header's quality value to the
- * accept_type_quality field of the variant, for later use in
- * determining the best matching variant.
- */
/* is_identity_encoding is included for back-compat, but does anyone
* use 7bit, 8bin or binary in their var files??
@@ -1454,25 +1670,38 @@
|| !strcmp(enc, "binary"));
}
+/* For a given variant, find the encoding quality using the
+ * Accept-Encoding header.
+ */
+
static void set_encoding_quality(negotiation_state *neg, var_rec *variant)
{
int i;
- accept_rec *accept_recs = (accept_rec *) neg->accept_encodings->elts;
+ accept_rec *accept_recs;
const char *enc = variant->content_encoding;
+ accept_rec *star = NULL;
if (!enc || is_identity_encoding(enc)) {
return;
}
- /* if no Accept: header, leave quality alone (will
- * remain at the default value of 1) */
- if (neg->accept_encodings->nelts == 0) {
- /* If we had an empty Accept-Encoding header, assume that
- * no encodings are acceptable, else all encodings are ok */
- variant->encoding_quality = neg->have_accept_header ? 0 : 1;
+ if (!neg->accept_encodings) {
+ /* We had no Accept-Encoding header, assume that all
+ * encodings are acceptable with a quality of 1.
+ *
+ * XXX: This assumption is unsafe, and the only thing which
+ * prevents it from causing serious harm is that the Apache
+ * negotiation algorithm (currently) always prefers unencoded
+ * variants over encoded ones, while the RVSA/1.0 algorithm
+ * (currently) ignores encoded variants entirely. -kh
+ */
+
+ variant->encoding_quality = 1;
return;
}
+ accept_recs = (accept_rec *) neg->accept_encodings->elts;
+
/* Go through each of the encodings on the Accept-Encoding: header,
* looking for a match with our encoding. x- prefixes are ignored.
*/
@@ -1480,16 +1709,26 @@
enc += 2;
}
for (i = 0; i < neg->accept_encodings->nelts; ++i) {
- char *name = accept_recs[i].type_name;
+ char *name = accept_recs[i].name;
if (name[0] == 'x' && name[1] == '-') {
name += 2;
}
if (!strcmp(name, enc)) {
- variant->encoding_quality = 1;
+ variant->encoding_quality = accept_recs[i].quality;
return;
}
+
+ if (strcmp(name, "*") == 0) {
+ star = &accept_recs[i];
+ }
+
+ }
+ /* No explicit match */
+ if (star) {
+ variant->encoding_quality = star->quality;
+ return;
}
/* Encoding not found on Accept-Encoding: header, so it is
@@ -1497,90 +1736,93 @@
variant->encoding_quality = 0;
}
-/* Possible results of the network algorithm */
+/*************************************************************
+ * Possible results of the variant selection algorithm
+ */
enum algorithm_results {
- na_not_applied = -1, /* net algorithm not used */
- na_choice = 1, /* choose variant */
- na_list /* list variants */
+ alg_choice = 1, /* choose variant */
+ alg_list /* list variants */
};
/*
- * This is a heavily-rewritten 'best_match' function. For a start, it
- * now returns an int, which has one of the three values: na_not_applied,
- * na_choice or na_list, which give the result of the network algorithm
- * (if it was not applied, the return value is na_not_applied).
- * The best variable is returned in *pbest. It also has two possible
- * algorithms for determining the best match: the network algorithm,
- * and the standard Apache algorithm. These are split out into
- * separate functions (is_variant_better_na() and is_variant_better()).
- *
- * Previously, best_match iterated first through the content_types
- * in the Accept: header, then checked each variant, and eliminated
- * those that didn't match the variant's type. We cannot do this because
- * we need full information, including language, charset, etc
- * quality for _every_ variant, for the Alternates: header,
- * and (possibly) the human-readable choice responses or 406 errors.
- *
- * After the 'best' (if any) is determined, the overall result of
- * the negotiation is obtained. If the network algorithm was not
- * in use, the result is na_not_applied. Else the result is
- * na_list if 'short accept header' is in use, else na_list
- * if _no_ best match was found, or na_choice if a best match
- * was found.
+ * Below is the 'best_match' function. It returns an int, which has
+ * one of the two values alg_choice or alg_list, which give the result
+ * of the variant selection algorithm. alg_list means that no best
+ * variant was found by the algorithm, alg_choice means that a best
+ * variant was found and should be returned. The list/choice
+ * terminology comes from TCN (rfc2295), but is used in a more generic
+ * way here. The best variant is returned in *pbest. best_match has
+ * two possible algorithms for determining the best variant: the
+ * RVSA/1.0 algorithm (from RFC2296), and the standard Apache
+ * algorithm. These are split out into separate functions
+ * (is_variant_better_rvsa() and is_variant_better()). Selection of
+ * one is through the neg->use_rvsa flag.
+ *
+ * The call to best_match also creates full information, including
+ * language, charset, etc quality for _every_ variant. This is needed
+ * for generating a correct Vary: header, and can be used for the
+ * Alternates: header, the human-readable list responses and 406
+ * errors. */
+
+/* Firstly, the RVSA/1.0 (HTTP Remote Variant Selection Algorithm
+ * v1.0) from rfc2296. This is the algorithm that goes together with
+ * transparent content negotiation (TCN).
*/
-/* Firstly, the negotiation 'network algorithm' from Holtman.
- */
-
-static int is_variant_better_na(negotiation_state *neg, var_rec *variant,
+static int is_variant_better_rvsa(negotiation_state *neg, var_rec *variant,
var_rec *best, float *p_bestq)
{
float bestq = *p_bestq, q;
- /* Note: Encoding is not negotiated in the Holtman
- * transparent neg draft, so we ignored it here. But
- * it does mean we could return encodings the UA
- * or proxy cannot handle. Eek. */
+ /* TCN does not cover negotiation on content-encoding. For now,
+ we ignore all variants which have a content-encoding, i.e. we
+ return 0 for them to signify that they are never better than
+ anything.
+ XXX Todo: improve on this, e.g. by adding a second negotiation
+ phase on content encoding if the RVSA is used.
+ */
+ if (variant->content_encoding) return 0;
- q = variant->accept_type_quality *
- variant->type_quality *
+ q = variant->mime_type_quality *
+ variant->source_quality *
variant->charset_quality *
variant->lang_quality;
+ /* We round to 5 decimal places, but make sure that variants with
+ a very low nonzero q value to not get rounded down to 0 */
+ if ((q<0.00001)&&(q>0)) {
+ q=0.00001;
+ }
+ else {
+ /* This roundabout way of rounding avoids use of the math library */
+ char qstr[10];
+ char qval = ap_snprintf(qstr, sizeof(qstr), "%1.5f", q);
+ sscanf(qstr, "%f", &q);
+ }
+
#ifdef NEG_DEBUG
- fprintf(stderr, "Variant: file=%s type=%s lang=%s acceptq=%1.3f "
- "langq=%1.3f typeq=%1.3f q=%1.3f definite=%d\n",
+ fprintf(stderr, "Variant: file=%s type=%s lang=%s sourceq=%1.3f "
+ "mimeq=%1.3f langq=%1.3f charq=%1.3f encq=%1.3f "
+ "q=%1.3f definite=%d\n",
(variant->file_name ? variant->file_name : ""),
- (variant->type_name ? variant->type_name : ""),
+ (variant->mime_type ? variant->mime_type : ""),
(variant->content_languages
? merge_string_array(neg->pool, variant->content_languages, ",")
: ""),
- variant->accept_type_quality,
+ variant->source_quality,
+ variant->mime_type_quality,
variant->lang_quality,
- variant->type_quality,
+ variant->charset_quality,
+ variant->encoding_quality,
q,
variant->definite);
#endif
+ if (q == 0) return 0;
if (q > bestq) {
*p_bestq = q;
return 1;
}
- if (q == bestq) {
- /* If the best variant's charset is ISO-8859-1 and this variant has
- * the same charset quality, then we prefer this variant
- */
- if (variant->charset_quality == best->charset_quality &&
- (variant->content_charset != NULL &&
- *variant->content_charset != '\0' &&
- strcmp(variant->content_charset, "iso-8859-1") != 0) &&
- (best->content_charset == NULL ||
- *best->content_charset == '\0' ||
- strcmp(best->content_charset, "iso-8859-1") == 0)) {
- *p_bestq = q;
- return 1;
- }
- }
return 0;
}
@@ -1619,13 +1861,13 @@
if (variant->encoding_quality == 0 ||
variant->lang_quality == 0 ||
- variant->type_quality == 0 ||
+ variant->source_quality == 0 ||
variant->charset_quality == 0 ||
- variant->accept_type_quality == 0) {
+ variant->mime_type_quality == 0) {
return 0; /* don't consider unacceptables */
}
- q = variant->accept_type_quality * variant->type_quality;
+ q = variant->mime_type_quality * variant->source_quality;
if (q == 0.0 || q < bestq) {
return 0;
}
@@ -1645,6 +1887,8 @@
/* if language qualities were equal, try the LanguagePriority
* stuff */
+ /* XXX: TODO: there is a slight discrepancy between how this
+ behaves and how it described in the documentation */
if (best->lang_index != -1 && variant->lang_index > best->lang_index) {
return 0;
}
@@ -1654,7 +1898,8 @@
return 1;
}
- /* content-type level (text/html only?) */
+ /* content-type level (sometimes used with text/html, though we
+ support it on other types too) */
levcmp = level_cmp(variant, best);
if (levcmp == -1) {
return 0;
@@ -1664,9 +1909,17 @@
return 1;
}
+#define OLD_ENCODING_NEGOTIATION
+#ifdef OLD_ENCODING_NEGOTIATION
+ /* XXX: Note that the encoding negotiation code below does not know
+ about Accept-Encoding q values. It should be upgraded at one
+ point.
+ */
+
/* encoding -- can only be 1 or 0, and if 0 we eliminated this
* variant at the start of this function. However we
* prefer variants with no encoding over those with encoding */
+
if (best->content_encoding == NULL && variant->content_encoding) {
return 0;
}
@@ -1674,6 +1927,31 @@
*p_bestq = q;
return 1;
}
+#else
+ /* XXX: The encoding negotiation code below knows about
+ Accept-Encoding q values, but has the same strange tie-breaking
+ as the version above if the qualities are equal.
+ */
+
+ /* Prefer the highest value for encoding_quality. If they are
+ * equal, prefer the variant without any encoding.
+ */
+ if (variant->encoding_quality < best->encoding_quality) {
+ return 0;
+ }
+ if (variant->encoding_quality > best->encoding_quality) {
+ *p_bestq = q;
+ return 1;
+ }
+
+ if (best->content_encoding == NULL && variant->content_encoding) {
+ return 0;
+ }
+ if (best->content_encoding && variant->content_encoding == NULL) {
+ *p_bestq = q;
+ return 1;
+ }
+#endif
/* charset */
if (variant->charset_quality < best->charset_quality) {
@@ -1682,6 +1960,9 @@
/* If the best variant's charset is ISO-8859-1 and this variant has
* the same charset quality, then we prefer this variant
*/
+ /* XXX: TODO: this specific tie-breaker is not described in the
+ documentation */
+
if (variant->charset_quality > best->charset_quality ||
((variant->content_charset != NULL &&
*variant->content_charset != '\0' &&
@@ -1710,98 +1991,105 @@
int j;
var_rec *best = NULL;
float bestq = 0.0f;
- enum algorithm_results algorithm_result = na_not_applied;
+ enum algorithm_results algorithm_result;
var_rec *avail_recs = (var_rec *) neg->avail_vars->elts;
set_default_lang_quality(neg);
/*
- * Find the 'best' variant
+ * Find the 'best' variant
*/
for (j = 0; j < neg->avail_vars->nelts; ++j) {
var_rec *variant = &avail_recs[j];
/* Find all the relevant 'quality' values from the
- * Accept... headers, and store in the variant
+ * Accept... headers, and store in the variant. This also
+ * prepares for sending an Alternates header etc so we need to
+ * do it even if we do not actually plan to find a best
+ * variant.
*/
set_accept_quality(neg, variant);
set_language_quality(neg, variant);
set_encoding_quality(neg, variant);
set_charset_quality(neg, variant);
- /* Now find out if this variant is better than the current
- * best, either using the network algorithm, or Apache's
- * internal server-driven algorithm. Presumably other
- * server-driven algorithms are possible, and could be
- * implemented here.
+ /* Only do variant selection if we may actually choose a
+ * variant for the client
*/
+ if (neg->may_choose) {
- if (neg->use_transparent_neg) {
- if (is_variant_better_na(neg, variant, best, &bestq)) {
- best = variant;
- }
- }
- else {
- if (is_variant_better(neg, variant, best, &bestq)) {
- best = variant;
- }
- }
+ /* Now find out if this variant is better than the current
+ * best, either using the RVSA/1.0 algorithm, or Apache's
+ * internal server-driven algorithm. Presumably other
+ * server-driven algorithms are possible, and could be
+ * implemented here.
+ */
+
+ if (neg->use_rvsa) {
+ if (is_variant_better_rvsa(neg, variant, best, &bestq)) {
+ best = variant;
+ }
+ }
+ else {
+ if (is_variant_better(neg, variant, best, &bestq)) {
+ best = variant;
+ }
+ }
+ }
}
/* We now either have a best variant, or no best variant
*/
- if (neg->use_transparent_neg) {
- if (neg->short_accept_headers) {
- algorithm_result = na_list;
- }
- else {
- /* From Holtman, result is:
- * If variant & URI are not neigbors, list_ua or list_os
- * Else
- * If UA can do trans neg
- * IF best is definite && best q > 0, choice_ua
- * ELSE list_ua
- * ELSE
- * IF best q > 0, choose_os
- * ELSE list_os (or forward_os on proxy)
- */
-
- /* assume variant and URI are neigbors (since URI in
- * var map must be in same directory) */
- if (neg->use_transparent_neg) {
- algorithm_result = (best && best->definite) && (bestq > 0)
- ? na_choice : na_list;
- }
- else {
- algorithm_result = bestq > 0 ? na_choice : na_list;
- }
- }
+ if (neg->use_rvsa) {
+ /* calculate result for RVSA/1.0 algorithm:
+ * only a choice response if the best variant has q>0
+ * and is definite
+ */
+ algorithm_result = (best && best->definite) && (bestq > 0)
+ ? alg_choice : alg_list;
+ }
+ else {
+ /* calculate result for Apache negotiation algorithm */
+ algorithm_result = bestq > 0 ? alg_choice : alg_list;
}
+ /* Returning a choice response with a non-neighboring variant is a
+ * protocol security error in TCN (see rfc2295). We do *not*
+ * verify here that the variant and URI are neighbors, even though
+ * we may return alg_choice. We depend on the environment (the
+ * caller) to only declare the resource transparently negotable if
+ * all variants are neighbors.
+ */
+
*pbest = best;
return algorithm_result;
}
/*
- * Sets the Alternates and Vary headers, used if we are going to
- * return 406 Not Acceptable status, a 300 Multiple Choice status,
- * or a Choice response.
- *
- * 'type' is the result of the network algorithm, if applied.
- * We do different things if the network algorithm was not applied
- * (type == na_not_applied): no Alternates header, and Vary:
- * does not include 'negotiate'.
- *
- * We should also add a max-age lifetime for the Alternates header,
- * but how long we we give it? Presumably this should be
- * configurable in the map file.
- */
+ * Sets response headers for a negotiated response.
+ * neg->is_tranparent determines whether a transparently negotiated
+ * response or a plain `server driven negotiation' response is
+ * created. Applicable headers are Alternates, Vary, and TCN.
+ */
+
+/* The Vary header we create is sometimes longer than is required for
+ the correct caching of negotiated results by HTTP/1.1 caches. For
+ example if we have 3 variants x.html, x.ps.en and x.ps.nl, and if
+ the Accept: header assignes a 0 quality to .ps, then the results of
+ the two server-side negotiation algorithms we currently implement
+ will never depend on Accept-Language so we could return `Vary:
+ negotiate, accept' in stead of the longer 'Vary: negotiate, accept,
+ accept-language' which the code below will return. A routine for
+ computing the exact minimal Vary header would be a huge pain to
+ code and maintain though, especially because we need to take all
+ possible twiddles in the server-side negotiation algorithms into
+ account. */
static void set_neg_headers(request_rec *r, negotiation_state *neg,
- int na_result)
+ int alg_result)
{
int j;
var_rec *avail_recs = (var_rec *) neg->avail_vars->elts;
@@ -1809,6 +2097,7 @@
char *sample_language = NULL;
const char *sample_encoding = NULL;
char *sample_charset = NULL;
+ int first_variant = 1;
int vary_by_type = 0;
int vary_by_language = 0;
int vary_by_charset = 0;
@@ -1817,6 +2106,10 @@
/* Put headers into err_headers_out, new send_http_header()
* outputs both headers_out and err_headers_out */
+
+ /* NB that we merge the headers in case some other module
+ * negotiates on something else. */
+
hdrs = r->err_headers_out;
for (j = 0; j < neg->avail_vars->nelts; ++j) {
@@ -1825,8 +2118,9 @@
char qstr[6];
long len;
char lenstr[22]; /* enough for 2^64 */
+ char *lang;
- ap_snprintf(qstr, sizeof(qstr), "%1.3f", variant->type_quality);
+ ap_snprintf(qstr, sizeof(qstr), "%1.3f", variant->source_quality);
/* Strip trailing zeros (saves those valuable network bytes) */
if (qstr[4] == '0') {
@@ -1840,82 +2134,135 @@
}
rec = ap_pstrcat(r->pool, "{\"", variant->file_name, "\" ", qstr,
NULL);
- if (variant->type_name) {
- if (*variant->type_name) {
- rec = ap_pstrcat(r->pool, rec, " {type ",
- variant->type_name, "}", NULL);
- }
- if (!sample_type) {
- sample_type = variant->type_name;
- }
- else if (strcmp(sample_type, variant->type_name)) {
- vary_by_type = 1;
- }
+
+ if (first_variant) {
+ sample_type = variant->mime_type;
+ } else if (strcmp(sample_type ? sample_type : "",
+ variant->mime_type ? variant->mime_type : "")) {
+ vary_by_type = 1;
+ }
+ if (variant->mime_type && *variant->mime_type) {
+ rec = ap_pstrcat(r->pool, rec, " {type ",
+ variant->mime_type, "}", NULL);
}
+
if (variant->content_languages && variant->content_languages->nelts) {
- char *langs = merge_string_array(r->pool,
- variant->content_languages, ",");
- rec = ap_pstrcat(r->pool, rec, " {language ", langs, "}", NULL);
- if (!sample_language) {
- sample_language = langs;
- }
- else if (strcmp(sample_language, langs)) {
- vary_by_language = 1;
- }
+ lang = merge_string_array(r->pool,
+ variant->content_languages, ",");
+ rec = ap_pstrcat(r->pool, rec, " {language ", lang, "}", NULL);
+ } else {
+ lang = NULL;
+ }
+ if (first_variant) {
+ sample_language = lang;
+ } else if (strcmp(sample_language ? sample_language : "",
+ lang ? lang : "")) {
+ vary_by_language = 1;
+ }
+
+ if (first_variant) {
+ sample_encoding = variant->content_encoding;
+ } else if (strcmp(sample_encoding ? sample_encoding : "",
+ variant->content_encoding ?
+ variant->content_encoding : "")) {
+ vary_by_encoding = 1;
+ }
+
+ if (first_variant) {
+ sample_charset = variant->content_charset;
+ } else if (strcmp(sample_charset ? sample_charset : "",
+ variant->content_charset ? variant->content_charset
+ : "")) {
+ vary_by_charset = 1;
}
- if (variant->content_encoding) {
- if (!sample_encoding) {
- sample_encoding = variant->content_encoding;
- }
- else if (strcmp(sample_encoding, variant->content_encoding)) {
- vary_by_encoding = 1;
- }
- }
- if (variant->content_charset) {
- if (*variant->content_charset) {
- rec = ap_pstrcat(r->pool, rec, " {charset ",
- variant->content_charset, "}", NULL);
- }
- if (!sample_charset) {
- sample_charset = variant->content_charset;
- }
- else if (strcmp(sample_charset, variant->content_charset)) {
- vary_by_charset = 1;
- }
+ if (variant->content_charset && *variant->content_charset) {
+ rec = ap_pstrcat(r->pool, rec, " {charset ",
+ variant->content_charset, "}", NULL);
}
- if ((len = find_content_length(neg, variant)) != 0) {
+
+ /* Note tat the Alternates specification (in rfc2295) does
+ not require that we include {length x}, so we could omit it
+ if determining the length is too expensive. We currently
+ always include it though.
+
+ If the variant is a CGI script, find_content_length would
+ return the length of the script, not the output it
+ produces, so we check for the presence of a handler and if
+ there is one we don't add a length.
+
+ XXX: TODO: This check does not detect a CGI script if we
+ get the variant from a type map. This needs to be fixed
+ (without breaking things if the type map specifies a
+ content-length, which currently leads to the correct result).
+ */
+
+ if (neg->send_alternates &&
+ !(variant->sub_req && variant->sub_req->handler) &&
+ (len = find_content_length(neg, variant)) != 0) {
ap_snprintf(lenstr, sizeof(lenstr), "%ld", len);
rec = ap_pstrcat(r->pool, rec, " {length ", lenstr, "}", NULL);
- }
-
+ }
+
rec = ap_pstrcat(r->pool, rec, "}", NULL);
+
+ if (neg->send_alternates) {
- if (na_result != na_not_applied) {
- ap_table_mergen(hdrs, "Alternates", rec);
- }
+ /* We only list the variant in the Alternates header if it
+ * has no content encoding, as there is no standard way of
+ * saying in the Alternates header that a variant is
+ * available in a content-encoded form (only). */
+ if (!variant->content_encoding)
+ ap_table_mergen(hdrs, "Alternates", rec);
+ }
+
+ first_variant = 0;
+
}
- if (na_result != na_not_applied) {
+ if (neg->is_transparent) {
ap_table_mergen(hdrs, "Vary", "negotiate");
}
+#define CORRECT_VARY_ACCEPT
+#ifdef CORRECT_VARY_ACCEPT
+ /* Currently the negotiation result _always_ has a depencence on
+ the contents of the Accept header because we do 'mxb='
+ processing in set_accept_quality(). So we _always_ inhave to
+ clude Accept in the Vary header. If we would not do this, a
+ html variant chosen because of client 1 sending
+ application/postscript;mxb=10000 can end up being returned by a
+ cache to all clients n>1 even if these do accept all postscript
+ no matter what the length. Oh what a pain it is. And always
+ adding Accept to Vary can significantly degrade the ability of
+ any HTTP/1.1 cache to return content without revalidation
+ too. */
+ ap_table_mergen(hdrs, "Vary", "accept");
+#else
if (vary_by_type) {
ap_table_mergen(hdrs, "Vary", "accept");
}
+#endif
if (vary_by_language) {
ap_table_mergen(hdrs, "Vary", "accept-language");
}
if (vary_by_charset) {
ap_table_mergen(hdrs, "Vary", "accept-charset");
}
- if (vary_by_encoding && na_result == na_not_applied) {
+ if (vary_by_encoding) {
ap_table_mergen(hdrs, "Vary", "accept-encoding");
}
+
+ if (neg->is_transparent) {
+ /* Create TCN response header */
+ ap_table_setn(hdrs, "TCN",
+ alg_result == alg_list ? "list" : "choice" );
+ }
+
}
/**********************************************************************
*
* Return an HTML list of variants. This is output as part of the
- * 300 or 406 status body.
+ * choice response or 406 status body.
*/
/* XXX: this is disgusting, this has O(n^2) behaviour! -djg */
@@ -1936,8 +2283,8 @@
* 'English'). */
t = ap_pstrcat(r->pool, t, "<li><a href=\"", filename, "\">",
filename, "</a> ", description, NULL);
- if (variant->type_name && *variant->type_name) {
- t = ap_pstrcat(r->pool, t, ", type ", variant->type_name, NULL);
+ if (variant->mime_type && *variant->mime_type) {
+ t = ap_pstrcat(r->pool, t, ", type ", variant->mime_type, NULL);
}
if (languages && languages->nelts) {
t = ap_pstrcat(r->pool, t, ", language ",
@@ -1948,6 +2295,10 @@
t = ap_pstrcat(r->pool, t, ", charset ", variant->content_charset,
NULL);
}
+ if (variant->content_encoding) {
+ t = ap_pstrcat(r->pool, t, ", encoding ",
+ variant->content_encoding, NULL);
+ }
t = ap_pstrcat(r->pool, t, "\n", NULL);
}
t = ap_pstrcat(r->pool, t, "</ul>\n", NULL);
@@ -1966,7 +2317,8 @@
}
}
-/* Called if we got a "Choice" response from the network algorithm.
+/* Called if we got a "Choice" response from the variant selection
+ * algorithm.
* It checks the result of the chosen variant to see if it
* is itself negotiated (if so, return error VARIANT_ALSO_VARIES).
* Otherwise, add the appropriate headers to the current response.
@@ -1982,7 +2334,9 @@
sub_req = ap_sub_req_lookup_file(variant->file_name, r);
status = sub_req->status;
- if (status != HTTP_OK && status != HTTP_MULTIPLE_CHOICES) {
+
+ if (status != HTTP_OK &&
+ !ap_table_get(sub_req->err_headers_out, "TCN")) {
ap_destroy_sub_req(sub_req);
return status;
}
@@ -1992,27 +2346,85 @@
sub_req = variant->sub_req;
}
- /* The network algorithm told us to return a "Choice"
+ /* The variant selection algorithm told us to return a "Choice"
* response. This is the normal variant response, with
* some extra headers. First, ensure that the chosen
- * variant did not itself return a "List" or "Choice" response.
+ * variant did or will not itself engage in transparent negotiation.
* If not, set the appropriate headers, and fall through to
* the normal variant handling
*/
- if ((sub_req->status == HTTP_MULTIPLE_CHOICES) ||
- (ap_table_get(sub_req->err_headers_out, "Alternates")) ||
- (ap_table_get(sub_req->err_headers_out, "Content-Location"))) {
+ /* This catches the error that a transparent type map selects a
+ transparent multiviews resource as the best variant.
+
+ XXX: We do not signal an error if a transparent type map
+ selects a _non_transparent multiviews resource as the best
+ variant, because we can generate a legal negotiation response
+ in this case. In this case, the vlist_validator of the
+ nontransparent subrequest will be lost however. This could
+ lead to cases in which a change in the set of variants or the
+ negotiation algorithm of the nontransparent resource is never
+ propagated up to a HTTP/1.1 cache which interprets Vary. To be
+ completely on the safe side we should return VARIANT_ALSO_VARIES
+ for this type of recursive negotiation too. */
+
+ if (neg->is_transparent &&
+ ap_table_get(sub_req->err_headers_out, "TCN")) {
+ return VARIANT_ALSO_VARIES;
+ }
+
+ /* This catches the error that a transparent type map recursively
+ selects, as the best variant, another type map which itself
+ causes transparent negotiation to be done.
+
+ XXX: Actually, we catch this error by catching all cases of
+ type map recursion. There are some borderline recursive type
+ map arrangements which would not produce transparent
+ negotiation protocol errors or lack of cache propagation
+ problems, but such arrangements are very hard to detect at this
+ point in the control flow, so we do not bother to single them
+ out.
+
+ Recursive type maps imply a recursive arrangement of negotiated
+ resources which is visible to outside clients, and this is not
+ supported by the transparent negotiation caching protocols, so
+ if we are to have generic support for recursive type maps, we
+ have to create some configuration setting which makes all type
+ maps non-transparent when recursion is enabled. Also, if we
+ want recursive type map support which ensures propagation of
+ type map changes into HTTP/1.1 caches that handle Vary, we
+ would have to extend the current mechanism for generating
+ variant list validators. */
+
+ if (sub_req->handler && strcmp(sub_req->handler, "type-map")==0) {
return VARIANT_ALSO_VARIES;
}
+ /* This adds an appropriate Variant-Vary header if the subrequest
+ is a multiviews resource.
+
+ XXX: TODO: Note that this does _not_ handle any Vary header
+ returned by a CGI if sub_req is a CGI script, because we don't
+ see that Vary header yet at this point in the control flow.
+ This won't cause any cache consistency problems _unless_ the
+ CGI script also returns a Cache-Control header marking the
+ response as cachable. This needs to be fixed, also there are
+ problems if a CGI returns an Etag header which also need to be
+ fixed.
+ */
if ((sub_vary = ap_table_get(sub_req->err_headers_out, "Vary")) != NULL) {
ap_table_setn(r->err_headers_out, "Variant-Vary", sub_vary);
+ /* Move the subreq Vary header into the main request to
+ prevent having two Vary headers in the response, which
+ would be legal but strange */
+ ap_table_setn(r->err_headers_out, "Vary", sub_vary);
+ ap_table_unset(sub_req->err_headers_out, "Vary");
}
+
ap_table_setn(r->err_headers_out, "Content-Location",
ap_pstrdup(r->pool, variant->file_name));
- set_neg_headers(r, neg, na_choice); /* add Alternates and Vary */
- /* to do: add Expires */
+ set_neg_headers(r, neg, alg_choice); /* add Alternates and Vary */
+ /* Still to do by caller: add Expires */
return 0;
}
@@ -2022,62 +2434,167 @@
* Executive...
*/
-static int handle_map_file(request_rec *r)
+static int do_negotiation(request_rec *r, negotiation_state *neg,
+ var_rec **bestp, int prefer_scripts)
{
- negotiation_state *neg = parse_accept_headers(r);
- var_rec *best;
+ int alg_result; /* result of variant selection algorithm */
int res;
- int na_result;
+ var_rec *avail_recs = (var_rec *) neg->avail_vars->elts;
+ int j;
+ int unencoded_variants=0;
- char *udir;
+ /* Decide if resource is transparently negotiable */
- if ((res = read_type_map(neg, r))) {
- return res;
+ /* GET or HEAD? (HEAD has same method number as GET) */
+ if (r->method_number == M_GET) {
+
+ /* maybe this should be configurable, see also the comment
+ about recursive type maps in setup_choice_response() */
+ neg->is_transparent = 1;
+
+ /* We can't be transparent if we are a map file in the middle
+ of the request URI */
+ if (r->path_info && *r->path_info) neg->is_transparent = 0;
+
+ for (j = 0; j < neg->avail_vars->nelts; ++j) {
+ var_rec *variant = &avail_recs[j];
+
+ /* We can't be transparent, because of internal
+ assumptions in best_match(), if there is a
+ non-neighboring variant. We can have a non-neighboring
+ variant when processing a type map.
+ */
+ if (strchr(variant->file_name,'/')) neg->is_transparent = 0;
+
+ if (!variant->content_encoding) unencoded_variants++;
+
+ }
+
+ /* If there are less than 2 unencoded variants, we always
+ switch to server-driven negotiation, regardless of whether
+ we are contacted by a client capable of transparent
+ negotiation. We do this because our current TCN
+ implementation does not deal well with the case of having 0
+ or 1 unencoded variants. */
+ if (unencoded_variants<2) neg->is_transparent = 0;
+
+ }
+
+ if (neg->is_transparent) {
+ parse_negotiate_header(r,neg);
+ } else {
+ /* configure negotiation on non-transparent resource */
+ neg->may_choose = 1;
}
- maybe_add_default_encodings(neg, 0);
+ maybe_add_default_encodings(neg, prefer_scripts);
- na_result = best_match(neg, &best);
+ alg_result = best_match(neg, bestp);
- /* na_result is one of
- * na_not_applied: we didn't use the network algorithm
- * na_choice: return a "Choice" response
- * na_list: return a "List" response (no variant chosen)
+ /* alg_result is one of
+ * alg_choice: a best variant is chosen
+ * alg_list: no best variant is chosen
*/
- if (na_result == na_list) {
- set_neg_headers(r, neg, na_list);
- store_variant_list(r, neg);
- return MULTIPLE_CHOICES;
- }
+ if (alg_result == alg_list) {
+ /* send a list response or NOT_ACCEPTABLE error response */
- if (!best) {
- ap_log_rerror(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, r,
- "no acceptable variant: %s", r->filename);
+ neg->send_alternates = 1; /* always include Alternates header */
+ set_neg_headers(r, neg, alg_result);
+ store_variant_list(r, neg);
+
+ if (neg->is_transparent && neg->ua_supports_trans) {
+ /* XXX todo: expires? cachability? */
+
+ /* Some HTTP/1.0 clients are known to choke when they get
+ a 300 (multiple choices) response without a Location
+ header. However the 300 code response we are are about
+ to generate will only reach 1.0 clients which support
+ transparent negotiation, and they should be OK. The
+ response should never reach older 1.0 clients, even if
+ we have CacheNegotiatedDocs enabled, because no 1.0
+ proxy cache (we know of) will cache and return 300
+ responses (they certainly won't if they conform to the
+ HTTP/1.0 specification). */
+ return MULTIPLE_CHOICES;
+ }
+
+ if (!*bestp) {
+ ap_log_rerror(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, r,
+ "no acceptable variant: %s", r->filename);
+ return NOT_ACCEPTABLE;
+ }
- set_neg_headers(r, neg, na_result);
- store_variant_list(r, neg);
- return NOT_ACCEPTABLE;
}
- if (na_result == na_choice) {
- if ((res = setup_choice_response(r, neg, best)) != 0) {
- return res;
- }
+ /* Variant selection chose a variant */
+
+ /* XXX todo: merge the two cases in the if statement below */
+ if (neg->is_transparent) {
+
+ if ((res = setup_choice_response(r, neg, *bestp)) != 0) {
+ return res; /* return if error */
+ }
+ }
+ else {
+ set_neg_headers(r, neg, alg_result);
}
/* Make sure caching works - Vary should handle HTTP/1.1, but for
- * HTTP/1.0, we can't allow caching at all. NB that we merge the
- * header in case some other module negotiates on something else.
- */
- if (!do_cache_negotiated_docs(r->server) && (r->proto_num <
HTTP_VERSION(1,1))) {
+ * HTTP/1.0, we can't allow caching at all. */
+
+ /* XXX: Note that we only set r->no_cache to 1, which causes
+ Expires: <now> to be added, when responding to a HTTP/1.0
+ client. If we return the response to a 1.1 client, we do not
+ add Expires <now>, because doing so would degrade 1.1 cache
+ performance by preventing re-use of the response withour prior
+ revalidation. On the other hand, if the 1.1 client is a proxy
+ which was itself contacted by a 1.0 client, or a proxy cache
+ which can be contacted later by 1.0 clients, then we currently
+ rely on this 1.1 proxy to add the Expires: <now> when it
+ forwards the response.
+
+ XXX: TODO: Find out if the 1.1 spec requires proxies and
+ tunnels to add Expires: <now> when forwarding the response to
+ 1.0 clients. I (kh) recall it is rather vague on this point.
+ Testing actual 1.1 proxy implementations would also be nice. If
+ Expires: <now> is not added by proxies then we need to always
+ include Expires: <now> ourselves to ensure correct caching, but
+ this would degrade HTTP/1.1 cache efficiency unless we also add
+ Cache-Control: max-age=N, which we currently don't. */
+
+ /* XXX: TODO: We need to fix things so that setting
+ r->no_cache to 1 overrides anything that mod_expires would
+ want to do. Currently r->no_cache==1 will not add expires:
+ <now> if mod_expires is used, instead mod_expires will add
+ some Expires: <future> header which breaks correct caching
+ of negotiated responses. */
+
+ if ((!do_cache_negotiated_docs(r->server)
+ && (r->proto_num < HTTP_VERSION(1,1)))
+ && neg->count_multiviews_variants != 1) {
r->no_cache = 1;
}
- if (na_result == na_not_applied) {
- set_neg_headers(r, neg, na_not_applied);
+ return OK;
+}
+
+
+static int handle_map_file(request_rec *r)
+{
+ negotiation_state *neg = parse_accept_headers(r);
+ var_rec *best;
+ int res;
+
+ char *udir;
+
+ if ((res = read_type_map(neg, r))) {
+ return res;
}
+ res = do_negotiation(r, neg, &best, 0);
+ if (res != 0) return res;
+
if (r->path_info && *r->path_info) {
r->uri[ap_find_path_info(r->uri, r->path_info)] = '\0';
}
@@ -2095,7 +2612,7 @@
request_rec *sub_req;
int res;
int j;
- int na_result; /* result of network algorithm */
+ char *variant_etag; /* TODO: check if still needed */
if (r->finfo.st_mode != 0 || !(ap_allow_options(r) & OPT_MULTI)) {
return DECLINED;
@@ -2119,45 +2636,21 @@
return DECLINED;
}
- maybe_add_default_encodings(neg,
- (r->method_number != M_GET) ||
- r->args || r->path_info);
-
- na_result = best_match(neg, &best);
- if (na_result == na_list) {
- /*
- * Network algorithm tols us to output a "List" response.
- * This is output at a 300 status code, which we will
- * return. The list of variants will be stored in r->notes
- * under the name "variants-list".
- */
- set_neg_headers(r, neg, na_list); /* set Alternates: and Vary: */
-
- store_variant_list(r, neg);
- res = MULTIPLE_CHOICES;
- goto return_from_multi;
- }
-
- if (!best) {
- ap_log_rerror(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, r,
- "no acceptable variant: %s", r->filename);
-
- set_neg_headers(r, neg, na_result);
- store_variant_list(r, neg);
- res = NOT_ACCEPTABLE;
- goto return_from_multi;
- }
-
- if (na_result == na_choice) {
- if ((res = setup_choice_response(r, neg, best)) != 0) {
- goto return_from_multi;
- }
- }
+ /* XXX: The calculation of the fourth (prefer_scripts) argument is
+ buggy, we always calculate 1 because r->path_info is never NULL
+ but always an empty string. (I don't think r->args and
+ r->path_info can be non-empty strings anyway if a multiviews is
+ invoked.) The bug in the calculation is offset by another bug
+ in maybe_add_default_encodings() however, see the comment there
+ for more info. -kh */
+ res = do_negotiation(r, neg, &best,
+ (r->method_number != M_GET) ||
+ r->args || r->path_info);
+ if (res!=0) goto return_from_multi;
if (!(sub_req = best->sub_req)) {
/* We got this out of a map file, so we don't actually have
- * a sub_req structure yet. Get one now.
- */
+ * a sub_req structure yet. Get one now. */
sub_req = ap_sub_req_lookup_file(best->file_name, r);
if (sub_req->status != HTTP_OK) {
@@ -2176,15 +2669,6 @@
/* Otherwise, use it. */
- if ((!do_cache_negotiated_docs(r->server) && (r->proto_num <
HTTP_VERSION(1,1)))
- && neg->count_multiviews_variants != 1) {
- r->no_cache = 1;
- }
-
- if (na_result == na_not_applied) {
- set_neg_headers(r, neg, na_not_applied);
- }
-
/* now do a "fast redirect" ... promote the sub_req into the main req */
/* We need to tell POOL_DEBUG that we're guaranteeing that sub_req->pool
* will exist as long as r->pool. Otherwise we run into troubles because
@@ -2192,6 +2676,7 @@
* sub_req->pool.
*/
ap_pool_join(r->pool, sub_req->pool);
+ r->mtime = 0; /* reset etag info for subrequest */
r->filename = sub_req->filename;
r->handler = sub_req->handler;
r->content_type = sub_req->content_type;
@@ -2218,7 +2703,8 @@
return OK;
}
-/* There is a problem with content-encoding, as some clients send and
+/**********************************************************************
+ * There is a problem with content-encoding, as some clients send and
* expect an x- token (e.g. x-gzip) while others expect the plain token
* (i.e. gzip). To try and deal with this as best as possible we do
* the following: if the client sent an Accept-Encoding header and it
@@ -2237,7 +2723,6 @@
const char *enc = r->content_encoding;
char *x_enc = NULL;
array_header *accept_encodings;
- accept_rec *accept_recs;
int i;
if (!enc || !*enc) {
@@ -2250,24 +2735,26 @@
accept_encodings = do_header_line(r->pool,
ap_table_get(r->headers_in,
"Accept-encoding"));
- accept_recs = (accept_rec *) accept_encodings->elts;
-
- for (i = 0; i < accept_encodings->nelts; ++i) {
- char *name = accept_recs[i].type_name;
-
- if (!strcmp(name, enc)) {
- r->content_encoding = name;
- return OK;
- }
-
- if (name[0] == 'x' && name[1] == '-' && !strcmp(name+2, enc)) {
- x_enc = name;
- }
- }
+ if (accept_encodings) {
+ accept_rec *accept_recs = (accept_rec *) accept_encodings->elts;
+
+ for (i = 0; i < accept_encodings->nelts; ++i) {
+ char *name = accept_recs[i].name;
+
+ if (!strcmp(name, enc)) {
+ r->content_encoding = name;
+ return OK;
+ }
+
+ if (name[0] == 'x' && name[1] == '-' && !strcmp(name+2, enc)) {
+ x_enc = name;
+ }
+ }
- if (x_enc) {
- r->content_encoding = x_enc;
- return OK;
+ if (x_enc) {
+ r->content_encoding = x_enc;
+ return OK;
+ }
}
return DECLINED;
>Audit-Trail:
>Unformatted:
[In order for any reply to be added to the PR database, ]
[you need to include <[EMAIL PROTECTED]> in the Cc line ]
[and leave the subject line UNCHANGED. This is not done]
[automatically because of the potential for mail loops. ]
[If you do not include this Cc, your reply may be ig- ]
[nored unless you are responding to an explicit request ]
[from a developer. ]
[Reply only with text; DO NOT SEND ATTACHMENTS! ]
