-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I've made some cleaning & fixes, updated patches attached.
kaalh
KaalH! a écrit :
> Nicolas Cannasse a écrit :
>>>> I'm not sure I'm following your here.
>>>>
>>>> Why exactly mod_tora for lighty or nginx cannot connect directly to
>>>> the tora host and use the protocol as mod_tora for Apache is doing ?
>>>>
>>> I maybe have a wrong vision of tora.
>>>
>>> You may have think mod_tora "only" as a http server plugin to solve
>>> some memory comsumption issues.
>>>
>>> But for me, programmer and sysadmin (can't choose only one, both are
>>> madly enjoying ), tora must be a scalable way to deploy neko
>>> application in web farms, so the lighttpd/nginx/... implementation
>>> should be able to connect to several tora instances, load balance and
>>> failover between them, like it's done with fastcgi on lighttpd and nginx.
>> I can understand that, but IMHO that's just a matter of configuration.
>> Once the tora server is selected (depending on balancing and failover),
>> you should be able to use tora protocol.
>
>> Or am I missing something else ?
>
> I don't think so :)
>
>>> The proxy paradigm make difficult to implement the tora multipart
>>> content scheme, be able to pass raw data to tora without have to wait
>>> for the CGetMultipart code would be a lot easier to implement in
>>> lighttpd, and seems to be required for nginx (
>>> http://marc.info/?l=nginx&m=121425082522327&w=2 )
>> Yes, this is similar to what mod_tora is doing, for similar reasons I
>> guess, which are the necessity to stream multipart/POST data only when
>> requested to do so.
>
>> I'm not sure however what exactly a nginx/lighty "proxy" (between?) is
>> doing here so you might have to explain it to me first :)
>
> - lighttpd core read the incoming data to a buffers chain (chunkqueue).
> - this "in" chunkqueue is converted by the tora proxy backend to another
> chunkqueue.
> - data is sent from this chunkqueue to the tora server.
> - lighttpd read the response from tora and pass it to the tora backend
> thru another chunckqueue.
> - the backend convert the data from tora to http to another chunkqueue
> - the lighttpd core send the response to the client
>
> I haven't played yet with nignx, but It seems to be the same, except
> that nginx does not wait for any state completion, chains of small
> buffers are piped along handler/filters/upstream(aka proxy) modules, so
> you could start to send the response to the client while still reading
> incoming data, looks amazing.
>
>> It might be possible to modify the tora protocol to let the server parse
>> and send the multipart data without waiting for the CGetMultipart : this
>> should not actually interfere with the client since it's not expecting
>> anything else from the server.
>
> Thanks, I was on the wrong way :)
>
> My lighttpd implementation is now a lot more efficient as it do no
> longer need to do tricky things in the proxy layer, I'm still in
> testing, I plan to use it in production environment in approx one month.
> The only missing feature is data flush, but I don't know yet if it's
> possible with lighttpd.
>
> You need to path the tora server with the attached "tora.diff".
> With CBoundary/CMultipartData/CMultipartDone, multipart data is saved by
> tora in a temporary directory ( TMPDIR environment variable, default set
> to "/tmp" ), and only decoded while calling parse_multipart_data.
>
> lighttpd patching instructions are the same as previous, updated patch
> attached.
>
> kaalh
>
>> Nicolas
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFI/yAbWRw620CDc80RApt5AJ0R63KlM5u2FZfUkQf8nHuRU9EHcQCfYaTe
OUact/gp9VJ2kiZTbkbswsU=
=znNy
-----END PGP SIGNATURE-----
diff -ru lighttpd-1.5.0/configure.in lighttpd/configure.in
--- lighttpd-1.5.0/configure.in 2007-09-02 21:46:03.000000000 +0200
+++ lighttpd/configure.in 2008-09-29 20:35:05.000000000 +0200
@@ -632,6 +632,7 @@
mod_proxy_backend_fastcgi \
mod_proxy_backend_scgi \
mod_proxy_backend_ajp13 \
+ mod_proxy_backend_tora \
mod_rrdtool \
mod_secdownload \
mod_setenv \
diff -ru lighttpd-1.5.0/src/Makefile.am lighttpd/src/Makefile.am
--- lighttpd-1.5.0/src/Makefile.am 2007-05-08 10:31:21.000000000 +0200
+++ lighttpd/src/Makefile.am 2008-09-29 20:32:10.000000000 +0200
@@ -243,6 +243,10 @@
mod_proxy_backend_ajp13_la_LDFLAGS = -module -export-dynamic -avoid-version -no-undefined
mod_proxy_backend_ajp13_la_LIBADD = $(common_libadd) $(PCRE_LIB)
+lib_LTLIBRARIES += mod_proxy_backend_tora.la
+mod_proxy_backend_tora_la_SOURCES = mod_proxy_backend_tora.c
+mod_proxy_backend_tora_la_LDFLAGS = -module -export-dynamic -avoid-version -no-undefined
+mod_proxy_backend_tora_la_LIBADD = $(common_libadd) $(PCRE_LIB)
lib_LTLIBRARIES += mod_ssi.la
mod_ssi_la_SOURCES = mod_ssi_exprparser.c mod_ssi_expr.c mod_ssi.c diff -Pru lighttpd-1.5.0/src/mod_proxy_backend_tora.c lighttpd/src/mod_proxy_backend_tora.c
diff -Pru lighttpd-1.5.0/src/plugin.c lighttpd/src/plugin.c
--- lighttpd-1.5.0/src/plugin.c 2007-05-08 12:35:14.000000000 +0200
+++ lighttpd/src/plugin.c 2008-10-06 16:48:35.000000000 +0200
@@ -134,6 +134,7 @@
PLUGIN_STATIC(mod_proxy_backend_fastcgi);
PLUGIN_STATIC(mod_proxy_backend_http);
PLUGIN_STATIC(mod_proxy_backend_scgi);
+PLUGIN_STATIC(mod_proxy_backend_tora);
PLUGIN_STATIC(mod_proxy_core);
PLUGIN_STATIC(mod_redirect);
PLUGIN_STATIC(mod_rewrite);
diff -Pru lighttpd-1.5.0/src/mod_proxy_backend_tora.c lighttpd/src/mod_proxy_backend_tora.c
--- lighttpd-1.5.0/src/mod_proxy_backend_tora.c 1970-01-01 01:00:00.000000000 +0100
+++ lighttpd/src/mod_proxy_backend_tora.c 2008-10-22 14:29:55.000000000 +0200
@@ -0,0 +1,660 @@
+/*
+ * Copyright (c) 2008, KaalH! <[EMAIL PROTECTED]>
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of the <organization> nor the
+ * names of its contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <copyright holder> ''AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL <copyright holder> BE LIABLE FOR ANY
+ * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "mod_proxy_core.h"
+#include "mod_proxy_core_protocol.h"
+#include "configfile.h"
+#include "buffer.h"
+#include "log.h"
+#include "inet_ntop_cache.h"
+
+#define CORE_PLUGIN "mod_proxy_core"
+
+#define TORA_HEADER_LEN 4
+
+#define BUFSIZE (1 << 16)
+
+#define PARSE_HEADER(start,cursor) \
+ cursor = start; \
+ if( *cursor == '"' ) { \
+ start++; \
+ cursor++; \
+ while( *cursor != '"' && *cursor != 0 ) \
+ cursor++; \
+ } else { \
+ while( *cursor != 0 && *cursor != '\r' && *cursor != '\n' && *cursor != '\t' ) \
+ cursor++; \
+ }
+
+typedef struct {
+ PLUGIN_DATA;
+ proxy_protocol *protocol;
+} protocol_plugin_data;
+
+typedef struct {
+ size_t len;
+ off_t offset;
+ int type;
+} tora_packet;
+
+/**
+ * The tora protocol decoder will use this struct for storing state variables
+ * used in decoding the stream
+ */
+typedef struct {
+ buffer *buf; /* holds raw header bytes or used to buffer STDERR */
+ off_t offset; /* parse offset into buffer. */
+ tora_packet packet; /* parsed info about current packet. */
+ size_t chunk_len; /* chunk length */
+ size_t requested_bytes; /* backend requested we send this many bytes of the request content. */
+ buffer *hdkey; /* temporary holds header key */
+} tora_state_data;
+
+typedef enum {
+ CODE_FILE = 1,
+ CODE_URI,
+ CODE_CLIENT_IP,
+ CODE_GET_PARAMS,
+ CODE_POST_DATA,
+ CODE_HEADER_KEY,
+ CODE_HEADER_VALUE,
+ CODE_HEADER_ADD_VALUE,
+ CODE_PARAM_KEY,
+ CODE_PARAM_VALUE,
+ CODE_HOST_NAME,
+ CODE_HTTP_METHOD,
+ CODE_EXECUTE,
+ CODE_ERROR,
+ CODE_PRINT,
+ CODE_LOG,
+ CODE_FLUSH,
+ CODE_REDIRECT,
+ CODE_RETURNCODE,
+ CODE_QUERY_MULTIPART,
+ CODE_PART_FILENAME,
+ CODE_PART_KEY,
+ CODE_PART_DATA,
+ CODE_PART_DONE,
+ CODE_TEST_CONNECT,
+ CODE_BOUNDARY,
+ CODE_MULTIPART_DATA
+} proto_code;
+
+typedef struct {
+ unsigned char type;
+ unsigned char lengthB0;
+ unsigned char lengthB1;
+ unsigned char lengthB2;
+} TORA_Header;
+
+
+void tora_encode_header(buffer *b, proto_code code,int len ) {
+ unsigned char h[4];
+ h[0] = (unsigned char)code;
+ h[1] = (unsigned char)len;
+ h[2] = (unsigned char)(len >> 8);
+ h[3] = (unsigned char)(len >> 16);
+ buffer_append_string_len(b,(char*)h,4);
+}
+void tora_encode_size(buffer *b, proto_code code, const char *str, int len ) {
+ tora_encode_header(b,code,len);
+ buffer_append_string_len(b,str,len);
+}
+void tora_encode(buffer *b, proto_code code, const char *str ) {
+ tora_encode_size(b,code,str,(int)strlen(str));
+}
+
+static char *memfind( char *mem, int mlen, const char *v ) {
+ char *found;
+ int len = (int)strlen(v);
+ if( len == 0 )
+ return mem;
+ while( (found = memchr(mem,*v,mlen)) != NULL ) {
+ if( (int)(found - mem) + len > mlen )
+ break;
+ if( memcmp(found,v,len) == 0 )
+ return found;
+ mlen -= (int)(found - mem + 1);
+ mem = found + 1;
+ }
+ return NULL;
+}
+
+int url_decode( const char *bin, int len, char *bout ) {
+ int pin = 0;
+ int pout = 0;
+ while( len-- > 0 ) {
+ char c = bin[pin++];
+ if( c == '+' )
+ c = ' ';
+ else if( c == '%' ) {
+ int p1, p2;
+ if( len < 2 )
+ break;
+ p1 = bin[pin++];
+ p2 = bin[pin++];
+ len -= 2;
+ if( p1 >= '0' && p1 <= '9' )
+ p1 -= '0';
+ else if( p1 >= 'a' && p1 <= 'f' )
+ p1 -= 'a' - 10;
+ else if( p1 >= 'A' && p1 <= 'F' )
+ p1 -= 'A' - 10;
+ else
+ continue;
+ if( p2 >= '0' && p2 <= '9' )
+ p2 -= '0';
+ else if( p2 >= 'a' && p2 <= 'f' )
+ p2 -= 'a' - 10;
+ else if( p2 >= 'A' && p2 <= 'F' )
+ p2 -= 'A' - 10;
+ else
+ continue;
+ c = (char)((unsigned char)((p1 << 4) + p2));
+ }
+ bout[pout++] = c;
+ }
+ bout[pout] = 0;
+ return pout;
+}
+
+#define DEFAULT_SIZE 256
+
+static void tora_param_encode( buffer *b, proto_code code, const char *str, int len ) {
+ char tmp[DEFAULT_SIZE];
+ char *buf = NULL;
+ int size;
+ if( len >= DEFAULT_SIZE )
+ buf = malloc(len+1);
+ size = url_decode(str,len,buf?buf:tmp);
+ tora_encode_size(b,code,buf?buf:tmp,size);
+ if( buf )
+ free(buf);
+}
+
+void tora_parse_params(buffer *b, buffer *r) {
+ char *aand, *aeq, *asep;
+ char *args = BUF_STR(b);
+
+ while( TRUE ) {
+ aand = strchr(args,'&');
+ if( aand == NULL ) {
+ asep = strchr(args,';');
+ aand = asep;
+ } else {
+ asep = strchr(args,';');
+ if( asep != NULL && asep < aand )
+ aand = asep;
+ }
+ if( aand != NULL )
+ *aand = 0;
+ aeq = strchr(args,'=');
+ if( aeq != NULL ) {
+ *aeq = 0;
+ tora_param_encode(r,CODE_PARAM_KEY,args,(int)(aeq-args));
+ tora_param_encode(r,CODE_PARAM_VALUE,aeq+1,(int)strlen(aeq+1));
+ *aeq = '=';
+ }
+ if( aand == NULL )
+ break;
+ *aand = (aand == asep)?';':'&';
+ args = aand+1;
+ }
+}
+
+tora_state_data *tora_state_data_init(void) {
+ tora_state_data *data;
+
+ data = calloc(1, sizeof(*data));
+ data->buf = buffer_init();
+ data->hdkey = buffer_init();
+ return data;
+}
+
+PROXY_CONNECTION_FUNC(proxy_tora_init) {
+
+ UNUSED(srv);
+
+ if(!proxy_con->protocol_data) {
+ proxy_con->protocol_data = tora_state_data_init();
+ }
+ return 1;
+}
+
+void tora_state_data_free(tora_state_data *data) {
+ buffer_free(data->buf);
+ buffer_free(data->hdkey);
+ free(data);
+}
+
+void tora_state_data_reset(tora_state_data *data) {
+ buffer_reset(data->buf);
+ data->packet.len = 0;
+ data->packet.offset = 0;
+ data->packet.type = 0;
+ data->offset = 0;
+ data->chunk_len = 0;
+}
+
+PROXY_CONNECTION_FUNC(proxy_tora_cleanup) {
+
+ UNUSED(srv);
+
+ if(proxy_con->protocol_data) {
+ tora_state_data_free((tora_state_data *)proxy_con->protocol_data);
+ proxy_con->protocol_data = NULL;
+ }
+ return 1;
+}
+
+/*
+ * copy len bytes from chunk-chain into buffer
+ */
+static int proxy_tora_fill_buffer(tora_state_data *data, chunkqueue *in, size_t len) {
+ off_t we_have = 0, we_need = len;
+ chunk *c;
+
+ for (c = in->first; c && we_need > 0; c = c->next) {
+ if(c->mem->used == 0) continue;
+ we_have = c->mem->used - c->offset - 1;
+ if (we_have == 0) continue;
+ if (we_have > we_need) we_have = we_need;
+ buffer_append_string_len(data->buf, c->mem->ptr + c->offset, we_have);
+ data->packet.offset += we_have;
+ c->offset += we_have;
+ in->bytes_out += we_have;
+ we_need -= we_have;
+ }
+ return we_need;
+}
+
+PROXY_STREAM_DECODER_FUNC(proxy_tora_stream_decoder_internal) {
+ proxy_connection *proxy_con = sess->proxy_con;
+ chunkqueue *in = proxy_con->recv;
+ tora_state_data *data = (tora_state_data *)proxy_con->protocol_data;
+ size_t we_have = 0, we_need = 0;
+ TORA_Header *header;
+ handler_t rc = HANDLER_GO_ON;
+ char buf[32];
+
+ UNUSED(srv);
+
+ if ((in->bytes_in == in->bytes_out) && in->is_closed) {
+ out->is_closed = 1;
+
+
+ TRACE("%ld / %ld -> %d",
+ in->bytes_in, in->bytes_out,
+ in->is_closed);
+
+ ERROR("looks like the tora backend (%s) terminated before it sent a FIN packet", BUF_STR(sess->request_uri));
+
+ return HANDLER_FINISHED;
+ }
+
+ /* no data ? */
+ if (!in->first) return HANDLER_GO_ON;
+
+ if(data->packet.type == 0) {
+ we_need = (TORA_HEADER_LEN - data->packet.offset);
+ we_need = proxy_tora_fill_buffer(data, in, we_need);
+ if(we_need > 0) {
+ /* we need more data to parse the header. */
+ chunkqueue_remove_finished_chunks(in);
+ return HANDLER_GO_ON;
+ }
+ header = (TORA_Header *)(data->buf->ptr);
+ data->packet.type = header->type;
+ data->packet.len = ((header->lengthB0 | header->lengthB1 << 8) | header->lengthB2 << 16);
+ data->packet.offset = 0;
+
+ /* Finished parsing raw header bytes. */
+ buffer_reset(data->buf);
+ }
+
+ /* proccess the packet's contents. */
+ we_need = data->packet.len - data->packet.offset;
+
+ /* for most packet types copy the content into the data buffer */
+ if (data->packet.type != CODE_PRINT) {
+ if(we_need > 0) {
+ /* copy tora packet contents to buffer */
+ we_need = proxy_tora_fill_buffer(data, in, we_need);
+ /* make sure we have the full tora packet content. */
+ if(we_need > 0) {
+ chunkqueue_remove_finished_chunks(in);
+ return HANDLER_GO_ON;
+ }
+ }
+ }
+
+ switch(data->packet.type) {
+ case CODE_EXECUTE:
+ LI_ltostr(buf, out->bytes_in);
+ array_append_key_value(sess->resp->headers, CONST_STR_LEN("Content-Length"), buf, strlen(buf));
+ if(sess->resp->status == -1) {
+ sess->resp->status = 200;
+ }
+ if(array_get_element(sess->resp->headers, CONST_STR_LEN("Content-Type")) == NULL) {
+ array_append_key_value(sess->resp->headers, CONST_STR_LEN("Content-Type"),CONST_STR_LEN("text/html"));
+ }
+ sess->have_response_headers = 1;
+ sess->is_request_finished = 1;
+ rc = HANDLER_FINISHED;
+ break;
+ case CODE_HEADER_KEY:
+ buffer_copy_string_buffer(data->hdkey,data->buf);
+ break;
+ case CODE_HEADER_VALUE:
+ array_append_key_value(sess->resp->headers, CONST_BUF_LEN(data->hdkey), CONST_BUF_LEN(data->buf));
+ break;
+ case CODE_HEADER_ADD_VALUE:
+ array_append_key_value(sess->resp->headers, CONST_BUF_LEN(data->hdkey), CONST_BUF_LEN(data->buf));
+ break;
+ case CODE_PRINT:
+ while(we_need > 0 && in->first != NULL) {
+ we_have = chunkqueue_steal_chunks_len(out, in->first, we_need);
+ we_need -= we_have;
+ in->bytes_out += we_have;
+ out->bytes_in += we_have;
+ chunkqueue_remove_finished_chunks(in);
+ }
+ break;
+ case CODE_LOG:
+ TRACE("[mod_tora] %s", BUF_STR(data->buf));
+ break;
+ case CODE_ERROR:
+ TRACE("[mod_tora] Error: %s", BUF_STR(data->buf));
+ rc = HANDLER_ERROR;
+ break;
+ case CODE_FLUSH:
+ /* TODO: how does lighty flush output ??? */
+ TRACE("[mod_tora] Notice: %s","flush not implemented");
+ break;
+ case CODE_REDIRECT:
+ array_append_key_value(sess->resp->headers, CONST_STR_LEN("Location"), CONST_BUF_LEN(data->buf));
+ sess->resp->status = 301;
+ sess->have_response_headers = 1;
+ break;
+ case CODE_RETURNCODE:
+ sess->resp->status = atoi(BUF_STR(data->buf));
+ break;
+ case CODE_QUERY_MULTIPART:
+ TRACE("[mod_tora] Warning: %s","You must used a patched tora server.");
+ break;
+ default:
+ TRACE("[mod_tora] %s","Unexpected code");
+ rc = HANDLER_ERROR;
+ break;
+ }
+
+ if(we_need == 0) {
+ /* packet finished, reset state for next packet */
+ tora_state_data_reset(data);
+ }
+
+ chunkqueue_remove_finished_chunks(in);
+
+ return rc;
+}
+
+PROXY_STREAM_DECODER_FUNC(proxy_tora_stream_decoder) {
+ proxy_connection *proxy_con = sess->proxy_con;
+ chunkqueue *in = proxy_con->recv;
+ int res;
+
+ if(out->is_closed) return 1;
+ /* decode the whole packet stream */
+ do {
+ /* decode the packet */
+ res = proxy_tora_stream_decoder_internal(srv, sess, out);
+ } while (in->first && res == HANDLER_GO_ON);
+ return res;
+}
+
+
+/**
+ * transform the content-stream into a valid TORA content-stream
+ *
+ * as we don't apply chunked-encoding here, pass it on AS IS
+ */
+PROXY_STREAM_ENCODER_FUNC(proxy_tora_stream_encoder) {
+ proxy_connection *proxy_con = sess->proxy_con;
+ connection *con = sess->remote_con;
+ chunkqueue *out = proxy_con->send;
+ handler_t rc = HANDLER_GO_ON;
+ size_t we_have = 0;
+ buffer *b, *bhm = buffer_init();
+ chunk *c;
+
+ UNUSED(srv);
+
+ /* output queue closed, can't encode any more data. */
+ if(out->is_closed) return HANDLER_FINISHED;
+
+ b = chunkqueue_get_append_buffer(out);
+
+ buffer_copy_string(bhm,get_http_method_name(con->request.http_method));
+
+ if (buffer_is_empty(con->uri.query)) {
+ char *qstr = malloc(con->request.orig_uri->used + 1);
+ if (NULL != (qstr = strchr(con->request.orig_uri->ptr, '?'))) {
+ /** extract query string from request.uri */
+ buffer_copy_string(con->uri.query, qstr + 1);
+ }
+ free(qstr);
+ }
+ if (strcmp(bhm->ptr,"POST") == 0) {
+
+ unsigned int is_multipart = 0;
+ data_string *ds;
+
+ ds = (data_string *) array_get_element(con->request.headers, CONST_STR_LEN("Content-Type"));
+ if(strstr(BUF_STR(ds->value),"multipart/form-data"))
+ is_multipart = 1;
+
+ if(is_multipart == 0) {
+ if(chunkqueue_length(in)> BUFSIZE) {
+ ERROR("[mod_tora] %s","Maximum POST data exceeded. Try using multipart encoding");
+ } else {
+
+ chunkqueue *tmp = chunkqueue_init();
+ buffer *r = buffer_init();
+
+ we_have = chunkqueue_steal_all_chunks(tmp, in);
+ in->bytes_out += we_have;
+ tmp->bytes_in += we_have;
+
+ for (c = tmp->first; c; c = c->next) {
+ buffer_append_string(r,c->mem->ptr + c->offset);
+ }
+
+ chunkqueue_free(tmp);
+
+ tora_encode(b,CODE_HTTP_METHOD, bhm->ptr);
+ tora_encode(b,CODE_POST_DATA, BUF_STR(r));
+ tora_parse_params(r,b);
+
+ buffer_free(r);
+ out->bytes_in += b->used-1;
+ rc = HANDLER_FINISHED;
+ }
+ tora_encode_size(b,CODE_EXECUTE, NULL,0);
+ out->bytes_in += b->used -1;
+
+ } else {
+
+ int len = 0;
+ char *boundstr = NULL;
+ unsigned int boundstr_len;
+ const char *boundary, *bend;
+
+ if( (boundary = strstr(BUF_STR(ds->value),"boundary=")) == NULL ) {
+ return HANDLER_ERROR;
+ }
+ boundary += 9;
+ PARSE_HEADER(boundary,bend);
+ len = (int)(bend - boundary);
+ boundstr_len = len + 2;
+
+ boundstr = (char*)malloc(boundstr_len + 1);
+ boundstr[0] = '-';
+ boundstr[1] = '-';
+ boundstr[boundstr_len] = 0;
+ memcpy(boundstr+2,boundary,len);
+ tora_encode(b,CODE_BOUNDARY, boundstr);
+ free(boundstr);
+ out->bytes_in += b->used -1;
+
+ do {
+ b = chunkqueue_get_append_buffer(out);
+ we_have = chunkqueue_steal_chunks_len(out, in->first,BUFSIZE);
+ if(we_have == 0) break;
+ tora_encode_header(b,CODE_MULTIPART_DATA,we_have);
+ out->bytes_in += b->used -1 + we_have;
+ in->bytes_out += we_have;
+ } while(in->first);
+ b = chunkqueue_get_append_buffer(out);
+ tora_encode_size(b,CODE_EXECUTE, NULL,0);
+ out->bytes_in += b->used -1;
+ rc = HANDLER_FINISHED;
+ }
+ } else {
+ if (!buffer_is_empty(con->uri.query)) {
+ tora_encode(b,CODE_GET_PARAMS, BUF_STR(con->uri.query));
+ tora_parse_params(con->uri.query,b);
+ }
+ tora_encode(b,CODE_HTTP_METHOD, bhm->ptr);
+ tora_encode_size(b,CODE_EXECUTE, NULL,0);
+ out->bytes_in += b->used -1;
+ rc = HANDLER_FINISHED;
+ }
+
+ buffer_free(bhm);
+ //out->is_closed = 1;
+ return rc;
+}
+
+/**
+ * generate a HTTP/1.1 proxy request from the set of request-headers
+ *
+ */
+PROXY_STREAM_ENCODER_FUNC(proxy_tora_encode_request_headers) {
+ proxy_connection *proxy_con = sess->proxy_con;
+ chunkqueue *out = proxy_con->send;
+ connection *con = sess->remote_con;
+ buffer *b;
+ unsigned int i;
+
+ //UNUSED(srv);
+ UNUSED(in);
+
+ b = chunkqueue_get_append_buffer(out);
+
+ tora_encode(b,CODE_FILE,BUF_STR(con->physical.path));
+
+ if (buffer_is_empty(con->request.pathinfo)) {
+ const char *qstr = memfind(con->request.orig_uri->ptr,con->request.orig_uri->used,"?");
+ if(qstr == NULL) {
+ tora_encode(b,CODE_URI,BUF_STR(con->request.orig_uri));
+ } else {
+ tora_encode_size(b,CODE_URI,BUF_STR(con->request.orig_uri),qstr - con->request.orig_uri->ptr);
+ }
+ } else {
+ tora_encode(b,CODE_URI,BUF_STR(con->request.pathinfo));
+ }
+
+ tora_encode(b,CODE_HOST_NAME,BUF_STR(con->server_name));
+ tora_encode(b,CODE_CLIENT_IP, inet_ntop_cache_get_ip(srv, &(con->dst_addr)));
+
+ for (i = 0; i < con->request.headers->used; i++) {
+ data_string *ds;
+ ds = (data_string *)con->request.headers->data[i];
+
+ if (buffer_is_empty(ds->value) || buffer_is_empty(ds->key)) continue;
+ tora_encode(b,CODE_HEADER_KEY,BUF_STR(ds->key));
+ tora_encode(b,CODE_HEADER_VALUE,BUF_STR(ds->value));
+ }
+
+ out->bytes_in += b->used - 1;
+
+ return HANDLER_FINISHED;
+}
+
+INIT_FUNC(mod_proxy_backend_tora_init) {
+ mod_proxy_core_plugin_data *core_data;
+ protocol_plugin_data *p;
+
+ /* get the plugin_data of the core-plugin */
+ core_data = plugin_get_config(srv, CORE_PLUGIN);
+ if(!core_data) return NULL;
+
+ p = calloc(1, sizeof(*p));
+
+ /* define protocol handler callbacks */
+ p->protocol = (core_data->proxy_register_protocol)("tora");
+
+ p->protocol->proxy_stream_init = proxy_tora_init;
+ p->protocol->proxy_stream_cleanup = proxy_tora_cleanup;
+ p->protocol->proxy_stream_decoder = proxy_tora_stream_decoder;
+ p->protocol->proxy_stream_encoder = proxy_tora_stream_encoder;
+ p->protocol->proxy_encode_request_headers = proxy_tora_encode_request_headers;
+
+ return p;
+}
+
+FREE_FUNC(mod_proxy_backend_tora_free) {
+ protocol_plugin_data *p = p_d;
+
+ UNUSED(srv);
+
+ if (!p) return HANDLER_GO_ON;
+
+ free(p);
+
+ return HANDLER_GO_ON;
+}
+
+LI_EXPORT int mod_proxy_backend_tora_plugin_init(plugin *p) {
+ data_string *ds;
+
+ p->version = LIGHTTPD_VERSION_ID;
+ p->name = buffer_init_string("mod_proxy_backend_tora");
+
+ p->init = mod_proxy_backend_tora_init;
+ p->cleanup = mod_proxy_backend_tora_free;
+
+ p->data = NULL;
+
+ ds = data_string_init();
+ buffer_copy_string_len(ds->value, CONST_STR_LEN(CORE_PLUGIN));
+ array_insert_unique(p->required_plugins, (data_unset *)ds);
+
+ return 0;
+}
+
+
Index: Client.hx
===================================================================
RCS file: /cvsroot/neko/libs/mod_tora/server/Client.hx,v
retrieving revision 1.7
diff -u -r1.7 Client.hx
--- Client.hx 7 Oct 2008 10:18:19 -0000 1.7
+++ Client.hx 22 Oct 2008 12:41:47 -0000
@@ -41,6 +41,8 @@
CPartData;
CPartDone;
CTestConnect;
+ CBoundary;
+ CMultipartData;
}
class Client {
@@ -66,6 +68,8 @@
public var httpMethod : String;
public var headersSent : Bool;
public var outputHeaders : List<{ code : Code, str : String }>;
+ public var boundary : String;
+ public var filenames : List<String>;
var key : String;
@@ -76,6 +80,7 @@
headers = new Array();
outputHeaders = new List();
params = new Array();
+ filenames = new List();
}
public function sendHeaders() {
@@ -129,6 +134,8 @@
case CHttpMethod: httpMethod = data;
case CExecute: execute = true; return true;
case CTestConnect: execute = false; return true;
+ case CBoundary: boundary = data;
+ case CMultipartData: storeTempData(data);
case CError: throw data;
default: throw "Unexpected "+Std.string(code);
}
@@ -144,4 +151,17 @@
return h + u;
}
+ function storeTempData(data:String)
+ {
+ var filename = Tora.tmpdir + "/tora-"+haxe.Md5.encode(ip+Std.random(10000000));
+ try {
+ var fo = neko.io.File.write(filename,true);
+ fo.writeString(data);
+ fo.close();
+ filenames.add(filename);
+ } catch(e:Dynamic) {
+ try neko.FileSystem.deleteFile(filename) catch(e:Dynamic) {};
+ neko.Lib.rethrow(e);
+ }
+ }
}
Index: ModNekoApi.hx
===================================================================
RCS file: /cvsroot/neko/libs/mod_tora/server/ModNekoApi.hx,v
retrieving revision 1.9
diff -u -r1.9 ModNekoApi.hx
--- ModNekoApi.hx 10 Oct 2008 17:01:00 -0000 1.9
+++ ModNekoApi.hx 22 Oct 2008 12:41:47 -0000
@@ -15,6 +15,7 @@
/* */
/* ************************************************************************ */
import neko.NativeString;
+import neko.io.File;
import Client.Code;
class ModNekoApi {
@@ -132,6 +133,10 @@
}
function parse_multipart_data( onPart : NativeString -> NativeString -> Void, onData : NativeString -> Int -> Int -> Void ) {
+ if(client.boundary != null) {
+ parse_raw_multipart_data(onPart,onData);
+ return;
+ }
var bufsize = 1 << 16;
client.sendMessage(CQueryMultipart,Std.string(bufsize));
var filename = null;
@@ -170,6 +175,69 @@
neko.Lib.rethrow(error.r);
}
+ function get_next_temp_file_contents ():String {
+ var filename = null;
+ var buffer = null;
+ try {
+ filename = client.filenames.pop();
+ if(filename != null) {
+ buffer = neko.io.File.getContent(filename);
+ try neko.FileSystem.deleteFile(filename) catch(e:Dynamic) {};
+ }
+ } catch(e:Dynamic) {
+ try neko.FileSystem.deleteFile(filename) catch(e:Dynamic) {};
+ for(f in client.filenames)
+ try neko.FileSystem.deleteFile(f) catch(e:Dynamic) {};
+ client.filenames.clear();
+ neko.Lib.rethrow(e);
+ }
+ return buffer;
+ }
+
+ function parse_raw_multipart_data( onPart : NativeString -> NativeString -> Void, onData : NativeString -> Int -> Int -> Void ) {
+ var bufsize = 1 << 16;
+ var buffer = "";
+ var name = null;
+ while( true ) {
+ var partOffset = 0;
+ var bodyOffset = 0;
+ if( name == null) {
+ partOffset = buffer.indexOf(client.boundary+"\r\n");
+ if(partOffset == -1) {
+ buffer += get_next_temp_file_contents();
+ continue;
+ }
+ partOffset += client.boundary.length + 2;
+ bodyOffset = buffer.indexOf("\r\n\r\n",partOffset);
+ if(bodyOffset == -1) {
+ buffer += get_next_temp_file_contents();
+ continue;
+ }
+ bodyOffset += 4;
+ var header = buffer.substr(partOffset,bodyOffset-partOffset);
+ var snpos = header.indexOf('name="') + 6;
+ var enpos = header.indexOf('"',snpos);
+ name = header.substr(snpos,enpos-snpos);
+ var sfpos = header.indexOf('filename="');
+ if(sfpos == -1) {
+ onPart( neko.Lib.haxeToNeko(name), null );
+ } else {
+ sfpos += 10;
+ var efpos = header.indexOf('"',sfpos);
+ onPart( neko.Lib.haxeToNeko(name), neko.Lib.haxeToNeko(header.substr(sfpos,efpos-sfpos)) );
+ }
+ } else {
+ buffer += get_next_temp_file_contents();
+ }
+ var partLen = buffer.indexOf(client.boundary,bodyOffset);
+ var len = (partLen == -1) ? buffer.length - bodyOffset : partLen - bodyOffset - 2;
+ onData( neko.Lib.haxeToNeko(buffer.substr(bodyOffset,len)),0,len);
+ buffer = buffer.substr(bodyOffset+len);
+ if(buffer == "\r\n"+client.boundary+"--\r\n") break;
+ if(partLen != -1) name = null;
+ }
+ }
+
function cgi_flush() {
client.sendHeaders();
client.sendMessage(CFlush,"");
@@ -210,4 +278,4 @@
return v;
}
-}
\ No newline at end of file
+}
Index: Tora.hx
===================================================================
RCS file: /cvsroot/neko/libs/mod_tora/server/Tora.hx,v
retrieving revision 1.34
diff -u -r1.34 Tora.hx
--- Tora.hx 16 Oct 2008 07:00:41 -0000 1.34
+++ Tora.hx 22 Oct 2008 12:41:47 -0000
@@ -342,6 +342,7 @@
}
public static var inst : Tora;
+ public static var tmpdir : String;
static function main() {
var args = neko.Sys.args();
@@ -353,10 +354,12 @@
if( nthreads == null ) nthreads = "32";
var port = Std.parseInt(port);
var nthreads = Std.parseInt(nthreads);
+ tmpdir = neko.Sys.getEnv('TMPDIR');
+ if(tmpdir == null) tmpdir = "/tmp";
inst = new Tora();
log("Starting Tora server on "+host+":"+port+" with "+nthreads+" threads");
inst.init(nthreads);
inst.run(host,port);
}
-}
\ No newline at end of file
+}
--
Neko : One VM to run them all
(http://nekovm.org)
