RE: Http Connector / Gzip
For some reason I've been added to this developer's list. Could you remove me, I have over 450 e-mails in my mailbox? I think it was due to the internet worm that went around in August. --- Peter Lin <[EMAIL PROTECTED]> wrote: > > interesting results. By any chance did you compare > it > to Perl regexp to see the difference? :) > > peter > > > > --- Chad Johnson <[EMAIL PROTECTED]> wrote: > > Some regex benchmark's I ran across: > > > > http://tusker.org/regex/regex_benchmark.html > > > > -Chad Johnson > > > > -Original Message- > > From: Henri Gomez [mailto:[EMAIL PROTECTED] > > Sent: Monday, October 06, 2003 11:15 AM > > To: Tomcat Developers List > > Subject: Re: Http Connector / Gzip > > > > > > Remy Maucherat a écrit : > > > > > Henri Gomez wrote: > > > > > >> Hi to all, > > >> > > >> What about using regexp in HTTP 1.1 connector > to > > include/exclude > > >> browser which could/couldn't use gzip > compression > > ? > > >> > > >> May be it could be use also to drop to HTTP 1.0 > > browser which > > >> claims to be 1.1 compatible, but are not. > > >> > > >> CF: Apache HTTPD server > > > > > > > > > Well, I thought regexp was rather slow in Java, > so > > it could be a problem > > > if using a generic regexp library. > > > > jakarta-regexp is too slow ? > > > > > > > - > > To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > For additional commands, e-mail: > > [EMAIL PROTECTED] > > > > > > > - > > To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > For additional commands, e-mail: > > [EMAIL PROTECTED] > > > > > __ > Do you Yahoo!? > The New Yahoo! Shopping - with improved product > search > http://shopping.yahoo.com > > - > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > __ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
Remy Maucherat a écrit : Henri Gomez wrote: Remy Maucherat a écrit : +0 if disabled by default; -1 otherwise. All this stuff is inefficient, as it uses Strings. Ie, we take the byte array we have for the header value, convert it to a String, and then regexp will likely convert it to something else to work with it (likely a char array). This is bad, obviously :-( I don't know if the JDK 1.4 stuff can work with bytes, but in any case, the benchmarks seem a lot faster. So we need a simple byte-based regexp :) (sort of like the mapper) May be CharacterIterator could be better in our case ? public boolean match(CharacterIterator search, int i) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
Remy Maucherat a écrit : Henri Gomez wrote: Could I commit my changes and start working on an optimizing version ? Yes, of course. There's no big rush to start optimizing this ;-) (it's really disabled by default, right ?) Yes, since the restrictedUserAgents and restrictedUserAgents are set to null in Http11Protocol until someone set the properties in server.xml. Trust me ;-) BTW, I'll update the build.xml but we need to have regexp.jar loaded somewhere in jtc... - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
Henri Gomez wrote: Could I commit my changes and start working on an optimizing version ? Yes, of course. There's no big rush to start optimizing this ;-) (it's really disabled by default, right ?) Remy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
Well I take my inspiration from what is done in some TC 4.1 Valves... I know ;-) Some problematic stuff went away, though (the various mappers), but some remain. This feature would really need to be seriously optimized, as ideally this would be enabled by default. We could do better as usually... Yes. Could I commit my changes and start working on an optimizing version ? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
Henri Gomez wrote: Remy Maucherat a écrit : +0 if disabled by default; -1 otherwise. All this stuff is inefficient, as it uses Strings. Ie, we take the byte array we have for the header value, convert it to a String, and then regexp will likely convert it to something else to work with it (likely a char array). This is bad, obviously :-( I don't know if the JDK 1.4 stuff can work with bytes, but in any case, the benchmarks seem a lot faster. So we need a simple byte-based regexp :) (sort of like the mapper) Well I take my inspiration from what is done in some TC 4.1 Valves... I know ;-) Some problematic stuff went away, though (the various mappers), but some remain. This feature would really need to be seriously optimized, as ideally this would be enabled by default. We could do better as usually... Yes. Remy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
Remy Maucherat a écrit : Henri Gomez wrote: Remy Maucherat a écrit : Henri Gomez wrote: Hi to all, What about using regexp in HTTP 1.1 connector to include/exclude browser which could/couldn't use gzip compression ? May be it could be use also to drop to HTTP 1.0 browser which claims to be 1.1 compatible, but are not. CF: Apache HTTPD server Well, I thought regexp was rather slow in Java, so it could be a problem if using a generic regexp library. You could find attached a patch to make use of regexp in HTTP 1.1. It check via jakarta-regexp which browser should be exclude from gzip but also from HTTP 1.1 (even if they claim to be compatible). (regexp is also used in some valves) With such add-on we mimic what HTTP teams does in Apache 2.0 via BrowserMatch. Nota their approach may be better since BrowserMatch set some properties like : nokeepalive, downgrade-1.0, force-response-1.0, nogzip Something which may be nice to implements (may we could use attributes or notes ?) You should note that if no restrictedUserAgents or noCompressionUserAgents are set in HTTP connector settings, there is no check done, so no performance penalty. There is case where it should be done so better have provision for +0 if disabled by default; -1 otherwise. All this stuff is inefficient, as it uses Strings. Ie, we take the byte array we have for the header value, convert it to a String, and then regexp will likely convert it to something else to work with it (likely a char array). This is bad, obviously :-( I don't know if the JDK 1.4 stuff can work with bytes, but in any case, the benchmarks seem a lot faster. So we need a simple byte-based regexp :) (sort of like the mapper) Well I take my inspiration from what is done in some TC 4.1 Valves... We could do better as usually... - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
Henri Gomez wrote: Remy Maucherat a écrit : Henri Gomez wrote: Hi to all, What about using regexp in HTTP 1.1 connector to include/exclude browser which could/couldn't use gzip compression ? May be it could be use also to drop to HTTP 1.0 browser which claims to be 1.1 compatible, but are not. CF: Apache HTTPD server Well, I thought regexp was rather slow in Java, so it could be a problem if using a generic regexp library. You could find attached a patch to make use of regexp in HTTP 1.1. It check via jakarta-regexp which browser should be exclude from gzip but also from HTTP 1.1 (even if they claim to be compatible). (regexp is also used in some valves) With such add-on we mimic what HTTP teams does in Apache 2.0 via BrowserMatch. Nota their approach may be better since BrowserMatch set some properties like : nokeepalive, downgrade-1.0, force-response-1.0, nogzip Something which may be nice to implements (may we could use attributes or notes ?) You should note that if no restrictedUserAgents or noCompressionUserAgents are set in HTTP connector settings, there is no check done, so no performance penalty. There is case where it should be done so better have provision for +0 if disabled by default; -1 otherwise. All this stuff is inefficient, as it uses Strings. Ie, we take the byte array we have for the header value, convert it to a String, and then regexp will likely convert it to something else to work with it (likely a char array). This is bad, obviously :-( I don't know if the JDK 1.4 stuff can work with bytes, but in any case, the benchmarks seem a lot faster. So we need a simple byte-based regexp :) (sort of like the mapper) Remy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
>>>
Here is the patch against latest remy changes ...
Index: build.properties.sample
===
RCS file: /home/cvs/jakarta-tomcat-connectors/http11/build.properties.sample,v
retrieving revision 1.2
diff -u -r1.2 build.properties.sample
--- build.properties.sample 15 Mar 2002 05:36:02 - 1.2
+++ build.properties.sample 7 Oct 2003 07:39:41 -
@@ -31,4 +31,7 @@
junit.jar=/java/junit/junit.jar
# commons-logging.jar -- Commons Logging 1.0 package
-commons-logging.jar=../lib/commons-logging.jar
\ No newline at end of file
+commons-logging.jar=../lib/commons-logging.jar
+
+# - Jakarta Regular Expressions Library, version 1.2 -
+regexp.jar=../lib/jakarta-regexp-1.2.jar
Index: build.xml
===
RCS file: /home/cvs/jakarta-tomcat-connectors/http11/build.xml,v
retrieving revision 1.14
diff -u -r1.14 build.xml
--- build.xml 4 May 2003 21:12:01 - 1.14
+++ build.xml 7 Oct 2003 07:39:41 -
@@ -38,6 +38,7 @@
+
@@ -82,6 +83,7 @@
+
Index: src/java/org/apache/coyote/http11/Http11Processor.java
===
RCS file:
/home/cvs/jakarta-tomcat-connectors/http11/src/java/org/apache/coyote/http11/Http11Processor.java,v
retrieving revision 1.81
diff -u -r1.81 Http11Processor.java
--- src/java/org/apache/coyote/http11/Http11Processor.java 1 Oct 2003 10:22:15
- 1.81
+++ src/java/org/apache/coyote/http11/Http11Processor.java 7 Oct 2003 07:39:42
-
@@ -81,6 +81,8 @@
import org.apache.coyote.http11.filters.IdentityOutputFilter;
import org.apache.coyote.http11.filters.VoidInputFilter;
import org.apache.coyote.http11.filters.VoidOutputFilter;
+import org.apache.regexp.RE;
+import org.apache.regexp.RESyntaxException;
import org.apache.tomcat.util.buf.Ascii;
import org.apache.tomcat.util.buf.ByteChunk;
import org.apache.tomcat.util.buf.HexUtils;
@@ -203,7 +205,7 @@
/**
* List of restricted user agents.
*/
-protected String[] restrictedUserAgents = null;
+protected RE[] restrictedUserAgents = null;
/**
@@ -281,9 +283,8 @@
/**
* List of user agents to not use gzip with
*/
-protected String[] noCompressionUserAgents = null;
-
-
+protected RE noCompressionUserAgents[] = null;
+
/**
* List of MIMES which could be gzipped
*/
@@ -364,7 +365,10 @@
* @param userAgent user-agent string
*/
public void addNoCompressionUserAgent(String userAgent) {
-addStringArray(noCompressionUserAgents, userAgent);
+try {
+RE nRule = new RE(userAgent);
+addREArray(noCompressionUserAgents, new RE(userAgent));
+} catch (RESyntaxException ree) {}
}
@@ -373,7 +377,7 @@
* a large number of connectors, where it would be better to have all of
* them referenced a single array).
*/
-public void setNoCompressionUserAgents(String[] noCompressionUserAgents) {
+public void setNoCompressionUserAgents(RE[] noCompressionUserAgents) {
this.noCompressionUserAgents = noCompressionUserAgents;
}
@@ -394,15 +398,6 @@
}
}
-
-/**
- * Return the list of no compression user agents.
- */
-public String[] findNoCompressionUserAgents() {
-return (noCompressionUserAgents);
-}
-
-
/**
* Add a mime-type which will be compressable
* The mime-type String will be exactly matched
@@ -483,13 +478,38 @@
* @param value string
*/
private void addStringArray(String sArray[], String value) {
-if (sArray == null)
-sArray = new String[0];
-String[] results = new String[sArray.length + 1];
-for (int i = 0; i < sArray.length; i++)
-results[i] = sArray[i];
-results[sArray.length] = value;
-sArray = results;
+if (sArray == null) {
+sArray = new String[1];
+sArray[0] = value;
+}
+else {
+String[] results = new String[sArray.length + 1];
+for (int i = 0; i < sArray.length; i++)
+results[i] = sArray[i];
+results[sArray.length] = value;
+sArray = results;
+}
+}
+
+
+/**
+ * General use method
+ *
+ * @param rArray the REArray
+ * @param value Obj
+ */
+private void addREArray(RE rArray[], RE value) {
+if (rArray == null) {
+rArray = new RE[1];
+rArray[0] = value;
+}
+else {
+RE[] results = new RE[rArray.length + 1];
+for (int i = 0; i < rArray.length; i++)
+results[i] = rArray[i];
+results[rArray.length] = value;
+rArray = results;
+}
}
@@ -529,13 +549,16 @@
/**
* Add restric
Re: Http Connector / Gzip
Remy Maucherat a écrit :
Henri Gomez wrote:
Hi to all,
What about using regexp in HTTP 1.1 connector to include/exclude
browser which could/couldn't use gzip compression ?
May be it could be use also to drop to HTTP 1.0 browser which
claims to be 1.1 compatible, but are not.
CF: Apache HTTPD server
Well, I thought regexp was rather slow in Java, so it could be a problem
if using a generic regexp library.
You could find attached a patch to make use of regexp in HTTP 1.1.
It check via jakarta-regexp which browser should be exclude from
gzip but also from HTTP 1.1 (even if they claim to be compatible).
(regexp is also used in some valves)
With such add-on we mimic what HTTP teams does in Apache 2.0 via
BrowserMatch.
Nota their approach may be better since BrowserMatch set some properties
like :
nokeepalive, downgrade-1.0, force-response-1.0, nogzip
Something which may be nice to implements (may we could use attributes
or notes ?)
You should note that if no restrictedUserAgents or
noCompressionUserAgents are set in HTTP connector settings,
there is no check done, so no performance penalty.
There is case where it should be done so better have provision for
Regards
Index: build.properties.sample
===
RCS file: /home/cvs/jakarta-tomcat-connectors/http11/build.properties.sample,v
retrieving revision 1.2
diff -u -r1.2 build.properties.sample
--- build.properties.sample 15 Mar 2002 05:36:02 - 1.2
+++ build.properties.sample 7 Oct 2003 07:31:39 -
@@ -31,4 +31,7 @@
junit.jar=/java/junit/junit.jar
# commons-logging.jar -- Commons Logging 1.0 package
-commons-logging.jar=../lib/commons-logging.jar
\ No newline at end of file
+commons-logging.jar=../lib/commons-logging.jar
+
+# - Jakarta Regular Expressions Library, version 1.2 -
+regexp.jar=../lib/jakarta-regexp-1.2.jar
Index: build.xml
===
RCS file: /home/cvs/jakarta-tomcat-connectors/http11/build.xml,v
retrieving revision 1.14
diff -u -r1.14 build.xml
--- build.xml 4 May 2003 21:12:01 - 1.14
+++ build.xml 7 Oct 2003 07:31:40 -
@@ -38,6 +38,7 @@
+
@@ -82,6 +83,7 @@
+
Index: src/java/org/apache/coyote/http11/Http11Processor.java
===
RCS file:
/home/cvs/jakarta-tomcat-connectors/http11/src/java/org/apache/coyote/http11/Http11Processor.java,v
retrieving revision 1.81
diff -u -r1.81 Http11Processor.java
--- src/java/org/apache/coyote/http11/Http11Processor.java 1 Oct 2003 10:22:15
- 1.81
+++ src/java/org/apache/coyote/http11/Http11Processor.java 7 Oct 2003 07:31:40
-
@@ -81,6 +81,8 @@
import org.apache.coyote.http11.filters.IdentityOutputFilter;
import org.apache.coyote.http11.filters.VoidInputFilter;
import org.apache.coyote.http11.filters.VoidOutputFilter;
+import org.apache.regexp.RE;
+import org.apache.regexp.RESyntaxException;
import org.apache.tomcat.util.buf.Ascii;
import org.apache.tomcat.util.buf.ByteChunk;
import org.apache.tomcat.util.buf.HexUtils;
@@ -203,7 +205,7 @@
/**
* List of restricted user agents.
*/
-protected String[] restrictedUserAgents = null;
+protected RE[] restrictedUserAgents = null;
/**
@@ -281,9 +283,8 @@
/**
* List of user agents to not use gzip with
*/
-protected String[] noCompressionUserAgents = null;
-
-
+protected RE noCompressionUserAgents[] = null;
+
/**
* List of MIMES which could be gzipped
*/
@@ -364,7 +365,10 @@
* @param userAgent user-agent string
*/
public void addNoCompressionUserAgent(String userAgent) {
-addStringArray(noCompressionUserAgents, userAgent);
+try {
+RE nRule = new RE(userAgent);
+addREArray(noCompressionUserAgents, new RE(userAgent));
+} catch (RESyntaxException ree) {}
}
@@ -373,7 +377,7 @@
* a large number of connectors, where it would be better to have all of
* them referenced a single array).
*/
-public void setNoCompressionUserAgents(String[] noCompressionUserAgents) {
+public void setNoCompressionUserAgents(RE[] noCompressionUserAgents) {
this.noCompressionUserAgents = noCompressionUserAgents;
}
@@ -394,15 +398,6 @@
}
}
-
-/**
- * Return the list of no compression user agents.
- */
-public String[] findNoCompressionUserAgents() {
-return (noCompressionUserAgents);
-}
-
-
/**
* Add a mime-type which will be compressable
* The mime-type String will be exactly matched
@@ -483,13 +478,38 @@
* @param value string
*/
private void addStringArray(String sArray[], String value) {
-if (sArray == null)
-sArray = new String[0];
-String[] results = new String[sArray.l
RE: Http Connector / Gzip
Peter, I did a quick search around the net looking for those benchmarks. Here's what I found: http://www.bagley.org/~doug/shootout/bench/regexmatch/ Looks like the "java" benchmarks used the ORO library (pre jakarta oro it looks like). -Chad Johnson -Original Message- From: Peter Lin [mailto:[EMAIL PROTECTED] Sent: Monday, October 06, 2003 11:29 AM To: Tomcat Developers List Subject: RE: Http Connector / Gzip interesting results. By any chance did you compare it to Perl regexp to see the difference? :) peter --- Chad Johnson <[EMAIL PROTECTED]> wrote: > Some regex benchmark's I ran across: > > http://tusker.org/regex/regex_benchmark.html > > -Chad Johnson > > -Original Message- > From: Henri Gomez [mailto:[EMAIL PROTECTED] > Sent: Monday, October 06, 2003 11:15 AM > To: Tomcat Developers List > Subject: Re: Http Connector / Gzip > > > Remy Maucherat a écrit : > > > Henri Gomez wrote: > > > >> Hi to all, > >> > >> What about using regexp in HTTP 1.1 connector to > include/exclude > >> browser which could/couldn't use gzip compression > ? > >> > >> May be it could be use also to drop to HTTP 1.0 > browser which > >> claims to be 1.1 compatible, but are not. > >> > >> CF: Apache HTTPD server > > > > > > Well, I thought regexp was rather slow in Java, so > it could be a problem > > if using a generic regexp library. > > jakarta-regexp is too slow ? > > > - > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > > > - > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > __ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
Peter Lin wrote: interesting results. By any chance did you compare it to Perl regexp to see the difference? :) That would be interesting indeed :) Do you know what the HTTPd uses, BTW ? Remy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Http Connector / Gzip
interesting results. By any chance did you compare it to Perl regexp to see the difference? :) peter --- Chad Johnson <[EMAIL PROTECTED]> wrote: > Some regex benchmark's I ran across: > > http://tusker.org/regex/regex_benchmark.html > > -Chad Johnson > > -Original Message- > From: Henri Gomez [mailto:[EMAIL PROTECTED] > Sent: Monday, October 06, 2003 11:15 AM > To: Tomcat Developers List > Subject: Re: Http Connector / Gzip > > > Remy Maucherat a écrit : > > > Henri Gomez wrote: > > > >> Hi to all, > >> > >> What about using regexp in HTTP 1.1 connector to > include/exclude > >> browser which could/couldn't use gzip compression > ? > >> > >> May be it could be use also to drop to HTTP 1.0 > browser which > >> claims to be 1.1 compatible, but are not. > >> > >> CF: Apache HTTPD server > > > > > > Well, I thought regexp was rather slow in Java, so > it could be a problem > > if using a generic regexp library. > > jakarta-regexp is too slow ? > > > - > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > > > - > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > __ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Http Connector / Gzip
Some regex benchmark's I ran across: http://tusker.org/regex/regex_benchmark.html -Chad Johnson -Original Message- From: Henri Gomez [mailto:[EMAIL PROTECTED] Sent: Monday, October 06, 2003 11:15 AM To: Tomcat Developers List Subject: Re: Http Connector / Gzip Remy Maucherat a écrit : > Henri Gomez wrote: > >> Hi to all, >> >> What about using regexp in HTTP 1.1 connector to include/exclude >> browser which could/couldn't use gzip compression ? >> >> May be it could be use also to drop to HTTP 1.0 browser which >> claims to be 1.1 compatible, but are not. >> >> CF: Apache HTTPD server > > > Well, I thought regexp was rather slow in Java, so it could be a problem > if using a generic regexp library. jakarta-regexp is too slow ? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
Remy Maucherat a écrit : Henri Gomez wrote: Hi to all, What about using regexp in HTTP 1.1 connector to include/exclude browser which could/couldn't use gzip compression ? May be it could be use also to drop to HTTP 1.0 browser which claims to be 1.1 compatible, but are not. CF: Apache HTTPD server Well, I thought regexp was rather slow in Java, so it could be a problem if using a generic regexp library. jakarta-regexp is too slow ? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http Connector / Gzip
Henri Gomez wrote: Hi to all, What about using regexp in HTTP 1.1 connector to include/exclude browser which could/couldn't use gzip compression ? May be it could be use also to drop to HTTP 1.0 browser which claims to be 1.1 compatible, but are not. CF: Apache HTTPD server Well, I thought regexp was rather slow in Java, so it could be a problem if using a generic regexp library. Remy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
