svn commit: r1553252 - /tomcat/tc6.0.x/trunk/STATUS.txt
Author: kkolinko Date: Tue Dec 24 06:32:59 2013 New Revision: 1553252 URL: http://svn.apache.org/r1553252 Log: Add a note Modified: tomcat/tc6.0.x/trunk/STATUS.txt Modified: tomcat/tc6.0.x/trunk/STATUS.txt URL: http://svn.apache.org/viewvc/tomcat/tc6.0.x/trunk/STATUS.txt?rev=1553252&r1=1553251&r2=1553252&view=diff == --- tomcat/tc6.0.x/trunk/STATUS.txt (original) +++ tomcat/tc6.0.x/trunk/STATUS.txt Tue Dec 24 06:32:59 2013 @@ -66,6 +66,8 @@ PATCHES PROPOSED TO BACKPORT: Update the Eclipse compiler to 4.3.1 http://svn.apache.org/viewvc?view=revision&revision=1539953 +1: markt + +0: kkolinko: A Tomcat6-specific patch is needed. It needs update to jar + exclude lists in TldConfig.java, TldLocationsCache.java, see r1476932. -1: * Add missing J2EE 5 resources - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Support RFC6265 cookie processing
In comments on issue #55917, there was suggestion for refactoring cookie support along the lines described in RFC6265. Reading this RFC, it appears to be more of an effort to standardize the actual behaviour seen on the Internet for different browser and server implementations. The observation is the RFC2109 has received limited adoption and RFC2965 virtually none at all, with most implementations falling back to the original specification released by Netscape that contains certain ambiguities. The Servlet spec’s JavaDoc for Cookie refers to RFC2109 behaviour with caveats around interoperability. It defines version 0 as complying with Netscape’s original specification and version 1 as complying RFC2109 (with the note “Since RFC 2109 is still somewhat new, consider version 1 as experimental; do not use it yet on production sites”). The current implementation uses a number of system properties to control how cookies are validated. In implementing RFC6265 I hope that some of these can be eliminated. If not, I would propose to add configuration options on the Connector or Host objects to allow the configuration to be set separately for different host domains. RFC6265 has separate sections in respect for generating and parsing cookie headers. It follows the practice that generation be strict but parsing be more tolerant of invalid input. Our current implementation generally follows that trend by suppressing invalid input data (after logging). However, for some input data, primary CTLs, it throws an IllegalArgumentException from the connector which does not allow the application to recover. In refactoring, I would propose to simply ignore that input thereby allowing the application to handle it, for example by parsing the header field manually. Cookie parsing in particular needs to be tolerant of cookies set by other sources, including different servers handling other parts of the domain and JavaScript or other client-side code setting values in the browser. In light of this, I propose separating the “Set-Cookie” generation side from the “Cookie” parsing side. Generation == The general principle here would be to use the version property of Cookie to determine the level of verification to perform: if 0 follow RFC6265, if 1 use RFC2109. The primary verification point would be in HttpServletRequest#addCookie() which would use the version in the Cookie instance. Characters will always be converted to octets using the ISO-8859-1 charset; unmappable values will result in an IAE. The Servlet spec requires an IAE be thrown in Cookie’s constructor if the name is not valid pre RFC2109. Both RFC6265 and RFC2109 define the name to be a “token” (per RFC2616 HTTP/1.1) so I would propose to always validate by those rules; this would allow US-ASCII characters except CTLs and separators. This will different from the current implementation that slash “/“ would be treated as a separator which would not be allowed in a name by default; this is consistent with the RFC’s and Glassfish’s implementation and I’m assuming that allowing it in our current implementation is a hangover from where we enabled use of “/“ in values. The spec allows vendors to provide "a configuration option that allows cookie names conforming to the original Netscape Cookie Specification to be accepted” and I propose to retain the system property “org.apache.tomcat.util.http.ServerCookie.STRICT_NAMING” for that. If explicitly set to false, it will verify names using Netscape’s rules and allow "a sequence of characters excluding semi-colon, comma and white space” but also excluding “=“ and CTLs per RFC2616; note this *would* allow 8-bit ISO-8859-1 characters in the name and relax the RFC2109 constraint that "NAMEs that begin with $ are reserved for other uses and must not be used by applications.” The value would not be checked until addCookie() was called and the cookie version is known. This would in principle use RFC6265’s “cookie-value” rule if version == 0 or RFC2109’s “value” rules if version == 1; values that do not conform would result in an IAE from addCookie(). Unlike the current implementation, this would not automatically upgrade the version or add quotes around RFC2109 “values” that did not match the “token” rule. If STRICT_SERVLET_COMPLIANCE is set, the rule for version 0 values would be relaxed to allow any value conforming to the Netscape specification except CTLs; this would effectively add DQUOTE, backslash, and 0x80-0xFF. For more granular control, I propose adding the system property “org.apache.tomcat.util.http.ServerCookie.ALLOW_IN_VALUE” which would take one of the following enum values to determine what octets were allowed: * Netscape * RFC2616_token * RFC2109_value * RFC6265_cookie_octet * Netscape_restricted (limits the permitted characters as recommended in the Servlet spec) * RFC6265_ISO-8859-1 (adds 0x80-0xff to cookie_octet) RFC6265 does allow value to be o
buildbot success in ASF Buildbot on tomcat-trunk
The Buildbot has detected a restored build on builder tomcat-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/tomcat-trunk/builds/5356 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: bb-vm_ubuntu Build Reason: scheduler Build Source Stamp: [branch tomcat/trunk] 1553187 Blamelist: jboynes,markt Build succeeded! sincerely, -The Buildbot
[Bug 55917] Cookie parsing fails hard with ISO-8859-1 values
https://issues.apache.org/bugzilla/show_bug.cgi?id=55917 --- Comment #6 from Jeremy Boynes --- Patch applied to trunk as r1553187 to be included in release 8.0.0 -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r1553187 - in /tomcat/trunk: java/org/apache/tomcat/util/http/Cookies.java test/org/apache/tomcat/util/http/TestCookies.java webapps/docs/changelog.xml
Author: jboynes
Date: Mon Dec 23 19:15:35 2013
New Revision: 1553187
URL: http://svn.apache.org/r1553187
Log:
fix #55917 by allowing 8-bit ISO-8859-1 characters in V0 cookie values
Modified:
tomcat/trunk/java/org/apache/tomcat/util/http/Cookies.java
tomcat/trunk/test/org/apache/tomcat/util/http/TestCookies.java
tomcat/trunk/webapps/docs/changelog.xml
Modified: tomcat/trunk/java/org/apache/tomcat/util/http/Cookies.java
URL:
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/tomcat/util/http/Cookies.java?rev=1553187&r1=1553186&r2=1553187&view=diff
==
--- tomcat/trunk/java/org/apache/tomcat/util/http/Cookies.java (original)
+++ tomcat/trunk/java/org/apache/tomcat/util/http/Cookies.java Mon Dec 23
19:15:35 2013
@@ -508,14 +508,7 @@ public final class Cookies {
private static final int getTokenEndPosition(byte bytes[], int off, int
end,
int version, boolean isName){
int pos = off;
-while (pos < end &&
-(!CookieSupport.isHttpSeparator((char)bytes[pos]) ||
- version == 0 &&
-CookieSupport.ALLOW_HTTP_SEPARATORS_IN_V0 &&
-bytes[pos] != '=' &&
-!CookieSupport.isV0Separator((char)bytes[pos]) ||
- !isName && bytes[pos] == '=' &&
- CookieSupport.ALLOW_EQUALS_IN_VALUE)) {
+while (pos < end && allowInToken(bytes[pos], version, isName)) {
pos++;
}
@@ -525,6 +518,34 @@ public final class Cookies {
return pos;
}
+private static boolean allowInToken(byte b, int version, boolean isName) {
+// byte is signed so cast into a positive int for comparisons
+int octet = ((int)b) & 0xff;
+
+// disallow all controls
+if (octet < 0x20 && octet != 0x09 || octet >= 0x7f && octet < 0xa0) {
+throw new IllegalArgumentException(
+"Control character in cookie value or attribute.");
+}
+
+// values 0xa0-0xff are allowed in V0 values, otherwise disallow
+if (octet >= 0x80) {
+if (isName || version != 0) {
+throw new IllegalArgumentException(
+"Control character in cookie value or attribute.");
+}
+return true;
+}
+
+return !CookieSupport.isHttpSeparator((char) b) ||
+version == 0 &&
+CookieSupport.ALLOW_HTTP_SEPARATORS_IN_V0 &&
+b != '=' &&
+!CookieSupport.isV0Separator((char) b) ||
+!isName && b == '=' &&
+CookieSupport.ALLOW_EQUALS_IN_VALUE;
+}
+
/**
* Given a starting position after an initial quote character, this gets
* the position of the end quote. This escapes anything after a '\' char
Modified: tomcat/trunk/test/org/apache/tomcat/util/http/TestCookies.java
URL:
http://svn.apache.org/viewvc/tomcat/trunk/test/org/apache/tomcat/util/http/TestCookies.java?rev=1553187&r1=1553186&r2=1553187&view=diff
==
--- tomcat/trunk/test/org/apache/tomcat/util/http/TestCookies.java (original)
+++ tomcat/trunk/test/org/apache/tomcat/util/http/TestCookies.java Mon Dec 23
19:15:35 2013
@@ -17,9 +17,113 @@
package org.apache.tomcat.util.http;
+import java.nio.charset.StandardCharsets;
+
+import javax.servlet.http.Cookie;
+
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Ignore;
import org.junit.Test;
public class TestCookies {
+private Cookies cookies;
+
+@Before
+public void init() {
+this.cookies = new Cookies(null);
+}
+
+@Test
+public void skipJsonInV0Value() {
+process("bad={\"v\":1,\"x\":2}; a=b");
+expect(makeCookie("a", "b", 0));
+}
+
+@Test(expected = IllegalArgumentException.class)
+public void disallow8bitInName() {
+process("f\u00f6o=bar");
+}
+
+@Test(expected = IllegalArgumentException.class)
+public void disallowControlInName() {
+process("f\010o=bar");
+}
+
+@Test(expected = IllegalArgumentException.class)
+public void disallow8BitControlInName() {
+process("f\210o=bar");
+}
+
+@Test
+public void allow8BitInV0Value() {
+process("foo=b\u00e1r");
+expect(makeCookie("foo", "b\u00e1r", 0));
+}
+
+@Test(expected = IllegalArgumentException.class)
+public void disallow8bitInV1UnquotedValue() {
+process("$Version=1; foo=b\u00e1r");
+}
+
+@Test
+public void allow8bitInV1QuotedValue() {
+process("$Version=1; foo=\"b\u00e1r\"");
+expect(makeCookie("foo", "b\u00e1r", 1));
+}
+
+@Test(expected = IllegalArgumentException.class)
+public void disallowControlInV0Value() {
+process("foo=b\010r");
+}
Re: [OT] [VOTE] Release Apache Tomcat 8.0.0-RC10
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 23/12/2013 17:35, Christopher Schultz wrote: > On 12/23/13, 6:31 AM, Konstantin Kolinko wrote: >> You cannot build a Windows Installer on an Unix system without >> wine. (The installer is built by NSIS which is a Windows >> executable). > > Does this mean that a complete Tomcat distribution can be built on > a non-Windows system? I didn't realize that. Does the installer > download the appropriate package(s) (NSIS) and run them, or does > NSIS need to be separately installed and configured? > > I think I may have been under the mistaken impression that I could > not build a complete release on my Mac (I do have Wine). Was I > wrong? Remy used to do release builds on a non-Windows system. The NSIS installer hasn't changed much since then so there is a reasonable chance it will work. Mark -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSuH1UAAoJEBDAHFovYFnnUS4QALj0Dz0u9e9wUKcLG7lF21yx nuzLb3AU7wjB9a6FdtwFW0SyKZ2L515w70BZ1PqiG9tBdhL+ZGbVouftUC6YHCza QpQreA4WhGolPVHvGn3yJCN/omFVN0v8SCtvxl8xIaNbGcpkVOFDGDCD7TjKjsvk 4WNfyM0G2fPHwWsCAzphtxxBH/VycONe5nqLBEqVw5g9hwDZcFKY0edD0ItiCLqp taacqfJY3oI3SBswuxVvmIqAFM8Uz5vorlILXv+JxT6hmlNgi9fsSwmIjyYjXPES 7M4wMTfco/gzEJXzsZsGKOC0GqUqaYUTceUUTnfdERA2bnEmX4vEGt1Tg650DThU EiG//FgURIc0BZjPUOFC0H+lnYERqp0Y0M+kwymIqCItggDOoX23gW5wWSYINJHL /I2wajpHR8DNWCX2egOnza7K7odSKI5GXwf27b0sQ1CaO45P75aFDKI7Qd8FkE9Z eQktyPbJe25+KAoSU1BBu07l9lo3xPLjS7rB8wXr1tm6h0Rbol6WHTHcn6EqwYaI DGlCaiLopemCXWFSaW+677qC1ioRo06FD48tCL5sTcwG0HH2spCP6TbvfUgwXtDx 6h80FClFPEJJ23UliHukKcZmMO7+PdTo9Ue+dfdrae6dnwkEawLDUHAuU3sfeHbi PBGsWIET+VvD84JjxT7d =I89c -END PGP SIGNATURE- - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [OT] [VOTE] Release Apache Tomcat 8.0.0-RC10
2013/12/23 Christopher Schultz : > Konstantin, > > On 12/23/13, 6:31 AM, Konstantin Kolinko wrote: >> 2013/12/23 Andrew Carr : >>> Wine not found?? im on a headless server, what did i do wrong? See >>> error below... >>> >>> Clean setup to test: >>> >>> Fresh VM with Ubuntu 13.10 64 bit server >>> Apache Ant(TM) version 1.9.2 compiled on July 14 2013 >>> OpenJDK Runtime Environment (IcedTea 2.3.12) (7u25-2.3.12-4ubuntu3) >>> fresh checkout of 8_0_0RC10 >>> [X] Broken - do not release [] Alpha - go ahead and release as 8.0.0-RC10 alpha >>> >>> >>> >>> BUILD FAILED >>> /home/support/TOMCAT_8_0_0_RC10/build.xml:2011: Execute failed: >>> java.io.IOException: Cannot run program "wine" (in directory >> >> You cannot build a Windows Installer on an Unix system without wine. >> (The installer is built by NSIS which is a Windows executable). > > Does this mean that a complete Tomcat distribution can be built on a > non-Windows system? I didn't realize that. Does the installer download > the appropriate package(s) (NSIS) and run them, or does NSIS need to be > separately installed and configured? > > I think I may have been under the mistaken impression that I could not > build a complete release on my Mac (I do have Wine). Was I wrong? > It does download and unpack NSIS as necessary, I think some of official releases of Tomcat 6 were built using WINE this way. (IIRC a fix in 6.0.25 "Ensure files in conf directory have CRLF line endings when using the Windows installer." and a fix in 6.0.24 ".zip distributions should have windows line endings." were related to this). Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [OT] [VOTE] Release Apache Tomcat 8.0.0-RC10
Konstantin, On 12/23/13, 6:31 AM, Konstantin Kolinko wrote: > 2013/12/23 Andrew Carr : >> Wine not found?? im on a headless server, what did i do wrong? See >> error below... >> >> Clean setup to test: >> >> Fresh VM with Ubuntu 13.10 64 bit server >> Apache Ant(TM) version 1.9.2 compiled on July 14 2013 >> OpenJDK Runtime Environment (IcedTea 2.3.12) (7u25-2.3.12-4ubuntu3) >> fresh checkout of 8_0_0RC10 >> >>> [X] Broken - do not release >>> [] Alpha - go ahead and release as 8.0.0-RC10 alpha >> >> >> >> BUILD FAILED >> /home/support/TOMCAT_8_0_0_RC10/build.xml:2011: Execute failed: >> java.io.IOException: Cannot run program "wine" (in directory > > You cannot build a Windows Installer on an Unix system without wine. > (The installer is built by NSIS which is a Windows executable). Does this mean that a complete Tomcat distribution can be built on a non-Windows system? I didn't realize that. Does the installer download the appropriate package(s) (NSIS) and run them, or does NSIS need to be separately installed and configured? I think I may have been under the mistaken impression that I could not build a complete release on my Mac (I do have Wine). Was I wrong? -chris signature.asc Description: OpenPGP digital signature
Re: [VOTE] Release Apache Tomcat 8.0.0-RC10
Thank you for the clarification Mark. That all makes sense. On Mon, Dec 23, 2013 at 10:08 AM, Mark Thomas wrote: > On 23/12/2013 14:54, Andrew Carr wrote: >> Hello Konstantin, >> >> Thank you very much for the reply. I understand what Wine is and why >> you can't build the installer without it. What I don't understand is >> why the configuration tries to build it by default. > > A release build needs to build the Windows installer. Failing if it > can't is intentional so a release manager knows something is wrong (we > used to have all sorts of issues with optional components that failed > silently during a release build). > >> Wine is not a prerequesite in the documentation. > > That can be fixed. > >> Maybe the default build.properties >> should contain the skip.installer=true. > > -1, for the reasons stated above. > > Mark > > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > -- With Regards, Andrew Carr e. andrewlanec...@gmail.com w. andrew.c...@openlogic.com h. 4235255668 c. 4239489852 a. 101 Francis Drive, Greeneville, TN, 37743 - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r1553126 - /tomcat/trunk/BUILDING.txt
Author: markt Date: Mon Dec 23 15:14:14 2013 New Revision: 1553126 URL: http://svn.apache.org/r1553126 Log: Add a note about creating release builds on non-Windows platforms. Modified: tomcat/trunk/BUILDING.txt Modified: tomcat/trunk/BUILDING.txt URL: http://svn.apache.org/viewvc/tomcat/trunk/BUILDING.txt?rev=1553126&r1=1553125&r2=1553126&view=diff == --- tomcat/trunk/BUILDING.txt (original) +++ tomcat/trunk/BUILDING.txt Mon Dec 23 15:14:14 2013 @@ -250,6 +250,12 @@ You can build them by using the followin (6) Building a full release (as provided via the ASF download pages) +A full release includes the Windows installer which requires a Windows +environment to be available to create it. If not building in a Windows +environment, the build scripts assume that WINE is available. If this is not +the case, the skip.installer property may be set to skip the creation of the +Windows installer. + 1. Configure GPG, if needed If the released artifacts have to be cryptographically signed with a - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 8.0.0-RC10
On 23/12/2013 14:54, Andrew Carr wrote: > Hello Konstantin, > > Thank you very much for the reply. I understand what Wine is and why > you can't build the installer without it. What I don't understand is > why the configuration tries to build it by default. A release build needs to build the Windows installer. Failing if it can't is intentional so a release manager knows something is wrong (we used to have all sorts of issues with optional components that failed silently during a release build). > Wine is not a prerequesite in the documentation. That can be fixed. > Maybe the default build.properties > should contain the skip.installer=true. -1, for the reasons stated above. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 8.0.0-RC10
Hello Konstantin, Thank you very much for the reply. I understand what Wine is and why you can't build the installer without it. What I don't understand is why the configuration tries to build it by default. Wine is not a prerequesite in the documentation. Maybe the default build.properties should contain the skip.installer=true. Also, retrying the build right now. Thank you again, Andrew Carr - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 55921] Cookie values in JSON format are not skipped correctly when parsing fails
https://issues.apache.org/bugzilla/show_bug.cgi?id=55921 --- Comment #1 from Christopher Schultz --- I'm pretty sure that an otherwise unencoded JSON value should be quoted when used in an HTTP header value. Tomcat might handle these cases in a more elegant way (e.g. not choking on the 'stray" comma), but I don't think Tomcat should go out of its way to read these kinds of cookies. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 8.0.0-RC10
2013/12/23 Andrew Carr : > Wine not found?? im on a headless server, what did i do wrong? See > error below... > > Clean setup to test: > > Fresh VM with Ubuntu 13.10 64 bit server > Apache Ant(TM) version 1.9.2 compiled on July 14 2013 > OpenJDK Runtime Environment (IcedTea 2.3.12) (7u25-2.3.12-4ubuntu3) > fresh checkout of 8_0_0RC10 > >> [X] Broken - do not release >> [] Alpha - go ahead and release as 8.0.0-RC10 alpha > > > > BUILD FAILED > /home/support/TOMCAT_8_0_0_RC10/build.xml:2011: Execute failed: > java.io.IOException: Cannot run program "wine" (in directory You cannot build a Windows Installer on an Unix system without wine. (The installer is built by NSIS which is a Windows executable). You can skip the step by setting "skip.installer" property in Ant configuration. (This option is not mentioned in BUILDING.txt though). Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
